Asset Systems and Scalability
Filter mode:
Topics
Media
asset system
caching
compression
file format
file io
game engine
hardware
language
memory
performance
threading
urban planning
(null topic)
🗩Speech🟉
Credits
?
?
W, K, P / S, J, N Jump to previous / next timestamp
t / T Toggle theatre / SUPERtheatre mode
V Revert filter to original state Y Select link (requires manual Ctrl-c)
X, ShiftSpace Toggle category and focus previous
v Invert topics / media as per focus
Keyboard Navigation
Global Keys
[, < / ], > Jump to previous / next episodeW, K, P / S, J, N Jump to previous / next timestamp
t / T Toggle theatre / SUPERtheatre mode
V Revert filter to original state Y Select link (requires manual Ctrl-c)
Menu toggling
q Quotes r References f Filter y Link c CreditsIn-Menu and Index Controls
a
w
s
s
d
h
j
k
l
←
↑
↓
↓
→
Esc Close menu / unfocus timestamp
Quotes and References Menus and Index
Enter Jump to timestampQuotes, References and Credits Menus
o Open URL (in new tab)Filter Menu
x, Space Toggle category and focus nextX, ShiftSpace Toggle category and focus previous
v Invert topics / media as per focus
Filter and Link Menus
z Toggle filter / linking modeCredits Menu
Enter Open URL (in new tab)⏫
Previous: 'Compression Followup'
⏫
0:04: Introducing Chris Butcher
0:04: Introducing Chris Butcher
0:04: Introducing Chris Butcher
1:55: Managing large teams is the opposite of pure compression programming
1:55: Managing large teams is the opposite of pure compression programming
1:55: Managing large teams is the opposite of pure compression programming
2:33: Show of hands: Programming team size
2:33: Show of hands: Programming team size
2:33: Show of hands: Programming team size
3:22: 680 people at Bungie working on Destiny
3:22: 680 people at Bungie working on Destiny
3:22: 680 people at Bungie working on Destiny
3:46: Take us back to before Destiny
3:46: Take us back to before Destiny
3:46: Take us back to before Destiny
4:15: Halo's engine and its tag file
4:15: Halo's engine and its tag file
4:15: Halo's engine and its tag file
7:07: Matching structure of the tag file and in memory
7:07: Matching structure of the tag file and in memory
7:07: Matching structure of the tag file and in memory
8:19: Data crawl and fix-up in the development build
8:19: Data crawl and fix-up in the development build
8:19: Data crawl and fix-up in the development build
10:17: Could you talk more about the asset update process?
10:17: Could you talk more about the asset update process?
10:17: Could you talk more about the asset update process?
11:03: Handle system, updating assets
11:03: Handle system, updating assets
11:03: Handle system, updating assets
12:09: Problem with Halo's development build: Slow startup
12:09: Problem with Halo's development build: Slow startup
12:09: Problem with Halo's development build: Slow startup
14:04: Two asset formats: 1) Individual tag format, for development
14:04: Two asset formats: 1) Individual tag format, for development
14:04: Two asset formats: 1) Individual tag format, for development
14:41: Two asset formats: 2) Cache file, for retail
14:41: Two asset formats: 2) Cache file, for retail
14:41: Two asset formats: 2) Cache file, for retail
15:11: Why couldn't you patch the retail file?
15:11: Why couldn't you patch the retail file?
15:11: Why couldn't you patch the retail file?
15:32: We didn't build that system or have the staff to maintain three versions of the game
15:32: We didn't build that system or have the staff to maintain three versions of the game
15:32: We didn't build that system or have the staff to maintain three versions of the game
16:18: The 20-minute rebuild was still okay, though?
16:18: The 20-minute rebuild was still okay, though?
16:18: The 20-minute rebuild was still okay, though?
16:38: Halo's simple asset pipeline
16:38: Halo's simple asset pipeline
16:38: Halo's simple asset pipeline
19:24: Starting work on Destiny in 2008-091 seeking to reconcile the problem of fast iteration vs fast loading
19:24: Starting work on Destiny in 2008-091 seeking to reconcile the problem of fast iteration vs fast loading
19:24: Starting work on Destiny in 2008-091 seeking to reconcile the problem of fast iteration vs fast loading
21:47: So developers and players use the same system?
21:47: So developers and players use the same system?
21:47: So developers and players use the same system?
22:44: Updating assets in Halo, by poking values into memory
22:44: Updating assets in Halo, by poking values into memory
22:44: Updating assets in Halo, by poking values into memory
24:29: Patching and arbitrary updates in Destiny, and platform convergence
24:29: Patching and arbitrary updates in Destiny, and platform convergence
24:29: Patching and arbitrary updates in Destiny, and platform convergence
27:48: Did you know in advance you'd need a different data pack for PS3?
27:48: Did you know in advance you'd need a different data pack for PS3?
27:48: Did you know in advance you'd need a different data pack for PS3?
28:34: Building the Destiny asset pipeline around differences between PS3 and workstations
28:34: Building the Destiny asset pipeline around differences between PS3 and workstations
28:34: Building the Destiny asset pipeline around differences between PS3 and workstations
30:05: If the transformation always has to be fast, maybe certain transformations could not be done at all?
30:05: If the transformation always has to be fast, maybe certain transformations could not be done at all?
30:05: If the transformation always has to be fast, maybe certain transformations could not be done at all?
30:20: Large team problems: Specialisation
30:20: Large team problems: Specialisation
30:20: Large team problems: Specialisation
32:29: Asset development example: Glowing crate
32:29: Asset development example: Glowing crate
32:29: Asset development example: Glowing crate
34:52: Halo tag system problem: Single-data edit and reload, prohibiting structural changes
34:52: Halo tag system problem: Single-data edit and reload, prohibiting structural changes
34:52: Halo tag system problem: Single-data edit and reload, prohibiting structural changes
36:13: So you'd need to look at a graph of data rather than a single piece of data?
36:13: So you'd need to look at a graph of data rather than a single piece of data?
36:13: So you'd need to look at a graph of data rather than a single piece of data?
36:53: Difference between Halo and Destiny: No graph analysis in Halo
36:53: Difference between Halo and Destiny: No graph analysis in Halo
36:53: Difference between Halo and Destiny: No graph analysis in Halo
37:56: Writing SPU code (micro programs) for the PS3
37:56: Writing SPU code (micro programs) for the PS3
37:56: Writing SPU code (micro programs) for the PS3
38:37: Pushing asset analysis into build time, with dependencies between objects and shaders
38:37: Pushing asset analysis into build time, with dependencies between objects and shaders
38:37: Pushing asset analysis into build time, with dependencies between objects and shaders
41:53: Observations: 1) The heavy price of individual freedom with regard to data dependencies
41:53: Observations: 1) The heavy price of individual freedom with regard to data dependencies
41:53: Observations: 1) The heavy price of individual freedom with regard to data dependencies
42:35: Dependency chains
42:35: Dependency chains
42:35: Dependency chains
43:21: Apparently benign dependencies
43:21: Apparently benign dependencies
43:21: Apparently benign dependencies
44:00: So the "meaningfulness" of a dependency is irrelevant?
44:00: So the "meaningfulness" of a dependency is irrelevant?
44:00: So the "meaningfulness" of a dependency is irrelevant?
44:34: Systematically identifying and removing dependencies
44:34: Systematically identifying and removing dependencies
44:34: Systematically identifying and removing dependencies
46:37: A system of one "collection" requires full rebuilds, whereas Halo had every node as its own rebuildable piece
46:37: A system of one "collection" requires full rebuilds, whereas Halo had every node as its own rebuildable piece
46:37: A system of one "collection" requires full rebuilds, whereas Halo had every node as its own rebuildable piece
47:15: The practicality of eliminating dependencies
47:15: The practicality of eliminating dependencies
47:15: The practicality of eliminating dependencies
47:54: City transit projects, without shutting everything down
47:54: City transit projects, without shutting everything down
47:54: City transit projects, without shutting everything down
48:19: Destiny's general-purpose work scheduler
48:19: Destiny's general-purpose work scheduler
48:19: Destiny's general-purpose work scheduler
52:10: Observations: 2) A general system for handling changes to particular data models is probably the wrong thing
52:10: Observations: 2) A general system for handling changes to particular data models is probably the wrong thing
52:10: Observations: 2) A general system for handling changes to particular data models is probably the wrong thing
52:58: Processing assets
52:58: Processing assets
52:58: Processing assets
54:33: Two-level non-blocking asset processing code
54:33: Two-level non-blocking asset processing code
54:33: Two-level non-blocking asset processing code
55:26: The problem of relieving programmers of the cost of crossing asset boundaries
55:26: The problem of relieving programmers of the cost of crossing asset boundaries
55:26: The problem of relieving programmers of the cost of crossing asset boundaries
56:38: Sorcerer's Apprentice job graphs
56:38: Sorcerer's Apprentice job graphs
56:38: Sorcerer's Apprentice job graphs
56:56: Abstract general code usage example: Rebuilding visibility when making an audio file change
56:56: Abstract general code usage example: Rebuilding visibility when making an audio file change
56:56: Abstract general code usage example: Rebuilding visibility when making an audio file change
58:19: Practically building the arbitrary data graph processor in C++11, having underappreciated graph cuts
58:19: Practically building the arbitrary data graph processor in C++11, having underappreciated graph cuts
58:19: Practically building the arbitrary data graph processor in C++11, having underappreciated graph cuts
1:04:18: Regressing to the serial processor to isolate bugs
1:04:18: Regressing to the serial processor to isolate bugs
1:04:18: Regressing to the serial processor to isolate bugs
1:04:46: 15 megatasks in a graph
1:04:46: 15 megatasks in a graph
1:04:46: 15 megatasks in a graph
1:07:08: The single-threaded graph processor's caching system (March 2011)
1:07:08: The single-threaded graph processor's caching system (March 2011)
1:07:08: The single-threaded graph processor's caching system (March 2011)
1:10:42: Two crash-started programs: 1) Port from 32-bit to 64-bit
1:10:42: Two crash-started programs: 1) Port from 32-bit to 64-bit
1:10:42: Two crash-started programs: 1) Port from 32-bit to 64-bit
1:11:05: Two crash-started programs: 2) Graph dump, relaunch and deserialise
1:11:05: Two crash-started programs: 2) Graph dump, relaunch and deserialise
1:11:05: Two crash-started programs: 2) Graph dump, relaunch and deserialise
1:14:00: Problems with the serial graph processor, and circular dependencies
1:14:00: Problems with the serial graph processor, and circular dependencies
1:14:00: Problems with the serial graph processor, and circular dependencies
1:17:46: Destiny's Asset System 2: Graph processor
1:17:46: Destiny's Asset System 2: Graph processor
1:17:46: Destiny's Asset System 2: Graph processor
1:19:45: Destiny's Asset System 2: Task dispatch engine
1:19:45: Destiny's Asset System 2: Task dispatch engine
1:19:45: Destiny's Asset System 2: Task dispatch engine
1:21:37: This all seems perfectly reasonable from the shader writer's standpoint
1:21:37: This all seems perfectly reasonable from the shader writer's standpoint
1:21:37: This all seems perfectly reasonable from the shader writer's standpoint
1:21:51: Nodes collapse in terms of execution but not metadata
1:21:51: Nodes collapse in terms of execution but not metadata
1:21:51: Nodes collapse in terms of execution but not metadata
1:25:07: Problems with Destiny's Asset System 2: Single-threaded graph diff
1:25:07: Problems with Destiny's Asset System 2: Single-threaded graph diff
1:25:07: Problems with Destiny's Asset System 2: Single-threaded graph diff
1:27:05: So parallelising the graph diff is high on your hit list?
1:27:05: So parallelising the graph diff is high on your hit list?
1:27:05: So parallelising the graph diff is high on your hit list?
1:27:16: Graph cuts
1:27:16: Graph cuts
1:27:16: Graph cuts
1:27:58: Can you do the graphs cuts separately from reducing the functionality of the system?
1:27:58: Can you do the graphs cuts separately from reducing the functionality of the system?
1:27:58: Can you do the graphs cuts separately from reducing the functionality of the system?
1:28:20: Implementation of graph cuts
1:28:20: Implementation of graph cuts
1:28:20: Implementation of graph cuts
1:30:02: Explicit links, reducing the changes of merging sub-graphs?
1:30:02: Explicit links, reducing the changes of merging sub-graphs?
1:30:02: Explicit links, reducing the changes of merging sub-graphs?
1:30:18: Reducing unintentional dirtying
1:30:18: Reducing unintentional dirtying
1:30:18: Reducing unintentional dirtying
1:31:14: Better performance, now addressing the problem of cutting the graph to defer asset transformations to runtime
1:31:14: Better performance, now addressing the problem of cutting the graph to defer asset transformations to runtime
1:31:14: Better performance, now addressing the problem of cutting the graph to defer asset transformations to runtime
1:32:04: So going back to the old Halo model, resolving at runtime?
1:32:04: So going back to the old Halo model, resolving at runtime?
1:32:04: So going back to the old Halo model, resolving at runtime?
1:32:30: Establishing an institutional understanding of asset dependencies
1:32:30: Establishing an institutional understanding of asset dependencies
1:32:30: Establishing an institutional understanding of asset dependencies
1:33:05: The moral of the story: Only build systems whose complexity you can totally understand before you start
1:33:05: The moral of the story: Only build systems whose complexity you can totally understand before you start
1:33:05: The moral of the story: Only build systems whose complexity you can totally understand before you start
1:33:59: Thank you, Chris
1:33:59: Thank you, Chris
1:33:59: Thank you, Chris
1:34:10: Shout-out to the team
1:34:10: Shout-out to the team
1:34:10: Shout-out to the team
⏬
Next: 'Technical Direction at Blizzard'
⏬