Stratifying and Multithreading the Lighting — Handmade Hero — Episode Guide

0:02Recap and set the stage for the day continuing with lighting mods

🗩

0:02Recap and set the stage for the day continuing with lighting mods

🗩

0:02Recap and set the stage for the day continuing with lighting mods

🗩

0:18Lighting mods

🖌

0:18Lighting mods

🖌

0:18Lighting mods

🖌

1:04"Stratified sampling" with a few words on random number generation and debug visualisation

🖌

1:04"Stratified sampling" with a few words on random number generation and debug visualisation

🖌

1:04"Stratified sampling" with a few words on random number generation and debug visualisation

🖌

4:50Run the game to see what we see, i.e. no debug visualisation of our ray casting

🏃

4:50Run the game to see what we see, i.e. no debug visualisation of our ray casting

🏃

4:50Run the game to see what we see, i.e. no debug visualisation of our ray casting

🏃

7:33Introduce PushDebugLine() and a debug_line struct to draw the lighting rays

17:10Run the game to see our wild rays

🏃

17:10Run the game to see our wild rays

🏃

17:10Run the game to see our wild rays

🏃

17:51Reduce the line thickness in OutputLightingPoints(), and fix the location of their origin in LightingTest()

20:37Run the game to see our stably located rays

🏃

20:37Run the game to see our stably located rays

🏃

20:37Run the game to see our stably located rays

🏃

21:47Reduce the near clip plane distance in UpdateAndRenderWorld() if in debug mode

22:13Run the game to see our debug visualisation

🏃

22:13Run the game to see our debug visualisation

🏃

22:13Run the game to see our debug visualisation

🏃

22:44Raise the ray's origin in LightingTest()

23:01Run the game and see our lighting ray debug visualisation

🏃

23:01Run the game and see our lighting ray debug visualisation

🏃

23:01Run the game and see our lighting ray debug visualisation

🏃

23:13Add a keybinding to pause the lighting ray debug visualisation

26:24Run the game, and pause and inspect our lighting ray debug visualisation

🏃

26:24Run the game, and pause and inspect our lighting ray debug visualisation

🏃

26:24Run the game, and pause and inspect our lighting ray debug visualisation

🏃

30:30Stratified Hemisphere Sampling

🖌

30:30Stratified Hemisphere Sampling

🖌

30:30Stratified Hemisphere Sampling

🖌

35:46Add XAxis and YAxis to lighting_point for SampleHemisphere() to take and distribute the samples into two halves

41:03Run the game to see our distributed lighting samples

🏃

41:03Run the game to see our distributed lighting samples

🏃

41:03Run the game to see our distributed lighting samples

🏃

44:03Add a SampleTable to lighting_solution for LightingTest() to fill with customised locations

52:47Run the game to see no flicker because the rays are not random

🏃

52:47Run the game to see no flicker because the rays are not random

🏃

52:47Run the game to see no flicker because the rays are not random

🏃

53:32Flatten down the midrange rays in LightingTest()

53:56Run the game to see our distributed rays

🏃

53:56Run the game to see our distributed rays

🏃

53:56Run the game to see our distributed rays

🏃

54:56Add some randomness into SampleHemisphere()

56:29Run the game to see our perturbed rays, with flicker

🏃

56:29Run the game to see our perturbed rays, with flicker

🏃

56:29Run the game to see our perturbed rays, with flicker

🏃

57:20Add a tweakable Ratio in SampleHemisphere()

57:49Run the game and tweak the Ratio, watching the effect on the flicker

🏃

57:49Run the game and tweak the Ratio, watching the effect on the flicker

🏃

57:49Run the game and tweak the Ratio, watching the effect on the flicker

🏃

59:41Increase the tUpdate in LightingTest(), to smooth the lighting over a longer time

1:00:10Run the game to inspect the lighting, and consider using predictable sampling, smoothed over time

🏃

1:00:10Run the game to inspect the lighting, and consider using predictable sampling, smoothed over time

🏃

1:00:10Run the game to inspect the lighting, and consider using predictable sampling, smoothed over time

🏃

1:02:07Replace the random Series with EntropyCounter in lighting_solution, to predictably seed the random number generator per lighting sample

1:05:16Run the game to see the effects of this

🏃

1:05:16Run the game to see the effects of this

🏃

1:05:16Run the game to see the effects of this

🏃

1:06:13Make ComputeLightPropagation() correctly pin the EntropyCounter to the lighting sample

1:07:04Run the game to see a repetitive pattern in the rays

🏃

1:07:04Run the game to see a repetitive pattern in the rays

🏃

1:07:04Run the game to see a repetitive pattern in the rays

🏃

1:08:27Make LightingTest() blend the lighting slower than it takes samples

1:10:09Run the game to see that this doesn't help us much

🏃

1:10:09Run the game to see that this doesn't help us much

🏃

1:10:09Run the game to see that this doesn't help us much

🏃

1:10:54Enable OutputLightingTextures() to factor in the direction to light

1:11:25Run the game to see how this affects the lighting solution

🏃

1:11:25Run the game to see how this affects the lighting solution

🏃

1:11:25Run the game to see how this affects the lighting solution

🏃

1:12:27Prevent LightingTest() from blending the light over time

1:13:08Run the game to see that this is incredibly noisy

🏃

1:13:08Run the game to see that this is incredibly noisy

🏃

1:13:08Run the game to see that this is incredibly noisy

🏃

1:15:10Tweak the EntropyCount and tUpdate in LightingTest(), and increase the RayCount

1:16:58Run the game to see our coverage

🏃

1:16:58Run the game to see our coverage

🏃

1:16:58Run the game to see our coverage

🏃

1:17:36Increase the Ratio in SampleHemisphere() and set the RayCount to 16 in ComputeLightPropagation()

1:18:01Run the game to see our more evenly distributed rays, and consider how to proceed

🏃

1:18:01Run the game to see our more evenly distributed rays, and consider how to proceed

🏃

1:18:01Run the game to see our more evenly distributed rays, and consider how to proceed

🏃

1:20:21Introduce a DoLightingWork() callback to thread the ray casting

1:28:47Run the game to see how this works

🏃

1:28:47Run the game to see how this works

🏃

1:28:47Run the game to see how this works

🏃

1:29:22Add a single-thread toggle to ComputeLightPropagation()

1:30:17Run the game to see that the threaded path has slowed us

🏃

1:30:17Run the game to see that the threaded path has slowed us

🏃

1:30:17Run the game to see that the threaded path has slowed us

🏃

1:30:29Try making ComputeLightPropagation() loop over fewer samples

1:31:40Run the game to see that we are still slow

🏃

1:31:40Run the game to see that we are still slow

🏃

1:31:40Run the game to see that we are still slow

🏃

1:32:37Try toggling SampleHemisphere() back to how it was

1:33:18Run the game to determine that this has nothing to do with it

🏃

1:33:18Run the game to determine that this has nothing to do with it

🏃

1:33:18Run the game to determine that this has nothing to do with it

🏃

1:34:00Distribute the work across multiple threads again

1:34:43Run the game to see that we do get a little bit faster, but not as much as expected

🏃

1:34:43Run the game to see that we do get a little bit faster, but not as much as expected

🏃

1:34:43Run the game to see that we do get a little bit faster, but not as much as expected

🏃

1:37:31Increase the RayCount to 64 and run the game comparing it in the single- and multithreaded paths

🏃

🖮

1:37:31Increase the RayCount to 64 and run the game comparing it in the single- and multithreaded paths

🏃

🖮

1:37:31Increase the RayCount to 64 and run the game comparing it in the single- and multithreaded paths

🏃

🖮

1:39:21Add a TIMED_FUNCTION() to LightingTest()

1:39:58Run the game to see that the heavy load is entirely in ComputeLightPropagation()

🏃

1:39:58Run the game to see that the heavy load is entirely in ComputeLightPropagation()

🏃

1:39:58Run the game to see that the heavy load is entirely in ComputeLightPropagation()

🏃

1:40:40Comment out all instances of values changing in the solution

1:42:11Run the game to see that this speeds us up, and add back in some of those value changes, noting that we were thrashing the cache

🏃

1:42:11Run the game to see that this speeds us up, and add back in some of those value changes, noting that we were thrashing the cache

🏃

1:42:11Run the game to see that this speeds us up, and add back in some of those value changes, noting that we were thrashing the cache

🏃

1:45:52Make ComputeLightPropagation() set PointsPerWork to a multiple of 16, for cache-line alignment

1:50:07Run the game to see that we are still speedy

🏃

1:50:07Run the game to see that we are still speedy

🏃

1:50:07Run the game to see that we are still speedy

🏃

1:52:08Q&A

🗩

1:52:08Q&A

🗩

1:52:08Q&A

🗩

1:52:32trevnewt Q: Can we temporarily turn off / turn down the frame-to-frame color interpolation to see how much the increased sample count stabilizes the solution on its own?

🗪

1:52:32trevnewt Q: Can we temporarily turn off / turn down the frame-to-frame color interpolation to see how much the increased sample count stabilizes the solution on its own?

🗪

1:52:32trevnewt Q: Can we temporarily turn off / turn down the frame-to-frame color interpolation to see how much the increased sample count stabilizes the solution on its own?

🗪

1:52:44Temporarily disable smoothing and tweak the RayCount, sample randomisation and EntropyFrameCount

🏃

🖮

1:52:44Temporarily disable smoothing and tweak the RayCount, sample randomisation and EntropyFrameCount

🏃

🖮

1:52:44Temporarily disable smoothing and tweak the RayCount, sample randomisation and EntropyFrameCount

🏃

🖮

1:56:39praet_a51 Q: Can someone explain why to store and pass around OnePastLastSamplePointIndex instead of calculating at the loop part?

🗪

1:56:39praet_a51 Q: Can someone explain why to store and pass around OnePastLastSamplePointIndex instead of calculating at the loop part?

🗪

1:56:39praet_a51 Q: Can someone explain why to store and pass around OnePastLastSamplePointIndex instead of calculating at the loop part?

🗪

1:59:29vaualbus Q: Why we not using the atomics when we modify data on the multithread pass?

🗪

1:59:29vaualbus Q: Why we not using the atomics when we modify data on the multithread pass?

🗪

1:59:29vaualbus Q: Why we not using the atomics when we modify data on the multithread pass?

🗪

2:00:05jamoflaw Q: Given the restricted number of rays should they be biased towards directions which are "interesting" to us, i.e. not vertically? Just wondering as are we interested in the light values gained from the ceiling compared to that from surfaces which the player can visually see on screen

🗪

2:00:05jamoflaw Q: Given the restricted number of rays should they be biased towards directions which are "interesting" to us, i.e. not vertically? Just wondering as are we interested in the light values gained from the ceiling compared to that from surfaces which the player can visually see on screen

🗪

2:00:05jamoflaw Q: Given the restricted number of rays should they be biased towards directions which are "interesting" to us, i.e. not vertically? Just wondering as are we interested in the light values gained from the ceiling compared to that from surfaces which the player can visually see on screen

🗪

2:01:03drzool Q: Can the raycasts be optimized more?

🗪

2:01:03drzool Q: Can the raycasts be optimized more?

🗪

2:01:03drzool Q: Can the raycasts be optimized more?

🗪

2:01:40sdanna Q: Would increasing the size of the SampleTable make sense if we are going to 64 rays?

🗪

2:01:40sdanna Q: Would increasing the size of the SampleTable make sense if we are going to 64 rays?

🗪

2:01:40sdanna Q: Would increasing the size of the SampleTable make sense if we are going to 64 rays?

🗪

2:02:01enyo_enev Q: Since you only use cubes, depending on the height of the light you may know exactly where to cast since there can be, say, only 4x4 square over you at a certain height? I think it could work even if the light is placed as a "wall" the generated table based on height would be flipped towards the normal?

🗪

2:02:01enyo_enev Q: Since you only use cubes, depending on the height of the light you may know exactly where to cast since there can be, say, only 4x4 square over you at a certain height? I think it could work even if the light is placed as a "wall" the generated table based on height would be flipped towards the normal?

🗪

2:02:01enyo_enev Q: Since you only use cubes, depending on the height of the light you may know exactly where to cast since there can be, say, only 4x4 square over you at a certain height? I think it could work even if the light is placed as a "wall" the generated table based on height would be flipped towards the normal?

🗪

2:02:38aquaxv Q: Less technical question: how are you today?

🗪

2:02:38aquaxv Q: Less technical question: how are you today?

🗪

2:02:38aquaxv Q: Less technical question: how are you today?

🗪

2:03:20enyo_enev Q: Normal of the wall

🗪

2:03:20enyo_enev Q: Normal of the wall

🗪

2:03:20enyo_enev Q: Normal of the wall

🗪

2:05:43jamoflaw Q: Now the threading is working well, how does it scale with more threads?

🗪

2:05:43jamoflaw Q: Now the threading is working well, how does it scale with more threads?

🗪

2:05:43jamoflaw Q: Now the threading is working well, how does it scale with more threads?

🗪

2:10:14staythirsty90 Q: What's the status of using LLVM?

🗪

2:10:14staythirsty90 Q: What's the status of using LLVM?

🗪

2:10:14staythirsty90 Q: What's the status of using LLVM?

🗪

2:10:37printf_armin Q: How is the light source stored?

🗪

2:10:37printf_armin Q: How is the light source stored?

🗪

2:10:37printf_armin Q: How is the light source stored?

🗪

2:10:56neitchzehrer Q: No more wrist supports?

🗪

2:10:56neitchzehrer Q: No more wrist supports?

🗪

2:10:56neitchzehrer Q: No more wrist supports?

🗪

2:10:59nxsy Q: Use the new calculator!

🗪

2:10:59nxsy Q: Use the new calculator!

🗪

2:10:59nxsy Q: Use the new calculator!

🗪

2:12:19longboolean Q: Are there any ways to make it clear in your profiler that you are having these cache issues (concerning threads, and shared memory) or is this just something you have to look for with trial and error, experience etc?

🗪

2:12:19longboolean Q: Are there any ways to make it clear in your profiler that you are having these cache issues (concerning threads, and shared memory) or is this just something you have to look for with trial and error, experience etc?

🗪

2:12:19longboolean Q: Are there any ways to make it clear in your profiler that you are having these cache issues (concerning threads, and shared memory) or is this just something you have to look for with trial and error, experience etc?

🗪

2:14:30nxsy Q: In menu, View -> History might help

🗪

2:14:30nxsy Q: In menu, View -> History might help

🗪

2:14:30nxsy Q: In menu, View -> History might help

🗪

2:15:05Time to shut down, with a glimpse into the future optimising the lighting and reducing its variance more intelligently

🗩

2:15:05Time to shut down, with a glimpse into the future optimising the lighting and reducing its variance more intelligently

🗩

2:15:05Time to shut down, with a glimpse into the future optimising the lighting and reducing its variance more intelligently

🗩

Keyboard Navigation

Global Keys

Menu toggling

In-Menu and Index Controls

Quotes and References Menus and Index

Quotes, References and Credits Menus

Filter Menu

Filter and Link Menus

Credits Menu