Optimizing Depth Peeling and Multisample Resolves — Handmade Hero — Episode Guide

0:02Plug Molly Rocket's Discord channel¹

🗩

0:02Plug Molly Rocket's Discord channel¹

🗩

0:02Plug Molly Rocket's Discord channel¹

🗩

1:44Recap and set the stage for the day with praise for RenderDoc

🗩

1:44Recap and set the stage for the day with praise for RenderDoc

🗩

1:44Recap and set the stage for the day with praise for RenderDoc

🗩

3:15Note that lots of our time is spent doing the multisample resolve

🗩

3:15Note that lots of our time is spent doing the multisample resolve

🗩

3:15Note that lots of our time is spent doing the multisample resolve

🗩

7:32Configure our project in RenderDoc

🗹

7:32Configure our project in RenderDoc

🗹

7:32Configure our project in RenderDoc

🗹

8:42Capture a frame in RenderDoc and consult the event timings to see that the multisample buffer took 10 times longer to resolve than to draw

🏃

8:42Capture a frame in RenderDoc and consult the event timings to see that the multisample buffer took 10 times longer to resolve than to draw

🏃

8:42Capture a frame in RenderDoc and consult the event timings to see that the multisample buffer took 10 times longer to resolve than to draw

🏃

10:54The possible bandwidth cost of resolving our multisample buffer

🏃

10:54The possible bandwidth cost of resolving our multisample buffer

🏃

10:54The possible bandwidth cost of resolving our multisample buffer

🏃

14:16Our best case solution: Resolve the multisample buffer independently of the depth peel

🏃

14:16Our best case solution: Resolve the multisample buffer independently of the depth peel

🏃

14:16Our best case solution: Resolve the multisample buffer independently of the depth peel

🏃

16:29Demo Milton's new grid feature, but its smoothing bug

🖌

16:29Demo Milton's new grid feature, but its smoothing bug

🖌

16:29Demo Milton's new grid feature, but its smoothing bug

🖌

17:32Rendering requirements of Sprites vs Geometry

🖌

17:32Rendering requirements of Sprites vs Geometry

🖌

17:32Rendering requirements of Sprites vs Geometry

🖌

20:16Demo some undesirable alpha blending when traversing stairs

🏃

20:16Demo some undesirable alpha blending when traversing stairs

🏃

20:16Demo some undesirable alpha blending when traversing stairs

🏃

22:15Consider two multisample resolves to may be necessary

🏃

22:15Consider two multisample resolves to may be necessary

🏃

22:15Consider two multisample resolves to may be necessary

🏃

24:00Plan to render the sprites with depth peel in a separate pass, then composite in the geometry with two multisample resolves

🖌

24:00Plan to render the sprites with depth peel in a separate pass, then composite in the geometry with two multisample resolves

🖌

24:00Plan to render the sprites with depth peel in a separate pass, then composite in the geometry with two multisample resolves

🖌

30:17Segregating our sprites and geometry into separate buffers

🖌

30:17Segregating our sprites and geometry into separate buffers

🖌

30:17Segregating our sprites and geometry into separate buffers

🖌

33:28Hesitate to impose this separation requirement on the renderer

🖌

33:28Hesitate to impose this separation requirement on the renderer

🖌

33:28Hesitate to impose this separation requirement on the renderer

🖌

35:31Producing a separate multisample buffer of the edge information

🖌

35:31Producing a separate multisample buffer of the edge information

🖌

35:31Producing a separate multisample buffer of the edge information

🖌

36:52The edge information in question

🏃

36:52The edge information in question

🏃

36:52The edge information in question

🏃

38:53Using conservative rasterization to enable recovery of our edge blend from a single high- / low-coverage multisample resolve

🏃

38:53Using conservative rasterization to enable recovery of our edge blend from a single high- / low-coverage multisample resolve

🏃

38:53Using conservative rasterization to enable recovery of our edge blend from a single high- / low-coverage multisample resolve

🏃

43:48Using conservative rasterization just to tell us how much a pixel is covered by a primitive

🏃

43:48Using conservative rasterization just to tell us how much a pixel is covered by a primitive

🏃

43:48Using conservative rasterization just to tell us how much a pixel is covered by a primitive

🏃

45:28Research conservative rasterization²^,3^,4

📖

45:28Research conservative rasterization²^,3^,4

📖

45:28Research conservative rasterization²^,3^,4

📖

49:14Hunt for a coverage-to-alpha function, in NV_shading_rate_image,⁵ ARB_sample_locations⁶ and ARB_sample_shading⁷

📖

49:14Hunt for a coverage-to-alpha function, in NV_shading_rate_image,⁵ ARB_sample_locations⁶ and ARB_sample_shading⁷

📖

49:14Hunt for a coverage-to-alpha function, in NV_shading_rate_image,⁵ ARB_sample_locations⁶ and ARB_sample_shading⁷

📖

58:13Find that conservative rasterization is not widely available⁸

📖

58:13Find that conservative rasterization is not widely available⁸

📖

58:13Find that conservative rasterization is not widely available⁸

📖

59:10Consider running the multisample routine without a multisample buffer, only recording which samples were covered⁹^,10^,11

📖

59:10Consider running the multisample routine without a multisample buffer, only recording which samples were covered⁹^,10^,11

📖

59:10Consider running the multisample routine without a multisample buffer, only recording which samples were covered⁹^,10^,11

📖

1:05:40Consider attaching a multisample and non-multisample render target at the same time

📖

1:05:40Consider attaching a multisample and non-multisample render target at the same time

📖

1:05:40Consider attaching a multisample and non-multisample render target at the same time

📖

1:07:05Consult glext.h for coverage-related functions¹²

📖

1:07:05Consult glext.h for coverage-related functions¹²

📖

1:07:05Consult glext.h for coverage-related functions¹²

📖

1:11:25Research NV_fragment_coverage_to_color¹³^,14

📖

1:11:25Research NV_fragment_coverage_to_color¹³^,14

📖

1:11:25Research NV_fragment_coverage_to_color¹³^,14

📖

1:14:11Research ARB_post_depth_coverage¹⁵ and gl_SampleMaskIn¹⁶

📖

1:14:11Research ARB_post_depth_coverage¹⁵ and gl_SampleMaskIn¹⁶

📖

1:14:11Research ARB_post_depth_coverage¹⁵ and gl_SampleMaskIn¹⁶

📖

1:17:53NVIDIA request: When enabling conservative rasterization, let us set the SampleMask to the number of samples we want

🗩

1:17:53NVIDIA request: When enabling conservative rasterization, let us set the SampleMask to the number of samples we want

🗩

1:17:53NVIDIA request: When enabling conservative rasterization, let us set the SampleMask to the number of samples we want

🗩

1:19:08Lament the tremendous amount of bandwidth required to smooth out our edges

🗩

1:19:08Lament the tremendous amount of bandwidth required to smooth out our edges

🗩

1:19:08Lament the tremendous amount of bandwidth required to smooth out our edges

🗩

1:20:36Reflect on the need for the multisample buffer to tell if primitives are coplanar

🏃

1:20:36Reflect on the need for the multisample buffer to tell if primitives are coplanar

🏃

1:20:36Reflect on the need for the multisample buffer to tell if primitives are coplanar

🏃

1:21:40Consider augmenting our Colour Pass shader to skip pixels whose prior pass produced a fully opaque pixel

🏃

1:21:40Consider augmenting our Colour Pass shader to skip pixels whose prior pass produced a fully opaque pixel

🏃

1:21:40Consider augmenting our Colour Pass shader to skip pixels whose prior pass produced a fully opaque pixel

🏃

1:28:10Make CompileResolveMultisample() set the gl_FragDepth of opaque pixels to 1.0f, enabling CompileZBiasProgram() to discard obscured pixels

1:36:24Capture a frame to see that depth sorting is still working, and our Colour Passes are now more efficient

🏃

1:36:24Capture a frame to see that depth sorting is still working, and our Colour Passes are now more efficient

🏃

1:36:24Capture a frame to see that depth sorting is still working, and our Colour Passes are now more efficient

🏃

1:38:28Switch to the non-multisampling fast path in CompileResolveMultisample()

1:39:50Capture a frame to see that our glDrawArrays() calls have sped up

🏃

1:39:50Capture a frame to see that our glDrawArrays() calls have sped up

🏃

1:39:50Capture a frame to see that our glDrawArrays() calls have sped up

🏃

1:41:01Options for creating our fast path in CompileResolveMultisample(): 1. Read from the previous depth peel and do not resolve opaque pixels; 2. Resolve into a separate "blend" buffer

🗩

1:41:01Options for creating our fast path in CompileResolveMultisample(): 1. Read from the previous depth peel and do not resolve opaque pixels; 2. Resolve into a separate "blend" buffer

🗩

1:41:01Options for creating our fast path in CompileResolveMultisample(): 1. Read from the previous depth peel and do not resolve opaque pixels; 2. Resolve into a separate "blend" buffer

🗩

1:43:30Make CompileResolveMultisample() only blend non-opaque pixels

1:44:42Reacquaint ourselves with the final CompilePeelComposite() with a view to instead accumulating the colour as we go

📖

1:44:42Reacquaint ourselves with the final CompilePeelComposite() with a view to instead accumulating the colour as we go

📖

1:44:42Reacquaint ourselves with the final CompilePeelComposite() with a view to instead accumulating the colour as we go

📖

1:46:39Introduce a MaskSampler in CompileResolveMultisample() to contain the opacity

1:48:49Crash the game under RenderDoc

🏃

1:48:49Crash the game under RenderDoc

🏃

1:48:49Crash the game under RenderDoc

🏃

1:49:11Hit a shader error "unable to find overloaded function texelFetch()"

🏃

1:49:11Hit a shader error "unable to find overloaded function texelFetch()"

🏃

1:49:11Hit a shader error "unable to find overloaded function texelFetch()"

🏃

1:49:45Fix CompileResolveMultisample() to fetch the Mask's texel from the 0th texture

1:50:24See that the multisampling is a little busted

🏃

1:50:24See that the multisampling is a little busted

🏃

1:50:24See that the multisampling is a little busted

🏃

1:51:01Just let CompileResolveMultisample() always blend, including some shader parser mayhem

1:53:16See that the multisampled artefacts are gone

🏃

1:53:16See that the multisampled artefacts are gone

🏃

1:53:16See that the multisampled artefacts are gone

🏃

1:53:42Make CompileResolveMultisample() initialise the samplers in the order in which they are passed to OpenGLLinkSamplers()

1:56:56Make OpenGLEndFrame() set the Mask for CompileResolveMultisample() to acquire pixel opacity, introducing a SinglePixelAllZeroesTexture

2:07:05See a few multisampling artefacts in there

🏃

2:07:05See a few multisampling artefacts in there

🏃

2:07:05See a few multisampling artefacts in there

🏃

2:08:08Quickly scrutinise our new Mask code

📖

2:08:08Quickly scrutinise our new Mask code

📖

2:08:08Quickly scrutinise our new Mask code

📖

2:09:18Capture a frame to see that our second and fourth Colour Passes are not as expected

🏃

2:09:18Capture a frame to see that our second and fourth Colour Passes are not as expected

🏃

2:09:18Capture a frame to see that our second and fourth Colour Passes are not as expected

🏃

2:10:01Check the Mask test in CompileResolveMultisample()

📖

2:10:01Check the Mask test in CompileResolveMultisample()

📖

2:10:01Check the Mask test in CompileResolveMultisample()

📖

2:11:00Take a close look at our first glDrawArrays() call, to see that our MaskSampler is 0×0 pixels

🏃

2:11:00Take a close look at our first glDrawArrays() call, to see that our MaskSampler is 0×0 pixels

🏃

2:11:00Take a close look at our first glDrawArrays() call, to see that our MaskSampler is 0×0 pixels

🏃

2:12:57Fix OpenGLInit() to correctly bind our SinglePixelAllZeroesTexture

2:13:19Capture a frame to see that our third Colour Pass drew much more than expected

🏃

2:13:19Capture a frame to see that our third Colour Pass drew much more than expected

🏃

2:13:19Capture a frame to see that our third Colour Pass drew much more than expected

🏃

2:15:29Make CompileResolveMultisample() set gl_FragDepth of opaque pixels

2:16:46Crash in RenderDoc

🏃

2:16:46Crash in RenderDoc

🏃

2:16:46Crash in RenderDoc

🏃

2:17:11Make CompileResolveMultisample() set BlendUnitColor of opaque pixels

2:17:36Battle with the shader parser

🏃

🖮

2:17:36Battle with the shader parser

🏃

🖮

2:17:36Battle with the shader parser

🏃

🖮

2:21:16Prevent CompileResolveMultisample() from setting the BlendUnitColor of opaque pixels

2:21:39Capture a frame to see that we are still busted

🏃

2:21:39Capture a frame to see that we are still busted

🏃

2:21:39Capture a frame to see that we are still busted

🏃

2:24:31Make CompileResolveMultisample() always blend

2:24:40Find that the shader parser is unhappy, and wonder if it's a problem with 4coder's virtual whitespace

🏃

2:24:40Find that the shader parser is unhappy, and wonder if it's a problem with 4coder's virtual whitespace

🏃

2:24:40Find that the shader parser is unhappy, and wonder if it's a problem with 4coder's virtual whitespace

🏃

2:25:14Revert CompileResolveMultisample() to a supposedly working state

2:25:48Find that the shader parser continues to be unhappy

🏃

2:25:48Find that the shader parser continues to be unhappy

🏃

2:25:48Find that the shader parser continues to be unhappy

🏃

2:26:48Change all the comments in CompileResolveMultisample() to be C-style ones

2:27:23Find that that doesn't solve the problem

🏃

2:27:23Find that that doesn't solve the problem

🏃

2:27:23Find that that doesn't solve the problem

🏃

2:28:00See how RemedyBG sees the code in CompileResolveMultisample()

🏃

2:28:00See how RemedyBG sees the code in CompileResolveMultisample()

🏃

2:28:00See how RemedyBG sees the code in CompileResolveMultisample()

🏃

2:30:06Trim out some possibly problematic code from CompileResolveMultisample()

2:30:18Find that the shader parser is happy now

🏃

2:30:18Find that the shader parser is happy now

🏃

2:30:18Find that the shader parser is happy now

🏃

2:30:35Make CompileResolveMultisample() set the Mask to 0.0

2:31:08Find that the shader parser remains happy, but we still see our multisampling artefact

🏃

2:31:08Find that the shader parser remains happy, but we still see our multisampling artefact

🏃

2:31:08Find that the shader parser remains happy, but we still see our multisampling artefact

🏃

2:31:23Make CompileResolveMultisample() always blend

2:31:33See our multisampling artefact

🏃

2:31:33See our multisampling artefact

🏃

2:31:33See our multisampling artefact

🏃

2:32:09Leave in our fast path in CompileResolveMultisample()

2:32:33See new artefacts

🏃

2:32:33See new artefacts

🏃

2:32:33See new artefacts

🏃

2:32:52Make CompileResolveMultisample() always blend

2:33:05See that everything looks nice and smooth

🏃

2:33:05See that everything looks nice and smooth

🏃

2:33:05See that everything looks nice and smooth

🏃

2:33:53Reinsert our fast path in CompileResolveMultisample()

2:34:06See new MIP map anisotropic filtering artefacts

🏃

2:34:06See new MIP map anisotropic filtering artefacts

🏃

2:34:06See new MIP map anisotropic filtering artefacts

🏃

2:34:44Capture a frame to see that our third and fourth Colour Passes are not as free as expected

🏃

2:34:44Capture a frame to see that our third and fourth Colour Passes are not as free as expected

🏃

2:34:44Capture a frame to see that our third and fourth Colour Passes are not as free as expected

🏃

2:35:11Fix CompileResolveMultisample() to black out the BlendUnitColor of opaque pixels

2:36:16Capture a frame to see that our Colour Passes are way more efficient

🏃

2:36:16Capture a frame to see that our Colour Passes are way more efficient

🏃

2:36:16Capture a frame to see that our Colour Passes are way more efficient

🏃

2:38:31Q&A

🗩

2:38:31Q&A

🗩

2:38:31Q&A

🗩

2:38:38dithinas Q: Are you exceeding the size of your buffer for the shader code text? The EOF error seems kind of suspicious

🗪

2:38:38dithinas Q: Are you exceeding the size of your buffer for the shader code text? The EOF error seems kind of suspicious

🗪

2:38:38dithinas Q: Are you exceeding the size of your buffer for the shader code text? The EOF error seems kind of suspicious

🗪

2:38:50Make CompileResolveMultisample() record the shader string BufferSize

2:39:42See that the shader string is 3877 characters

🏃

2:39:42See that the shader string is 3877 characters

🏃

2:39:42See that the shader string is 3877 characters

🏃

2:39:59Increase the FragmentCode buffer size from 4096 to 16000

2:41:04Find that the shader parser is happy, with a few words on tooling

🏃

2:41:04Find that the shader parser is happy, with a few words on tooling

🏃

2:41:04Find that the shader parser is happy, with a few words on tooling

🏃

2:43:33sagian2005 Q: Couldn't you just not define the dimension and let the compiler figure out the size?

🗪

2:43:33sagian2005 Q: Couldn't you just not define the dimension and let the compiler figure out the size?

🗪

2:43:33sagian2005 Q: Couldn't you just not define the dimension and let the compiler figure out the size?

🗪

2:44:10letambourinroyal Q: Why can’t you just run the preprocessor on the shaders, though?

🗪

2:44:10letambourinroyal Q: Why can’t you just run the preprocessor on the shaders, though?

🗪

2:44:10letambourinroyal Q: Why can’t you just run the preprocessor on the shaders, though?

🗪

2:46:53dithinas Q: Do you have any concept of like a string builder that you could use with your format string function to have it pass you back a string allocated from temp memory? It's a small thing, but you won't need to think / worry about that anymore

🗪

2:46:53dithinas Q: Do you have any concept of like a string builder that you could use with your format string function to have it pass you back a string allocated from temp memory? It's a small thing, but you won't need to think / worry about that anymore

🗪

2:46:53dithinas Q: Do you have any concept of like a string builder that you could use with your format string function to have it pass you back a string allocated from temp memory? It's a small thing, but you won't need to think / worry about that anymore

🗪

2:48:06enyo_enev Q: I think that Dual Depth Peeling.¹⁷^,18 This paper may be able to help you out, because I looked at their OpenGL code and it seems possible to resolve only once when you render to the screen. You can download the code and see. I am not sure. It is pure OpenGL. They have not enabled multisampling, however I think it is possible to work

🗪

2:48:06enyo_enev Q: I think that Dual Depth Peeling.¹⁷^,18 This paper may be able to help you out, because I looked at their OpenGL code and it seems possible to resolve only once when you render to the screen. You can download the code and see. I am not sure. It is pure OpenGL. They have not enabled multisampling, however I think it is possible to work

🗪

2:48:06enyo_enev Q: I think that Dual Depth Peeling.¹⁷^,18 This paper may be able to help you out, because I looked at their OpenGL code and it seems possible to resolve only once when you render to the screen. You can download the code and see. I am not sure. It is pure OpenGL. They have not enabled multisampling, however I think it is possible to work

🗪

2:53:20samyuutsu Q: Have you considered using RenderDoc directly to recompile the shaders at runtime during rapid development?

🗪

2:53:20samyuutsu Q: Have you considered using RenderDoc directly to recompile the shaders at runtime during rapid development?

🗪

2:53:20samyuutsu Q: Have you considered using RenderDoc directly to recompile the shaders at runtime during rapid development?

🗪

2:54:40eddiesutrecht Q: Hi Casey, some episodes ago you said you didn't understand why Jon credited you on Braid, since you didn't do any work on it. But you wrote the collision detector, right? Maybe I misunderstood and it was about the rewind only

🗪

2:54:40eddiesutrecht Q: Hi Casey, some episodes ago you said you didn't understand why Jon credited you on Braid, since you didn't do any work on it. But you wrote the collision detector, right? Maybe I misunderstood and it was about the rewind only

🗪

2:54:40eddiesutrecht Q: Hi Casey, some episodes ago you said you didn't understand why Jon credited you on Braid, since you didn't do any work on it. But you wrote the collision detector, right? Maybe I misunderstood and it was about the rewind only

🗪

2:59:08vaualbus Q: How you would implement in debug systems the way of changing also v3 / v4, values?

🗪

2:59:08vaualbus Q: How you would implement in debug systems the way of changing also v3 / v4, values?

🗪

2:59:08vaualbus Q: How you would implement in debug systems the way of changing also v3 / v4, values?

🗪

2:59:56zrizi Q: I just watched the video about sub-pixel sampling for pixel art assets. Thank you for explaining it, was really good. I have two related questions, though: 1) When you viewed the sprite sheet you noticed that it’s not alpha pre-multiplied and kind of mentioned that it might be a problem. I was wondering why. 2) You mentioned that they don’t use MIP maps and that got me thinking… How should we produce MIPs for pixel art assets? Box-filtering would not work since that’s made for bilinear

🗪

2:59:56zrizi Q: I just watched the video about sub-pixel sampling for pixel art assets. Thank you for explaining it, was really good. I have two related questions, though: 1) When you viewed the sprite sheet you noticed that it’s not alpha pre-multiplied and kind of mentioned that it might be a problem. I was wondering why. 2) You mentioned that they don’t use MIP maps and that got me thinking… How should we produce MIPs for pixel art assets? Box-filtering would not work since that’s made for bilinear

🗪

2:59:56zrizi Q: I just watched the video about sub-pixel sampling for pixel art assets. Thank you for explaining it, was really good. I have two related questions, though: 1) When you viewed the sprite sheet you noticed that it’s not alpha pre-multiplied and kind of mentioned that it might be a problem. I was wondering why. 2) You mentioned that they don’t use MIP maps and that got me thinking… How should we produce MIPs for pixel art assets? Box-filtering would not work since that’s made for bilinear

🗪

3:02:44zrizi Q: 1) You're right. But I think Unity's shaders were set up for point sampling

🗪

3:02:44zrizi Q: 1) You're right. But I think Unity's shaders were set up for point sampling

🗪

3:02:44zrizi Q: 1) You're right. But I think Unity's shaders were set up for point sampling

🗪

3:03:47zrizi Q: And thank you very much for the skin mesh video. I still have to watch it

🗪

3:03:47zrizi Q: And thank you very much for the skin mesh video. I still have to watch it

🗪

3:03:47zrizi Q: And thank you very much for the skin mesh video. I still have to watch it

🗪

3:04:00Time for bed, with a plug of Molly Rocket's Discord channel¹⁹

🗩

3:04:00Time for bed, with a plug of Molly Rocket's Discord channel¹⁹

🗩

3:04:00Time for bed, with a plug of Molly Rocket's Discord channel¹⁹

🗩

Keyboard Navigation

Global Keys

Menu toggling

In-Menu and Index Controls

Quotes and References Menus and Index

Quotes, References and Credits Menus

Filter Menu

Filter and Link Menus

Credits Menu