Game Computing Applications - Home -...
Transcript of Game Computing Applications - Home -...
![Page 1: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/1.jpg)
Mixed Resolution Rendering
Jeremy Shopf
AMDGame Computing Applications
![Page 2: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/2.jpg)
Outline
» Motivation
» Uniform Upsampling
» Adaptive Upsampling
» Demo
![Page 3: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/3.jpg)
Offscreen Particles
Used in many shipping titles
Method:
Render scene without particles
Downsample depth buffer
Render particles to low-res color buffer, using downsampled depth buffer for depth testing
Composite low-res particles
Fix up gaps caused by low-res depth testing
[Nguyen04]
![Page 4: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/4.jpg)
High-Res Version (20 fps@1280x960)
Offscreen Particles
![Page 5: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/5.jpg)
Offscreen Particles
2x Downsampled Offscreen Version (32 fps@1280x960)
![Page 6: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/6.jpg)
Fix-up pixels (rest are low-res)
Offscreen Particles
![Page 7: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/7.jpg)
Offscreen Particles
Tradeoff:
+ Save on Fill Rate
- Lose resolution
![Page 8: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/8.jpg)
Mixed Resolution Rendering
» For relatively low frequency effects, mixed resolution rendering results in barely perceivable loss in quality
» Dramatic performance improvement
Scales linearly minus small overhead for upsampling
» Allows scaling of effects to maintain framerate
![Page 9: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/9.jpg)
Bilinear Interpolation
Used in Offscreen Particles
+ Efficient and simple
- Doesn’t respect depth discontinuities
Requires separate discontinuity detection pass
Hi-Res Texels
Low-Res Texels
![Page 10: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/10.jpg)
Bilateral Upsampling
» Modify bilinear weights by normal and depth discontinuities
» Weight each coarse sample by:
Bilinear weight
Normal similarity weight
Depth similarity weight
[Sloan07]
![Page 11: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/11.jpg)
float4 vBilinearWeights[4] = {
// 0 1 2 3ddddddfloat4( 9/16, 3/16, 3/16, 1/16 ), // 0float4( 3/16, 9/16, 1/16, 3/16 ), // 1float4( 3/16, 1/16, 9/16, 3/16 ), // 2float4( 1/16, 3/16, 3/16, 9/16 ) // 3
};
Computing bilinear weights:
Hi-Res Texels
Low-Res Texels
Bilateral Upsampling
0
2
1
0
3
1
2 3
![Page 12: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/12.jpg)
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
0 0.2 0.4 0.6 0.8 1 1.2
float3 vNormalsCoarse[4] = {…}
float vNormalHiRes = ...
for(int i=0;i<4;i++)
{vNormalWeights[i] = dot( vNormalsCoarse[i],
vNormalHiRes );vNormalWeights[i] = pow(vNormalWeights[i] , 32 );
}
Computing normal weights:
Bilateral Upsampling
Hi-Res Texels
Low-Res Texels
0
2
1
0
3
1
2 3
![Page 13: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/13.jpg)
float fDepthsCoarse[4] = {…};float fDepthHiRes = …;
for(int i=0;i<4;i++){
float fDepthDiff = fDepthHiRes – fDepthsCoarse[i];vDepthWeights[i] = 1.0/( EPSILON + abs(fDepthDiff ));
}
Computing depth weights:
Bilateral Upsampling
Hi-Res Texels
Low-Res Texels
0
2
1
0
3
1
2 3
![Page 14: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/14.jpg)
Putting it all together:
for(int nSample=0; nSample<4; nSample++){
float fWeight = vNormalWeights[nSample] *vDepthWeights[nSample] *vBilinearWeights[nTexel][nSample];
fTotalWeight += fWeight;vUpsampled += vShadingCoarse[nSample]*fWeight;
}
vUpsampled /= fTotalWeight;
Hi-Res Texels
Low-Res Texels
Bilateral Upsampling
0
2
1
0
3
1
2 3
![Page 15: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/15.jpg)
Degenerate Case
No valid coarse samplesEasy to detect this situationHappens infrequently
Options: - Pick one coarse sample - Average coarse samples- Set stencil and do an additional pass to calculate these at full res
![Page 16: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/16.jpg)
Illustration of Degenerate Case
![Page 17: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/17.jpg)
Crease Shading
Diffuse only
Bilateral Upsampling Results
![Page 18: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/18.jpg)
High-Res Shading(67 fps@1024x1024)
Bilateral Upsampling Results
Crease Shading
![Page 19: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/19.jpg)
2x Downsampled Shading(166 fps@1024x1024)
Bilateral Upsampling Results
Crease Shading
![Page 20: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/20.jpg)
Bilateral Upsampling Results
Sphere AO
No AO
![Page 21: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/21.jpg)
High-Res AO (62 fps@1280x960)
Sphere AO
Bilateral Upsampling Results
![Page 22: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/22.jpg)
2x Downsampled AO (135 fps@1280x960)
Sphere AO
Bilateral Upsampling Results
![Page 23: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/23.jpg)
Bilateral Upsampling
+ Small amount of overhead
+ Easily to implement
- Uniform sampling
Undersampling small features
Oversampling low resolution regions
![Page 24: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/24.jpg)
Pull-push Upsampling
» Used to reconstruct images from sparse pixels
» Allows adaptive sampling
» Used for lots of stuff:
DoF [Krauss and Strengert 07]
Imperfect Shadow Maps [Ritschel et al. 08]
….
[Mitchell87,Grossman97]
![Page 25: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/25.jpg)
Pull-push Upsampling
Two phase algorithm fills in unknown values hierarchically
Pull : Bottom-up averaging of valid values
![Page 26: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/26.jpg)
Two phase algorithm fills in unknown values hierarchically
Push : Top-down interpolation of higher levels
Pull-push Upsampling
![Page 27: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/27.jpg)
Soft Shadow Example
» “Real-time Soft Shadow Mapping by Backprojection” [Guennebaud06,07]
» Use a min/max pyramid of the shadow depth buffer to accelerate computation by culling unimportant shadowmap texels
![Page 28: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/28.jpg)
Pull-push Upsampling
Select pixels to be shaded based on penumbra width
8 1 4 1
1 2 1 2
4 1 ∞ 1
1 2 1 2
Estimated Penumbra Width
<
Skip Computation
[Günnebaud06]
![Page 29: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/29.jpg)
Screen Resolution Soft Shadows
![Page 30: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/30.jpg)
Adaptive Sampling
![Page 31: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/31.jpg)
Screen Resolution Soft Shadows
26 fps @1024x1024
![Page 32: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/32.jpg)
Adaptive Sampling
58 fps @1024x1024
![Page 33: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/33.jpg)
» Shading sparse points under-utilizes GPU
» Need to pack points to be shaded
» Unpack before pull-push interpolation
GPU Implementation
![Page 34: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/34.jpg)
Discussion
» Selection function must be cheap to compute and the shading function expensive
» Shading in packed texture may be limiting Proxy-based shading (e.g. Sphere AO)
incompatible
» Discontinuities have to be handled at full resolution
![Page 35: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/35.jpg)
MultiRes Upsampling
» Adaptive upsampling that respects discontinuities
» Compute shading at several different resolutions
Upsample and combine hierarchically
Use bilinear interpolation in regions with no discontinuities
[NicholsWyman09]
![Page 36: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/36.jpg)
Multiresolution Indirect Illumination
» Distribute Virtual Point Lights in scene via shadow map
» Compute direct illumination from VPLs
» 400 VPLs = 400 lighting calculations per pixel
![Page 37: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/37.jpg)
Choosing a resolution
» For each resolution (lowest to highest)
For each texel If there is no discontinuity
Compute shading
Else
Wait to compute this region at a higher level
![Page 38: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/38.jpg)
Detecting Discontinuities
» Construct hierarchical Min/Max depth and normal buffers of scene from viewer
Like a Mip Map, but stores min and max rather than average for all resolutions
» If Max-Min > some threshold, there is a discontinuity at this resolution
![Page 39: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/39.jpg)
UpsamplingFor each resolution (2nd lowest to highest)
Upsample previous levels
If all four coarse samples are available, use bilinear interpolation, otherwise point sample
Combine with current res
![Page 40: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/40.jpg)
Demo
~22 fps full res~57 fps multi res
400 VPLs per pixel1024x1024 resolution
![Page 41: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/41.jpg)
Conclusions
» Mixed resolution rendering can save you perf with a modicum of overhead
» Uniform Upsampling
Lightweight and scales well
» Adaptive Upsampling
Adaptive but with more overhead
![Page 42: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/42.jpg)
More Details
Read my blog, level of detail
http://www.jshopf.com/blog
"Mixed Resolution Rendering"
Contact me:
Thank you: Greg Nichols and Chris Wyman,
GCAG
![Page 43: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/43.jpg)
Thanks!
![Page 44: Game Computing Applications - Home - AMDdeveloper.amd.com/wordpress/media/.../10/ShopfMixedResolutionRe… · Render particles to low-res color buffer, using downsampled depth buffer](https://reader033.fdocuments.net/reader033/viewer/2022052801/5f144f19f39fc56de07a6cf1/html5/thumbnails/44.jpg)
References
Peter-Pike Sloan, Naga K. Govindaraju, Derek Nowrouzezahrai, John Snyder. "Image-Based Proxy Accumulation for Real-Time Soft Global Illumination". Pacific Graphics 2007.
Greg Nichols and Chris Wyman. "Multiresolution Splatting for Indirect Illumination." ACM Symposium on Interactive 3D Graphics and Games, 83-90. 2009.
D.P. Mitchell. "Generating Antialiased Images at Low Sampling Densities". In Proceedings of SIGGRAPH 1987, pages 65-72
H. Nguyen. "Fire In the Vulcan Demo." GPU Gems.
J.P. Grossman. "Point Sample Rendering". In 9th Eurographics Workshop on Rendering 1998. Pages 181-192.
Gael Guennebaud, Loic Barthe, Mathias Paulin. "High-Quality Adaptive Soft Shadow Mapping." Eurographics 2007.
Gael Guennebaud, Loic Barthe, Mathias Paulin. "Real-time Soft Shadow Mapping by Backprojection." Eurographics Symposium on Rendering 2006.
M. Krauss and M. Strengert. "Depth-of-Field Rendering using Pyramidal Image Processing". Computer Graphics Forum Volume 26, Issue 3. 2007.
T. Ritschel, T. Grosch, M. H. Kim, H.-P. Seidel, C. Dachsbacher, J. Kautz ACM Trans. on Graphics (Proceedings SIGGRAPH Asia 2008), 27(5), 2008.