Abstract
We present a technique that adaptively adjusts the shading rate during rendering based on the scene content and motion. Our goal is to improve performance with no loss in perceived quality. We determine per-screen-tile shading rate by testing an error estimate against a perceptually-corrected just-noticeable difference threshold. Our design features an effective and efficient error estimate using spatial and frequency analysis of half and quarter rate shading. We also study the effect of motion in reducing perceived error, a consequence of display-persistence and/or motion blur effects. Our implementation uses the computed per-tile shading rate with variable rate shading (a recent GPU feature) to lower shading cost. We demonstrate the quality and performance of our technique on two high-end game engines and shipped games.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Gradient Estimation for Realtime Adaptive Temporal Filtering
- Kurt Akeley. 1993. Reality engine graphics. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. ACM, 109--116. Google Scholar
Digital Library
- Tomas Akenine-Moller, Eric Haines, and Naty Hoffman. 2018. Real-time rendering. AK Peters/CRC Press. Google Scholar
Digital Library
- Magnus Andersson, Jon Hasselgren, Robert Toth, and Tomas Akenine-Möiler. 2014. Adaptive texture space shading for stochastic rendering. Computer Graphics Forum 33, 2 (2014), 341--350. Google Scholar
Digital Library
- Swaroop Bhonde. 2018. Turing Variable Rate Shading in VRWorks. https://devblogs.nvidia.com/turing-variable-rate-shading-vrworks/. (2018).Google Scholar
- Mark R Bolin and Gary W Meyer. 1998. A perceptually based adaptive sampling algorithm. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. ACM, 299--309. Google Scholar
Digital Library
- Huw Bowles, Kenny Mitchell, Robert W Sumner, Jeremy Moore, and Markus Gross. 2012. Iterative image warping. In Computer graphics forum, Vol. 31. Wiley Online Library, 237--246.Google Scholar
- Christopher A Burns, Kayvon Fatahalian, and William R Mark. 2010. A lazy object-space shading architecture with decoupled sampling. In Proceedings of the Conference on High Performance Graphics. Eurographics Association, 19--28. Google Scholar
Digital Library
- George Casella and Roger L Berger. 2002. Statistical inference. Vol. 2. Duxbury Pacific Grove, CA.Google Scholar
- Stanley H Chan and Truong Q Nguyen. 2011. LCD motion blur: modeling, analysis, and algorithm. IEEE Transactions on Image Processing 20, 8 (2011), 2352--2365. Google Scholar
Digital Library
- Petrik Clarberg, Robert Toth, Jon Hasselgren, Jim Nilsson, and Tomas Akenine-Möller. 2014. AMFS: adaptive multi-frequency shading for future graphics processors. ACM Transactions on Graphics (TOG) 33, 4 (2014), 141. Google Scholar
Digital Library
- Petrik Clarberg, Robert Toth, and Jacob Munkberg. 2013. A sort-based deferred shading architecture for decoupled sampling. ACM Transactions on Graphics (TOG) 32, 4 (2013), 141. Google Scholar
Digital Library
- Jean-Philippe Farrugia and Bernard Péroche. 2004. A progressive rendering algorithm using an adaptive perceptually based image metric. In Computer Graphics Forum, Vol. 23. Wiley Online Library, 605--614.Google Scholar
- David J Field. 1987. Relations between the statistics of natural images and the response properties of cortical cells. Josa a 4, 12 (1987), 2379--2394.Google Scholar
- Yong He, Yan Gu, and Kayvon Fatahalian. 2014. Extending the graphics pipeline with adaptive, multi-rate shading. ACM Transactions on Graphics (TOG) 33, 4 (2014), 142. Google Scholar
Digital Library
- Karl E Hillesland and JC Yang. 2016. Texel shading. In Proceedings of the 37th Annual Conference of the European Association for Computer Graphics: Short Papers. Eurographics Association, 73--76. Google Scholar
Digital Library
- Gábor Liktor and Carsten Dachsbacher. 2012. Decoupled deferred shading for hardware rasterization. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 143--150. Google Scholar
Digital Library
- Justin Luitjens. 2016. Faster Parallel Reductions on Kepler. https://devblogs.nvidia.com/faster-parallel-reductions-kepler. (2016).Google Scholar
- Don P Mitchell. 1987. Generating antialiased images at low sampling densities. In ACM SIGGRAPH Computer Graphics, Vol. 21. ACM, 65--72. Google Scholar
Digital Library
- Diego Nehab, Pedro V. Sander, Jason D. Lawrence, Natalya Tatarchuk, and John R. Isidoro. 2007. Accelerating Real-Time Shading with Reverse Reprojection Caching. In ACM SIGGRAPH/Eurographics Symposium on Graphics Hardware. 25--35. Google Scholar
Digital Library
- NVIDIA. 2015. VRWorks - Multi-Res Shading. https://developer.nvidia.com/vrworks/graphics/multiresshading. (2015).Google Scholar
- NVIDIA. 2016. VRWorks - Lens-Matched Shading. https://developer.nvidia.com/vrworks/graphics/lensmatchedshading. (2016).Google Scholar
- NVIDIA. 2018. NVIDIA Turing GPU Architecture Whitepaper. https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf. (2018).Google Scholar
- Alan V. Oppenheim and Ronald W. Schafer. 2009. Discrete-Time Signal Processing (3rd ed.). Prentice Hall Press, Upper Saddle River, NJ, USA. Google Scholar
Digital Library
- Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics (TOG) 35, 6 (2016), 179. Google Scholar
Digital Library
- Olivier Penacchio and Arnold J Wilkins. 2015. Visual discomfort and the spatial distribution of Fourier energy. Vision research 108 (2015), 1--7.Google Scholar
- Jonathan Ragan-Kelley, Jaakko Lehtinen, Jiawen Chen, Michael Doggett, and Frédo Durand. 2011. Decoupled sampling for graphics pipelines. ACM Transactions on Graphics (TOG) 30, 3 (2011), 17. Google Scholar
Digital Library
- Jeremy Shopf. 2009. Mixed resolution rendering. In Game Developers Conference.Google Scholar
- Michael Stengel, Steve Grogorick, Martin Eisemann, and Marcus Magnor. 2016. Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 129--139. Google Scholar
Digital Library
- Ee-Leng Tan and Woon-Seng Gan. 2015. Computational Models for Just-Noticeable Differences. In Perceptual Image Coding with Discrete Cosine Transform. Springer, 3--19.Google Scholar
- TestUFO.com. 2017. Blur Busters UFO Motion Tests. https://www.testufo.com/. (2017).Google Scholar
- Robert Toth, Jim Nilsson, and Tomas Akenine-Möller. 2016. Comparison of projection methods for rendering virtual reality. In Proceedings of High Performance Graphics. Eurographics Association, 163--171. Google Scholar
Digital Library
- Karthik Vaidyanathan, Marco Salvi, Robert Toth, Tim Foley, Tomas Akenine-Möller, Jim Nilsson, Jacob Munkberg, Jon Hasselgren, Masamichi Sugihara, Petrik Clarberg, et al. 2014. Coarse pixel shading. In Proceedings of High Performance Graphics. Eurographics Association, 9--18. Google Scholar
Digital Library
- Karthik Vaidyanathan, Robert Toth, Marco Salvi, Solomon Boulos, and Aaron Lefohn. 2012. Adaptive image space shading for motion and defocus blur. In Proceedings of the Fourth ACM SIGGRAPH/Eurographics conference on High-Performance Graphics. Eurographics Association, 13--21. Google Scholar
Digital Library
- Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612. Google Scholar
Digital Library
- Andrew B Watson, Albert J Ahumada, and Joyce E Farrell. 1986. Window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays. JOSA A 3, 3 (1986), 300--307.Google Scholar
Cross Ref
- Kai Xiao, Gabor Liktor, and Karthik Vaidyanathan. 2018. Coarse pixel shading with temporal supersampling. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 1. Google Scholar
Digital Library
- Lei Yang, Pedro V. Sander, and Jason Lawrence. 2008. Geometry-Aware Framebuffer Level of Detail. Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering 2008) 27, 4 (2008), 1183--1188. Google Scholar
Digital Library
- Lei Yang, Yu-Chiu Tse, Pedro V Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L Wilkins. 2011. Image-based bidirectional scene reprojection. In ACM Transactions on Graphics (TOG), Vol. 30. ACM, 150. Google Scholar
Digital Library
- Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and S-E Yoon. 2015. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 667--681. Google Scholar
Digital Library
Index Terms
Visually Lossless Content and Motion Adaptive Shading in Games
Recommendations
Improved Specular Highlights With Adaptive Shading
CGI '96: Proceedings of the 1996 Conference on Computer Graphics InternationalGouraud shading and Phong shading are widely used interpolation methods to render a polygon mesh of a curved surface. When an illumination equation has a specular reflection term Phong shading produces more realistic results than Gouraud shading. The ...
Micropolygon ray tracing with defocus and motion blur
SIGGRAPH '10: ACM SIGGRAPH 2010 papersWe present a micropolygon ray tracing algorithm that is capable of efficiently rendering high quality defocus and motion blur effects. A key component of our algorithm is a BVH (bounding volume hierarchy) based on 4D hyper-trapezoids that project into ...
Towards multi-perspective rasterization
We present a novel framework for real-time multi-perspective rendering. While most existing approaches are based on ray-tracing, we present an alternative approach by emulating multi-perspective rasterization on the classical perspective graphics ...






Comments