skip to main content
research-article

Visually Lossless Content and Motion Adaptive Shading in Games

Published:03 June 2019Publication History
Skip Abstract Section

Abstract

We present a technique that adaptively adjusts the shading rate during rendering based on the scene content and motion. Our goal is to improve performance with no loss in perceived quality. We determine per-screen-tile shading rate by testing an error estimate against a perceptually-corrected just-noticeable difference threshold. Our design features an effective and efficient error estimate using spatial and frequency analysis of half and quarter rate shading. We also study the effect of motion in reducing perceived error, a consequence of display-persistence and/or motion blur effects. Our implementation uses the computed per-tile shading rate with variable rate shading (a recent GPU feature) to lower shading cost. We demonstrate the quality and performance of our technique on two high-end game engines and shipped games.

Skip Supplemental Material Section

Supplemental Material

References

  1. Kurt Akeley. 1993. Reality engine graphics. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. ACM, 109--116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Tomas Akenine-Moller, Eric Haines, and Naty Hoffman. 2018. Real-time rendering. AK Peters/CRC Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Magnus Andersson, Jon Hasselgren, Robert Toth, and Tomas Akenine-Möiler. 2014. Adaptive texture space shading for stochastic rendering. Computer Graphics Forum 33, 2 (2014), 341--350. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Swaroop Bhonde. 2018. Turing Variable Rate Shading in VRWorks. https://devblogs.nvidia.com/turing-variable-rate-shading-vrworks/. (2018).Google ScholarGoogle Scholar
  5. Mark R Bolin and Gary W Meyer. 1998. A perceptually based adaptive sampling algorithm. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. ACM, 299--309. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Huw Bowles, Kenny Mitchell, Robert W Sumner, Jeremy Moore, and Markus Gross. 2012. Iterative image warping. In Computer graphics forum, Vol. 31. Wiley Online Library, 237--246.Google ScholarGoogle Scholar
  7. Christopher A Burns, Kayvon Fatahalian, and William R Mark. 2010. A lazy object-space shading architecture with decoupled sampling. In Proceedings of the Conference on High Performance Graphics. Eurographics Association, 19--28. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. George Casella and Roger L Berger. 2002. Statistical inference. Vol. 2. Duxbury Pacific Grove, CA.Google ScholarGoogle Scholar
  9. Stanley H Chan and Truong Q Nguyen. 2011. LCD motion blur: modeling, analysis, and algorithm. IEEE Transactions on Image Processing 20, 8 (2011), 2352--2365. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Petrik Clarberg, Robert Toth, Jon Hasselgren, Jim Nilsson, and Tomas Akenine-Möller. 2014. AMFS: adaptive multi-frequency shading for future graphics processors. ACM Transactions on Graphics (TOG) 33, 4 (2014), 141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Petrik Clarberg, Robert Toth, and Jacob Munkberg. 2013. A sort-based deferred shading architecture for decoupled sampling. ACM Transactions on Graphics (TOG) 32, 4 (2013), 141. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Jean-Philippe Farrugia and Bernard Péroche. 2004. A progressive rendering algorithm using an adaptive perceptually based image metric. In Computer Graphics Forum, Vol. 23. Wiley Online Library, 605--614.Google ScholarGoogle Scholar
  13. David J Field. 1987. Relations between the statistics of natural images and the response properties of cortical cells. Josa a 4, 12 (1987), 2379--2394.Google ScholarGoogle Scholar
  14. Yong He, Yan Gu, and Kayvon Fatahalian. 2014. Extending the graphics pipeline with adaptive, multi-rate shading. ACM Transactions on Graphics (TOG) 33, 4 (2014), 142. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Karl E Hillesland and JC Yang. 2016. Texel shading. In Proceedings of the 37th Annual Conference of the European Association for Computer Graphics: Short Papers. Eurographics Association, 73--76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Gábor Liktor and Carsten Dachsbacher. 2012. Decoupled deferred shading for hardware rasterization. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 143--150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Justin Luitjens. 2016. Faster Parallel Reductions on Kepler. https://devblogs.nvidia.com/faster-parallel-reductions-kepler. (2016).Google ScholarGoogle Scholar
  18. Don P Mitchell. 1987. Generating antialiased images at low sampling densities. In ACM SIGGRAPH Computer Graphics, Vol. 21. ACM, 65--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Diego Nehab, Pedro V. Sander, Jason D. Lawrence, Natalya Tatarchuk, and John R. Isidoro. 2007. Accelerating Real-Time Shading with Reverse Reprojection Caching. In ACM SIGGRAPH/Eurographics Symposium on Graphics Hardware. 25--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. NVIDIA. 2015. VRWorks - Multi-Res Shading. https://developer.nvidia.com/vrworks/graphics/multiresshading. (2015).Google ScholarGoogle Scholar
  21. NVIDIA. 2016. VRWorks - Lens-Matched Shading. https://developer.nvidia.com/vrworks/graphics/lensmatchedshading. (2016).Google ScholarGoogle Scholar
  22. NVIDIA. 2018. NVIDIA Turing GPU Architecture Whitepaper. https://www.nvidia.com/content/dam/en-zz/Solutions/design-visualization/technologies/turing-architecture/NVIDIA-Turing-Architecture-Whitepaper.pdf. (2018).Google ScholarGoogle Scholar
  23. Alan V. Oppenheim and Ronald W. Schafer. 2009. Discrete-Time Signal Processing (3rd ed.). Prentice Hall Press, Upper Saddle River, NJ, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics (TOG) 35, 6 (2016), 179. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Olivier Penacchio and Arnold J Wilkins. 2015. Visual discomfort and the spatial distribution of Fourier energy. Vision research 108 (2015), 1--7.Google ScholarGoogle Scholar
  26. Jonathan Ragan-Kelley, Jaakko Lehtinen, Jiawen Chen, Michael Doggett, and Frédo Durand. 2011. Decoupled sampling for graphics pipelines. ACM Transactions on Graphics (TOG) 30, 3 (2011), 17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Jeremy Shopf. 2009. Mixed resolution rendering. In Game Developers Conference.Google ScholarGoogle Scholar
  28. Michael Stengel, Steve Grogorick, Martin Eisemann, and Marcus Magnor. 2016. Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 129--139. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ee-Leng Tan and Woon-Seng Gan. 2015. Computational Models for Just-Noticeable Differences. In Perceptual Image Coding with Discrete Cosine Transform. Springer, 3--19.Google ScholarGoogle Scholar
  30. TestUFO.com. 2017. Blur Busters UFO Motion Tests. https://www.testufo.com/. (2017).Google ScholarGoogle Scholar
  31. Robert Toth, Jim Nilsson, and Tomas Akenine-Möller. 2016. Comparison of projection methods for rendering virtual reality. In Proceedings of High Performance Graphics. Eurographics Association, 163--171. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Karthik Vaidyanathan, Marco Salvi, Robert Toth, Tim Foley, Tomas Akenine-Möller, Jim Nilsson, Jacob Munkberg, Jon Hasselgren, Masamichi Sugihara, Petrik Clarberg, et al. 2014. Coarse pixel shading. In Proceedings of High Performance Graphics. Eurographics Association, 9--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Karthik Vaidyanathan, Robert Toth, Marco Salvi, Solomon Boulos, and Aaron Lefohn. 2012. Adaptive image space shading for motion and defocus blur. In Proceedings of the Fourth ACM SIGGRAPH/Eurographics conference on High-Performance Graphics. Eurographics Association, 13--21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Andrew B Watson, Albert J Ahumada, and Joyce E Farrell. 1986. Window of visibility: a psychophysical theory of fidelity in time-sampled visual motion displays. JOSA A 3, 3 (1986), 300--307.Google ScholarGoogle ScholarCross RefCross Ref
  36. Kai Xiao, Gabor Liktor, and Karthik Vaidyanathan. 2018. Coarse pixel shading with temporal supersampling. In Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Lei Yang, Pedro V. Sander, and Jason Lawrence. 2008. Geometry-Aware Framebuffer Level of Detail. Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering 2008) 27, 4 (2008), 1183--1188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Lei Yang, Yu-Chiu Tse, Pedro V Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L Wilkins. 2011. Image-based bidirectional scene reprojection. In ACM Transactions on Graphics (TOG), Vol. 30. ACM, 150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and S-E Yoon. 2015. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 667--681. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Visually Lossless Content and Motion Adaptive Shading in Games

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!