skip to main content
research-article

Gradient Estimation for Real-time Adaptive Temporal Filtering

Published:24 August 2018Publication History
Skip Abstract Section

Abstract

With the push towards physically based rendering, stochastic sampling of shading, e.g. using path tracing, is becoming increasingly important in real-time rendering. To achieve high performance, only low sample counts are viable, which necessitates the use of sophisticated reconstruction filters. Recent research on such filters has shown dramatic improvements in both quality and performance. They exploit the coherence of consecutive frames by reusing temporal information to achieve stable, denoised results. However, existing temporal filters often create objectionable artifacts such as ghosting and lag. We propose a novel temporal filter which analyzes the signal over time to derive adaptive temporal accumulation factors per pixel. It repurposes a subset of the shading budget to sparsely sample and reconstruct the temporal gradient. This allows us to reliably detect sudden changes of the sampled signal and to drop stale history information. We create gradient samples through forward-projection of surface samples from the previous frame into the current frame and by reevaluating the shading samples using the same random sequence. We apply our filter to improve real-time path tracers. Compared to previous work, we show a significant reduction of lag and ghosting as well as improved temporal stability. Our temporal filter runs in 2 ms at 1080p on modern graphics hardware and can be integrated into deferred renderers.

Skip Supplemental Material Section

Supplemental Material

References

  1. Colin Barré-Brisebois. 2017. A Certain Slant of Light: Past, Present and Future Challenges of Global Illumination in Games, Open Problems in Real-time Rendering. In ACM SIGGRAPH 2017 Courses (SIGGRAPH '17). ACM.Google ScholarGoogle Scholar
  2. Nir Benty, Kai-Hwa Yao, Tim Foley, Conor Lavelle, and Chris Wyman. 2017. The Falcor Rendering Framework. https://github.com/NVIDIAGameWorks/FalcorGoogle ScholarGoogle Scholar
  3. Christopher A. Burns and Warren A. Hunt. 2013. The Visibility Buffer: A Cache-Friendly Approach to Deferred Shading. Journal of Computer Graphics Techniques (JCGT) 2, 2 (2013), 55--69. http://jcgt.org/published/0002/02/04/Google ScholarGoogle Scholar
  4. Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder. ACM Transactions on Graphics (Proc. SIGGRAPH) 36, 4 (2017), 98:1--98:12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Alessandro Dal Corso, Marco Salvi, Craig Kolb, Jeppe Revall Frisvad, Aaron Lefohn, and David Luebke. 2017. Interactive Stable Ray Tracing. In Proc. of High Performance Graphics (HPG '17). 1:1--1:10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Holger Dammertz, Daniel Sewtz, Johannes Hanika, and Hendrik Lensch. 2010. Edge-Avoiding À-Trous Wavelet Transform for fast Global Illumination Filtering. In Proc. of High Performance Graphics (HPG '10). 67--75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Mauricio Delbracio, Pablo Musé, Antoni Buades, Julien Chauvier, Nicholas Phelps, and Jean-Michel Morel. 2014. Boosting Monte Carlo Rendering by Ray Histogram Fusion. ACM Transactions on Graphics 33, 1 (2014), 8:1--8:15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Iliyan Georgiev and Marcos Fajardo. 2016. Blue-noise Dithered Sampling. In ACM SIGGRAPH 2016 Talks. 35:1--35:1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Tobias Günther and Thorsten Grosch. 2015. Consistent Scene Editing by Progressive Difference Images. Computer Graphics Forum 34, 4 (2015), 41--51. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. John H. Halton. 1964. Algorithm 247: Radical-Inverse Quasi-Random Point Sequence. Commun. ACM 7, 12 (1964), 701--702. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Jose A. Iglesias-Guitian, Bochang Moon, Charalampos Koniaris, Eric Smolikowski, and Kenny Mitchell. 2016. Pixel History Linear Models for Real-time Temporal Filtering. Computer Graphics Forum (Proc. Pacific Graphics) 35, 7 (2016), 363--372. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Anton S. Kaplanyan and Carsten Dachsbacher. 2013. Path Space Regularization for Holistic and Robust Light Transport. Computer Graphics Forum (Proc. of Eurographics) 32, 2 (2013), 63--72.Google ScholarGoogle ScholarCross RefCross Ref
  13. Brian Karis. 2014. High-Quality Temporal Supersampling. In ACM SIGGRAPH 2014 Courses: Advances in Real-time Rendering in Games, Part I (SIGGRAPH '14).Google ScholarGoogle Scholar
  14. Marco Manzi, Markus Kettunen, Frédo Durand, Matthias Zwicker, and Jaakko Lehtinen. 2016. Temporal Gradient-domain Path Tracing. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 35, 6 (2016), 246:1--246:9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Michael Mara, Morgan McGuire, Benedikt Bitterli, and Wojciech Jarosz. 2017. An Efficient Denoising Algorithm for Global Illumination. In Proc. of High Performance Graphics (HPG '17). 3:1--3:7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mark Meyer and John Anderson. 2006. Statistical Acceleration for Animated Global Illumination. ACM Transactions on Graphics 25, 3 (2006), 1075--1080. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Bochang Moon, Jose A Iglesias-Guitian, Sung-Eui Yoon, and Kenny Mitchell. 2015. Adaptive Rendering with Linear Predictions. ACM Transactions on Graphics 34, 4 (2015), 121:1--121:11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Jacob Munkberg, Jon Hasselgren, Petrik Clarberg, Magnus Andersson, and Tomas Akenine-Möller. 2016. Texture Space Caching and Reconstruction for Ray Tracing. ACM Transactions on Graphics 35, 6 (2016), 249:1--249:13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Diego Nehab, Pedro V. Sander, Jason Lawrence, Natalya Tatarchuk, and John R. Isidoro. 2007. Accelerating Real-time Shading with Reverse Reprojection Caching. In SIGGRAPH/Eurographics Workshop on Graphics Hardware. 25--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Steven G. Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, and Martin Stich. 2010. OptiX: A General Purpose Ray Tracing Engine. ACM Transactions on Graphics (Proc. SIGGRAPH) 29, 4 (2010), 66:1--66:13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards Foveated Rendering for Gaze-Tracked Virtual Reality. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 35, 6 (2016), 179:1--179:12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Fabrice Rousselle, Wojciech Jarosz, and Jan Novák. 2016. Image-space Control Variates for Rendering. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 35, 6 (2016), 169:1--169:12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Takafumi Saito and Tokiichiro Takahashi. 1990. Comprehensible Rendering of 3-D Shapes. Computer Graphics (Proc. SIGGRAPH) (1990), 197--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Daniel Scherzer, Lei Yang, Oliver Mattausch, Diego Nehab, Pedro V. Sander, Michael Wimmer, and Elmar Eisemann. 2012. Temporal Coherence Methods in Real-time Rendering. Computer Graphics Forum 31, 8 (2012), 2378--2408. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Christoph Schied, Anton Kaplanyan, Chris Wyman, Anjul Patney, Chakravarty R. Alla Chaitanya, John Burgess, Shiqiu Liu, Carsten Dachsbacher, Aaron Lefohn, and Marco Salvi. 2017. Spatiotemporal Variance-guided Filtering: Real-time Reconstruction for Path-traced Global Illumination. In Proc. of High Performance Graphics (HPG '17). 2:1--2:12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Robert A. Ulichney. 1993. Void-and-cluster method for dither array generation. Proc. SPIE 1913 (1993), 1913 - 1913 - 12.Google ScholarGoogle Scholar
  27. Bruce Walter, George Drettakis, and Steven Parker. 1999. Interactive Rendering Using the Render Cache. In Eurographics Workshop on Rendering, Vol. 10. 19--30. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Kai Xiao, Gabor Liktor, and Karthik Vaidyanathan. 2018. Coarse Pixel Shading with Temporal Supersampling. In Proc. ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D '18). ACM, Article 1, 7 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Lei Yang, Diego Nehab, Pedro V Sander, Pitchaya Sitthi-amorn, Jason Lawrence, and Hugues Hoppe. 2009. Amortized Supersampling. ACM Transactions on Graphics (Proc. SIGGRAPH Asia) 28, 5 (2009), 135:1--135:12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Henning Zimmer, Fabrice Rousselle, Wenzel Jakob, Oliver Wang, David Adler, Wojciech Jarosz, Olga Sorkine-Hornung, and Alexander Sorkine-Hornung. 2015. Path-space Motion Estimation and Decomposition for Robust Animation Filtering. Proc. Eurographics Symposium on Rendering 34, 4 (2015), 131--142.Google ScholarGoogle ScholarCross RefCross Ref
  32. Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and S-E Yoon. 2015. Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum (STAR) 34, 2 (2015), 667--681. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Gradient Estimation for Real-time Adaptive Temporal Filtering

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!