Blockwise Multi-Order Feature Regression for Real-Time Path-Tracing Reconstruction

Abstract

Path tracing produces realistic results including global illumination using a unified simple rendering pipeline. Reducing the amount of noise to imperceptible levels without post-processing requires thousands of samples per pixel (spp), while currently it is only possible to render extremely noisy 1 spp frames in real time with desktop GPUs. However, post-processing can utilize feature buffers, which contain noise-free auxiliary data available in the rendering pipeline. Previously, regression-based noise filtering methods have only been used in offline rendering due to their high computational cost. In this article we propose a novel regression-based reconstruction pipeline, called Blockwise Multi-Order Feature Regression (BMFR), tailored for path-traced 1 spp inputs that runs in real time. The high speed is achieved with a fast implementation of augmented QR factorization and by using stochastic regularization to address rank-deficient feature data. The proposed algorithm is 1.8× faster than the previous state-of-the-art real-time path-tracing reconstruction method while producing better quality frame sequences.

References

1. Anne Aaron, Zhi Li, Megha Manohara, Joe Yuchieh Lin, Eddy Chi-Hao Wu, and C.-C. Jay Kuo. 2015. Challenges in cloud based ingest and encoding for high quality streaming media. In Proceedings of the Annual Conference on Image Processing.
2. Timo Aila and Tero Karras. 2010. Architecture considerations for tracing incoherent rays. In Proceedings of the Annual Conference on High Performance Graphics.
3. Chakravarty Alla Chaitanya, Anton Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. Trans. Graph. 36, 4 (2017).
4. AMD. 2017. RadeonRays SDK. Online. Retrieved January 23, 2018 from https://github.com/GPUOpen-LibrariesAndSDKs/RadeonRays_SDK.
5. Steve Bako, Thijs Vogels, Brian Mcwilliams, Mark Meyer, Jan NováK, Alex Harvill, Pradeep Sen, Tony Derose, and Fabrice Rousselle. 2017. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. Trans. Graph. 36, 4 (2017).
6. Pablo Bauszat, Martin Eisemann, and Marcus Magnor. 2011. Guided image filtering for interactive high-quality global illumination. Comput. Graph. Forum 30, 4 (2011).
7. Benedikt Bitterli. 2016. Rendering Resources. Retrieved from https://benedikt-bitterli.me/resources/.
8. Benedikt Bitterli, Fabrice Rousselle, Bochang Moon, José A Iglesias-Guitián, David Adler, Kenny Mitchell, Wojciech Jarosz, and Jan Novák. 2016. Nonlinearly weighted first-order regression for denoising monte carlo renderings. Comput. Graph. Forum 35, 4 (2016).
9. Peter Burt. 1981. Fast filter transform for image processing. Comput. Graph. Image Process. 16, 1 (1981).
10. Holger Dammertz, Daniel Sewtz, Johannes Hanika, and Hendrik Lensch. 2010. Edge-avoiding À-Trous wavelet transform for fast global illumination filtering. In Proceedings of the Annual Conference on High Performance Graphics.
11. Kevin Egan, Yu-Ting Tseng, Nicolas Holzschuch, Frédo Durand, and Ravi Ramamoorthi. 2009. Frequency analysis and sheared reconstruction for rendering motion blur. Trans. Graph. 28, 3 (2009), 93.
12. Luke Goddard. 2014. Silencing the noise on elysium. In Proceedings of the ACM SIGGRAPH 2014 Talks.
13. Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided image filtering. Trans. Pattern Anal. Mach. Intell. 35, 6 (2013).
14. Michael Heath. 1997. Scientific Computing. McGraw–Hill.
15. Jorge Jimenez, Jose I. Echevarria, Tiago Sousa, and Diego Gutierrez. 2012. SMAA: Enhanced morphological antialiasing. Comput. Graph. Forum (Proc. EUROGRAPHICS 2012) 31, 2 (2012).
16. Jorge Jiménez, X. Wu, A. Pesce, and A. Jarabo. 2016. Practical real-time strategies for accurate indirect occlusion. SIGGRAPH 2016 Courses: Physically Based Shading in Theory and Practice (2016).
17. Nima Khademi Kalantari, Steve Bako, and Pradeep Sen. 2015. A machine learning approach for filtering Monte Carlo noise. Trans. Graph. 34, 4 (2015).
18. Anton Kaplanyan and Carsten Dachsbacher. 2013. Path space regularization for holistic and robust light transport. Comput. Graph. Forum 32, 2pt1 (2013).
19. Brian Karis. 2014. High-quality temporal supersampling. In Proceedings of the ACM SIGGRAPH Advances in Real-Time Rendering in Games 2014.
20. Samuli Laine, Tero Karras, and Timo Aila. 2013. Megakernels considered harmful: Wavefront path tracing on GPUs. In Proceedings of the Annual Conference on High Performance Graphics.
21. Tzu-Mao Li, Yu-Ting Wu, and Yung-Yu Chuang. 2012. SURE-based optimization for adaptive sampling and reconstruction. Trans. Graph. 31, 6 (2012).
22. Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. 2016. Toward a Practical Perceptual Video Quality Metric. Retrieved January 23, 2018 from https://medium.com/netflix-techblog/toward-a-practical-perceptual-video-quality-metric-653f208b9652.
23. Yu Liu, Changwen Zheng, Quan Zheng, and Hongliang Yuan. 2017. Removing Monte Carlo noise using a Sobel operator and a guided image filter. Vis. Comput. 34, 4 (2017).
24. Michael Mara, Morgan McGuire, Benedikt Bitterli, and Wojciech Jarosz. 2017. An efficient denoising algorithm for global illumination. In Proceedings of the Annual Conference on High Performance Graphics.
25. Morgan McGuire. 2017. Computer Graphics Archive. Retrieved from https://casual-effects.com/data.
26. Bochang Moon, Nathan Carr, and Sung-Eui Yoon. 2014. Adaptive rendering based on weighted local regression. Trans. Graph. 33, 5 (2014).
27. Bochang Moon, Jose A Iglesias-Guitian, Sung-Eui Yoon, and Kenny Mitchell. 2015. Adaptive rendering with linear predictions. Trans. Graph. 34, 4 (2015).
28. Bochang Moon, Steven McDonagh, Kenny Mitchell, and Markus Gross. 2016. Adaptive polynomial rendering. Trans. Graph. 35, 4 (2016).
29. Steven G. Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, et al. 2010. Optix: A general purpose ray tracing engine. Trans. Graph. 29, 4 (2010).
30. Amar Patel. 2018. D3D12 Raytracing Functional Spec, v0.09. Microsoft. Retrieved March 23, 2018 Available: http://forums.directxtech.com/index.php?topic=5860.0.
31. Matt Pharr and Greg Humphreys. 2010. Physically Based Rendering: From Theory to Implementation (2nd ed.). Morgan Kaufmann.
32. Amy R. Reibman and David Poole. 2007. Predicting packet-loss visibility using scene characteristics. In Proceedings of the Annual Conference on Packet Video.
33. Gilberto Rosado. 2007. Motion blur as a post-processing effect. In GPU Gems 3. Addison-Wesley Professional, Chapter 27.
34. Christoph Schied, Anton Kaplanyan, Chris Wyman, Anjul Patney, Chakravarty R. Alla Chaitanya, John Burgess, Shiqiu Liu, Carsten Dachsbacher, Aaron Lefohn, and Marco Salvi. 2017. Spatiotemporal variance-guided filtering: Real-time reconstruction for path-traced global illumination. In Proceedings of the Annual Conference on High Performance Graphics.
35. Christoph Schied, Christoph Peters, and Carsten Dachsbacher. 2018. Gradient estimation for real-time adaptive temporal filtering. Proceedings of the ACM Annual Conference on Computer Graphics and Interactive Techniques 1, 2 (2018), 24.
36. Carlo Tomasi and Roberto Manduchi. 1998. Bilateral filtering for gray and color images. In Proceedings of the Annual Conference on Computer Vision.
37. Eric Veach and Leonidas J Guibas. 1995. Optimally combining sampling techniques for Monte Carlo rendering. In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques.
38. Timo Viitanen, Matias Koskela, Kalle Immonen, Markku Mäkitalo, Pekka Jääskeläinen, and Jarmo Takala. 2018. Sparse sampling for real-time ray tracing. In Proceedings of the International Conference on Computer Graphics Theory and Applications (GRAPP’18).
39. Ingo Wald, Sven Woop, Carsten Benthin, Gregory S. Johnson, and Manfred Ernst. 2014. Embree: A kernel framework for efficient CPU ray tracing. Trans. Graph. 33, 4 (2014).
40. Yong Wang, Xiaofeng Liao, Di Xiao, and Kwok-Wo Wong. 2008. One-way hash function construction based on 2D coupled map lattices. Inf. Sci. 178, 5 (2008).
41. Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: From error visibility to structural similarity. Trans. Image Process. 13, 4 (2004).
42. Ling-Qi Yan, Soham Uday Mehta, Ravi Ramamoorthi, and Fredo Durand. 2015. Fast 4D sheared filtering for interactive rendering of distribution effects. Trans. Graph. 35, 1 (2015), 7.
43. Lei Yang, Diego Nehab, Pedro V. Sander, Pitchaya Sitthi-amorn, Jason Lawrence, and Hugues Hoppe. 2009. Amortized supersampling. Trans. Graph. 28, 5 (2009).
44. Henning Zimmer, Fabrice Rousselle, Wenzel Jakob, Oliver Wang, David Adler, Wojciech Jarosz, Olga Sorkine-Hornung, and Alexander Sorkine-Hornung. 2015. Path-space motion estimation and decomposition for robust animation filtering. Comput. Graph. Forum 34, 4 (2015).
45. Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and S.-E. Yoon. 2015. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Comput. Graph. Forum 34, 2 (2015).

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

      • Image processing

    • Rendering

      • Ray tracing