Using moments to represent bounded signals for spectral rendering

Abstract

We present a compact and efficient representation of spectra for accurate rendering using more than three dimensions. While tristimulus color spaces are sufficient for color display, a spectral renderer has to simulate light transport per wavelength. Consequently, emission spectra and surface albedos need to be known at each wavelength. It is practical to store dense samples for emission spectra but for albedo textures, the memory requirements of this approach are unreasonable. Prior works that approximate dense spectra from tristimulus data introduce strong errors under illuminants with sharp peaks and in indirect illumination. We represent spectra by an arbitrary number of Fourier coefficients. However, we do not use a common truncated Fourier series because its ringing could lead to albedos below zero or above one. Instead, we present a novel approach for reconstruction of bounded densities based on the theory of moments. The core of our technique is our bounded maximum entropy spectral estimate. It uses an efficient closed form to compute a smooth signal between zero and one that matches the given Fourier coefficients exactly. Still, a ground truth that localizes all of its mass around a few wavelengths can be reconstructed adequately. Therefore, our representation covers the full gamut of valid reflectances. The resulting textures are compact because each coefficient can be stored in 10 bits. For compatibility with existing tristimulus assets, we implement a mapping from tristimulus color spaces to three Fourier coefficients. Using three coefficients, our technique gives state of the art results without some of the drawbacks of related work. With four to eight coefficients, our representation is superior to all existing representations. Our focus is on offline rendering but we also demonstrate that the technique is fast enough for real-time rendering.

References

1. Jonas Aeschbacher, Jiqing Wu, and Radu Timofte. 2017. In Defense of Shallow Learned Spectral Reconstruction from RGB Images. In 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). IEEE, 471--479.
2. Laurent Belcour and Pascal Barla. 2017. A Practical Extension to Microfacet Theory for the Modeling of Varying Iridescence. ACM Trans. Graph. (Proc. SIGGRAPH) 36, 4 (2017), 65:1--65:14.
3. Nir Benty, Kai-Hwa Yao, Tim Foley, Conor Lavelle, and Chris Wyman. 2018. The Falcor Rendering Framework 3.2. https://github.com/NVIDIAGameWorks/Falcor
4. Steven Bergner, Mark S. Drew, and Torsten Möller. 2009. A Tool to Create Illuminant and Reflectance Spectra for Light-driven Graphics and Visualization. ACM Trans. Graph. 28, 1, Article 5 (2009), 5:1--5:11 pages.
5. John Parker Burg. 1975. Maximum Entropy Spectral Analysis. Ph.D. dissertation. Stanford University, Department of Geophysics. http://sepwww.stanford.edu/data/media/public/oldreports/sep06/
6. Tenn F. Chen, Gladimir V. G. Baranoski, Bradley W. Kimmel, and Erik Miranda. 2015. Hyperspectral Modeling of Skin Appearance. ACM Trans. Graph. 34, 3, Article 31 (2015), 31:1--31:14 pages.
7. Oskar Elek, Pablo Bauszat, Tobias Ritschel, Marcus Magnor, and Hans-Peter Seidel. 2014. Spectral Ray Differentials. Computer Graphics Forum 33, 4 (2014), 113--122.
8. Luca Fascione, Johannes Hanika, Marcos Fajardo, Per Christensen, Brent Burley, and Brian Green. 2017. Path Tracing in Production - Part 1: Production Renderers. In ACM SIGGRAPH 2017 Courses (SIGGRAPH '17). ACM, Article 13, 13:1--13:39 pages.
9. David Geisler-Moroder and Arne Dür. 2010. A New Ward BRDF Model with Bounded Albedo. Computer Graphics Forum 29, 4 (2010), 1391--1398.
10. Andrew S. Glassner. 1989. How to derive a spectrum from an RGB triplet. IEEE Computer Graphics and Applications 9, 4 (1989), 95--99.
11. Gene H. Golub and Charles F. Van Loan. 2012. Matrix Computations, Fourth Edition. The Johns Hopkins University Press. https://jhupbooks.press.jhu.edu/content/matrix-computations-0
12. Björn Gustafsson and Mihai Putinar. 2017. Hyponormal Quantization of Planar Domains, Exponential Transform in Dimension Two. Lecture Notes in Mathematics, Vol. 2199. Springer International Publishing.
13. Ville Heikkinen, Reiner Lenz, Tuija Jetsu, Jussi Parkkinen, Markku Hauta-Kasari, and Timo Jääskeläinen. 2008. Evaluation and unification of some methods for estimating reflectance spectra from RGB images. J. Opt. Soc. Am. A 25, 10 (2008), 2444--2458.
14. Wenzel Jakob and Johannes Hanika. 2019. A Low-Dimensional Function Space for Efficient Spectral Upsampling. Computer Graphics Forum 38, 2 (2019).
15. Johan Karlsson and Tryphon T. Georgiou. 2013. Uncertainty Bounds for Spectral Estimation. IEEE Trans. Automat. Control 58, 7 (2013), 1659--1673.
16. Bradley W. Kimmel, Gladimir V. G. Baranoski, T. F. Chen, Daniel Yim, and Erik Miranda. 2013. Spectral Appearance Changes Induced by Light Exposure. ACM Trans. Graph. 32, 1 (2013), 10:1--10:13.
17. I. V. Kovalishina and Vladimir Petrovich Potapov. 1982. Integral representation of Hermitian positive functions. Hokkaido University, Sapporo, Japan. https://gso.gbv.de/DB=2.1/PPNSET?PPN=014282208 Private translation by Tsuyoshi Ando of a Russian monograph. Copies were gifted to several libraries.
18. Boris Kravchenko, Gladimir V. G. Baranoski, Tenn Francis Chen, Erik Miranda, and Spencer R. Van Leeuwen. 2017. High-fidelity iridal light transport simulations at interactive rates. Computer Animation and Virtual Worlds 28, 3--4 (2017).
19. Mark Grigorievich Kreĭn and Adol'f Abramovich Nudel'man. 1977. The Markov Moment Problem and Extremal Problems. Translations of Mathematical Monographs, Vol. 50. American Mathematical Society.
20. Peter Kutz, Ralf Habel, Yining Karl Li, and Jan Novák. 2017. Spectral and Decomposition Tracking for Rendering Heterogeneous Volumes. ACM Trans. Graph. (Proc. SIGGRAPH) 36, 4 (2017).
21. David L. MacAdam. 1935a. Maximum Visual Efficiency of Colored Materials. J. Opt. Soc. Am. 25, 11 (1935).
22. David L. MacAdam. 1935b. The Theory of the Maximum Visual Efficiency of Colored Materials. J. Opt. Soc. Am. 25, 8 (1935).
23. André Markoff. 1896. Nouvelles applications des fractions continues. Math. Ann. 47, 4 (1896), 579--597.
24. Johannes Meng, Florian Simon, Johannes Hanika, and Carsten Dachsbacher. 2015. Physically Meaningful Rendering using Tristimulus Colours. Computer Graphics Forum 34, 4 (2015).
25. Michal Mojzík, Alban Fichet, and Alexander Wilkie. 2018. Handling Fluorescence in a Uni-directional Spectral Path Tracer. Computer Graphics Forum 37, 4 (2018), 77--94.
26. Cedrick Münstermann, Stefan Krumpen, Reinhard Klein, and Christoph Peters. 2018. Moment-Based Order-Independent Transparency. Proc. ACM Comput. Graph. Interact. Tech. (Proc. i3D) 1, 1 (2018), 7:1--7:20.
27. Hisanari Otsu, Masafumi Yamamoto, and Toshiya Hachisuka. 2018. Reproducing Spectral Reflectances From Tristimulus Colours. Computer Graphics Forum 37, 6 (2018), 370--381.
28. Jong-Il Park, Moon-Hyun Lee, Michael D. Grossberg, and Shree K. Nayar. 2007. Multispectral Imaging Using Multiplexed Illumination. In IEEE 11th International Conference on Computer Vision. 1--8.
29. Mark S. Peercy. 1993. Linear Color Representations for Full Speed Spectral Rendering. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '93). ACM, 191--198.
30. Christoph Peters, Jonathan Klein, Matthias B. Hullin, and Reinhard Klein. 2015. Solving Trigonometric Moment Problems for Fast Transient Imaging. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 34, 6 (2015).
31. Christoph Peters and Reinhard Klein. 2015. Moment Shadow Mapping. In Proceedings of the 19th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (i3D '15). ACM, 7--14.
32. Christoph Peters, Cedrick Münstermann, Nico Wetzstein, and Reinhard Klein. 2017. Improved Moment Shadow Maps for Translucent Occluders, Soft Shadows and Single Scattering. Journal of Computer Graphics Techniques (JCGT) 6, 1 (2017), 17--67. http://jcgt.org/published/0006/01/03/
33. Victor Petitjean, Pablo Bauszat, and Elmar Eisemann. 2018. Spectral Gradient Sampling for Path Tracing. Computer Graphics Forum 37, 4 (2018).
34. Michal Radziszewski, Krzysztof Boryczko, and Witold Alda. 2009. An Improved Technique for Full Spectral Rendering. Journal of WSCG 17 (2009), 9--16.
35. Brian Sharpe. 2018. Moment Transparency. In Proceedings of the Conference on High-Performance Graphics (HPG '18). ACM.
36. Brian Smits. 1999. An RGB-to-spectrum Conversion for Reflectances. Journal of Graphics Tools 4, 4 (1999), 11--22.
37. Antoine Toisoul, Daljit Singh Dhillon, and Abhijeet Ghosh. 2018. Acquiring Spatially Varying Appearance of Printed Holographic Surfaces. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018).
38. Antoine Toisoul and Abhijeet Ghosh. 2017a. Practical Acquisition and Rendering of Diffraction Effects in Surface Reflectance. ACM Trans. Graph. 36, 5 (2017).
39. Antoine Toisoul and Abhijeet Ghosh. 2017b. Real-time Rendering of Realistic Surface Diffraction with Low Rank Factorisation. In Proceedings of the 14th European Conference on Visual Media Production. ACM.
40. Brian A. Wandell. 1987. The Synthesis and Analysis of Color Images. IEEE Transactions on Pattern Analysis and Machine Intelligence 9, 1 (1987), 2--13.
41. Alexander Wilkie, Sehera Nawaz, Marc Droske, Andrea Weidlich, and Johannes Hanika. 2014. Hero Wavelength Spectral Sampling. Computer Graphics Forum 33, 4 (2014).
42. Ling-Qi Yan, Miloš Hašan, Bruce Walter, Steve Marschner, and Ravi Ramamoorthi. 2018. Rendering Specular Microgeometry with Wave Optics. ACM Trans. Graph. (Proc. SIGGRAPH) 37, 4 (2018), 75:1--75:10.

Index Terms

• Computing methodologies

  • Computer graphics

    • Rendering

      • Reflectance modeling