A non-exponential transmittance model for volumetric scene representations

Abstract

We introduce a novel transmittance model to improve the volumetric representation of 3D scenes. The model can represent opaque surfaces in the volumetric light transport framework. Volumetric representations are useful for complex scenes, and become increasingly popular for level of detail and scene reconstruction. The traditional exponential transmittance model found in volumetric light transport cannot capture correlations in visibility across volume elements. When representing opaque surfaces as volumetric density, this leads to both bloating of silhouettes and light leaking artifacts. By introducing a parametric non-exponential transmittance model, we are able to approximate these correlation effects and significantly improve the accuracy of volumetric appearance representation of opaque scenes. Our parametric transmittance model can represent a continuum between the linear transmittance that opaque surfaces exhibit and the traditional exponential transmittance encountered in participating media and unstructured geometries. This covers a large part of the spectrum of geometric structures encountered in complex scenes. In order to handle the spatially varying transmittance correlation effects, we further extend the theory of non-exponential participating media to a heterogeneous transmittance model. Our model is compact in storage and computationally efficient both for evaluation and for reverse-mode gradient computation. Applying our model to optimization algorithms yields significant improvements in volumetric scene appearance quality. We further show improvements for relevant applications, such as scene appearance prefiltering, image-based scene reconstruction using differentiable rendering, neural representations, and compare it to a conventional exponential model.

References

1. Johanna Beyer, Markus Hadwiger, and Hanspeter Pfister. 2015. State-of-the-Art in GPU-Based Large-Scale Volume Visualization. Computer Graphics Forum 34, 8 (Dec. 2015), 13--37.
2. Sai Bi, Zexiang Xu, Pratul Srinivasan, Ben Mildenhall, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, and Ravi Ramamoorthi. 2020a. Neural Reflectance Fields for Appearance Acquisition. arXiv:2008.03824
3. Sai Bi, Zexiang Xu, Kalyan Sunkavalli, Miloš Hašan, Yannick Hold-Geoffroy, David Kriegman, and Ravi Ramamoorthi. 2020b. Deep Reflectance Volumes: Relightable Reconstructions from Multi-view Photometric Images. In ECCV. 25 pages.
4. Benedikt Bitterli, Srinath Ravichandran, Thomas Müller, Magnus Wrenninge, Jan Novák, Steve Marschner, and Wojciech Jarosz. 2018. A radiative transfer framework for non-exponential media. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 225:1--225:17.
5. Emanuele Caglioti and Francois Golse. 2003. On the Distribution of Free Path Lengths for the Periodic Lorentz Gas III. Communications in Mathematical Physics 236, 2 (May 2003), 199--221.
6. Thomas Camminady, Martin Frank, and Edward W. Larsen. 2017. Nonclassical particle transport in heterogeneous materials. In M&C 2017 : International Conference on Mathematics & Computational Methods Applied to Nuclear Science & Engineering.
7. Subrahmanyan Chandrasekhar. 1960. Radiative transfer. Dover publications, New York.
8. Cyril Crassin, Fabrice Neyret, Sylvain Lefebvre, and Elmar Eisemann. 2009. GigaVoxels: Ray-Guided Streaming for Efficient and Detailed Voxel Rendering. In Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games (Boston, Massachusetts). Association for Computing Machinery, New York, NY, USA, 15--22.
9. Cyril Crassin, Fabrice Neyret, Miguel Sainz, Simon Green, and Elmar Eisemann. 2011. Interactive indirect illumination using voxel cone tracing. Computer Graphics Forum 30, 7 (2011), 1921--1930.
10. Anthony Davis and Mark Mineev-Weinstein. 2011. Radiation propagation in random media: From positive to negative correlations in high-frequency fluctuations. Journal of Quantitative Spectroscopy and Radiative Transfer 112 (03 2011), 632--645.
11. Eugene d'Eon. 2019. A Reciprocal Formulation of Nonexponential Radiative Transfer. 3: Binary Mixtures. arXiv:1903.08783
12. Jonathan Dupuy, Eric Heitz, and Eugene d'Eon. 2016. Additional Progress towards the Unification of Microfacet and Microflake Theories. In Proceedings of the Eurographics Symposium on Rendering: Experimental Ideas and Implementations (Dublin, Ireland) (Computer Graphics Forum (Proceedings of EGSR)). Eurographics Association, Goslar, DEU, 55--63.
13. Eugene d'Eon. 2018. A Reciprocal Formulation of Nonexponential Radiative Transfer. 1: Sketch and Motivation. Journal of Computational and Theoretical Transport 47, 1-3 (2018), 84--115.
14. Eugene d'Eon. 2019. A Reciprocal Formulation of Nonexponential Radiative Transfer. 2: Monte Carlo Estimation and Diffusion Approximation. Journal of Computational and Theoretical Transport 48, 6 (2019), 201--262.
15. Iliyan Georgiev, Zackary Misso, Toshiya Hachisuka, Derek Nowrouzezahrai, Ja roslav Křivánek, and Wojciech Jarosz. 2019. Integral formulations of volumetric transmittance. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 6 (2019), 17 pages.
16. Ioannis Gkioulekas, Anat Levin, and Todd Zickler. 2016. An Evaluation of Computational Imaging Techniques for Heterogeneous Inverse Scattering. In ECCV, Bastian Leibe, Jiri Matas, Nicu Sebe, and Max Welling (Eds.). Springer International Publishing, 685--701.
17. Aidan N. Gomez, Mengye Ren, Raquel Urtasun, and Roger B. Grosse. 2017. The Reversible Residual Network: Backpropagation without Storing Activations. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, USA) (NIPS'17). Curran Associates Inc., Red Hook, NY, USA, 2211--2221.
18. Jie Guo, Yanjun Chen, Bingyang Hu, Ling-Qi Yan, Yanwen Guo, and Yuntao Liu. 2019. Fractional Gaussian Fields for Modeling and Rendering of Spatially-Correlated Media. ACM Trans. Graph. (Proc. SIGGRAPH) 38, 4, Article 45 (July 2019), 13 pages.
19. Markus Hadwiger, Ali K. Al-Awami, Johanna Beyer, Marco Agus, and Hanspeter Pfister. 2018. SparseLeap: Efficient Empty Space Skipping for Large-Scale Volume Rendering. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 974--983.
20. Eric Heitz, Jonathan Dupuy, Cyril Crassin, and Carsten Dachsbacher. 2015. The SGGX Microflake Distribution. ACM Trans. Graph. (Proc. SIGGRAPH) 34, 4, Article 48 (July 2015), 11 pages.
21. Eric Heitz and Fabrice Neyret. 2012. Representing Appearance and Pre-Filtering Sub-pixel Data in Sparse Voxel Octrees. In Proceedings of the Fourth ACM SIGGRAPH / Eurographics Conference on High-Performance Graphics (Paris, France) (EGGH-HPG'12). Eurographics Association, Goslar, DEU, 125--134.
22. Homan Igehy. 1999. Tracing Ray Differentials. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 99). ACM Press/Addison-Wesley Publishing Co., USA, 179--186.
23. Wenzel Jakob. 2019. Enoki: structured vectorization and differentiation on modern processor architectures. https://github.com/mitsuba-renderer/enoki.
24. Wenzel Jakob, Jonathan T. Moon, Adam Arbree, Kavita Bala, and Steve Marschner. 2010. A Radiative Transfer Framework for Rendering Materials with Anisotropic Structure. ACM Trans. Graph. (Proc. SIGGRAPH) 29, 10 (July 2010), 53:1--53:13.
25. Adrian Jarabo, Carlos Aliaga, and Diego Gutierrez. 2018. A Radiative Transfer Framework for Spatially-Correlated Materials. ACM Trans. Graph. (Proc. SIGGRAPH) 37, 4, Article 83 (July 2018), 13 pages.
26. Anton Kaplanyan, Anton Sochenov, Thomas Leimkuehler, Mikhail Okunev, Todd Goodall, and Rufo Gizem. 2019. DeepFovea: Neural Reconstruction for Foveated Rendering and Video Compression using Learned Statistics of Natural Videos. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 4 (2019), 212:1--212:13.
27. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, Conference Track Proceedings.
28. Tzu-Mao Li, Miika Aittala, Frédo Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo Ray Tracing through Edge Sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 222:1--222:11.
29. Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. 2019. Neural Volumes: Learning Dynamic Renderable Volumes from Images. ACM Trans. Graph. (Proc. SIGGRAPH) 38, 4, Article 65 (July 2019), 14 pages.
30. Guillaume Loubet and Fabrice Neyret. 2017. Hybrid Mesh-Volume LoDs for All-Scale Pre-Filtering of Complex 3D Assets. Computer Graphics Forum 36, 2 (2017), 431--442.
31. David Luebke, Martin Reddy, Jonathan D. Cohen, Amitabh Varshney, Benjamin Watson, and Robert Huebner. 2002. Level of Detail for 3D Graphics. Morgan Kaufmann Publishers Inc.
32. Johannes Meng, Marios Papas, Ralf Habel, Carsten Dachsbacher, Steve Marschner, Markus Gross, and Wojciech Jarosz. 2015. Multi-scale Modeling and Rendering of Granular Materials. ACM Trans. Graph. (Proc. SIGGRAPH) 34, 4, Article 49 (July 2015), 13 pages.
33. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV. 25 pages.
34. Patrick Min. 2004 - 2021. binvox. http://www.patrickmin.com/binvox. Accessed: 2021-04-20.
35. Jonathan T. Moon, Bruce Walter, and Stephen R. Marschner. 2007. Rendering Discrete Random Media Using Precomputed Scattering Solutions. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (Grenoble, France) (Computer Graphics Forum (Proceedings of EGSR)). Eurographics Association, Goslar, DEU, 231--242.
36. Nate Morrical, Will Usher, Ingo Wald, and Valerio Pascucci. 2019. Efficient Space Skipping and Adaptive Sampling of Unstructured Volumes Using Hardware Accelerated Ray Tracing. In 2019 IEEE Visualization Conference (VIS). 256--260.
37. Thomas Müller, Marios Papas, Markus Gross, Wojciech Jarosz, and Jan Novák. 2016. Efficient Rendering of Heterogeneous Polydisperse Granular Media. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 35, 6 (Dec. 2016), 168:1--168:14.
38. Merlin Nimier-David, Sébastien Speierer, Benoît Ruiz, and Wenzel Jakob. 2020. Radiative Backpropagation: An Adjoint Method for Lightning-Fast Differentiable Rendering. ACM Trans. Graph. (Proc. SIGGRAPH) 39, 4, Article 146 (July 2020), 15 pages.
39. Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: A Retargetable Forward and Inverse Renderer. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 38, 6 (Nov. 2019), 17 pages.
40. Fakir S. Nooruddin and Greg Turk. 2003. Simplification and Repair of Polygonal Models Using Volumetric Techniques. IEEE Transactions on Visualization and Computer Graphics 9, 2 (2003), 191--205.
41. Jan Novák, Iliyan Georgiev, Johannes Hanika, and Wojciech Jarosz. 2018. Monte Carlo Methods for Volumetric Light Transport Simulation. Computer Graphics Forum (Proceedings of Eurographics - State of the Art Reports) 37, 2 (May 2018), 26 pages.
42. Steven G. Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, and Martin Stich. 2010. OptiX: A General Purpose Ray Tracing Engine. ACM Trans. Graph. (Proc. SIGGRAPH), Article 66 (2010), 13 pages.
43. Alexander Schwank, Callum James James, and Tony Micilotta. 2016. The Trees of The Jungle Book. In ACM SIGGRAPH Talks (Anaheim, California). Association for Computing Machinery, New York, NY, USA, Article 21, 2 pages.
44. Jos Stam. 2020. Computing Light Transport Gradients using the Adjoint Method. CoRR (2020), 23 pages. arXiv:2006.15059
45. Ingo Wald, Sven Woop, Carsten Benthin, Gregory S. Johnson, and Manfred Ernst. 2014. Embree: a kernel framework for efficient CPU ray tracing. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--8.
46. Shuang Zhao, Wenzel Jakob, Steve Marschner, and Kavita Bala. 2012. Structure-aware synthesis for predictive woven fabric appearance. ACM Trans. Graph. (Proc. SIGGRAPH) 31, 4 (2012), 75.
47. Shuang Zhao, Lifan Wu, Frédo Durand, and Ravi Ramamoorthi. 2016. Downsampling Scattering Parameters for Rendering Anisotropic Media. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 35, 6, Article 166 (Nov. 2016), 11 pages.
48. Quan Zheng, Vahid Babaei, Gordon Wetzstein, Hans-Peter Seidel, Matthias Zwicker, and Gurprit Singh. 2020. Neural Light Field 3D Printing. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 39, 6, Article 207 (2020), 12 pages.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

      • Antialiasing

    • Rendering

      • Ray tracing

      • Visibility

    • Shape modeling

      • Volumetric models