Abstract
Rendering of realistic-looking hair is in general still too costly to do in real-time applications, from simulating the physics to rendering the fine details required for it to look natural, including self-shadowing.
We show how an autoencoder network, that can be evaluated in real time, can be trained to filter an image of few stochastic samples, including self-shadowing, to produce a much more detailed image that takes into account real hair thickness and transparency.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Real-Time Hair Filtering with Convolutional Neural Networks
- Hendrik Baatz, Jonathan Granskog, Marios Papas, Fabrice Rousselle, and Jan Novák. 2021. NeRF-Tex: Neural Reflectance Field Textures. In Eurographics Symposium on Rendering - DL-Only Track. The Eurographics Association, 13. https://doi.org/10.2312/sr.20211285Google Scholar
- Steve Bako, Thijs Vogels, Brian Mcwilliams, Mark Meyer, Jan NováK, Alex Harvill, Pradeep Sen, Tony Derose, and Fabrice Rousselle. 2017. Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings. ACM Trans. Graph. 36, 4 (July 2017), 1--14. https://doi.org/10.1145/3072959.3073708Google Scholar
Digital Library
- P. Bauszat, M. Eisemann, S. John, and M. Magnor. 2015. Sample-Based Manifold Filtering for Interactive Global Illumination and Depth of Field. Comput. Graph. Forum 34, 1 (feb 2015), 265--276. https://doi.org/10.1111/cgf.12511Google Scholar
Digital Library
- Louis Bavoil, Steven P. Callahan, Aaron Lefohn, João L. D. Comba, and Cláudio T. Silva. 2007. Multi-Fragment Effects on the GPU Using the k-Buffer. In Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games (I3D '07). Association for Computing Machinery, New York, NY, USA, 97--104. https://doi.org/10.1145/1230100.1230117Google Scholar
Digital Library
- Menglei Chai, Jian Ren, and Sergey Tulyakov. 2020. Neural Hair Rendering. In Computer Vision - ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Vol. 12363. Springer International Publishing, Cham, 371--388. https://doi.org/10.1007/978-3-030-58523-5_22Google Scholar
Digital Library
- Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder. ACM Transactions on Graphics 36, 4 (July 2017), 1--12. https://doi.org/10.1145/3072959.3073601Google Scholar
- Holger Dammertz, Daniel Sewtz, Johannes Hanika, and Hendrik P. A. Lensch. 2010. Edge-Avoiding À-Trous Wavelet Transform for Fast Global Illumination Filtering. In Proceedings of the Conference on High Performance Graphics (Saarbrucken, Germany) (HPG '10). Eurographics Association, Goslar, DEU, 67--75.Google Scholar
- Charles de Rousiers, Gaelle Morand, and Michael Forot. 2020. An Early Look at Next-Generation Real-Time Hair and Fur. https://www.unrealengine.com/en-US/tech-blog/an-early-look-at-next-generation-real-time-hair-and-fur.Google Scholar
- Eric Enderton, Erik Sintorn, Peter Shirley, and David Luebke. 2011. Stochastic Transparency. IEEE Transactions on Visualization and Computer Graphics 17 (2011), 1036--1047. https://doi.org/10.1109/TVCG.2010.123Google Scholar
Digital Library
- Eduardo S. L. Gastal and Manuel M. Oliveira. 2012. Adaptive Manifolds for Real-Time High-Dimensional Filtering. ACM Trans. Graph. 31, 4, Article 33 (jul 2012), 13 pages. https://doi.org/10.1145/2185520.2185529Google Scholar
Digital Library
- Johannes Hanika, Holger Dammertz, and Hendrik Lensch. 2011. Edge-optimized À-trous wavelets for local contrast enhancement with robust denoising. Comput. Graph. Forum 30, 7 (2011), 1879--1886. https://doi.org/10.1111/j.1467-8659.2011.02054.xGoogle Scholar
Cross Ref
- Erik Sven Vasconcelos Jansson, Matthäus G. Chajdas, Jason Lacroix, and Ingemar Ragnemalm. 2019. Real-Time Hybrid Hair Rendering. Eurographics Symposium on Rendering - DL-only and Industry Track (2019). https://doi.org/10.2312/SR.20191215Google Scholar
- J. T. Kajiya and T. L. Kay. 1989. Rendering Fur with Three Dimensional Textures (SIGGRAPH '89). Association for Computing Machinery, New York, NY, USA, 271--280. https://doi.org/10.1145/74333.74361Google Scholar
- Nima Khademi Kalantari, Steve Bako, and Pradeep Sen. 2015. A Machine Learning Approach for Filtering Monte Carlo Noise. ACM Trans. Graph. 34, 4, Article 122 (jul 2015), 12 pages. https://doi.org/10.1145/2766977Google Scholar
Digital Library
- Brian Karis. 2016. Physically Based Hair Shading in Unreal.Google Scholar
- Alexander Keller, Ken Dahm, and Nikolaus Binder. 2014. Path Space Filtering. In ACM SIGGRAPH 2014 Talks (Vancouver, Canada) (SIGGRAPH '14). Association for Computing Machinery, New York, NY, USA, Article 68, 1 pages. https://doi.org/10.1145/2614106.2614149Google Scholar
- Michael Kern, Christoph Neuhauser, Torben Maack, Mengjiao Han, Will Usher, and Rüdiger Westermann. 2021. A Comparison of Rendering Techniques for 3D Line Sets With Transparency. IEEE Transactions on Visualization and Computer Graphics 27, 8 (Aug. 2021), 3361--3376. https://doi.org/10.1109/TVCG.2020.2975795Google Scholar
Digital Library
- Tae-Yong Kim and Ulrich Neumann. 2001. Opacity Shadow Maps. In Rendering Techniques 2001. Springer Vienna, 177--182. https://doi.org/10.1007/978-3-7091-6242-2_16Google Scholar
Cross Ref
- Samuli Laine and Tero Karras. 2011. Stratified Sampling for Stochastic Transparency. Computer Graphics Forum 30, 4 (June 2011), 1197--1204. https://doi.org/10.1111/j.1467-8659.2011.01978.xGoogle Scholar
Digital Library
- Tom Lokovic and Eric Veach. 2000. Deep Shadow Maps. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH '00. ACM Press, Not Known, 385--392. https://doi.org/10.1145/344779.344958Google Scholar
Digital Library
- Stephen R. Marschner, Henrik Wann Jensen, Mike Cammarano, Steve Worley, and Pat Hanrahan. 2003. Light Scattering from Human Hair Fibers. ACM Trans. Graph. 22, 3 (July 2003), 780--791. https://doi.org/10.1145/882262.882345Google Scholar
Digital Library
- Marilena Maule, João L. D. Comba, Rafael P. Torchelsen, and Rui Bastos. 2011. A Survey of Raster-Based Transparency Techniques. Computers & Graphics 35, 6 (Dec. 2011), 1023--1034. https://doi.org/10.1016/j.cag.2011.07.006Google Scholar
Digital Library
- Morgan McGuire and Eric Enderton. 2011. Colored Stochastic Shadow Maps. In Symposium on Interactive 3D Graphics and Games (I3D '11). Association for Computing Machinery, New York, NY, USA, 89--96. https://doi.org/10.1145/1944745.1944760Google Scholar
- Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. arXiv:2003.08934 [cs] (Aug. 2020). arXiv:2003.08934 [cs]Google Scholar
- J.D. Mulder, F.C.A. Groen, and J.J. van Wijk. 1998. Pixel masks for screen-door transparency. In Proceedings Visualization '98 (Cat. No.98CB36276). IEEE, 351-358,. https://doi.org/10.1109/VISUAL.1998.745323Google Scholar
- Jacob Munkberg, Jon Hasselgren, Petrik Clarberg, Magnus Andersson, and Tomas Akenine-Möller. 2016. Texture space caching and reconstruction for ray tracing. ACM Trans. Graph. 35, 6 (2016), 1--13. https://doi.org/10.1145/2980179.2982407Google Scholar
Digital Library
- Cedrick Münstermann, Stefan Krumpen, Reinhard Klein, and Christoph Peters. 2018. Moment-Based Order-Independent Transparency. Proc. ACM Comput. Graph. Interact. Tech. 1, 1 (July 2018), 7:1-7:20. https://doi.org/10.1145/3203206Google Scholar
Digital Library
- Zhi Qiao and Takashi Kanai. 2021. A GAN-Based Temporally Stable Shading Model for Fast Animation of Photorealistic Hair. Comp. Visual Media 7, 1 (March 2021), 127--138. https://doi.org/10.1007/s41095-020-0201-9Google Scholar
- H. Qiu, C. Wang, H. Zhu, X. Zhu, J. Gu, and X. Han. 2019. Two-Phase Hair Image Synthesis by Self-Enhancing Generative Model. Computer Graphics Forum 38, 7 (2019), 403--412. https://doi.org/10.1111/cgf.13847Google Scholar
Cross Ref
- Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597 [cs.CV]Google Scholar
- Iman Sadeghi, Heather Pritchett, Henrik Wann Jensen, and Rasmus Tamstorf. 2010. An Artist Friendly Hair Shading System. ACM Trans. Graph. 29, 4 (July 2010), 56:1-56:10. https://doi.org/10.1145/1778765.1778793Google Scholar
Digital Library
- Marco Salvi. 2016. An Excursion In Temporal Supersampling. In Game Developers Conference.Google Scholar
- Marco Salvi, Jefferson Montgomery, and Aaron Lefohn. 2011. Adaptive Transparency. In Proceedings of the ACM SIGGRAPH Symposium on High Performance Graphics (HPG '11). Association for Computing Machinery, New York, NY, USA, 119--126. https://doi.org/10.1145/2018323.2018342Google Scholar
- Thorsten Scheuermann. 2004a. Hair Rendering and Shading. https://developer.amd.com/wordpress/media/2012/10/Scheuermann_HairRendering.pdfGoogle Scholar
- Thorsten Scheuermann. 2004b. Practical Real-Time Hair Rendering and Shading. In ACM SIGGRAPH 2004 Sketches (Los Angeles, California) (SIGGRAPH '04). Association for Computing Machinery, New York, NY, USA, 147. https://doi.org/10.1145/1186223.1186408Google Scholar
- Erik Sintorn and Ulf Assarsson. 2008. Real-Time Approximate Sorting for Self Shadowing and Transparency in Hair Rendering. In Proceedings of the 2008 Symposium on Interactive 3D Graphics and Games - SI3D '08. ACM Press, Redwood City, California, 157. https://doi.org/10.1145/1342250.1342275Google Scholar
Digital Library
- Erik Sintorn and Ulf Assarsson. 2009. Hair self shadowing and transparency depth ordering using occupancy maps. In Proceedings of the 2009 symposium on Interactive 3D graphics and games (Boston, Massachusetts) (I3D '09). ACM, New York, NY, USA, 67--74. https://doi.org/10.1145/1507149.1507160Google Scholar
Digital Library
- Sebastian Tafuri. 2019. Strand-Based Hair Rendering in Frostbite. https://doi.org/10.1145/3305366.3335035Google Scholar
- Lingyu Wei, Liwen Hu, Vladimir Kim, Ersin Yumer, and Hao Li. 2018. Real-Time Hair Rendering Using Sequential Adversarial Networks. In Computer Vision - ECCV 2018. Vol. 11208. Springer International Publishing, Cham, 105--122. https://doi.org/10.1007/978-3-030-01225-0_7Google Scholar
Cross Ref
- Lance Williams. 1978. Casting curved shadows on curved surfaces. In Proceedings of the 5th annual conference on Computer graphics and interactive techniques (SIGGRAPH '78). Association for Computing Machinery. https://doi.org/10.1145/800248.807402Google Scholar
Digital Library
- Chris Wyman and Morgan McGuire. 2017. Hashed Alpha Testing. In Proceedings of the 21st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, San Francisco California, 1--9. https://doi.org/10.1145/3023368.3023370Google Scholar
Digital Library
- Lei Yang, Shiqiu Liu, and Marco Salvi. 2020. A Survey of Temporal Antialiasing Techniques. Comput. Graph. Forum 39, 2 (2020), 607--621. https://doi.org/10.1111/cgf.14018Google Scholar
Cross Ref
- Cem Yuksel. [n.d.]. HAIR Model Files - Cem Yuksel. http://www.cemyuksel.com/research/hairmodels/Google Scholar
- Cem Yuksel and John Keyser. 2008. Deep Opacity Maps. Computer Graphics Forum 27, 2 (2008), 675--680. https://doi.org/10.1111/j.1467-8659.2008.01165.xGoogle Scholar
Cross Ref
- Arno Zinke, Cem Yuksel, Andreas Weber, and John Keyser. 2008. Dual Scattering Approximation for Fast Multiple Scattering in Hair. In ACM SIGGRAPH 2008 Papers. ACM Press, Los Angeles, California, 1. https://doi.org/10.1145/1399504.1360631Google Scholar
- Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and Sung-Eui Yoon. 2015. Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum (Proceedings of Eurographics - State of the Art Reports) 34, 2 (May 2015), 667--681. https://doi.org/10/f7k6kjGoogle Scholar
Cross Ref
Index Terms
Real-Time Hair Filtering with Convolutional Neural Networks
Recommendations
Simulating multiple scattering in hair using a photon mapping approach
Simulating multiple scattering correctly is important for accurate rendering of hair. However, a volume of hair is a difficult scene to simulate because scattering from an individual fiber is very structured and forward directed, and because the ...
Simulating multiple scattering in hair using a photon mapping approach
SIGGRAPH '06: ACM SIGGRAPH 2006 PapersSimulating multiple scattering correctly is important for accurate rendering of hair. However, a volume of hair is a difficult scene to simulate because scattering from an individual fiber is very structured and forward directed, and because the ...
Precomputed illuminance composition for real-time global illumination
I3D '16: Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and GamesIn this paper we present a new real-time approach for indirect global illumination under dynamic lighting conditions. We use surfels to gather a sampling of the local illumination and propagate the light through the scene using a hierarchy and a set of ...






Comments