skip to main content
research-article
Open Access

Real-Time Hair Filtering with Convolutional Neural Networks

Authors Info & Claims
Published:04 May 2022Publication History
Skip Abstract Section

Abstract

Rendering of realistic-looking hair is in general still too costly to do in real-time applications, from simulating the physics to rendering the fine details required for it to look natural, including self-shadowing.

We show how an autoencoder network, that can be evaluated in real time, can be trained to filter an image of few stochastic samples, including self-shadowing, to produce a much more detailed image that takes into account real hair thickness and transparency.

Skip Supplemental Material Section

Supplemental Material

References

  1. Hendrik Baatz, Jonathan Granskog, Marios Papas, Fabrice Rousselle, and Jan Novák. 2021. NeRF-Tex: Neural Reflectance Field Textures. In Eurographics Symposium on Rendering - DL-Only Track. The Eurographics Association, 13. https://doi.org/10.2312/sr.20211285Google ScholarGoogle Scholar
  2. Steve Bako, Thijs Vogels, Brian Mcwilliams, Mark Meyer, Jan NováK, Alex Harvill, Pradeep Sen, Tony Derose, and Fabrice Rousselle. 2017. Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings. ACM Trans. Graph. 36, 4 (July 2017), 1--14. https://doi.org/10.1145/3072959.3073708Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. P. Bauszat, M. Eisemann, S. John, and M. Magnor. 2015. Sample-Based Manifold Filtering for Interactive Global Illumination and Depth of Field. Comput. Graph. Forum 34, 1 (feb 2015), 265--276. https://doi.org/10.1111/cgf.12511Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Louis Bavoil, Steven P. Callahan, Aaron Lefohn, João L. D. Comba, and Cláudio T. Silva. 2007. Multi-Fragment Effects on the GPU Using the k-Buffer. In Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games (I3D '07). Association for Computing Machinery, New York, NY, USA, 97--104. https://doi.org/10.1145/1230100.1230117Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Menglei Chai, Jian Ren, and Sergey Tulyakov. 2020. Neural Hair Rendering. In Computer Vision - ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Vol. 12363. Springer International Publishing, Cham, 371--388. https://doi.org/10.1007/978-3-030-58523-5_22Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder. ACM Transactions on Graphics 36, 4 (July 2017), 1--12. https://doi.org/10.1145/3072959.3073601Google ScholarGoogle Scholar
  7. Holger Dammertz, Daniel Sewtz, Johannes Hanika, and Hendrik P. A. Lensch. 2010. Edge-Avoiding À-Trous Wavelet Transform for Fast Global Illumination Filtering. In Proceedings of the Conference on High Performance Graphics (Saarbrucken, Germany) (HPG '10). Eurographics Association, Goslar, DEU, 67--75.Google ScholarGoogle Scholar
  8. Charles de Rousiers, Gaelle Morand, and Michael Forot. 2020. An Early Look at Next-Generation Real-Time Hair and Fur. https://www.unrealengine.com/en-US/tech-blog/an-early-look-at-next-generation-real-time-hair-and-fur.Google ScholarGoogle Scholar
  9. Eric Enderton, Erik Sintorn, Peter Shirley, and David Luebke. 2011. Stochastic Transparency. IEEE Transactions on Visualization and Computer Graphics 17 (2011), 1036--1047. https://doi.org/10.1109/TVCG.2010.123Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Eduardo S. L. Gastal and Manuel M. Oliveira. 2012. Adaptive Manifolds for Real-Time High-Dimensional Filtering. ACM Trans. Graph. 31, 4, Article 33 (jul 2012), 13 pages. https://doi.org/10.1145/2185520.2185529Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Johannes Hanika, Holger Dammertz, and Hendrik Lensch. 2011. Edge-optimized À-trous wavelets for local contrast enhancement with robust denoising. Comput. Graph. Forum 30, 7 (2011), 1879--1886. https://doi.org/10.1111/j.1467-8659.2011.02054.xGoogle ScholarGoogle ScholarCross RefCross Ref
  12. Erik Sven Vasconcelos Jansson, Matthäus G. Chajdas, Jason Lacroix, and Ingemar Ragnemalm. 2019. Real-Time Hybrid Hair Rendering. Eurographics Symposium on Rendering - DL-only and Industry Track (2019). https://doi.org/10.2312/SR.20191215Google ScholarGoogle Scholar
  13. J. T. Kajiya and T. L. Kay. 1989. Rendering Fur with Three Dimensional Textures (SIGGRAPH '89). Association for Computing Machinery, New York, NY, USA, 271--280. https://doi.org/10.1145/74333.74361Google ScholarGoogle Scholar
  14. Nima Khademi Kalantari, Steve Bako, and Pradeep Sen. 2015. A Machine Learning Approach for Filtering Monte Carlo Noise. ACM Trans. Graph. 34, 4, Article 122 (jul 2015), 12 pages. https://doi.org/10.1145/2766977Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Brian Karis. 2016. Physically Based Hair Shading in Unreal.Google ScholarGoogle Scholar
  16. Alexander Keller, Ken Dahm, and Nikolaus Binder. 2014. Path Space Filtering. In ACM SIGGRAPH 2014 Talks (Vancouver, Canada) (SIGGRAPH '14). Association for Computing Machinery, New York, NY, USA, Article 68, 1 pages. https://doi.org/10.1145/2614106.2614149Google ScholarGoogle Scholar
  17. Michael Kern, Christoph Neuhauser, Torben Maack, Mengjiao Han, Will Usher, and Rüdiger Westermann. 2021. A Comparison of Rendering Techniques for 3D Line Sets With Transparency. IEEE Transactions on Visualization and Computer Graphics 27, 8 (Aug. 2021), 3361--3376. https://doi.org/10.1109/TVCG.2020.2975795Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Tae-Yong Kim and Ulrich Neumann. 2001. Opacity Shadow Maps. In Rendering Techniques 2001. Springer Vienna, 177--182. https://doi.org/10.1007/978-3-7091-6242-2_16Google ScholarGoogle ScholarCross RefCross Ref
  19. Samuli Laine and Tero Karras. 2011. Stratified Sampling for Stochastic Transparency. Computer Graphics Forum 30, 4 (June 2011), 1197--1204. https://doi.org/10.1111/j.1467-8659.2011.01978.xGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  20. Tom Lokovic and Eric Veach. 2000. Deep Shadow Maps. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques - SIGGRAPH '00. ACM Press, Not Known, 385--392. https://doi.org/10.1145/344779.344958Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Stephen R. Marschner, Henrik Wann Jensen, Mike Cammarano, Steve Worley, and Pat Hanrahan. 2003. Light Scattering from Human Hair Fibers. ACM Trans. Graph. 22, 3 (July 2003), 780--791. https://doi.org/10.1145/882262.882345Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Marilena Maule, João L. D. Comba, Rafael P. Torchelsen, and Rui Bastos. 2011. A Survey of Raster-Based Transparency Techniques. Computers & Graphics 35, 6 (Dec. 2011), 1023--1034. https://doi.org/10.1016/j.cag.2011.07.006Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Morgan McGuire and Eric Enderton. 2011. Colored Stochastic Shadow Maps. In Symposium on Interactive 3D Graphics and Games (I3D '11). Association for Computing Machinery, New York, NY, USA, 89--96. https://doi.org/10.1145/1944745.1944760Google ScholarGoogle Scholar
  24. Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. arXiv:2003.08934 [cs] (Aug. 2020). arXiv:2003.08934 [cs]Google ScholarGoogle Scholar
  25. J.D. Mulder, F.C.A. Groen, and J.J. van Wijk. 1998. Pixel masks for screen-door transparency. In Proceedings Visualization '98 (Cat. No.98CB36276). IEEE, 351-358,. https://doi.org/10.1109/VISUAL.1998.745323Google ScholarGoogle Scholar
  26. Jacob Munkberg, Jon Hasselgren, Petrik Clarberg, Magnus Andersson, and Tomas Akenine-Möller. 2016. Texture space caching and reconstruction for ray tracing. ACM Trans. Graph. 35, 6 (2016), 1--13. https://doi.org/10.1145/2980179.2982407Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Cedrick Münstermann, Stefan Krumpen, Reinhard Klein, and Christoph Peters. 2018. Moment-Based Order-Independent Transparency. Proc. ACM Comput. Graph. Interact. Tech. 1, 1 (July 2018), 7:1-7:20. https://doi.org/10.1145/3203206Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Zhi Qiao and Takashi Kanai. 2021. A GAN-Based Temporally Stable Shading Model for Fast Animation of Photorealistic Hair. Comp. Visual Media 7, 1 (March 2021), 127--138. https://doi.org/10.1007/s41095-020-0201-9Google ScholarGoogle Scholar
  29. H. Qiu, C. Wang, H. Zhu, X. Zhu, J. Gu, and X. Han. 2019. Two-Phase Hair Image Synthesis by Self-Enhancing Generative Model. Computer Graphics Forum 38, 7 (2019), 403--412. https://doi.org/10.1111/cgf.13847Google ScholarGoogle ScholarCross RefCross Ref
  30. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597 [cs.CV]Google ScholarGoogle Scholar
  31. Iman Sadeghi, Heather Pritchett, Henrik Wann Jensen, and Rasmus Tamstorf. 2010. An Artist Friendly Hair Shading System. ACM Trans. Graph. 29, 4 (July 2010), 56:1-56:10. https://doi.org/10.1145/1778765.1778793Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Marco Salvi. 2016. An Excursion In Temporal Supersampling. In Game Developers Conference.Google ScholarGoogle Scholar
  33. Marco Salvi, Jefferson Montgomery, and Aaron Lefohn. 2011. Adaptive Transparency. In Proceedings of the ACM SIGGRAPH Symposium on High Performance Graphics (HPG '11). Association for Computing Machinery, New York, NY, USA, 119--126. https://doi.org/10.1145/2018323.2018342Google ScholarGoogle Scholar
  34. Thorsten Scheuermann. 2004a. Hair Rendering and Shading. https://developer.amd.com/wordpress/media/2012/10/Scheuermann_HairRendering.pdfGoogle ScholarGoogle Scholar
  35. Thorsten Scheuermann. 2004b. Practical Real-Time Hair Rendering and Shading. In ACM SIGGRAPH 2004 Sketches (Los Angeles, California) (SIGGRAPH '04). Association for Computing Machinery, New York, NY, USA, 147. https://doi.org/10.1145/1186223.1186408Google ScholarGoogle Scholar
  36. Erik Sintorn and Ulf Assarsson. 2008. Real-Time Approximate Sorting for Self Shadowing and Transparency in Hair Rendering. In Proceedings of the 2008 Symposium on Interactive 3D Graphics and Games - SI3D '08. ACM Press, Redwood City, California, 157. https://doi.org/10.1145/1342250.1342275Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Erik Sintorn and Ulf Assarsson. 2009. Hair self shadowing and transparency depth ordering using occupancy maps. In Proceedings of the 2009 symposium on Interactive 3D graphics and games (Boston, Massachusetts) (I3D '09). ACM, New York, NY, USA, 67--74. https://doi.org/10.1145/1507149.1507160Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Sebastian Tafuri. 2019. Strand-Based Hair Rendering in Frostbite. https://doi.org/10.1145/3305366.3335035Google ScholarGoogle Scholar
  39. Lingyu Wei, Liwen Hu, Vladimir Kim, Ersin Yumer, and Hao Li. 2018. Real-Time Hair Rendering Using Sequential Adversarial Networks. In Computer Vision - ECCV 2018. Vol. 11208. Springer International Publishing, Cham, 105--122. https://doi.org/10.1007/978-3-030-01225-0_7Google ScholarGoogle ScholarCross RefCross Ref
  40. Lance Williams. 1978. Casting curved shadows on curved surfaces. In Proceedings of the 5th annual conference on Computer graphics and interactive techniques (SIGGRAPH '78). Association for Computing Machinery. https://doi.org/10.1145/800248.807402Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Chris Wyman and Morgan McGuire. 2017. Hashed Alpha Testing. In Proceedings of the 21st ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, San Francisco California, 1--9. https://doi.org/10.1145/3023368.3023370Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Lei Yang, Shiqiu Liu, and Marco Salvi. 2020. A Survey of Temporal Antialiasing Techniques. Comput. Graph. Forum 39, 2 (2020), 607--621. https://doi.org/10.1111/cgf.14018Google ScholarGoogle ScholarCross RefCross Ref
  43. Cem Yuksel. [n.d.]. HAIR Model Files - Cem Yuksel. http://www.cemyuksel.com/research/hairmodels/Google ScholarGoogle Scholar
  44. Cem Yuksel and John Keyser. 2008. Deep Opacity Maps. Computer Graphics Forum 27, 2 (2008), 675--680. https://doi.org/10.1111/j.1467-8659.2008.01165.xGoogle ScholarGoogle ScholarCross RefCross Ref
  45. Arno Zinke, Cem Yuksel, Andreas Weber, and John Keyser. 2008. Dual Scattering Approximation for Fast Multiple Scattering in Hair. In ACM SIGGRAPH 2008 Papers. ACM Press, Los Angeles, California, 1. https://doi.org/10.1145/1399504.1360631Google ScholarGoogle Scholar
  46. Matthias Zwicker, Wojciech Jarosz, Jaakko Lehtinen, Bochang Moon, Ravi Ramamoorthi, Fabrice Rousselle, Pradeep Sen, Cyril Soler, and Sung-Eui Yoon. 2015. Recent Advances in Adaptive Sampling and Reconstruction for Monte Carlo Rendering. Computer Graphics Forum (Proceedings of Eurographics - State of the Art Reports) 34, 2 (May 2015), 667--681. https://doi.org/10/f7k6kjGoogle ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Real-Time Hair Filtering with Convolutional Neural Networks

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image Proceedings of the ACM on Computer Graphics and Interactive Techniques
            Proceedings of the ACM on Computer Graphics and Interactive Techniques  Volume 5, Issue 1
            May 2022
            252 pages
            EISSN:2577-6193
            DOI:10.1145/3535313
            Issue’s Table of Contents

            Copyright © 2022 Owner/Author

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 4 May 2022
            Published in pacmcgit Volume 5, Issue 1

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed
          • Article Metrics

            • Downloads (Last 12 months)184
            • Downloads (Last 6 weeks)18

            Other Metrics

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!