George Ash
ABSTRACT
We present a method to infer physically-based material properties for free-viewpoint video. Given a multi-camera image feed and reconstructed geometry, our method infers material properties, such as albedo, surface normal, metallic and roughness maps. We use a physically based, differentiable renderer to generate candidate images which are compared against the image feed. Our method searches for material textures which minimise an image-space loss metric between candidate renders and the ground truth image feed. Our method produces results that approximate state of the art reflectance capture, and produces texture maps that are compatible with common real-time and offline shading models.
Supplemental Material
Available for Download
- Alvaro Collet, Ming Chuang, Pat Sweeney, Don Gillett, Dennis Evseev, David Calabrese, Hugues Hoppe, Adam Kirk, and Steve Sullivan. 2015. High-Quality Streamable Free-Viewpoint Video. ACM Trans. Graph. 34, 4, Article 69 (July 2015), 13 pages. https://doi.org/10.1145/2766945Google Scholar
Digital Library
- Kaiwen Guo, Peter Lincoln, Philip Davidson, Jay Busch, Xueming Yu, Matt Whalen, Geoff Harvey, Sergio Orts-Escolano, Rohit Pandey, Jason Dourgarian, Danhang Tang, Anastasia Tkach, Adarsh Kowdle, Emily Cooper, Mingsong Dou, Sean Fanello, Graham Fyffe, Christoph Rhemann, Jonathan Taylor, Paul Debevec, and Shahram Izadi. 2019. The Relightables: Volumetric Performance Capture of Humans with Realistic Relighting. ACM Trans. Graph. 38, 6, Article 217 (Nov. 2019), 19 pages. https://doi.org/10.1145/3355089.3356571Google ScholarDigital Library
- Brian Karis. 2013. Real Shading in Unreal Engine 4. In ACM SIGGRAPH 2013 Courses: Physically Based Shading in Theory and Practice (Anaheim, California) (SIGGRAPH ’13). Association for Computing Machinery, New York, NY, USA, Article 22, 8 pages. https://doi.org/10.1145/2504435.2504457Google Scholar
- Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).Google Scholar
- Samuli Laine, Janne Hellsten, Tero Karras, Yeongho Seol, Jaakko Lehtinen, and Timo Aila. 2020. Modular Primitives for High-Performance Differentiable Rendering. ACM Trans. Graph. 39, 6, Article 194 (Nov. 2020), 14 pages. https://doi.org/10.1145/3414685.3417861Google ScholarDigital Library
- Erik Reinhard, Michael Stark, Peter Shirley, and James Ferwerda. 2002. Photographic Tone Reproduction for Digital Images. ACM Trans. Graph. 21, 3 (July 2002), 267–276. https://doi.org/10.1145/566654.566575Google ScholarDigital Library
Index Terms
Reflectance Estimation for Free-viewpoint Video
Recommendations
Free-viewpoint video rendering for mobile devices
MIRAGE '13: Proceedings of the 6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and ApplicationsFree-viewpoint video renderers (FVVR) allow a user to view captured video footage from any position and direction. Despite the obvious appeal of such systems, they have yet to make a major impact on digital entertainment. Current FVVR implementations ...
Free-viewpoint video of human actors
SIGGRAPH '03: ACM SIGGRAPH 2003 PapersIn free-viewpoint video, the viewer can interactively choose his viewpoint in 3-D space to observe the action of a dynamic real-world scene from arbitrary perspectives. The human body and its motion plays a central role in most visual media and its ...
Printing reflectance functions
The reflectance function of a scene point captures the appearance of that point as a function of lighting direction. We present an approach to printing the reflectance functions of an object or scene so that its appearance is modified correctly as a ...
Comments