Thomas Müller
ABSTRACT
We extend our instant NeRF implementation [Müller et al. 2022] to allow training from an incremental stream of images and camera poses, provided by a realtime Simultaneous Localization And Mapping (SLAM) system. Camera poses are refined end-to-end by back-propagating the gradients from NeRF training. Reconstruction quality is further improved by compensating for various camera properties, such as rolling shutter, non-linear lens distortion, and variable exposure typical of digital cameras.
Static scenes can be scanned, the NeRF model trained, and the reconstruction verified in an interactive fashion, in under a minute.
- Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, and Andrew Fitzgibbon. 2011. KinectFusion: Real-Time 3D Reconstruction and Interaction Using a Moving Depth Camera. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). Association for Computing Machinery, New York, NY, USA, 559–568. https://doi.org/10.1145/2047196.2047270Google Scholar
Digital Library
- Maik Keller, Damien Lefloch, Martin Lambers, Shahram Izadi, Tim Weyrich, and Andreas Kolb. 2013. Real-time 3d reconstruction in dynamic scenes using point-based fusion. In 2013 International Conference on 3D Vision-3DV 2013. IEEE, 1–8.Google ScholarDigital Library
- Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 (June 2014).Google Scholar
- Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey. 2021. BARF: Bundle-Adjusting Neural Radiance Fields. CoRR abs/2104.06405(2021). arXiv:2104.06405https://arxiv.org/abs/2104.06405Google Scholar
- Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.Google Scholar
- Thomas Müller, Alex Evans, Christoph Schied, and Alexander Keller. 2022. Instant Neural Graphics Primitives with a Multiresolution Hash Encoding. ACM Trans. Graph. 41, 4, Article 102 (July 2022), 15 pages. https://doi.org/10.1145/3528223.3530127Google ScholarDigital Library
- Thomas Müller, Fabrice Rousselle, Jan Novák, and Alexander Keller. 2021. Real-time Neural Radiance Caching for Path Tracing. ACM Trans. Graph. 40, 4, Article 36 (Aug. 2021), 16 pages. https://doi.org/10.1145/3450626.3459812Google ScholarDigital Library
Index Terms
Instant Neural Radiance Fields
Recommendations
VMRF: View Matching Neural Radiance Fields
MM '22: Proceedings of the 30th ACM International Conference on MultimediaNeural Radiance Fields (NeRF) has demonstrated very impressive performance in novel view synthesis via implicitly modelling 3D representations from multi-view 2D images. However, most existing studies train NeRF models with either reasonable camera pose ...
StyleTRF: Stylizing Tensorial Radiance Fields✱
ICVGIP '22: Proceedings of the Thirteenth Indian Conference on Computer Vision, Graphics and Image ProcessingStylized view generation of scenes captured casually using a camera has received much attention recently. The geometry and appearance of the scene are typically captured as neural point sets or neural radiance fields in the previous work. An image ...
PI-NeRF: A Partial-Invertible Neural Radiance Fields for Pose Estimation
MM '23: Proceedings of the 31st ACM International Conference on MultimediaIn recent years, Neural Radiance Fields (NeRF) have been used as a map of 3D scene to estimate the 6-DoF pose of new observed images - given an image, estimate the relative rotation and translation of a camera using a trained NeRF. However, existing ...
Comments