Ulrike Bath
Amir Semmo
ABSTRACT
3D photography has emerged as a medium that provides an immersive dimension to 2D photos. We present Trios, an interactive mobile app that combines the vividness of image-based artistic rendering with 3D photos by implementing an end-to-end pipeline for their generation and stylization. Trios uses Apple’s accelerated image-processing APIs and dedicated Neural Engine for depth-generation and learning-based artistic rendering. The pipeline runs at interactive frame rates and outputs a compact video, which can easily be shared. Thus, it serves as a unique interactive tool for digital artists interested in creating immersive artistic content.
Supplemental Material
- Peter Hedman, Suhib Alsisan, Richard Szeliski, and Johannes Kopf. 2017. Casual 3D Photography. ACM Trans. Graph. 36, 6, Article 234 (2017), 15 pages. https://doi.org/10.1145/3130800.3130828Google Scholar
Digital Library
- Peter Hedman and Johannes Kopf. 2018. Instant 3D Photography. ACM Trans. Graph. 37, 4, Article 101 (jul 2018), 12 pages. https://doi.org/10.1145/3197517.3201384Google ScholarDigital Library
- Johannes Kopf, Kevin Matzen, Suhib Alsisan, Ocean Quigley, Francis Ge, Yangming Chong, Josh Patterson, Jan-Michael Frahm, Shu Wu, Matthew Yu, Peizhao Zhang, Zijian He, Peter Vajda, Ayush Saraf, and Michael Cohen. 2020. One Shot 3D Photography. ACM Trans. Graph. 39, 4, Article 76 (jul 2020), 13 pages. https://doi.org/10.1145/3386569.3392420Google ScholarDigital Library
- Sebastian Pasewaldt, Amir Semmo, Jürgen Döllner, and Frank Schlegel. 2016. Becasso: Artistic Image Processing and Editing on Mobile Devices. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications(SA ’16). Article 14, 1 pages. https://doi.org/10.1145/2999508.2999518Google ScholarDigital Library
- René Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. 2022. Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-Shot Cross-Dataset Transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 3(2022), 1623–1637. https://doi.org/10.1109/TPAMI.2020.3019967Google ScholarCross Ref
- Amir Semmo and Sebastian Pasewaldt. 2020. Graphite: Interactive Photo-to-Drawing Stylization on Mobile Devices. In Proceedings SIGGRAPH Appy Hour (Virtual Event, USA). ACM, New York. https://doi.org/10.1145/3388529.3407306Google ScholarDigital Library
- Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. 1998. Layered Depth Images. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH ’98). Association for Computing Machinery, New York, NY, USA, 231–242. https://doi.org/10.1145/280814.280882Google ScholarDigital Library
- Meng-Li Shih, Shih-Yang Su, Johannes Kopf, and Jia-Bin Huang. 2020. 3D Photography Using Context-Aware Layered Depth Inpainting. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8025–8035. https://doi.org/10.1109/CVPR42600.2020.00805Google ScholarCross Ref
Recommendations
Free-viewpoint video rendering for mobile devices
MIRAGE '13: Proceedings of the 6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and ApplicationsFree-viewpoint video renderers (FVVR) allow a user to view captured video footage from any position and direction. Despite the obvious appeal of such systems, they have yet to make a major impact on digital entertainment. Current FVVR implementations ...
Real-time 3D rendering with hatching
We present an approach for real-time pen-and-ink hatching renderings on large scenes. Starting with 3D models including photorealistic textures and materials, we aim to propose a solution that produces hatched renderings. As we consider scene objects ...
Comments