skip to main content
research-article

View-Region Optimized Image-Based Scene Simplification

Published:24 August 2018Publication History
Skip Abstract Section

Abstract

We present a new algorithm for image-based simplification of complex scenes for virtual reality (VR). The algorithm transforms geometrically-detailed environments into a layered quad tile representation that is optimized for a specified viewing region and is renderable on low-power mobile-class VR devices. A novel constrained optimization formulation ensures that the scene can be rendered within a predetermined compute budget, with limits on both primitive count and fill rate. Furthermore, we introduce a new method for texturing from point samples of the original scene geometry that generates high-quality silhouettes without the drawbacks of traditional point splatting.

The resulting representation achieves a visual fidelity that was previously impossible on mobile graphics hardware; our algorithm can typically generate a high-quality representation of visually-rich scenes with billions of triangles using just 72k triangles and a single high-resolution texture map (with generally only about 50% more texels than a stereo panorama). The effectiveness of the approach is demonstrated with a set of challenging test cases.

Skip Supplemental Material Section

Supplemental Material

References

  1. Sameer Agarwal, Keir Mierle, and Others. 2018. Ceres Solver. (2018). http://ceres-solver.orgGoogle ScholarGoogle Scholar
  2. Alcon Interactive. 2018. Blade Runner: Revelations. (2018). https://play.google.com/store/apps/details?id=com.alconinteractive.brvrGoogle ScholarGoogle Scholar
  3. Robert Anderson, David Gallup, Jonathan T. Barron, Janne Kontkanen, Noah Snavely, Carlos Hernández, Sameer Agarwal, and Steven M. Seitz. 2016. Jump: Virtual Reality Video. ACM Trans. Graph. 35, 6, Article 198, 13 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. C. Andujar, P. Brunet, A. Chica, I. Navazo, J. Rossignac, and A. Vinacua. 2004. Computing Maximal Tiles and Application to Impostor-Based Simplification. Computer Graphics Forum 23, 3 (2004), 401--410.Google ScholarGoogle ScholarCross RefCross Ref
  5. S. Behrendt, C. Colditz, O. Franzke, J. Kopf, and O. Deussen. 2005. Realistic real-time rendering of landscapes using billboard clouds. Computer Graphics Forum (2005).Google ScholarGoogle Scholar
  6. Marius Bjørge, Sam Martin, Sandeep Kakarlapudi, and Jan-Harald Fredriksen. 2014. Efficient Rendering with Tile Local Storage. In ACM SIGGRAPH 2014 Talks (SIGGRAPH '14). Article 51, 1 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Kevin Boos, David Chu, and Eduardo Cuervo. 2016. FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys '16). 291--304. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. David Cohen-Steiner, Pierre Alliez, and Mathieu Desbrun. 2004. Variational Shape Approximation. ACM Trans. Graph. 23, 3 (Aug. 2004), 905--914. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Cyril Crassin, Fabrice Neyret, Sylvain Lefebvre, and Elmar Eisemann. 2009. GigaVoxels: Ray-guided Streaming for Efficient and Detailed Voxel Rendering. In Proceedings of the 2009 Symposium on Interactive 3D Graphics and Games (I3D '09). 15--22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Philippe Decaudin and Fabrice Neyret. 2009. Volumetric Billboards. Computer Graphics Forum 28, 8 (2009), 2079--2089.Google ScholarGoogle Scholar
  11. Xavier Décoret, Frédo Durand, François X. Sillion, and Julie Dorsey. 2003. Billboard Clouds for Extreme Model Simplification. ACM Trans. Graph. 22, 3 (July 2003), 689--696. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Hugh Everett. 1963. Generalized Lagrange Multiplier Method for Solving Problems of Optimum Allocation of Resources. Operations Research 11, 3 (1963), 399--417. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Marshall L. Fisher. 2004. The Lagrangian Relaxation Method for Solving Integer Programming Problems. Management Science 50, 12 (2004), 1861--1871. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Anton L. Fuhrmann, Eike Umlauf, and Stephan Mantler. 2005. Extreme Model Simplification for Forest Rendering. In Proceedings of the First Eurographics Conference on Natural Phenomena (NPH'05). Eurographics Association, 57--67. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Michael Garland and Paul S. Heckbert. 1997. Surface Simplification Using Quadric Error Metrics. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '97). 209--216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Enrico Gobbetti and Fabio Marton. 2005. Far Voxels: A Multiresolution Framework for Interactive Rendering of Huge Complex 3D Models on Commodity Graphics Platforms. In ACM SIGGRAPH 2005 Papers (SIGGRAPH '05). 878--885. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Google Inc. 2018. Expeditions. (2018). https://play.google.com/store/apps/details?id=com.google.vr.expeditionsGoogle ScholarGoogle Scholar
  18. Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael F. Cohen. 1996. The Lumigraph. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). 43--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. J. Huang, Z. Chen, D. Ceylan, and H. Jin. 2017. 6-DOF VR videos with a single 360-camera. In 2017 IEEE Virtual Reality (VR). 37--44.Google ScholarGoogle Scholar
  20. H. Ishiguro, M. Yamamoto, and S. Tsuji. 1990. Omni-directional stereo for making global map. In Proceedings Third International Conference on Computer Vision. 540--547.Google ScholarGoogle Scholar
  21. Stefan Jeschke and Michael Wimmer. 2002. Textured Depth Meshes for Real-time Rendering of Arbitrary Scenes. In Eurographics Workshop on Rendering, P. Debevec and S. Gibson (Eds.). The Eurographics Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Alexander Kharlamov, Iain Cantlay, and Yury Stepanenko. 2007. Next-Generation SpeedTree Rendering. In GPU Gems 3 (first ed.), Hubert Nguyen (Ed.). Addison-Wesley Professional.Google ScholarGoogle Scholar
  23. Babis Koniaris, Ivan Huerta, Maggie Kosek Karen Darragh, Charles Malleson, Joanna Jamrozy, Nick Swafford, Jose Guitian, Bochang Moon, Ali Israr, and Kenny Mitchell. 2016. IRIDiuM: Immersive Rendered Interactive Deep Media. In ACM SIGGRAPH 2016 VR Village (SIGGRAPH '16). Article 11, 2 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Babis Koniaris, Maggie Kosek, David Sinclair, and Kenny Mitchell. 2017. Real-time Rendering with Compressed Animated Light Fields. In Proceedings of the 43rd Graphics Interface Conference (GI '17). 33--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Martin Kraus and Thomas Ertl. 2002. Adaptive Texture Maps. In Proceedings of the ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware (HWWS '02). Eurographics Association, 7--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. K.N. Kutulakos and S.M. Seitz. 1999. A Theory of Shape by Space Carving. In The Proceedings of the Seventh IEEE International Conference on Computer Vision. 307--314.Google ScholarGoogle Scholar
  27. Dylan Lacewell, David Edwards, Peter Shirley, and William B. Thompson. 2006. Stochastic Billboard Clouds for Interactive Foliage Rendering. Journal of Graphics Tools 11 (1 2006), 1--12.Google ScholarGoogle Scholar
  28. Marc Levoy and Pat Hanrahan. 1996. Light Field Rendering. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). 31--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Bruno Lévy, Sylvain Petitjean, Nicolas Ray, and Jérome Maillot. 2002. Least Squares Conformal Maps for Automatic Texture Atlas Generation. ACM Trans. Graph. 21, 3 (July 2002), 362--371. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. A. Lodi, S. Martello, and D. Vigo. 2004. Models and Bounds for Two-Dimensional Level Packing Problems. Journal of Combinatorial Optimization 8, 3 (9 2004), 363--379.Google ScholarGoogle ScholarCross RefCross Ref
  31. B. Luo, F. Xu, C. Richardt, and J. H. Yong. 2018. Parallax360: Stereoscopic 360° Scene Representation for Head-Motion Parallax. IEEE Transactions on Visualization and Computer Graphics (2018). Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Manufactura K4. 2015. Asia -- Far East Environment. https://assetstore.unity.com/packages/3d/environments/asia-far-east-environment-21298. (2015). Accessed on 2018-01-22.Google ScholarGoogle Scholar
  33. Harald Niederreiter. 1992. Random Number Generation and Quasi-Monte Carlo Methods. Society for Industrial and Applied Mathematics. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. J. Nystad, A. Lassen, A. Pomianowski, S. Ellis, and T. Olson. 2012. Adaptive Scalable Texture Compression. In Proceedings of the Fourth ACM SIGGRAPH/Eurographics Conference on High-Performance Graphics (EGGH-HPG'12). Eurographics Association, 105--114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. S. Peleg, M. Ben-Ezra, and Y. Pritch. 2001. Omnistereo: panoramic stereo imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence 23, 3 (Mar 2001), 279--290. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Eric Penner and Li Zhang. 2017. Soft 3D Reconstruction for View Synthesis. ACM Trans. Graph. 36, 6, Article 235 (Nov. 2017), 11 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Hanspeter Pfister, Matthias Zwicker, Jeroen van Baar, and Markus Gross. 2000. Surfels: Surface Elements As Rendering Primitives. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '00). 335--342. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Matt Pharr. 2017. Mobile VR: Challenges and Opportunities. (2017). http://pharr.org/matt/mobilevr.pdf ACM I3D Keynote.Google ScholarGoogle Scholar
  39. Thomas Porter and Tom Duff. 1984. Compositing Digital Images. SIGGRAPH Comput. Graph. 18, 3 (Jan. 1984), 253--259. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Liu Ren, Hanspeter Pfister, and Matthias Zwicker. 2002. Object Space EWA Surface Splatting: A Hardware Accelerated Approach to High Quality Point Rendering. Computer Graphics Forum 21, 3 (2002), 461--470.Google ScholarGoogle ScholarCross RefCross Ref
  41. C. Richardt, Y. Pritch, H. Zimmer, and A. Sorkine-Hornung. 2013. Megastereo: Constructing High-Resolution Stereo Panoramas. In 2013 IEEE Conference on Computer Vision and Pattern Recognition. 1256--1263. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Takafumi Saito and Tokiichiro Takahashi. 1991. NC Machining with G-buffer Method. In Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '91). 207--216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Gernot Schaufler. 1998. Image-based Object Representation by Layered Impostors. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (VRST '98). 99--104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Mark Segal, Carl Korobkin, Rolf van Widenfelt, Jim Foran, and Paul Haeberli. 1992. Fast Shadows and Lighting Effects Using Texture Mapping. In Proceedings of the 19th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '92). 249--252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Jonathan Shade, Steven Gortler, Li-Wei He, and Richard Szeliski. 1998. Layered Depth Images. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98). 231--242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Unity Technologies. 2015. The Blacksmith Environments. https://assetstore.unity.com/packages/3d/environments/asia-far-east-environment-21298. (2015). Accessed on 2018-01-22.Google ScholarGoogle Scholar
  47. Alex Vlachos. 2016. Advanced VR Rendering Performance. (2016). https://alex.vlachos.com/graphics/Alex_Vlachos_Advanced_VR_Rendering_Performance_GDC2016.pdf GDC Talk.Google ScholarGoogle Scholar
  48. Zhou Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. Image Quality Assessment: From Error Visibility to Structural Similarity. Trans. Img. Proc. 13, 4 (April 2004), 600--612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Andrew Wilson and Dinesh Manocha. 2003. Simplifying Complex Environments Using Incremental Textured Depth Meshes. In ACM SIGGRAPH 2003 Papers (SIGGRAPH '03). 678--688. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Daniel N. Wood, Daniel I. Azuma, Ken Aldinger, Brian Curless, Tom Duchamp, David H. Salesin, and Werner Stuetzle. 2000. Surface Light Fields for 3D Photography. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '00). 287--296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Matthias Zwicker, Hanspeter Pfister, Jeroen van Baar, and Markus Gross. 2001. Surface Splatting. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01). 371--378. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. View-Region Optimized Image-Based Scene Simplification

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image Proceedings of the ACM on Computer Graphics and Interactive Techniques
          Proceedings of the ACM on Computer Graphics and Interactive Techniques  Volume 1, Issue 2
          August 2018
          223 pages
          EISSN:2577-6193
          DOI:10.1145/3273023
          Issue’s Table of Contents

          Copyright © 2018 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 24 August 2018
          Published in pacmcgit Volume 1, Issue 2

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!