skip to main content
research-article
Public Access

Designing and Evaluating a Mesh Simplification Algorithm for Virtual Reality

Published:27 June 2018Publication History
Skip Abstract Section

Abstract

With the increasing accessibility of the mobile head-mounted displays (HMDs), mobile virtual reality (VR) systems are finding applications in various areas. However, mobile HMDs are highly constrained with limited graphics processing units (GPUs) and low processing power and onboard memory. Hence, VR developers must be cognizant of the number of polygons contained within their virtual environments to avoid rendering at low frame rates and inducing simulator sickness. The most robust and rapid approach to keeping the overall number of polygons low is to use mesh simplification algorithms to create low-poly versions of pre-existing, high-poly models. Unfortunately, most existing mesh simplification algorithms cannot adequately handle meshes with lots of boundaries or nonmanifold meshes, which are common attributes of many 3D models.

In this article, we present QEM4VR, a high-fidelity mesh simplification algorithm specifically designed for VR. This algorithm addresses the deficiencies of prior quadric error metric (QEM) approaches by leveraging the insight that the most relevant boundary edges lie along curvatures while linear boundary edges can be collapsed. Additionally, our algorithm preserves key surface properties, such as normals, texture coordinates, colors, and materials, as it preprocesses 3D models and generates their low-poly approximations offline.

We evaluated the effectiveness of our QEM4VR algorithm by comparing its simplified-mesh results to those of prior QEM variations in terms of geometric approximation error, texture error, progressive approximation errors, frame rate impact, and perceptual quality measures. We found that QEM4VR consistently yielded simplified meshes with less geometric approximation error and texture error than the prior QEM variations. It afforded better frame rates than QEM variations with boundary preservation constraints that create unnecessary lower bounds on overall polygon count reduction. Our evaluation revealed that QEM4VR did not fair well in terms of existing perceptual distance measurements, but human-based inspections demonstrate that these algorithmic measurements are not suitable substitutes for actual human perception. In turn, we present a user-based methodology for evaluating the perceptual qualities of mesh simplification algorithms.

Skip Supplemental Material Section

Supplemental Material

References

  1. K. Boos, D. Chu, and E. Cuervo. 2016. Flashback: Immersive virtual reality on mobile devices via rendering memoization. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 291--304. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. D. Pohl and C. F. de Tejada Quemada. 2016. See what I see: Concepts to improve the social acceptance of HMDs. In 2016 IEEE Virtual Reality (VR’16). IEEE, 267--268.Google ScholarGoogle Scholar
  3. A. Gargantini, F. Terzi, M. Zambelli, and S. Bonfanti. 2015. A low-cost virtual reality game for amblyopia rehabilitation. In Proceedings of the 3rd 2015 Workshop on ICTs for Improving Patients Rehabilitation Research Techniques. ACM, 81--84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. A. Steed, S. Frlston, M. M. Lopez, J. Drummond, Y. Pan, and D. Swapp. 2016. An’in the wild’ experiment on presence and embodiment using consumer virtual reality equipment. IEEE Transactions on Visualization and Computer Graphics 22, 4, 1406--1414. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. X. Tong, D. Gromala, A. Amin, and A. Choo. 2015. The design of an immersive mobile virtual reality serious game in cardboard head-mounted display for pain management. In International Symposium on Pervasive Computing Paradigms for Mental Health. Springer International Publishing, 284--293.Google ScholarGoogle Scholar
  6. C. Lai, R. P. McMahan, M. Kitagawa, and I. Connolly. 2016. Geometry explorer: Facilitating geometry education with virtual reality. In International Conference on Virtual, Augmented and Mixed Reality. Springer International Publishing, 702--713.Google ScholarGoogle Scholar
  7. D. J. Zielinski, H. M. Rao, N. D. Potter, M. A. Sommer, L. G. Appelbaum, and R. Kopper. 2016. Evaluating the effects of image persistence on dynamic target acquisition in low frame rate virtual environments. In 2016 IEEE Symposium on 3D User Interfaces (3DUI’16). IEEE, 133--140.Google ScholarGoogle Scholar
  8. C. Pruett. Squeezing performance out of your unity gear VR game. Retrieved from https://developer.oculus.com/blog/squeezing-performance-out-of-yourunity-gear-vr-game.Google ScholarGoogle Scholar
  9. Microsoft Developers Resources. Performance recommendations for Microsoft Hololens. Retrieved from https://developer.microsoft.com/en-us/windows/holographic/performance-recommendations.Google ScholarGoogle Scholar
  10. D. P. Luebke. 2001. A developer's survey of polygonal simplification algorithms. IEEE Computer Graphics and Applications 21, 3, 24--35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. A. Guéziec, G. Taubin, F. Lazarus, and W. Horn. 1998. Converting sets of polygons to manifold surfaces by cutting and stitching. In Proceedings of Visualization. IEEE, 383--390. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. H. Hoppe. 1999. New quadric metric for simplifying meshes with appearance attributes. In Proceedings of the Conference on Visualization: Celebrating Ten Years. IEEE Computer Society Press, 59--66. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. M. Garland and P. S. Heckbert. 1997. Surface simplification using quadric error metrics. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., 209--216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. M. Garland and P. S. Heckbert. 1998. Simplifying surfaces with color and texture using quadric error metrics. In Proceedings of Visualization. IEEE, 263--269. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. K. Bahirat, C. Lai, R. P. McMahan, and B. Prabhakaran. 2017. A boundary and texture preserving mesh simplification algorithm for virtual reality. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 50--61. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. P. Cignoni, C. Montani, and R. Scopigno. 1998. A comparison of mesh simplification algorithms. Computers & Graphics 22, 1, 37--54.Google ScholarGoogle ScholarCross RefCross Ref
  17. H. Hoppe, T. DeRose, T. Duchamp, J. McDonald, and W. Stuetzle. 1993. Mesh optimization. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques. ACM, 19--26. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. H. Hoppe. 1996. Progressive meshes. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. ACM, 99--108. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. G. Turk. 1992. Re-tiling polygonal surfaces. ACM SIGGRAPH Computer Graphics 26, 2, 55--64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. T. He, L. Hong, A. Kaufman, A. Varshney, and S. Wang. 1995. Voxel based object simplification. In Proceedings of the 6th Conference on Visualization. IEEE Computer Society, 296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. W. E. Lorensen and H. E. Cline. 1987. Marching cubes: A high resolution 3D surface construction algorithm. In ACM Siggraph Computer Graphics 21, 4, 163--169. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. Cohen, A. Varshney, D. Manocha, G. Turk, H. Weber, P. Agarwal, and W. Wright. 1996. Simplification envelopes. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques. ACM, 119--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. Lounsbery, T. D. DeRose, and J. Warren. 1997. Multiresolution analysis for surfaces of arbitrary topological type. ACM Transactions on Graphics (TOG) 16, 1, 34--73. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Eck, T. DeRose, T. Duchamp, H. Hoppe, M. Lounsbery, and W. Stuetzle. 1995. Multiresolution analysis of arbitrary meshes. In Proceedings of the 22nd Annual Conference on Computer GRAPHICS and Interactive Techniques. ACM, 173--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. M. Hosseini, D. T. Ahmed, and S. Shirmohammadi. 2012. Adaptive 3D texture streaming in M3G-based mobile games. In Proceedings of the 3rd Multimedia Systems Conference. ACM, 143--148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. W. J. Schroeder, J. A. Zarge, and W. E. Lorensen. 1992. Decimation of triangle meshes. ACM Siggraph Computer Graphics 26, 2, 65--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. J. Rossignac and P. Borrel. 1993. Multi-resolution 3D approximations for rendering complex scenes. Modeling in Computer Graphics: Methods and Applications. Springer, Berlin, Heidelberg, 455--465.Google ScholarGoogle Scholar
  28. D. Luebke and C. Erikson. 1997. View-dependent simplification of arbitrary polygonal environments. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., 199--208. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. P. Lindstrom and G. Turk. 1998. Fast and memory efficient polygonal simplification. In Proceedings of Visualization. IEEE, 279--286. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. Garland. 1999. Quadric-based polygonal surface simplification (No. CMU-CS-99-105). Carnegie-Mellon University of Pittsburgh, School of Computer Science.Google ScholarGoogle Scholar
  31. J. H. Wu, S. M. Hu, C. L. Tai, and J. G. Sun. 2001. An effective feature-preserving mesh simplification scheme based on face constriction. In Proceedings of the 9th Pacific Conference on Computer Graphics and Applications, 2001. IEEE, 12--21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. E. Ovreiu. 2012. Accurate 3D mesh simplification. Doctoral Dissertation, INSA de Lyon.Google ScholarGoogle Scholar
  33. E. Pojar and D. Schmalstieg. 2003. User-controlled creation of multiresolution meshes. In Proceedings of the 2003 Symposium on Interactive 3D Graphics. ACM, 127--130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. T. Ebrahimi. 2009. Quality of multimedia experience: Past, present and future. In Proceedings of the 17th ACM International Conference on Multimedia (No. MMSPL-CONF-2010-003) (MM’09). ACM, 3--4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. C. Timmerer, T. Ebrahimi, and F. Pereira. 2015. Toward a new assessment of quality. Computer 48, 3, 108--110.Google ScholarGoogle ScholarCross RefCross Ref
  36. A. Raake and S. Egger. 2014. Quality and quality of experience. In Quality of Experience. Springer, Cham, 11--33.Google ScholarGoogle Scholar
  37. J. Puig, A. Perkis, F. Lindseth, and T. Ebrahimi. 2012. Towards an efficient methodology for evaluation of quality of experience in Augmented Reality. In 2012 4th International Workshop on Quality of Multimedia Experience (QoMEX’12). IEEE, 188--193.Google ScholarGoogle Scholar
  38. R. P. McMahan, D. A. Bowman, D. J. Zielinski, and R. B. Brady. 2012. Evaluating display fidelity and interaction fidelity in a virtual reality game. IEEE Transactions on Visualization and Computer Graphics 18, 4, 626--633. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. D. A. Bowman and R. P. McMahan. 2007. Virtual reality: How much immersion is enough? Computer 40, 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. M. Meehan, B. Insko, M. Whitton, and F. P. Brooks Jr. 2002. Physiological measures of presence in stressful virtual environments. ACM Transactions on Graphics (TOG) 21, 3, 645--652. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. D. Egan, S. Brennan, J. Barrett, Y. Qiao, C. Timmerer, and N. Murray. 2016. An evaluation of heart rate and electrodermal activity as an objective QoE evaluation method for immersive virtual reality environments. In 2016 8th International Conference on Quality of Multimedia Experience (QoMEX’16). IEEE, 1--6.Google ScholarGoogle Scholar
  42. C. Keighrey, R. Flynn, S. Murray, and N. Murray. 2017. A QoE evaluation of immersive augmented and virtual reality speech & language assessment applications. In 2017 9th International Conference on Quality of Multimedia Experience (QoMEX’17). IEEE, 1--6.Google ScholarGoogle Scholar
  43. A. Lecuyer, S. Coquillart, A. Kheddar, P. Richard, and P. Coiffet. 2000. Pseudo-haptic feedback: Can isometric input devices simulate force feedback? In IEEE Proceedings of Virtual Reality, 2000. IEEE, 83--90. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. X. K. Yang, W. S. Ling, Z. K. Lu, E. P. Ong, and S. S. Yao. 2005. Just noticeable distortion model and its applications in video coding. Signal Processing: Image Communication 20, 7, 662--680.Google ScholarGoogle ScholarCross RefCross Ref
  45. X. Zhang, W. Lin, and P. Xue. 2008. Just-noticeable difference estimation with pixels in images. Journal of Visual Communication and Image Representation 19, 1, 30--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Y. Zhao, Z. Chen, C. Zhu, Y. P. Tan, and L. Yu. 2011. Binocular just-noticeable-difference model for stereoscopic images. IEEE Signal Processing Letters 18, 1, 19--22.Google ScholarGoogle ScholarCross RefCross Ref
  47. W. Hachicha, A. Beghdadi, and F. A. Cheikh. 2013. Stereo image quality assessment using a binocular just noticeable difference model. In 2013 20th IEEE International Conference on Image Processing (ICIP’13). IEEE, 113--117.Google ScholarGoogle Scholar
  48. W. Wu, A. Arefin, G. Kurillo, P. Agarwal, K. Nahrstedt, and R. Bajcsy. 2011. Color-plus-depth level-of-detail in 3D tele-immersive video: A psychophysical approach. In Proceedings of the 19th ACM International Conference on Multimedia. ACM, 13--22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. R. Goldman. 2005. Curvature formulas for implicit curves and surfaces. Computer Aided Geometric Design 22, 7, 632--658. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. P. Cignoni, M. Callieri, M. Corsini, M. Dellepiane, F. Ganovelli, and G. Ranzuglia. 2008. Meshlab: An open-source mesh processing tool. In Eurographics Italian Chapter Conference. 129--136.Google ScholarGoogle Scholar
  51. P. Cignoni, C. Rocchini, and R. Scopigno. 1998. Metro: Measuring error on simplified surfaces. Computer Graphics Forum 17, 2, 167--174.Google ScholarGoogle Scholar
  52. G. Lavoué, E. D. Gelasca, F. Dupont, A. Baskurt, and T. Ebrahimi. 2006. Perceptually driven 3D distance metrics with application to watermarking. In SPIE Optics+ Photonics (63120L-63120L). International Society for Optics and Photonics.Google ScholarGoogle Scholar
  53. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4, 600--612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. G. Lavoué. 2011. A multiscale metric for 3D mesh visual quality assessment. Computer Graphics Forum 30, 5, 1427--1437.Google ScholarGoogle ScholarCross RefCross Ref
  55. K. Wang, F. Torkhani, and A. Montanvert. 2012. A fast roughness-based approach to the assessment of 3D mesh visual quality. Computers & Graphics 36, 7, 808--818. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. F. Torkhani, K. Wang, and J. M. Chassery. 2014. A curvature-tensor-based perceptual quality metric for 3D triangular meshes. Machine Graphics & Vision 23, 1.Google ScholarGoogle Scholar
  57. K. Wang. 2012. Source code for fast mesh perceptual distance (FMPD). Retrieved from http://www.gipsa-lab.grenoble-inp.fr/∼kai.wang/publications_en.html.Google ScholarGoogle Scholar
  58. I. Cheng, R. Shen, X. D. Yang, and P. Boulanger. 2006. Perceptual analysis of level-of-detail: The JND approach. In 8th IEEE International Symposium on Multimedia (ISM'06). IEEE, 533--540. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. H. Wang, I. Katsavounidis, J. Zhou, J. Park, S. Lei, X. Zhou, M. O. Pun, X. Jin, R. Wang, X. Wang, and Y. Zhang. 2017. VideoSet: A large-scale compressed video quality dataset based on JND measurement. Journal of Visual Communication and Image Representation 46, 292--302. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. S. Qin, S. Ge, H. Yin, L. Liu, and I. Heynderickx. 2009. Effect of experimental methodology on the JND of the black level for natural images. Journal of the Society for Information Display 17, 8, 687--694.Google ScholarGoogle ScholarCross RefCross Ref
  61. A. B. Watson and A. Fitzhugh. 1990. The method of constant stimuli is inefficient. Perception & Psychophysics 47, 1, 87--91.Google ScholarGoogle ScholarCross RefCross Ref
  62. I. Cheng and P. Boulanger. 2005. Feature extraction on 3-D TexMesh using scale-space analysis and perceptual evaluation. IEEE Transactions on Circuits and Systems for Video Technology 15, 10, 1234--1244. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Link for the code: http://cs.utdallas.edu/multimedialab/qem4vr/.Google ScholarGoogle Scholar

Index Terms

  1. Designing and Evaluating a Mesh Simplification Algorithm for Virtual Reality

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 14, Issue 3s
      Special Section on Delay-Sensitive Video Computing in the Cloud and Special Section on Extended MMSys-NOSSDAV Best Papers
      June 2018
      317 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3233173
      Issue’s Table of Contents

      Copyright © 2018 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 27 June 2018
      • Accepted: 1 April 2018
      • Revised: 1 March 2018
      • Received: 1 September 2017
      Published in tomm Volume 14, Issue 3s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!