skip to main content
research-article

SCeVE: A Component-based Framework to Author Mixed Reality Tours

Published:22 May 2020Publication History
Skip Abstract Section

Abstract

Authoring a collaborative, interactive Mixed Reality (MR) tour requires flexible design and development of various software modules for tasks such as managing geographically distributed participants, adaptable travel and virtual camera techniques, data logging for assessment of the incorporated techniques, as well as for evaluating the Quality of Experiences (QoE). In most cases, authors might have to develop all these software modules, instead of focusing only on the virtual environment design. In this article, we propose SCeVE, a component-based framework that supports flexible design and authoring of interactive MR tours by offering ease of access to four major design choices: (i) Synchronization, (ii) Collaborative exploration, (iii) Visualization, and (iv) Evaluation. Based on tour requirements, an author can access one or more components (or software libraries) of design choices via SCeVE’s API (Application Programming Interface) services, as demonstrated by the two case studies on group travel in a plant walk MR tour.

SCeVE framework is innovative in the sense that it facilitates group travel in virtual environments involving “live” models of participants from geographically distributed sites. SCeVE empowers authors to focus only on the design of the required virtual environments. They can quickly build a diverse set of collaborative MR tours by utilizing the flexibility of SCeVE in terms of the various available options for traveling, rendering on multiple devices, and virtual camera viewpoint computation strategies. By providing data logs of various components, SCeVE facilitates performance evaluation of the various strategies used as well as the user experience in collaborative MR tours. SCeVE is designed in an extensible manner, allowing authors to add devices and software services as additional components.

References

  1. Majed Al Zayer, Paul MacNeilage, and Eelke Folmer. 2018. Virtual locomotion: A survey. IEEE Trans. Vis. Comput. Graph. (2018). DOI:10.1109/TVCG.2018.2887379Google ScholarGoogle Scholar
  2. Robert S. Allison, Laurence R. Harris, Michael Jenkin, Urszula Jasiobedzka, and James E. Zacher. 2001. Tolerance of temporal delay in virtual environments. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 247--254.Google ScholarGoogle Scholar
  3. AltSpaceVR. 2015. AltSpaceVR. Retrieved from https://altvr.com/.Google ScholarGoogle Scholar
  4. Abdenour Amamra, Yacine Amara, Redha Benaissa, and Billal Merabti. 2016. Optimal camera path planning for 3D visualisation. In Proceedings of the SAI Computing Conference (SAI’16). IEEE, 388--393.Google ScholarGoogle ScholarCross RefCross Ref
  5. Amazon. [2004]. Amazon Sumerian. Retrieved from https://aws.amazon.com/sumerian/.Google ScholarGoogle Scholar
  6. Hirokazu Kato. [2007]. Inside ARToolKit. In Proceedings of the 1st IEEE International Workshop on Augmented Reality Toolkit.Google ScholarGoogle Scholar
  7. Anders Backman. 2005. Colosseum3D-authoring framework for virtual environments. In Proceedings of the International Workshop on Immersive Projection Technology and Eurographics Symposium on Virtual Environments (IPT/EGVE’05). Citeseer, 225--226.Google ScholarGoogle Scholar
  8. Martin Bauer, Bernd Bruegge, Gudrun Klinker, Asa MacWilliams, Thomas Reicher, Stefan Riss, Christian Sandor, and Martin Wagner. 2001. Design of a component-based augmented reality framework. In Proceedings of the IEEE and ACM International Symposium on Augmented Reality. IEEE, 45--54.Google ScholarGoogle ScholarCross RefCross Ref
  9. Tomasz Bednarz, Craig James, Eleonora Widzyk-Capehart, Con Caris, and Leila Alem. 2015. Distributed collaborative immersive virtual reality framework for the mining industry. In Machine Vision and Mechatronics in Practice. Springer, 39--48.Google ScholarGoogle Scholar
  10. Ian Bickerstaff. 2012. Case study: The introduction of stereoscopic games on the Sony Playstation 3. In Stereoscopic Displays and Applications XXIII, Vol. 8288. International Society for Optics and Photonics, 828815.Google ScholarGoogle ScholarCross RefCross Ref
  11. Allen Bierbaum, Christopher Just, Patrick Hartling, Kevin Meinert, Albert Baker, and Carolina Cruz-Neira. 2001. VR Juggler: A virtual platform for virtual reality application development. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 89--96.Google ScholarGoogle ScholarCross RefCross Ref
  12. Christoph W. Borst, Nicholas G. Lipari, and Jason W. Woodworth. 2018. Teacher-guided educational VR: Assessment of live and prerecorded teachers guiding virtual field trips. In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces (VR’18). IEEE, 467--474.Google ScholarGoogle Scholar
  13. Kjell Brunnström, Mårten Sjöström, Muhammad Imran, Magnus Pettersson, and Mathias Johanson. 2018. Quality of experience for a virtual reality simulator. Electron. Imag. 2018, 14 (2018), 1--9.Google ScholarGoogle ScholarCross RefCross Ref
  14. Andrew S. Cantino, David L. Roberts, and Charles L. Isbell. 2007. Autonomous nondeterministic tour guides: Improving quality of experience with TTD-MDPS. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, 22.Google ScholarGoogle Scholar
  15. Ki Hun Cho and Wan Hee Lee. 2013. Virtual walking training program using a real-world video recording for patients with chronic stroke: A pilot study. Amer. J. Phys. Med. Rehab. 92, 5 (2013), 371--384.Google ScholarGoogle ScholarCross RefCross Ref
  16. Marc Christie, Rumesh Machap, Jean-Marie Normand, Patrick Olivier, and Jonathan Pickering. 2005. Virtual camera planning: A survey. In Proceedings of the International Symposium on Smart Graphics. Springer, 40--52.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Steven H. Collins, Peter G. Adamczyk, and Arthur D. Kuo. 2009. Dynamic arm swinging in human walking. Proc. Roy. Soc. B: Biol. Sci. 276, 1673 (2009), 3679--3688.Google ScholarGoogle Scholar
  18. Noah Coomer, Sadler Bullard, William Clinton, and Betsy Williams-Sanders. 2018. Evaluating the effects of four VR locomotion methods: Joystick, arm-cycling, point-tugging, and teleporting. In Proceedings of the 15th ACM Symposium on Applied Perception. ACM, 7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Nguyen-Thong Dang, Céline Chatelain, Jean-Marie Pergandi, and Daniel Mestre. 2008. A framework for design and evaluation of collaborative virtual environments. In Proceedings of the Journées de lâ’Association Française de Réalité Virtuelle. 119--126.Google ScholarGoogle Scholar
  20. Demo. [2019]. Demo. Retrieved from https://youtu.be/ie9TMDKZ750.Google ScholarGoogle Scholar
  21. Desiree DePriest and Karlyn Barilovits. 2011. LIVE: Xbox Kinect©; virtual realities to learning games. In Proceedings of the Theory of Cryptography Conference (TCC’11). TCCHawaii, 48--54.Google ScholarGoogle Scholar
  22. Kevin Desai, Balakrishnan Prabhakaran, and Suraj Raghuraman. 2018. Skeleton-based continuous extrinsic calibration of multiple RGB-D kinect cameras. In Proceedings of the 9th ACM Multimedia Systems Conference. ACM, 250--257.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Stephen R. Ellis, Bernard D. Adelstein, S. Baumeler, G. J. Jense, and Richard H. Jacoby. 1999. Sensor spatial distortion, visual latency, and update rate effects on 3D tracking in virtual environments. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 218--221.Google ScholarGoogle Scholar
  24. Framework. 2019. Code. Retrieved from https://github.com/utd-multimedia/PlantWalk.Google ScholarGoogle Scholar
  25. Sebastian Friston and Anthony Steed. 2014. Measuring latency in virtual environments. IEEE Trans. Vis. Comput. Graph. 20, 4 (2014), 616--625.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Tobias Fritsch, Carsten Magerkurth, Benjamin Voigt, and Jochen Schiller. 2007. 4MOG--massive multiplayer middleware for mobile online games. In Proceedings of the 1st International Workshop on Intercultural Collaboration (IWIC’07).Google ScholarGoogle Scholar
  27. Hania Gajewska, David P. Mendenhall, Peter A. Korn, Michael C. Albers, and Lynn Monsanto. 2003. Keyboard navigation of non-focusable components. US Patent 6,654,038.Google ScholarGoogle Scholar
  28. getReal3D. [n.d.]. getReal3D. Retrieved from https://projects.vrac.iastate.edu/.Google ScholarGoogle Scholar
  29. Benjamin Goldstein. 2000. Tandem: A Component-based Framework for Interactive, Collaborative Virtual Reality. Master’s thesis. University of Illinois at Chicago.Google ScholarGoogle Scholar
  30. Michael Haller, Jürgen Zauner, Werner Hartmann, and Thomas Luckeneder. 2003. A Generic Framework for a Training Application Based on Mixed Reality. Technical Report. Upper Austria University of Applied Sciences.Google ScholarGoogle Scholar
  31. Abdelwahab Hamam and Abdulmotaleb El Saddik. 2013. Toward a mathematical model for quality of experience evaluation of haptic applications. IEEE Trans. Instrument. Meas. 62, 12 (2013), 3315--3322.Google ScholarGoogle ScholarCross RefCross Ref
  32. Sylvia Irawati, Sangchul Ahn, Jinwook Kim, and Heedong Ko. 2008. Varu framework: Enabling rapid prototyping of VR, AR, and ubiquitous applications. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 201--208.Google ScholarGoogle ScholarCross RefCross Ref
  33. Yutaka Ishibashi, Sosuke Hoshino, Qi Zeng, Norishige Fukushima, and Shinji Sugawara. 2012. QoE assessment of fairness between players in networked game with olfaction. In Proceedings of the 11th Workshop on Network and Systems Support for Games (NetGames’12). IEEE, 1--2.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Jeff Feasel, Mary C. Whitton, Jeremy D. Wendt. 2008. LLCM-WIP: Low-latency, continuous-motion walking-in-place. In Proceedings of the 2008 IEEE Symposium on 3D User Interfaces. IEEE, 97--104.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Dennis Joele, G. van der Mast, and Marfa Carmen Juan-Lizandra. 2005. Development of an Augmented Reality System Using ARToolKit and User Invisible Markers. Research Assignment, Master Programme Media 8 Knowledge Engineering, Technical University of Valencia, Spain.Google ScholarGoogle Scholar
  36. Konstantina Kilteni, Jean-Marie Normand, Maria V. Sanchez-Vives, and Mel Slater. 2012. Extending body space in immersive virtual reality: A very long arm illusion. PloS One 7, 7 (2012), e40867.Google ScholarGoogle ScholarCross RefCross Ref
  37. Yuji Kusunose, Yutaka Ishibashi, Norishige Fukushima, and Shinji Sugawara. 2010. QoE assessment in networked air hockey game with haptic media. In Proceedings of the 9th Workshop on Network and Systems Support for Games. IEEE, 1--2.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. J. LaViola. 1999. Whole-hand and Speech Input in Virtual Environments. Unpublished Master’s Thesis, Department of Computer Science, Brown University, CS-99-15 (1999).Google ScholarGoogle Scholar
  39. Joseph J. LaViola Jr., Ernst Kruijff, Doug Bowman, Ivan P. Poupyrev, and Ryan P. McMahan. 2017. 3D User Interfaces: Theory and Practice (Second Edition). Addison-Wesley.Google ScholarGoogle Scholar
  40. Gun A. Lee and Mark Billinghurst. 2013. A component based framework for mobile outdoor AR applications. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. ACM, 207--210.Google ScholarGoogle Scholar
  41. Wei-Po Lee, Li-Jen Liu, and Jeng-An Chiou. 2011. A component-based framework for rapidly developing online board games. Int. J. Comput. Applic. 33, 4 (2011), 293--302.Google ScholarGoogle Scholar
  42. Tsai-Yen Li and Chung-Chiang Cheng. 2008. Real-time camera planning for navigation in virtual environments. In Proceedings of the International Symposium on Smart Graphics. Springer, 118--129.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Tsai-Yen Li . 1999. Automatically generating virtual guided tours. In Proceedings Computer Animation. IEEE, 99--106. DOI:10.1109/CA.1999.781203Google ScholarGoogle Scholar
  44. Joan Llobera, Mar González-Franco, Daniel Perez-Marcos, Josep Valls-Solé, Mel Slater, and Maria V. Sanchez-Vives. 2013. Virtual reality for assessment of patients suffering chronic pain: A case study. Experim. Brain Res. 225, 1 (2013), 105--117.Google ScholarGoogle ScholarCross RefCross Ref
  45. Gale M. Lucas, Evan Szablowski, Jonathan Gratch, Andrew Feng, Tiffany Huang, Jill Boberg, and Ari Shapiro. 2016. Do avatars that look like their users improve performance in a simulation? In Proceedings of the International Conference on Intelligent Virtual Agents. Springer, 351--354.Google ScholarGoogle ScholarCross RefCross Ref
  46. Mary Lou Maher and John S. Gero. 2002. Agent models of 3D virtual worlds. CUMINCAD, IEEE. DOI:10.1109/CW.2005.13Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Ryan P. McMahan, Chengyuan Lai, and Swaroop K. Pal. 2016. Interaction fidelity: The uncanny valley of virtual reality interactions. In Proceedings of the International Conference on Virtual, Augmented and Mixed Reality. Springer, 59--70.Google ScholarGoogle Scholar
  48. David L. Mills. 1991. Internet time synchronization: The network time protocol. IEEE Trans. Commun. 39, 10 (1991), 1482--1493.Google ScholarGoogle ScholarCross RefCross Ref
  49. Shailey Minocha and Christopher Hardy. 2016. Navigation and wayfinding in learning spaces in 3D virtual worlds. Learn. Virt. Worlds: Res. Applic. (2016), 3--41. DOI:10.15215/aupress/9781771991339.01dGoogle ScholarGoogle Scholar
  50. Alessandro Mulloni, Hartmut Seichter, and Dieter Schmalstieg. 2011. Handheld augmented reality indoor navigation with activity-based instructions. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. ACM, 211--220.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Mahdi Nabiyouni, Ayshwarya Saktheeswaran, Doug A. Bowman, and Ambika Karanth. 2015. Comparing the performance of natural, semi-natural, and non-natural locomotion techniques in virtual reality. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI’15). IEEE, 3--10.Google ScholarGoogle Scholar
  52. Niels C. Nilsson, Stefania Serafin, Morten H. Laursen, Kasper S. Pedersen, Erik Sikström, and Rolf Nordahl. 2013. Tapping-in-place: Increasing the naturalness of immersive walking-in-place locomotion through novel gestural input. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI’13). IEEE, 31--38.Google ScholarGoogle ScholarCross RefCross Ref
  53. Niels Christian Nilsson, Stefania Serafin, Frank Steinicke, and Rolf Nordahl. 2018. Natural walking in virtual reality: A review. Comput. Entert. 16, 2 (2018), 8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Jan Ohlenburg, Iris Herbst, Irma Lindt, Thorsten Fröhlich, and Wolfgang Broll. 2004. The MORGAN framework: Enabling dynamic multi-user AR and VR projects. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology. ACM, 166--169.Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Jong-Seung Park. 2011. AR-Room: A rapid prototyping framework for augmented reality applications. Multim. Tools Applic. 55, 3 (2011), 725--746.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Roma Patel and Deborah Tuck. 2008. Narrating the past: Virtual environments and narrative. 249--258. http://irep.ntu.ac.uk/id/eprint/15426/.Google ScholarGoogle Scholar
  57. Ryan A. Pavlik and Judy M. Vance. 2012. VR JuggLua: A framework for VR applications combining Lua, openscenegraph, and VR Juggler. In Proceedings of the 5th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS’12). IEEE, 29--35.Google ScholarGoogle Scholar
  58. Francesca Pazzaglia, Chiara Meneghetti, Enia Labate, and Lucia Ronconi. 2016. Are wayfinding self-efficacy, and pleasure in exploring related to shortcut finding? A study in a virtual environment. In Spatial Cognition X. Springer, 55--68.Google ScholarGoogle Scholar
  59. Wayne Piekarski and Bruce H. Thomas. 2003. An object-oriented software architecture for 3D mixed reality applications. In Proceedings of the 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE, 247--256.Google ScholarGoogle Scholar
  60. Suraj Raghuraman. 2017. i3DTI: Interactive 3D Tele-Immersion. Ph.D. Dissertation. The University of Texas at Dallas, Richardson, Texas, USA.Google ScholarGoogle Scholar
  61. M. J. Rorke. 2000. Designing and Implementing a Virtual Reality Interaction Framework. Ph.D. Dissertation. Rhodes University.Google ScholarGoogle Scholar
  62. Maria Roussou and Mel Slater. 2005. A virtual playground for the study of the role of interactivity in virtual learning environments. Perspectives 8 (2005), 9.Google ScholarGoogle Scholar
  63. Andrew Sears, Jinhuan Feng, Kwesi Oseitutu, and Claire-Marie Karat. 2003. Hands-free, speech-based navigation during dictation: Difficulties, consequences, and solutions. Hum.-comput. Interact. 18, 3 (2003), 229--257.Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. Ari Shapiro, Andrew Feng, Ruizhe Wang, Hao Li, Mark Bolas, Gerard Medioni, and Evan Suma. 2014. Rapid avatar capture and simulation using commodity depth sensors. Comput. Anim. Virt. Worlds 25, 3--4 (2014), 201--211.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Mya Sithu, Yutaka Ishibashi, Pingguo Huang, and Norishige Fukushima. 2015. QoE assessment of operability and fairness for soft objects in networked real-time game with haptic sense. In Proceedings of the 21st Asia-Pacific Conference on Communications (APCC’15). IEEE, 570--574.Google ScholarGoogle ScholarCross RefCross Ref
  66. Bernhard Spanlang, Xavi Navarro, Jean-Marie Normand, Sameer Kishore, Rodrigo Pizarro, and Mel Slater. 2013. Real time whole body motion mapping for avatars and robots. In Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology. ACM, 175--178.Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. SteamVR. 2016. SteamVR. Retrieved from https://steamcommunity.com/steamvr.Google ScholarGoogle Scholar
  68. Richard Stoakley, Matthew J. Conway, and Randy Pausch. 1995. Virtual reality on a WIM: Interactive worlds in miniature. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI’95), Vol. 95. 265--272.Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Ayano Tatematsu, Yutaka Ishibashi, Norishige Fukushima, and Shinji Sugawara. 2010. QoE assessment in haptic media, sound, and video transmission: Influences of network latency. In Proceedings of the IEEE International Workshop on Communications Quality and Reliability (CQR’10). IEEE, 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  70. Unity Technologies. 2005. Unity3D Game Engine. Retrieved from https://unity3d.com/.Google ScholarGoogle Scholar
  71. Unreal. 2004. Unreal game engine. Retrieved from https://www.unrealengine.com.Google ScholarGoogle Scholar
  72. Narasimha Raghavan Veeraragavan, Leonardo Montecchi, Nicola Nostro, Roman Vitenberg, Hein Meling, and Andrea Bondavalli. 2016. Modeling QoE in dependable tele-immersive applications: A case study of world opera. IEEE Trans. Parallel Distrib. Syst. 27, 9 (2016), 2667--2681.Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Shanthi Vellingiri and Prabhakaran Balakrishnan. 2017. Modeling user quality of experience (QoE) through position discrepancy in multi-sensorial, immersive, collaborative environments. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 296--307.Google ScholarGoogle ScholarDigital LibraryDigital Library
  74. Shanthi Vellingiri and Balakrishnan Prabhakaran. 2018. Quantifying group navigation experience in collaborative augmented virtuality tours. In Proceedings of the 3rd International Workshop on Multimedia Alternate Realities. ACM, 3--8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. Shanthi Vellingiri, Yuan Tian, and Balakrishnan Prabhakaran. 2014. A real time, distributed system with haptic interfaces for fine motor skill rehabilitation and its quality of experience. In Proceedings of the IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE’14). IEEE, 53--58.Google ScholarGoogle ScholarCross RefCross Ref
  76. Norman G. Vinson. 1999. Design guidelines for landmarks to support navigation in virtual environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 278--285.Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. VRTK. 2016. VRTK. Retrieved from https://vrtoolkit.readme.io/.Google ScholarGoogle Scholar
  78. Daniel Wagner and Dieter Schmalstieg. 2007. Muddleware for prototyping mixed reality multiuser games. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 235--238.Google ScholarGoogle ScholarCross RefCross Ref
  79. Martin White, Emmanuel Jay, Fotis Liarokapis, Costas Kostakis, and Paul Lister. 2001. A virtual interactive teaching environment using XML and augmented reality. Int. J. Elect. Eng. Educ. 38, 4 (2001), 316--329.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. SCeVE: A Component-based Framework to Author Mixed Reality Tours

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!