Abstract
Authoring a collaborative, interactive Mixed Reality (MR) tour requires flexible design and development of various software modules for tasks such as managing geographically distributed participants, adaptable travel and virtual camera techniques, data logging for assessment of the incorporated techniques, as well as for evaluating the Quality of Experiences (QoE). In most cases, authors might have to develop all these software modules, instead of focusing only on the virtual environment design. In this article, we propose SCeVE, a component-based framework that supports flexible design and authoring of interactive MR tours by offering ease of access to four major design choices: (i) Synchronization, (ii) Collaborative exploration, (iii) Visualization, and (iv) Evaluation. Based on tour requirements, an author can access one or more components (or software libraries) of design choices via SCeVE’s API (Application Programming Interface) services, as demonstrated by the two case studies on group travel in a plant walk MR tour.
SCeVE framework is innovative in the sense that it facilitates group travel in virtual environments involving “live” models of participants from geographically distributed sites. SCeVE empowers authors to focus only on the design of the required virtual environments. They can quickly build a diverse set of collaborative MR tours by utilizing the flexibility of SCeVE in terms of the various available options for traveling, rendering on multiple devices, and virtual camera viewpoint computation strategies. By providing data logs of various components, SCeVE facilitates performance evaluation of the various strategies used as well as the user experience in collaborative MR tours. SCeVE is designed in an extensible manner, allowing authors to add devices and software services as additional components.
- Majed Al Zayer, Paul MacNeilage, and Eelke Folmer. 2018. Virtual locomotion: A survey. IEEE Trans. Vis. Comput. Graph. (2018). DOI:10.1109/TVCG.2018.2887379Google Scholar
- Robert S. Allison, Laurence R. Harris, Michael Jenkin, Urszula Jasiobedzka, and James E. Zacher. 2001. Tolerance of temporal delay in virtual environments. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 247--254.Google Scholar
- AltSpaceVR. 2015. AltSpaceVR. Retrieved from https://altvr.com/.Google Scholar
- Abdenour Amamra, Yacine Amara, Redha Benaissa, and Billal Merabti. 2016. Optimal camera path planning for 3D visualisation. In Proceedings of the SAI Computing Conference (SAI’16). IEEE, 388--393.Google Scholar
Cross Ref
- Amazon. [2004]. Amazon Sumerian. Retrieved from https://aws.amazon.com/sumerian/.Google Scholar
- Hirokazu Kato. [2007]. Inside ARToolKit. In Proceedings of the 1st IEEE International Workshop on Augmented Reality Toolkit.Google Scholar
- Anders Backman. 2005. Colosseum3D-authoring framework for virtual environments. In Proceedings of the International Workshop on Immersive Projection Technology and Eurographics Symposium on Virtual Environments (IPT/EGVE’05). Citeseer, 225--226.Google Scholar
- Martin Bauer, Bernd Bruegge, Gudrun Klinker, Asa MacWilliams, Thomas Reicher, Stefan Riss, Christian Sandor, and Martin Wagner. 2001. Design of a component-based augmented reality framework. In Proceedings of the IEEE and ACM International Symposium on Augmented Reality. IEEE, 45--54.Google Scholar
Cross Ref
- Tomasz Bednarz, Craig James, Eleonora Widzyk-Capehart, Con Caris, and Leila Alem. 2015. Distributed collaborative immersive virtual reality framework for the mining industry. In Machine Vision and Mechatronics in Practice. Springer, 39--48.Google Scholar
- Ian Bickerstaff. 2012. Case study: The introduction of stereoscopic games on the Sony Playstation 3. In Stereoscopic Displays and Applications XXIII, Vol. 8288. International Society for Optics and Photonics, 828815.Google Scholar
Cross Ref
- Allen Bierbaum, Christopher Just, Patrick Hartling, Kevin Meinert, Albert Baker, and Carolina Cruz-Neira. 2001. VR Juggler: A virtual platform for virtual reality application development. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 89--96.Google Scholar
Cross Ref
- Christoph W. Borst, Nicholas G. Lipari, and Jason W. Woodworth. 2018. Teacher-guided educational VR: Assessment of live and prerecorded teachers guiding virtual field trips. In Proceedings of the IEEE Conference on Virtual Reality and 3D User Interfaces (VR’18). IEEE, 467--474.Google Scholar
- Kjell Brunnström, Mårten Sjöström, Muhammad Imran, Magnus Pettersson, and Mathias Johanson. 2018. Quality of experience for a virtual reality simulator. Electron. Imag. 2018, 14 (2018), 1--9.Google Scholar
Cross Ref
- Andrew S. Cantino, David L. Roberts, and Charles L. Isbell. 2007. Autonomous nondeterministic tour guides: Improving quality of experience with TTD-MDPS. In Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. ACM, 22.Google Scholar
- Ki Hun Cho and Wan Hee Lee. 2013. Virtual walking training program using a real-world video recording for patients with chronic stroke: A pilot study. Amer. J. Phys. Med. Rehab. 92, 5 (2013), 371--384.Google Scholar
Cross Ref
- Marc Christie, Rumesh Machap, Jean-Marie Normand, Patrick Olivier, and Jonathan Pickering. 2005. Virtual camera planning: A survey. In Proceedings of the International Symposium on Smart Graphics. Springer, 40--52.Google Scholar
Digital Library
- Steven H. Collins, Peter G. Adamczyk, and Arthur D. Kuo. 2009. Dynamic arm swinging in human walking. Proc. Roy. Soc. B: Biol. Sci. 276, 1673 (2009), 3679--3688.Google Scholar
- Noah Coomer, Sadler Bullard, William Clinton, and Betsy Williams-Sanders. 2018. Evaluating the effects of four VR locomotion methods: Joystick, arm-cycling, point-tugging, and teleporting. In Proceedings of the 15th ACM Symposium on Applied Perception. ACM, 7.Google Scholar
Digital Library
- Nguyen-Thong Dang, Céline Chatelain, Jean-Marie Pergandi, and Daniel Mestre. 2008. A framework for design and evaluation of collaborative virtual environments. In Proceedings of the Journées de lâ’Association Française de Réalité Virtuelle. 119--126.Google Scholar
- Demo. [2019]. Demo. Retrieved from https://youtu.be/ie9TMDKZ750.Google Scholar
- Desiree DePriest and Karlyn Barilovits. 2011. LIVE: Xbox Kinect©; virtual realities to learning games. In Proceedings of the Theory of Cryptography Conference (TCC’11). TCCHawaii, 48--54.Google Scholar
- Kevin Desai, Balakrishnan Prabhakaran, and Suraj Raghuraman. 2018. Skeleton-based continuous extrinsic calibration of multiple RGB-D kinect cameras. In Proceedings of the 9th ACM Multimedia Systems Conference. ACM, 250--257.Google Scholar
Digital Library
- Stephen R. Ellis, Bernard D. Adelstein, S. Baumeler, G. J. Jense, and Richard H. Jacoby. 1999. Sensor spatial distortion, visual latency, and update rate effects on 3D tracking in virtual environments. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 218--221.Google Scholar
- Framework. 2019. Code. Retrieved from https://github.com/utd-multimedia/PlantWalk.Google Scholar
- Sebastian Friston and Anthony Steed. 2014. Measuring latency in virtual environments. IEEE Trans. Vis. Comput. Graph. 20, 4 (2014), 616--625.Google Scholar
Digital Library
- Tobias Fritsch, Carsten Magerkurth, Benjamin Voigt, and Jochen Schiller. 2007. 4MOG--massive multiplayer middleware for mobile online games. In Proceedings of the 1st International Workshop on Intercultural Collaboration (IWIC’07).Google Scholar
- Hania Gajewska, David P. Mendenhall, Peter A. Korn, Michael C. Albers, and Lynn Monsanto. 2003. Keyboard navigation of non-focusable components. US Patent 6,654,038.Google Scholar
- getReal3D. [n.d.]. getReal3D. Retrieved from https://projects.vrac.iastate.edu/.Google Scholar
- Benjamin Goldstein. 2000. Tandem: A Component-based Framework for Interactive, Collaborative Virtual Reality. Master’s thesis. University of Illinois at Chicago.Google Scholar
- Michael Haller, Jürgen Zauner, Werner Hartmann, and Thomas Luckeneder. 2003. A Generic Framework for a Training Application Based on Mixed Reality. Technical Report. Upper Austria University of Applied Sciences.Google Scholar
- Abdelwahab Hamam and Abdulmotaleb El Saddik. 2013. Toward a mathematical model for quality of experience evaluation of haptic applications. IEEE Trans. Instrument. Meas. 62, 12 (2013), 3315--3322.Google Scholar
Cross Ref
- Sylvia Irawati, Sangchul Ahn, Jinwook Kim, and Heedong Ko. 2008. Varu framework: Enabling rapid prototyping of VR, AR, and ubiquitous applications. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 201--208.Google Scholar
Cross Ref
- Yutaka Ishibashi, Sosuke Hoshino, Qi Zeng, Norishige Fukushima, and Shinji Sugawara. 2012. QoE assessment of fairness between players in networked game with olfaction. In Proceedings of the 11th Workshop on Network and Systems Support for Games (NetGames’12). IEEE, 1--2.Google Scholar
Digital Library
- Jeff Feasel, Mary C. Whitton, Jeremy D. Wendt. 2008. LLCM-WIP: Low-latency, continuous-motion walking-in-place. In Proceedings of the 2008 IEEE Symposium on 3D User Interfaces. IEEE, 97--104.Google Scholar
Digital Library
- Dennis Joele, G. van der Mast, and Marfa Carmen Juan-Lizandra. 2005. Development of an Augmented Reality System Using ARToolKit and User Invisible Markers. Research Assignment, Master Programme Media 8 Knowledge Engineering, Technical University of Valencia, Spain.Google Scholar
- Konstantina Kilteni, Jean-Marie Normand, Maria V. Sanchez-Vives, and Mel Slater. 2012. Extending body space in immersive virtual reality: A very long arm illusion. PloS One 7, 7 (2012), e40867.Google Scholar
Cross Ref
- Yuji Kusunose, Yutaka Ishibashi, Norishige Fukushima, and Shinji Sugawara. 2010. QoE assessment in networked air hockey game with haptic media. In Proceedings of the 9th Workshop on Network and Systems Support for Games. IEEE, 1--2.Google Scholar
Digital Library
- J. LaViola. 1999. Whole-hand and Speech Input in Virtual Environments. Unpublished Master’s Thesis, Department of Computer Science, Brown University, CS-99-15 (1999).Google Scholar
- Joseph J. LaViola Jr., Ernst Kruijff, Doug Bowman, Ivan P. Poupyrev, and Ryan P. McMahan. 2017. 3D User Interfaces: Theory and Practice (Second Edition). Addison-Wesley.Google Scholar
- Gun A. Lee and Mark Billinghurst. 2013. A component based framework for mobile outdoor AR applications. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry. ACM, 207--210.Google Scholar
- Wei-Po Lee, Li-Jen Liu, and Jeng-An Chiou. 2011. A component-based framework for rapidly developing online board games. Int. J. Comput. Applic. 33, 4 (2011), 293--302.Google Scholar
- Tsai-Yen Li and Chung-Chiang Cheng. 2008. Real-time camera planning for navigation in virtual environments. In Proceedings of the International Symposium on Smart Graphics. Springer, 118--129.Google Scholar
Digital Library
- Tsai-Yen Li . 1999. Automatically generating virtual guided tours. In Proceedings Computer Animation. IEEE, 99--106. DOI:10.1109/CA.1999.781203Google Scholar
- Joan Llobera, Mar González-Franco, Daniel Perez-Marcos, Josep Valls-Solé, Mel Slater, and Maria V. Sanchez-Vives. 2013. Virtual reality for assessment of patients suffering chronic pain: A case study. Experim. Brain Res. 225, 1 (2013), 105--117.Google Scholar
Cross Ref
- Gale M. Lucas, Evan Szablowski, Jonathan Gratch, Andrew Feng, Tiffany Huang, Jill Boberg, and Ari Shapiro. 2016. Do avatars that look like their users improve performance in a simulation? In Proceedings of the International Conference on Intelligent Virtual Agents. Springer, 351--354.Google Scholar
Cross Ref
- Mary Lou Maher and John S. Gero. 2002. Agent models of 3D virtual worlds. CUMINCAD, IEEE. DOI:10.1109/CW.2005.13Google Scholar
Digital Library
- Ryan P. McMahan, Chengyuan Lai, and Swaroop K. Pal. 2016. Interaction fidelity: The uncanny valley of virtual reality interactions. In Proceedings of the International Conference on Virtual, Augmented and Mixed Reality. Springer, 59--70.Google Scholar
- David L. Mills. 1991. Internet time synchronization: The network time protocol. IEEE Trans. Commun. 39, 10 (1991), 1482--1493.Google Scholar
Cross Ref
- Shailey Minocha and Christopher Hardy. 2016. Navigation and wayfinding in learning spaces in 3D virtual worlds. Learn. Virt. Worlds: Res. Applic. (2016), 3--41. DOI:10.15215/aupress/9781771991339.01dGoogle Scholar
- Alessandro Mulloni, Hartmut Seichter, and Dieter Schmalstieg. 2011. Handheld augmented reality indoor navigation with activity-based instructions. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. ACM, 211--220.Google Scholar
Digital Library
- Mahdi Nabiyouni, Ayshwarya Saktheeswaran, Doug A. Bowman, and Ambika Karanth. 2015. Comparing the performance of natural, semi-natural, and non-natural locomotion techniques in virtual reality. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI’15). IEEE, 3--10.Google Scholar
- Niels C. Nilsson, Stefania Serafin, Morten H. Laursen, Kasper S. Pedersen, Erik Sikström, and Rolf Nordahl. 2013. Tapping-in-place: Increasing the naturalness of immersive walking-in-place locomotion through novel gestural input. In Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI’13). IEEE, 31--38.Google Scholar
Cross Ref
- Niels Christian Nilsson, Stefania Serafin, Frank Steinicke, and Rolf Nordahl. 2018. Natural walking in virtual reality: A review. Comput. Entert. 16, 2 (2018), 8.Google Scholar
Digital Library
- Jan Ohlenburg, Iris Herbst, Irma Lindt, Thorsten Fröhlich, and Wolfgang Broll. 2004. The MORGAN framework: Enabling dynamic multi-user AR and VR projects. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology. ACM, 166--169.Google Scholar
Digital Library
- Jong-Seung Park. 2011. AR-Room: A rapid prototyping framework for augmented reality applications. Multim. Tools Applic. 55, 3 (2011), 725--746.Google Scholar
Digital Library
- Roma Patel and Deborah Tuck. 2008. Narrating the past: Virtual environments and narrative. 249--258. http://irep.ntu.ac.uk/id/eprint/15426/.Google Scholar
- Ryan A. Pavlik and Judy M. Vance. 2012. VR JuggLua: A framework for VR applications combining Lua, openscenegraph, and VR Juggler. In Proceedings of the 5th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS’12). IEEE, 29--35.Google Scholar
- Francesca Pazzaglia, Chiara Meneghetti, Enia Labate, and Lucia Ronconi. 2016. Are wayfinding self-efficacy, and pleasure in exploring related to shortcut finding? A study in a virtual environment. In Spatial Cognition X. Springer, 55--68.Google Scholar
- Wayne Piekarski and Bruce H. Thomas. 2003. An object-oriented software architecture for 3D mixed reality applications. In Proceedings of the 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE, 247--256.Google Scholar
- Suraj Raghuraman. 2017. i3DTI: Interactive 3D Tele-Immersion. Ph.D. Dissertation. The University of Texas at Dallas, Richardson, Texas, USA.Google Scholar
- M. J. Rorke. 2000. Designing and Implementing a Virtual Reality Interaction Framework. Ph.D. Dissertation. Rhodes University.Google Scholar
- Maria Roussou and Mel Slater. 2005. A virtual playground for the study of the role of interactivity in virtual learning environments. Perspectives 8 (2005), 9.Google Scholar
- Andrew Sears, Jinhuan Feng, Kwesi Oseitutu, and Claire-Marie Karat. 2003. Hands-free, speech-based navigation during dictation: Difficulties, consequences, and solutions. Hum.-comput. Interact. 18, 3 (2003), 229--257.Google Scholar
Digital Library
- Ari Shapiro, Andrew Feng, Ruizhe Wang, Hao Li, Mark Bolas, Gerard Medioni, and Evan Suma. 2014. Rapid avatar capture and simulation using commodity depth sensors. Comput. Anim. Virt. Worlds 25, 3--4 (2014), 201--211.Google Scholar
Digital Library
- Mya Sithu, Yutaka Ishibashi, Pingguo Huang, and Norishige Fukushima. 2015. QoE assessment of operability and fairness for soft objects in networked real-time game with haptic sense. In Proceedings of the 21st Asia-Pacific Conference on Communications (APCC’15). IEEE, 570--574.Google Scholar
Cross Ref
- Bernhard Spanlang, Xavi Navarro, Jean-Marie Normand, Sameer Kishore, Rodrigo Pizarro, and Mel Slater. 2013. Real time whole body motion mapping for avatars and robots. In Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology. ACM, 175--178.Google Scholar
Digital Library
- SteamVR. 2016. SteamVR. Retrieved from https://steamcommunity.com/steamvr.Google Scholar
- Richard Stoakley, Matthew J. Conway, and Randy Pausch. 1995. Virtual reality on a WIM: Interactive worlds in miniature. In Proceedings of the International Conference on Human Factors in Computing Systems (CHI’95), Vol. 95. 265--272.Google Scholar
Digital Library
- Ayano Tatematsu, Yutaka Ishibashi, Norishige Fukushima, and Shinji Sugawara. 2010. QoE assessment in haptic media, sound, and video transmission: Influences of network latency. In Proceedings of the IEEE International Workshop on Communications Quality and Reliability (CQR’10). IEEE, 1--6.Google Scholar
Cross Ref
- Unity Technologies. 2005. Unity3D Game Engine. Retrieved from https://unity3d.com/.Google Scholar
- Unreal. 2004. Unreal game engine. Retrieved from https://www.unrealengine.com.Google Scholar
- Narasimha Raghavan Veeraragavan, Leonardo Montecchi, Nicola Nostro, Roman Vitenberg, Hein Meling, and Andrea Bondavalli. 2016. Modeling QoE in dependable tele-immersive applications: A case study of world opera. IEEE Trans. Parallel Distrib. Syst. 27, 9 (2016), 2667--2681.Google Scholar
Digital Library
- Shanthi Vellingiri and Prabhakaran Balakrishnan. 2017. Modeling user quality of experience (QoE) through position discrepancy in multi-sensorial, immersive, collaborative environments. In Proceedings of the 8th ACM on Multimedia Systems Conference. ACM, 296--307.Google Scholar
Digital Library
- Shanthi Vellingiri and Balakrishnan Prabhakaran. 2018. Quantifying group navigation experience in collaborative augmented virtuality tours. In Proceedings of the 3rd International Workshop on Multimedia Alternate Realities. ACM, 3--8.Google Scholar
Digital Library
- Shanthi Vellingiri, Yuan Tian, and Balakrishnan Prabhakaran. 2014. A real time, distributed system with haptic interfaces for fine motor skill rehabilitation and its quality of experience. In Proceedings of the IEEE International Symposium on Haptic, Audio and Visual Environments and Games (HAVE’14). IEEE, 53--58.Google Scholar
Cross Ref
- Norman G. Vinson. 1999. Design guidelines for landmarks to support navigation in virtual environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 278--285.Google Scholar
Digital Library
- VRTK. 2016. VRTK. Retrieved from https://vrtoolkit.readme.io/.Google Scholar
- Daniel Wagner and Dieter Schmalstieg. 2007. Muddleware for prototyping mixed reality multiuser games. In Proceedings of the IEEE Virtual Reality Conference. IEEE, 235--238.Google Scholar
Cross Ref
- Martin White, Emmanuel Jay, Fotis Liarokapis, Costas Kostakis, and Paul Lister. 2001. A virtual interactive teaching environment using XML and augmented reality. Int. J. Elect. Eng. Educ. 38, 4 (2001), 316--329.Google Scholar
Cross Ref
Index Terms
SCeVE: A Component-based Framework to Author Mixed Reality Tours
Recommendations
User Experience Guidelines for Designing HMD Extended Reality Applications
Human-Computer Interaction – INTERACT 2019AbstractWith the rise of Extended Reality (XR) technologies, such as head mounted displays (HMD) for Virtual Reality (VR), Mixed Reality (MR), and Augmented Reality (AR), designers are presented with many unique challenges and opportunities when creating ...
Through Tinted Eyeglasses
EIC Roy Want introduces the special issue on cross-reality environments and discusses alternate realities including virtual reality, augmented reality, embodied virtuality, cross-reality, and mixed reality.
An approach for enabling the use of immersive virtual reality in desktop hybrid interfaces
IASTED-HCI '07: Proceedings of the Second IASTED International Conference on Human Computer InteractionHybrid User Interfaces, which create a heterogeneous environment providing different interaction forms and devices, may be enhanced by exploring more extensively the mixed reality continuum, which ranges from the real world to the complete virtual world,...






Comments