Abstract

The availability of consumer-level devices for both visualising and interacting with Virtual Reality (VR) environments opens the opportunity to introduce more immersive contents and experiences, even on the web. For reaching a wider audience, developing VR applications in a web environment requires a flexible adaptation to the different input and output devices that are currently available. This paper examines the required support and explores how to develop VR applications based on web technologies that can adapt to different VR devices. We summarize the main engineering challenges and we describe a flexible framework for integrating and exploiting various VR devices for both input and output. Using such framework, we describe how we re-implemented four manipulation techniques from the literature to enable them within the same application, providing details on how we adapted its parts for different input and output devices such as Kinect and Leap Motion. Finally, we briefly examine the usability of the final application using our framework.
- Ferran Argelaguet and Carlos Andujar. 2013. A survey of 3D object selection techniques for virtual environments. Computers & Graphics 37, 3 (2013), 121 -- 136. Google Scholar
Digital Library
- Stephen Block, Rich Tibbett, Andrei Popescu, and Tim Volodine. 2016. DeviceOrientation Event Specification. Candidate Recommendation. W3C. https://www.w3.org/TR/2016/CR-orientation-event-20160818/.Google Scholar
- Doug Bowman, Ernst Kruijff, Joseph J LaViola Jr, and Ivan Poupyrev. 2004. 3D User Interfaces: Theory and Practice, CourseSmart eTextbook. Addison-Wesley. Google Scholar
Digital Library
- Doug A. Bowman and Larry F. Hodges. 1997. An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments. In Proceedings of the 1997 Symposium on Interactive 3D Graphics (I3D '97). ACM, New York, NY, USA, 35--ff. Google Scholar
Digital Library
- Doug A. Bowman, Ryan P. McMahan, and Eric D. Ragan. 2012. Questioning Naturalism in 3D User Interfaces. Commun. ACM 55, 9 (Sept. 2012), 78--88. Google Scholar
Digital Library
- Alessandro Carcangiu, Gianni Fenu, and Lucio Davide Spano. 2016. A Design Pattern for Multimodal and Multidevice User Interfaces. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '16). ACM, New York, NY, USA, 177--182. Google Scholar
Digital Library
- David Catuhe and David Rousset. 2013. Babylon JS. https://github.com/BabylonJS. (2013).Google Scholar
- Cooper Davies, Jade White, Alec McAllister, Adam Saroka, Omar Addam, Fatemeh Hendijani Fard, and Frank Maurer. 2016. A Toolkit for Building Collaborative Immersive Multi-Surface Applications. In Proceedings of the 2016 ACM on Interactive Surfaces and Spaces (ISS '16). ACM, New York, NY, USA, 485--488. Google Scholar
Digital Library
- I. Fette and A. Melnikov. 2011. The WebSocket Protocol. RFC 6455. RFC Editor. 1--70 pages. https://tools.ietf.org/html/ rfc6455Google Scholar
- Simon Fothergill, Helena Mentis, Pushmeet Kohli, and Sebastian Nowozin. 2012. Instructing People for Training Gestural Interactive Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 1737--1746. Google Scholar
Digital Library
- Mozilla Foundation. 2017. A-Frame. https://aframe.io/. (2017).Google Scholar
- Giuseppe Ghiani, Marco Manca, and Fabio Paternò. 2015. Authoring Context-dependent Cross-device User Interfaces Based on Trigger/Action Rules. In Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia (MUM '15). ACM, New York, NY, USA, 313--322. Google Scholar
Digital Library
- Khronos Group. 2016. OpenGL ES Specification. http://www.khronos.org/registry/OpenGL/index_es.php. (2016).Google Scholar
- Maria Husmann, Nina Heyder, and Moira C. Norrie. 2016. Is a Framework Enough?: Cross-device Testing and Debugging. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '16). ACM, New York, NY, USA, 251--262. Google Scholar
Digital Library
- Dean Jackson and Jeff Gilbert. 2017. WebGL 2 Specification. Editor's Draft. Kronos Group. https://www.khronos.org/registry/webgl/specs/latest/2.0/.Google Scholar
- Taeho Kim and Jinah Park. 2014. 3D Object Manipulation Using Virtual Handles with a Grabbing Metaphor. IEEE Computer Graphics and Applications 34, 3 (May 2014), 30--38. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm? arnumber=6762796Google Scholar
Cross Ref
- Joseph J. LaViola and Daniel F. Keefe. 2011. 3D Spatial Interaction: Applications for Art, Design, and Science. In ACM SIGGRAPH 2011 Courses (SIGGRAPH '11). ACM, New York, NY, USA, Article 1, 75 pages. Google Scholar
Digital Library
- Shahzad Malik and Joe Laszlo. 2004. Visual Touchpad: A Two-handed Gestural Input Device. In Proceedings of the 6th International Conference on Multimodal Interfaces (ICMI '04). ACM, New York, NY, USA, 289--296. Google Scholar
Digital Library
- Daniel Mendes, Filipe Relvas, Alfredo Ferreira, and Joaquim Jorge. 2016. The benefits of DOF separation in mid-air 3D object manipulation. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology. ACM, 261--268. Google Scholar
Digital Library
- Mark R Mine, Frederick P Brooks Jr, and Carlo H Sequin. 1997. Moving objects in space: exploiting proprioception in virtual-environment interaction. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 19--26. Google Scholar
Digital Library
- Michael Nebeling, Theano Mintsi, Maria Husmann, and Moira Norrie. 2014. Interactive Development of Cross-device User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2793--2802. Google Scholar
Digital Library
- Michael Nebeling, Elena Teunissen, Maria Husmann, and Moira C. Norrie. 2014. XDKinect: Development Framework for Cross-device Interaction Using Kinect. In Proceedings of the 2014 ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '14). ACM, New York, NY, USA, 65--74. Google Scholar
Digital Library
- Thi Thuong Huyen Nguyen, Thierry Duval, and Charles Pontonnier. 2014. A new direct manipulation technique for immersive 3d virtual environments. In ICAT-EGVE 2014: the 24th International Conference on Artificial Reality and Telexistence and the 19th Eurographics Symposium on Virtual Environments. 8. Google Scholar
Digital Library
- Jeffrey S. Pierce and Jeffrey Nichols. 2008. An Infrastructure for Extending Applications' User Experiences Across Multiple Personal Devices. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (UIST '08). ACM, New York, NY, USA, 101--110. Google Scholar
Digital Library
- Playcanvas. 2017. Playcanvas game engine. https://playcanvas.com/. (2017).Google Scholar
- Ivan Poupyrev, Mark Billinghurst, Suzanne Weghorst, and Tadao Ichikawa. 1996. The Go-go Interaction Technique: Non-linear Mapping for Direct Manipulation in VR. In Proceedings of the 9th Annual ACM Symposium on User Interface Software and Technology (UIST '96). ACM, New York, NY, USA, 79--80. Google Scholar
Digital Library
- Teddy Seyed, Alaa Azazi, Edwin Chan, Yuxi Wang, and Frank Maurer. 2015. SoD-Toolkit: A Toolkit for Interactively Prototyping and Developing Multi-Sensor, Multi-Device Environments. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). ACM, New York, NY, USA, 171--180. Google Scholar
Digital Library
- Peng Song, Wooi Boon Goh, William Hutama, Chi-Wing Fu, and Xiaopei Liu. 2012. A Handle Bar Metaphor for Virtual Object Manipulation with Mid-air Interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 1297--1306. Google Scholar
Digital Library
- Richard Stoakley, Matthew J. Conway, and Randy Pausch. 1995. Virtual Reality on a WIM: Interactive Worlds in Miniature. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 265--272. Google Scholar
Digital Library
- Three JS. 2017. Three JS documentation. https://threejs.org/. (2017). Accessed: 2017-07--16.Google Scholar
- Radu-Daniel Vatavu. 2012. User-defined Gestures for Free-hand TV Control. In Proceedings of the 10th European Conference on Interactive Tv and Video (EuroiTV '12). ACM, New York, NY, USA, 45--48. Google Scholar
Digital Library
- Radu-Daniel Vatavu and Stefan-Gheorghe Pentiuc. 2008. Interactive Coffee Tables: Interfacing TV within an Intuitive, Fun and Shared Experience. Springer Berlin Heidelberg, Berlin, Heidelberg, 183--187.Google Scholar
- Radu-Daniel Vatavu, Ştefan-Gheorghe Pentiuc, Christophe Chaillou, Laurent Grisoni, and Samuel Degrande. 2006. Visual recognition of hand postures for interacting with virtual environments. Advances in Electrical and Computer Engineering 6, 13 (2006), 55--58.Google Scholar
- Radu-Daniel Vatavu and Ionut-Alexandru Zaiti. 2014. Leap Gestures for TV: Insights from an Elicitation Study. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video (TVX '14). ACM, New York, NY, USA, 131--138. Google Scholar
Digital Library
- Manuel Veit, Antonio Capobianco, and Dominique Bechmann. 2009. Influence of Degrees of Freedom's Manipulation on Performances During Orientation Tasks in Virtual Reality Environments. In Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology (VRST '09). ACM, New York, NY, USA, 51--58. Google Scholar
Digital Library
- Vladimir Vukicevic, Brandon Jones, Kearwood Gilbert, and Chris Van Wiemeersch. 2017. WebVR Editor's Draft. https://w3c.github.io/webvr/spec/1.1/. (2017). Accessed: 2017-07--16.Google Scholar
- Rick Waldron, Mikhail Pozdnyakov, and Alexander Shalamov. 2017. Generic Sensor API. Editor's Draft. W3C. https://w3c.github.io/sensors/.Google Scholar
- Andrew D. Wilson. 2006. Robust Computer Vision-based Detection of Pinching for One and Two-handed Gesture Input. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (UIST '06). ACM, New York, NY, USA, 255--258. Google Scholar
Digital Library
- Chadwick A Wingrave, Doug A Bowman, and Naren Ramakrishnan. 2002. Towards preferences in virtual environment interfaces. In EGVE, Vol. 2. 63--72 Google Scholar
Digital Library
Index Terms
Web5VR: A Flexible Framework for Integrating Virtual Reality Input and Output Devices on the Web
Recommendations
TranSection: Hand-Based Interaction for Playing a Game within a Virtual Reality Game
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing SystemsTranSection is a new VR interface design concept with cross-dimensional virtual reality interface. It is a 2D puzzle platformer game within a 3D virtual environment. By incorporating the head tracking and visualization capabilities of Oculus Rift and ...
Evaluation of visual feedback techniques for virtual grasping with bare hands using Leap Motion and Oculus Rift
Bare hand interaction (BHI) allows users to use their hands and fingers to interact with digital content without any attached devices or accessories. For BHI to realize widespread adoption, interaction techniques for fundamental operations, like grasp-...
Copyright page
VR '10: Proceedings of the 2010 IEEE Virtual Reality ConferenceThe following topics are dealt with: augmented reality; augmented virtual environments; egocentric space-distorting visualizations; immersive virtual reality; virtual experience test; and rendering method.






Comments