Abstract

In this paper, we present a system that allows a user with a head-mounted display (HMD) to communicate and collaborate with spectators outside of the headset. We evaluate its impact on task performance, immersion, and collaborative interaction. Our solution targets scenarios like live presentations or multi-user collaborative systems, where it is not convenient to develop a VR multiplayer experience and supply each user (and spectator) with an HMD. The spectator views the virtual world on a large-scale tiled video wall and is given the ability to control the orientation of their own virtual camera. This allows spectators to stay focused on the immersed user's point of view or freely look around the environment. To improve collaboration between users, we implemented a pointing system where a spectator can point at objects on the screen, which maps an indicator directly onto the objects in the virtual world. We conducted a user study to investigate the influence of rotational camera decoupling and pointing gestures in the context of HMD-immersed and non-immersed users utilizing a large-scale display. Our results indicate that camera decoupling and pointing positively impacts collaboration. A decoupled view is preferable in situations where both users need to indicate objects of interest in the scene, such as presentations and joint-task scenarios, as it requires a shared reference space. A coupled view, on the other hand, is preferable in synchronous interactions such as remote-assistant scenarios.
Supplemental Material
- Richard A. Armstrong. 2014. When to use the Bonferroni correction. Ophthalmic and Physiological Optics , Vol. 34, 5 (2014), 502--508. https://doi.org/10.1111/opo.12131Google Scholar
Cross Ref
- Teo Babic, Harald Reiterer, and Michael Haller. 2018. Pocket6: A 6DoF Controller Based On A Simple Smartphone Application. In Proceedings of the Symposium on Spatial User Interaction (Berlin, Germany) (SUI '18). ACM, New York, NY, USA, 2--10. https://doi.org/10.1145/3267782.3267785Google Scholar
Digital Library
- Frank Biocca, C. Harms, and Jennifer Gregg. 2001. The Networked Minds Measure of Social Presence: Pilot Test of the Factor Structure and Concurrent Validity. Technical Report. Michigan State University. https://www.researchgate.net/publication/200772411_The_Networked_Minds_Measure_of_Social_Presence_Pilot_Test_of_the_Factor_Structure_and_Concurrent_ValidityGoogle Scholar
- Doug A. Bowman, Brian Badillo, and Dhruv Manek. 2007. Evaluating the Need for Display-Specific and Device-Specific 3D Interaction Techniques. In Virtual Reality (Lecture Notes in Computer Science ), , Randall Shumaker (Ed.). Springer, Berlin, Heidelberg, 195--204. https://doi.org/10.1007/978--3--540--73335--5_22Google Scholar
- John Brooke. 1996. SUS: A 'Quick and Dirty' Usability Scale. CRC Press, London, England. https://doi.org/10.1201/9781498710411--35Google Scholar
- Liwei Chan and Kouta Minamizawa. 2017. FrontFace: Facilitating Communication between HMD Users and Outsiders Using Front-Facing-Screen HMDs. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (Vienna, Austria) (MobileHCI '17). ACM, New York, NY, USA, 1--5. https://doi.org/10.1145/3098279.3098548Google Scholar
Digital Library
- Carolina Cruz-Neira, Daniel J. Sandin, Thomas A. DeFanti, Robert V. Kenyon, and John C. Hart. 1992. The CAVE: Audio Visual Experience Automatic Virtual Environment . Commun. ACM , Vol. 35, 6 (June 1992), 64--72. https://doi.org/10.1145/129888.129892Google Scholar
Digital Library
- James J. Cummings and Jeremy N. Bailenson. 2016. How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology , Vol. 19, 2 (2016), 272--309. https://doi.org/10.1080/15213269.2015.1015740Google Scholar
Cross Ref
- Alex Endert, Lauren Bradel, Jessica Zeitz, Christopher Andrews, and Chris North. 2012. Designing Large High-resolution Display Workspaces. In Proceedings of the International Working Conference on Advanced Visual Interfaces (Capri Island, Italy) (AVI '12). ACM, New York, NY, USA, 58--65. https://doi.org/10.1145/2254556.2254570Google Scholar
Digital Library
- Epic Games. 2020 a. UE4 Documentation: Using a Single Line Trace (Raycast) by Channel. Epic Games. https://docs.unrealengine.com/en-US/InteractiveExperiences/Tracing/HowTo/SingleLineTraceByChannel/index.htmlGoogle Scholar
- Epic Games. 2020 b. UE4 Documentation: Virtual Reality Spectator Screen . Epic Games. https://docs.unrealengine.com/en-US/Platforms/VR/DevelopVR/VRSpectatorScreen/index.htmlGoogle Scholar
- Syafizwan Faroque, Ben Horan, Michael Mortimer, and Mulyoto Pangestu. 2016. Large-scale Virtual Reality micro-robotic cell injection training. In 2016 World Automation Congress (WAC) (Rio Grande, PR, USA). IEEE, New York, NY, USA, 1--6. https://doi.org/10.1109/WAC.2016.7583006Google Scholar
Cross Ref
- Paulo Gallotti, Alberto Raposo, and Luciano Soares. 2011. v-Glove: A 3D Virtual Touch Interface . In Proceedings of the 2011 XIII Symposium on Virtual Reality (Washington, DC, USA) (SVR '11). IEEE, New York, NY, USA, 242--251. https://doi.org/10.1109/svr.2011.21Google Scholar
Digital Library
- Jerônimo Gustavo Grandi, Henrique Galvan Debarba, and Anderson Maciel. 2019. Characterizing Asymmetric Collaborative Interactions in Virtual and Augmented Realities. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Osaka, Japan). IEEE, New York, NY, USA, 127--135. https://doi.org/10.1109/VR.2019.8798080 ISSN: 2642--5254.Google Scholar
- Jan Gugenheimer, Evgeny Stemasov, Julian Frommel, and Enrico Rukzio. 2017. ShareVR: Enabling Co-Located Experiences for Virtual Reality Between HMD and non-HMD Users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). ACM, New York, NY, USA, 4021--4033. https://doi.org/10.1145/3025453.3025683Google Scholar
Digital Library
- Jan Gugenheimer, Evgeny Stemasov, Harpreet Sareen, and Enrico Rukzio. 2018. FaceDisplay: Towards Asymmetric Multi-User Interaction for Nomadic Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). ACM, New York, NY, USA, 1--13. https://doi.org/10.1145/3173574.3173628Google Scholar
Digital Library
- Joe Guna, Gregor Gerak, Iztok Humar, Jeungeun Song, Janko Drnovek, and Matev Poganik. 2019. Influence of video content type on users' virtual reality sickness perception and physiological response. Future Generation Computer Systems , Vol. 91 (2019), 263 -- 276. https://doi.org/10.1016/j.future.2018.08.049Google Scholar
Digital Library
- Kunal Gupta, Gun A Lee, and Mark Billinghurst. 2016a. Do you see what I see? The effect of gaze tracking on task space remote collaboration. IEEE transactions on visualization and computer graphics , Vol. 22, 11 (2016), 2413--2422.Google Scholar
Digital Library
- Kunal Gupta, Gun A. Lee, and Mark Billinghurst. 2016b. Do You See What I See? The Effect of Gaze Tracking on Task Space Remote Collaboration. IEEE Transactions on Visualization and Computer Graphics , Vol. 22, 11 (2016), 2413--2422. https://doi.org/10.1109/TVCG.2016.2593778Google Scholar
Digital Library
- Carl Gutwin, Saul Greenberg, and Mark Roseman. 1996. Workspace awareness in real-time distributed groupware: Framework, widgets, and evaluation. In People and Computers XI . Springer, London, 281--298.Google Scholar
- Hikaru Ibayashi, Yuta Sugiura, Daisuke Sakamoto, Natsuki Miyata, Mitsunori Tada, Takashi Okuma, Takeshi Kurata, Masaaki Mochimaru, and Takeo Igarashi. 2015. Dollhouse VR: a multi-view, multi-user collaborative design workspace with VR technology. In SIGGRAPH Asia 2015 Emerging Technologies (Kobe, Japan) (SA '15). ACM, New York, NY, USA, 1--2. https://doi.org/10.1145/2818466.2818480Google Scholar
Digital Library
- Akira Ishii, Ippei Suzuki, Masaya Tsuruta, Shuta Nakamae, Junichi Suzuki, and Yoichi Ochiai. 2018. ReverseCAVE: CAVE-Based Visualization Methods of Public VR Towards Shareable VR Experience. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA '18). Association for Computing Machinery, New York, NY, USA, 1. https://doi.org/10.1145/3170427.3186593Google Scholar
Digital Library
- Ricardo Jota, Bruno R. de Araújo, Luís C. Bruno, João M. Pereira, and Joaquim A. Jorge. 2010a. IMMIView: a multi-user solution for design review in real-time. Journal of Real-Time Image Processing , Vol. 5, 2 (June 2010), 91--107. https://doi.org/10.1007/s11554-009-0141--1Google Scholar
Cross Ref
- Ricardo Jota, Miguel A. Nacenta, Joaquim A. Jorge, Sheelagh Carpendale, and Saul Greenberg. 2010b. A comparison of ray pointing techniques for very large displays. In Proceedings of Graphics Interface 2010 (GI '10). Canadian Information Processing Society, CAN, 269--276. https://dl.acm.org/doi/10.5555/1839214.1839261Google Scholar
Digital Library
- Behrang Keshavarz and Heiko Hecht. 2011. Axis rotation and visually induced motion sickness: the role of combined roll, pitch, and yaw motion. Aviation, space, and environmental medicine , Vol. 82, 11 (2011), 1023--1029.Google Scholar
- Andre Kunert, Tim Weissker, Bernd Froehlich, and Alexander Kulik. 2019. Multi-window 3d interaction for collaborative virtual reality. IEEE transactions on visualization and computer graphics , Vol. 26, 11 (2019), 3271--3284.Google Scholar
- Andre Kunert, Tim Weissker, Bernd Froehlich, and Alexander Kulik. 2020. Multi-Window 3D Interaction for Collaborative Virtual Reality. IEEE Transactions on Visualization and Computer Graphics , Vol. 26, 11 (2020), 3271--3284. https://doi.org/10.1109/TVCG.2019.2914677Google Scholar
Cross Ref
- Ed Lantz. 1996. The future of virtual reality: head mounted displays versus spatially immersive displays (panel). In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques (SIGGRAPH '96). ACM, New York, NY, USA, 485--486. https://doi.org/10.1145/237170.237289Google Scholar
Digital Library
- Ed Lantz. 2007. A survey of large-scale immersive displays. In Proceedings of the 2007 workshop on Emerging displays technologies: images and beyond: the future of displays and interacton (San Diego, California, USA) (EDT '07). ACM, New York, NY, USA, 1--es. https://doi.org/10.1145/1278240.1278241Google Scholar
Digital Library
- JJ-W Lin, Henry Been-Lirn Duh, Donald E. Parker, Habib Abi-Rached, and Thomas A. Furness. 2002. Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. In Proceedings IEEE Virtual Reality 2002 (Orlando, FL, USA). IEEE, New York, NY, USA, 164--171. https://doi.org/10.1109/VR.2002.996519Google Scholar
Cross Ref
- Lars Lischke, Sven Mayer, Katrin Wolf, Niels Henze, Harald Reiterer, and Albrecht Schmidt. 2016. Screen Arrangements and Interaction Areas for Large Display Work Places. In Proceedings of the 5th ACM International Symposium on Pervasive Displays (Oulu, Finland) (PerDis '16). ACM, New York, NY, USA, 228--234. https://doi.org/10.1145/2914920.2915027Google Scholar
Digital Library
- Sreejith M, Siddharth Rakesh, Samik Gupta, Samprit Biswas, and Partha Pratim Das. 2015. Real-time hands-free immersive image navigation system using Microsoft Kinect 2.0 and Leap Motion Controller. In 2015 Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG) (Patna, India). IEEE, New York, NY, USA, 1--4. https://doi.org/10.1109/NCVPRIPG.2015.7489999Google Scholar
Cross Ref
- Narges Mahyar and Melanie Tory. 2014. Supporting communication and coordination in collaborative sensemaking. IEEE transactions on visualization and computer graphics , Vol. 20, 12 (2014), 1633--1642.Google Scholar
Cross Ref
- Christian Mai, Lukas Rambold, and Mohamed Khamis. 2017. TransparentHMD: Revealing the HMD User's Face to Bystanders. In Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia (Stuttgart, Germany) (MUM '17). ACM, New York, NY, USA, 515--520. https://doi.org/10.1145/3152832.3157813Google Scholar
Digital Library
- Marilyn M. Mantei, Ronald M. Baecker, Abigail J. Sellen, William A. S. Buxton, Thomas Milligan, and Barry Wellman. 1991. Experiences in the Use of a Media Space. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New Orleans, Louisiana, USA) (CHI '91). Association for Computing Machinery, New York, NY, USA, 203--208. https://doi.org/10.1145/108844.108888Google Scholar
Digital Library
- Anders Markussen, Sebastian Boring, Mikkel R. Jakobsen, and Kasper Hornbæk. 2016. Off-Limits: Interacting Beyond the Boundaries of Large Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). ACM, New York, NY, USA, 5862--5873. https://doi.org/10.1145/2858036.2858083Google Scholar
Digital Library
- Daniel Medeiros, Rafael K. dos Anjos, Daniel Mendes, Jo ao Madeiras Pereira, Alberto Raposo, and Joaquim Jorge. 2018. Keep My Head on My Shoulders! Why Third-Person is Bad for Navigation in VR. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (Tokyo, Japan) (VRST '18). Association for Computing Machinery, New York, NY, USA, Article 16, bibinfonumpages10 pages. https://doi.org/10.1145/3281505.3281511Google Scholar
Digital Library
- Daniel Medeiros, Eduardo Ribeiro, Peter Dam, Rodrigo Pinheiro, Thiago Motta, Manuel Loaiza, and Alberto B. Raposo. 2012. A Case Study on the Implementation of the 3C Collaboration Model in Virtual Environments. In 2012 14th Symposium on Virtual and Augmented Reality (Rio de Janeiro, Brazil). IEEE, New York, NY, USA, 147--154. https://doi.org/10.1109/SVR.2012.28Google Scholar
Digital Library
- Daniel Medeiros, Lucas Teixeira, Felipe Carvalho, Ismael Santos, and Alberto Raposo. 2013. A tablet-based 3D interaction tool for virtual engineering environments. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (Hong Kong, Hong Kong) (VRCAI '13). ACM, New York, NY, USA, 211--218. https://doi.org/10.1145/2534329.2534349Google Scholar
Digital Library
- Luís A. R. Neng and Teresa Chambel. 2010. Get around 360° Hypervideo. In Proceedings of the 14th International Academic MindTrek Conference: Envisioning Future Media Environments (Tampere, Finland) (MindTrek '10). ACM, New York, NY, USA, 119--122. https://doi.org/10.1145/1930488.1930512Google Scholar
Digital Library
- Ohan Oda, Carmine Elvezio, Mengu Sukan, Steven Feiner, and Barbara Tversky. 2015. Virtual Replicas for Remote Assistance in Virtual and Augmented Reality. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (Charlotte, NC, USA) (UIST '15). Association for Computing Machinery, New York, NY, USA, 405--415. https://doi.org/10.1145/2807442.2807497Google Scholar
Digital Library
- Emilee Patrick, Dennis Cosgrove, Aleksandra Slavkovic, Jennifer A. Rode, Thom Verratti, and Greg Chiselko. 2000. Using a Large Projection Screen As an Alternative to Head-mounted Displays for Virtual Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (The Hague, The Netherlands) (CHI '00). ACM, New York, NY, USA, 478--485. https://doi.org/10.1145/332040.332479Google Scholar
Digital Library
- Amy Pavel, Björn Hartmann, and Maneesh Agrawala. 2017. Shot Orientation Controls for Interactive Cinematography with 360 Video. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST '17). ACM, New York, NY, USA, 289--297. https://doi.org/10.1145/3126594.3126636Google Scholar
Digital Library
- Taehyun Rhee, Stephen Thompson, Daniel Medeiros, Rafael Kuffner Dos Anjos, and Andrew Chalmers. 2020. Augmented Virtual Teleportation for High-Fidelity Telecollaboration . IEEE Transactions on Visualization and Computer Graphics , Vol. 26, 5 (2020), 1923--1933. https://doi.org/10.1109/tvcg.2020.2973065Google Scholar
Cross Ref
- Sarah Sharples, Sue Cobb, Amanda Moody, and John R. Wilson. 2008. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays , Vol. 29, 2 (2008), 58 -- 69. https://doi.org/10.1016/j.displa.2007.09.005Google Scholar
Cross Ref
- Andreas Simon. 2005. First-person Experience and Usability of Co-located Interaction in a Projection-based Virtual Environment . In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (Monterey, CA, USA) (VRST '05). ACM, New York, NY, USA, 23--30. https://doi.org/10.1145/1101616.1101622Google Scholar
Digital Library
- Mel Slater, Martin Usoh, and Anthony Steed. 1994. Depth of presence in virtual environments. Presence: Teleoperators & Virtual Environments , Vol. 3, 2 (1994), 130--144.Google Scholar
Digital Library
- Maurício Sousa, Rafael Kuffner dos Anjos, Daniel Mendes, Mark Billinghurst, and Joaquim Jorge. 2019. Warping Deixis: Distorting Gestures to Enhance Collaboration (CHI '19). ACM, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300838Google Scholar
Digital Library
- Desney S. Tan, Darren Gergle, Peter Scupelli, and Randy Pausch. 2003. With Similar Visual Angles, Larger Displays Improve Spatial Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI '03). ACM, New York, NY, USA, 217--224. https://doi.org/10.1145/642611.642650Google Scholar
Digital Library
- Ikuko Tsubaki, Toshiya Morita, Takahiro Saito, and Kiyoharu Aizawa. 2005. An adaptive video stabilization method for reducing visually induced motion sickness. In IEEE International Conference on Image Processing 2005 (Genova, Italy), Vol. 3. IEEE, New York, NY, USA, III--497. https://doi.org/10.1109/ICIP.2005.1530437Google Scholar
Cross Ref
- Colin Ware and Steven Osborne. 1990. Exploration and Virtual Camera Control in Virtual Three Dimensional Environments . SIGGRAPH Comput. Graph. , Vol. 24, 2 (Feb. 1990), 175--183. https://doi.org/10.1145/91394.91442Google Scholar
Digital Library
- Haijun Xia, Sebastian Herscher, Ken Perlin, and Daniel Wigdor. 2018. Spacetime: Enabling Fluid Individual and Collaborative Editing in Virtual Reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST '18). ACM, New York, NY, USA, 853--866. https://doi.org/10.1145/3242587.3242597Google Scholar
Digital Library
Index Terms
Spectator View: Enabling Asymmetric Interaction between HMD Wearers and Spectators with a Large Display
Recommendations
HMD Light: Sharing In-VR Experience via Head-Mounted Projector for Asymmetric Interaction
UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and TechnologyWe present HMD Light, a proof-of-concept Head-Mounted Display (HMD) implementation that reveals the Virtual Reality (VR) user's experience in the physical environment to facilitate communication between VR and external users in a mobile VR context. ...
Effects of dynamic field-of-view restriction on cybersickness and presence in HMD-based virtual reality
AbstractThe phenomenon of cybersickness is currently hindering the mass market adoption of head-mounted display (HMD) virtual reality (VR) technologies. This study examined the effects of dynamic field-of-view (FOV) restriction on the cybersickness ...
VARiable HMD: Optical See-Through HMD for AR and VR
UIST '19 Adjunct: Adjunct Proceedings of the 32nd Annual ACM Symposium on User Interface Software and TechnologyVirtual reality (VR) and augmented reality (AR) with head-mounted displays (HMDs) are rapidly becoming popular. Unfortunately, optical see-through HMDs for AR are not compatible with VR and users must prepare the other type of HMD to experience VR. We ...






Comments