skip to main content
research-article

Spectator View: Enabling Asymmetric Interaction between HMD Wearers and Spectators with a Large Display

Authors Info & Claims
Published:05 November 2021Publication History
Skip Abstract Section

Abstract

In this paper, we present a system that allows a user with a head-mounted display (HMD) to communicate and collaborate with spectators outside of the headset. We evaluate its impact on task performance, immersion, and collaborative interaction. Our solution targets scenarios like live presentations or multi-user collaborative systems, where it is not convenient to develop a VR multiplayer experience and supply each user (and spectator) with an HMD. The spectator views the virtual world on a large-scale tiled video wall and is given the ability to control the orientation of their own virtual camera. This allows spectators to stay focused on the immersed user's point of view or freely look around the environment. To improve collaboration between users, we implemented a pointing system where a spectator can point at objects on the screen, which maps an indicator directly onto the objects in the virtual world. We conducted a user study to investigate the influence of rotational camera decoupling and pointing gestures in the context of HMD-immersed and non-immersed users utilizing a large-scale display. Our results indicate that camera decoupling and pointing positively impacts collaboration. A decoupled view is preferable in situations where both users need to indicate objects of interest in the scene, such as presentations and joint-task scenarios, as it requires a shared reference space. A coupled view, on the other hand, is preferable in synchronous interactions such as remote-assistant scenarios.

Skip Supplemental Material Section

Supplemental Material

V5iss485wVF.mp4

Supplemental video

References

  1. Richard A. Armstrong. 2014. When to use the Bonferroni correction. Ophthalmic and Physiological Optics , Vol. 34, 5 (2014), 502--508. https://doi.org/10.1111/opo.12131Google ScholarGoogle ScholarCross RefCross Ref
  2. Teo Babic, Harald Reiterer, and Michael Haller. 2018. Pocket6: A 6DoF Controller Based On A Simple Smartphone Application. In Proceedings of the Symposium on Spatial User Interaction (Berlin, Germany) (SUI '18). ACM, New York, NY, USA, 2--10. https://doi.org/10.1145/3267782.3267785Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Frank Biocca, C. Harms, and Jennifer Gregg. 2001. The Networked Minds Measure of Social Presence: Pilot Test of the Factor Structure and Concurrent Validity. Technical Report. Michigan State University. https://www.researchgate.net/publication/200772411_The_Networked_Minds_Measure_of_Social_Presence_Pilot_Test_of_the_Factor_Structure_and_Concurrent_ValidityGoogle ScholarGoogle Scholar
  4. Doug A. Bowman, Brian Badillo, and Dhruv Manek. 2007. Evaluating the Need for Display-Specific and Device-Specific 3D Interaction Techniques. In Virtual Reality (Lecture Notes in Computer Science ), , Randall Shumaker (Ed.). Springer, Berlin, Heidelberg, 195--204. https://doi.org/10.1007/978--3--540--73335--5_22Google ScholarGoogle Scholar
  5. John Brooke. 1996. SUS: A 'Quick and Dirty' Usability Scale. CRC Press, London, England. https://doi.org/10.1201/9781498710411--35Google ScholarGoogle Scholar
  6. Liwei Chan and Kouta Minamizawa. 2017. FrontFace: Facilitating Communication between HMD Users and Outsiders Using Front-Facing-Screen HMDs. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (Vienna, Austria) (MobileHCI '17). ACM, New York, NY, USA, 1--5. https://doi.org/10.1145/3098279.3098548Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Carolina Cruz-Neira, Daniel J. Sandin, Thomas A. DeFanti, Robert V. Kenyon, and John C. Hart. 1992. The CAVE: Audio Visual Experience Automatic Virtual Environment . Commun. ACM , Vol. 35, 6 (June 1992), 64--72. https://doi.org/10.1145/129888.129892Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. James J. Cummings and Jeremy N. Bailenson. 2016. How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology , Vol. 19, 2 (2016), 272--309. https://doi.org/10.1080/15213269.2015.1015740Google ScholarGoogle ScholarCross RefCross Ref
  9. Alex Endert, Lauren Bradel, Jessica Zeitz, Christopher Andrews, and Chris North. 2012. Designing Large High-resolution Display Workspaces. In Proceedings of the International Working Conference on Advanced Visual Interfaces (Capri Island, Italy) (AVI '12). ACM, New York, NY, USA, 58--65. https://doi.org/10.1145/2254556.2254570Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Epic Games. 2020 a. UE4 Documentation: Using a Single Line Trace (Raycast) by Channel. Epic Games. https://docs.unrealengine.com/en-US/InteractiveExperiences/Tracing/HowTo/SingleLineTraceByChannel/index.htmlGoogle ScholarGoogle Scholar
  11. Epic Games. 2020 b. UE4 Documentation: Virtual Reality Spectator Screen . Epic Games. https://docs.unrealengine.com/en-US/Platforms/VR/DevelopVR/VRSpectatorScreen/index.htmlGoogle ScholarGoogle Scholar
  12. Syafizwan Faroque, Ben Horan, Michael Mortimer, and Mulyoto Pangestu. 2016. Large-scale Virtual Reality micro-robotic cell injection training. In 2016 World Automation Congress (WAC) (Rio Grande, PR, USA). IEEE, New York, NY, USA, 1--6. https://doi.org/10.1109/WAC.2016.7583006Google ScholarGoogle ScholarCross RefCross Ref
  13. Paulo Gallotti, Alberto Raposo, and Luciano Soares. 2011. v-Glove: A 3D Virtual Touch Interface . In Proceedings of the 2011 XIII Symposium on Virtual Reality (Washington, DC, USA) (SVR '11). IEEE, New York, NY, USA, 242--251. https://doi.org/10.1109/svr.2011.21Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Jerônimo Gustavo Grandi, Henrique Galvan Debarba, and Anderson Maciel. 2019. Characterizing Asymmetric Collaborative Interactions in Virtual and Augmented Realities. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (Osaka, Japan). IEEE, New York, NY, USA, 127--135. https://doi.org/10.1109/VR.2019.8798080 ISSN: 2642--5254.Google ScholarGoogle Scholar
  15. Jan Gugenheimer, Evgeny Stemasov, Julian Frommel, and Enrico Rukzio. 2017. ShareVR: Enabling Co-Located Experiences for Virtual Reality Between HMD and non-HMD Users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI '17). ACM, New York, NY, USA, 4021--4033. https://doi.org/10.1145/3025453.3025683Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jan Gugenheimer, Evgeny Stemasov, Harpreet Sareen, and Enrico Rukzio. 2018. FaceDisplay: Towards Asymmetric Multi-User Interaction for Nomadic Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). ACM, New York, NY, USA, 1--13. https://doi.org/10.1145/3173574.3173628Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Joe Guna, Gregor Gerak, Iztok Humar, Jeungeun Song, Janko Drnovek, and Matev Poganik. 2019. Influence of video content type on users' virtual reality sickness perception and physiological response. Future Generation Computer Systems , Vol. 91 (2019), 263 -- 276. https://doi.org/10.1016/j.future.2018.08.049Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kunal Gupta, Gun A Lee, and Mark Billinghurst. 2016a. Do you see what I see? The effect of gaze tracking on task space remote collaboration. IEEE transactions on visualization and computer graphics , Vol. 22, 11 (2016), 2413--2422.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Kunal Gupta, Gun A. Lee, and Mark Billinghurst. 2016b. Do You See What I See? The Effect of Gaze Tracking on Task Space Remote Collaboration. IEEE Transactions on Visualization and Computer Graphics , Vol. 22, 11 (2016), 2413--2422. https://doi.org/10.1109/TVCG.2016.2593778Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Carl Gutwin, Saul Greenberg, and Mark Roseman. 1996. Workspace awareness in real-time distributed groupware: Framework, widgets, and evaluation. In People and Computers XI . Springer, London, 281--298.Google ScholarGoogle Scholar
  21. Hikaru Ibayashi, Yuta Sugiura, Daisuke Sakamoto, Natsuki Miyata, Mitsunori Tada, Takashi Okuma, Takeshi Kurata, Masaaki Mochimaru, and Takeo Igarashi. 2015. Dollhouse VR: a multi-view, multi-user collaborative design workspace with VR technology. In SIGGRAPH Asia 2015 Emerging Technologies (Kobe, Japan) (SA '15). ACM, New York, NY, USA, 1--2. https://doi.org/10.1145/2818466.2818480Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Akira Ishii, Ippei Suzuki, Masaya Tsuruta, Shuta Nakamae, Junichi Suzuki, and Yoichi Ochiai. 2018. ReverseCAVE: CAVE-Based Visualization Methods of Public VR Towards Shareable VR Experience. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA '18). Association for Computing Machinery, New York, NY, USA, 1. https://doi.org/10.1145/3170427.3186593Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Ricardo Jota, Bruno R. de Araújo, Luís C. Bruno, João M. Pereira, and Joaquim A. Jorge. 2010a. IMMIView: a multi-user solution for design review in real-time. Journal of Real-Time Image Processing , Vol. 5, 2 (June 2010), 91--107. https://doi.org/10.1007/s11554-009-0141--1Google ScholarGoogle ScholarCross RefCross Ref
  24. Ricardo Jota, Miguel A. Nacenta, Joaquim A. Jorge, Sheelagh Carpendale, and Saul Greenberg. 2010b. A comparison of ray pointing techniques for very large displays. In Proceedings of Graphics Interface 2010 (GI '10). Canadian Information Processing Society, CAN, 269--276. https://dl.acm.org/doi/10.5555/1839214.1839261Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Behrang Keshavarz and Heiko Hecht. 2011. Axis rotation and visually induced motion sickness: the role of combined roll, pitch, and yaw motion. Aviation, space, and environmental medicine , Vol. 82, 11 (2011), 1023--1029.Google ScholarGoogle Scholar
  26. Andre Kunert, Tim Weissker, Bernd Froehlich, and Alexander Kulik. 2019. Multi-window 3d interaction for collaborative virtual reality. IEEE transactions on visualization and computer graphics , Vol. 26, 11 (2019), 3271--3284.Google ScholarGoogle Scholar
  27. Andre Kunert, Tim Weissker, Bernd Froehlich, and Alexander Kulik. 2020. Multi-Window 3D Interaction for Collaborative Virtual Reality. IEEE Transactions on Visualization and Computer Graphics , Vol. 26, 11 (2020), 3271--3284. https://doi.org/10.1109/TVCG.2019.2914677Google ScholarGoogle ScholarCross RefCross Ref
  28. Ed Lantz. 1996. The future of virtual reality: head mounted displays versus spatially immersive displays (panel). In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques (SIGGRAPH '96). ACM, New York, NY, USA, 485--486. https://doi.org/10.1145/237170.237289Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Ed Lantz. 2007. A survey of large-scale immersive displays. In Proceedings of the 2007 workshop on Emerging displays technologies: images and beyond: the future of displays and interacton (San Diego, California, USA) (EDT '07). ACM, New York, NY, USA, 1--es. https://doi.org/10.1145/1278240.1278241Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. JJ-W Lin, Henry Been-Lirn Duh, Donald E. Parker, Habib Abi-Rached, and Thomas A. Furness. 2002. Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment. In Proceedings IEEE Virtual Reality 2002 (Orlando, FL, USA). IEEE, New York, NY, USA, 164--171. https://doi.org/10.1109/VR.2002.996519Google ScholarGoogle ScholarCross RefCross Ref
  31. Lars Lischke, Sven Mayer, Katrin Wolf, Niels Henze, Harald Reiterer, and Albrecht Schmidt. 2016. Screen Arrangements and Interaction Areas for Large Display Work Places. In Proceedings of the 5th ACM International Symposium on Pervasive Displays (Oulu, Finland) (PerDis '16). ACM, New York, NY, USA, 228--234. https://doi.org/10.1145/2914920.2915027Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Sreejith M, Siddharth Rakesh, Samik Gupta, Samprit Biswas, and Partha Pratim Das. 2015. Real-time hands-free immersive image navigation system using Microsoft Kinect 2.0 and Leap Motion Controller. In 2015 Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG) (Patna, India). IEEE, New York, NY, USA, 1--4. https://doi.org/10.1109/NCVPRIPG.2015.7489999Google ScholarGoogle ScholarCross RefCross Ref
  33. Narges Mahyar and Melanie Tory. 2014. Supporting communication and coordination in collaborative sensemaking. IEEE transactions on visualization and computer graphics , Vol. 20, 12 (2014), 1633--1642.Google ScholarGoogle ScholarCross RefCross Ref
  34. Christian Mai, Lukas Rambold, and Mohamed Khamis. 2017. TransparentHMD: Revealing the HMD User's Face to Bystanders. In Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia (Stuttgart, Germany) (MUM '17). ACM, New York, NY, USA, 515--520. https://doi.org/10.1145/3152832.3157813Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Marilyn M. Mantei, Ronald M. Baecker, Abigail J. Sellen, William A. S. Buxton, Thomas Milligan, and Barry Wellman. 1991. Experiences in the Use of a Media Space. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New Orleans, Louisiana, USA) (CHI '91). Association for Computing Machinery, New York, NY, USA, 203--208. https://doi.org/10.1145/108844.108888Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Anders Markussen, Sebastian Boring, Mikkel R. Jakobsen, and Kasper Hornbæk. 2016. Off-Limits: Interacting Beyond the Boundaries of Large Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). ACM, New York, NY, USA, 5862--5873. https://doi.org/10.1145/2858036.2858083Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Daniel Medeiros, Rafael K. dos Anjos, Daniel Mendes, Jo ao Madeiras Pereira, Alberto Raposo, and Joaquim Jorge. 2018. Keep My Head on My Shoulders! Why Third-Person is Bad for Navigation in VR. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (Tokyo, Japan) (VRST '18). Association for Computing Machinery, New York, NY, USA, Article 16, bibinfonumpages10 pages. https://doi.org/10.1145/3281505.3281511Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Daniel Medeiros, Eduardo Ribeiro, Peter Dam, Rodrigo Pinheiro, Thiago Motta, Manuel Loaiza, and Alberto B. Raposo. 2012. A Case Study on the Implementation of the 3C Collaboration Model in Virtual Environments. In 2012 14th Symposium on Virtual and Augmented Reality (Rio de Janeiro, Brazil). IEEE, New York, NY, USA, 147--154. https://doi.org/10.1109/SVR.2012.28Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Daniel Medeiros, Lucas Teixeira, Felipe Carvalho, Ismael Santos, and Alberto Raposo. 2013. A tablet-based 3D interaction tool for virtual engineering environments. In Proceedings of the 12th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry (Hong Kong, Hong Kong) (VRCAI '13). ACM, New York, NY, USA, 211--218. https://doi.org/10.1145/2534329.2534349Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Luís A. R. Neng and Teresa Chambel. 2010. Get around 360° Hypervideo. In Proceedings of the 14th International Academic MindTrek Conference: Envisioning Future Media Environments (Tampere, Finland) (MindTrek '10). ACM, New York, NY, USA, 119--122. https://doi.org/10.1145/1930488.1930512Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Ohan Oda, Carmine Elvezio, Mengu Sukan, Steven Feiner, and Barbara Tversky. 2015. Virtual Replicas for Remote Assistance in Virtual and Augmented Reality. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (Charlotte, NC, USA) (UIST '15). Association for Computing Machinery, New York, NY, USA, 405--415. https://doi.org/10.1145/2807442.2807497Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Emilee Patrick, Dennis Cosgrove, Aleksandra Slavkovic, Jennifer A. Rode, Thom Verratti, and Greg Chiselko. 2000. Using a Large Projection Screen As an Alternative to Head-mounted Displays for Virtual Environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (The Hague, The Netherlands) (CHI '00). ACM, New York, NY, USA, 478--485. https://doi.org/10.1145/332040.332479Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Amy Pavel, Björn Hartmann, and Maneesh Agrawala. 2017. Shot Orientation Controls for Interactive Cinematography with 360 Video. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (Québec City, QC, Canada) (UIST '17). ACM, New York, NY, USA, 289--297. https://doi.org/10.1145/3126594.3126636Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Taehyun Rhee, Stephen Thompson, Daniel Medeiros, Rafael Kuffner Dos Anjos, and Andrew Chalmers. 2020. Augmented Virtual Teleportation for High-Fidelity Telecollaboration . IEEE Transactions on Visualization and Computer Graphics , Vol. 26, 5 (2020), 1923--1933. https://doi.org/10.1109/tvcg.2020.2973065Google ScholarGoogle ScholarCross RefCross Ref
  45. Sarah Sharples, Sue Cobb, Amanda Moody, and John R. Wilson. 2008. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays , Vol. 29, 2 (2008), 58 -- 69. https://doi.org/10.1016/j.displa.2007.09.005Google ScholarGoogle ScholarCross RefCross Ref
  46. Andreas Simon. 2005. First-person Experience and Usability of Co-located Interaction in a Projection-based Virtual Environment . In Proceedings of the ACM Symposium on Virtual Reality Software and Technology (Monterey, CA, USA) (VRST '05). ACM, New York, NY, USA, 23--30. https://doi.org/10.1145/1101616.1101622Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Mel Slater, Martin Usoh, and Anthony Steed. 1994. Depth of presence in virtual environments. Presence: Teleoperators & Virtual Environments , Vol. 3, 2 (1994), 130--144.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Maurício Sousa, Rafael Kuffner dos Anjos, Daniel Mendes, Mark Billinghurst, and Joaquim Jorge. 2019. Warping Deixis: Distorting Gestures to Enhance Collaboration (CHI '19). ACM, New York, NY, USA, 1--12. https://doi.org/10.1145/3290605.3300838Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Desney S. Tan, Darren Gergle, Peter Scupelli, and Randy Pausch. 2003. With Similar Visual Angles, Larger Displays Improve Spatial Performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI '03). ACM, New York, NY, USA, 217--224. https://doi.org/10.1145/642611.642650Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Ikuko Tsubaki, Toshiya Morita, Takahiro Saito, and Kiyoharu Aizawa. 2005. An adaptive video stabilization method for reducing visually induced motion sickness. In IEEE International Conference on Image Processing 2005 (Genova, Italy), Vol. 3. IEEE, New York, NY, USA, III--497. https://doi.org/10.1109/ICIP.2005.1530437Google ScholarGoogle ScholarCross RefCross Ref
  51. Colin Ware and Steven Osborne. 1990. Exploration and Virtual Camera Control in Virtual Three Dimensional Environments . SIGGRAPH Comput. Graph. , Vol. 24, 2 (Feb. 1990), 175--183. https://doi.org/10.1145/91394.91442Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Haijun Xia, Sebastian Herscher, Ken Perlin, and Daniel Wigdor. 2018. Spacetime: Enabling Fluid Individual and Collaborative Editing in Virtual Reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (Berlin, Germany) (UIST '18). ACM, New York, NY, USA, 853--866. https://doi.org/10.1145/3242587.3242597Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Spectator View: Enabling Asymmetric Interaction between HMD Wearers and Spectators with a Large Display

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!