Abstract
Mixed reality (MR) cooperation scenarios are more and more interesting for business and research as powerful wearable devices like head mounted displays (HMD) become commercially available. A lot of work focuses on remote MR cooperation settings like remote maintenance support and co-located scenarios in which participants cooperate over a longer period time. Despite this, MR also has great potential for real-time co-located cooperation support with the need of short-term decisions and interactions. However, little is known on how this support can be provided. To bridge this gap, we conducted an experiment using a MR visual search task performed by dyads. Based on related work, visual search was chosen to represent typical challenges of short-term cooperative MR tasks. The aim of the experiment was to explore how the participants coordinate their searches and how this influences their performance in a task. We found that participants mainly used embodied and verbal cues to coordinate their searches (rather than virtual cues provided by the HMD) and that less communication worked significantly better, which is in (partial) contrast to existing findings. We discuss potential reasons for and impacts of these findings.
- Ahish Agrawal, Gourav Acharya, Krishna Balasubramanian, Nehal Agrawal, and Ratnesh Chaturvedi. 2016. A Review on the use of Augmented Reality to Generate Safety Awareness and Enhance Emergency Response. International Journal of Current Engineering and Technology 6, 3 (June 2016), 813--820.Google Scholar
- David G. Armstrong, Timothy M. Rankin, Nicholas A. Giovinco, Joseph L. Mills, and Yoky Matsuoka. 2014. A Heads-Up Display for Diabetic Limb Salvage Surgery A View Through the Google Looking Glass. Journal of diabetes science and technology 8, 5 (2014), 951--956.Google Scholar
Cross Ref
- Ronald T. Azuma. 1997. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 6, 4 (August 1997), 355--385. DOI:https://doi.org/10.1162/pres.1997.6.4.355Google Scholar
Digital Library
- L. Bannon and K. Schmidt. 1992. Taking CSCW Seriously: Supporting articulation work. Computer Supported Cooperative Work 1, 1 (1992), 7--40.Google Scholar
Cross Ref
- Patrick Baudisch and Ruth Rosenholtz. 2003. Halo: a technique for visualizing off-screen objects. In Proceedings of the SIGCHI conference on Human factors in computing systems, 481--488. Retrieved June 27, 2017 from http://dl.acm.org/citation.cfm?id=642695Google Scholar
Digital Library
- Martin Bauer, Gerd Kortuem, and Zary Segall. 1999. "Where Are You Pointing At?" A Study of Remote Collaboration in a Wearable Videoconference System. In Proceedings of the 3rd IEEE International Symposium on Wearable Computers (ISWC '99), 151--. Retrieved September 21, 2016 from http://dl.acm.org/citation.cfm?id=519309.856501Google Scholar
Cross Ref
- Michel Beaudouin-Lafon and Alain Karsenty. 1992. Transparency and Awareness in a Real-time Groupware System. In Proceedings of the 5th Annual ACM Symposium on User Interface Software and Technology (UIST '92), 171--180. DOI:https://doi.org/10.1145/142621.142646Google Scholar
Digital Library
- Mark Billinghurst and Hirokazu Kato. 2002. Collaborative Augmented Reality. Commun. ACM 45, 7 (July 2002), 64--70. DOI:https://doi.org/10.1145/514236.514265Google Scholar
Digital Library
- Susan E. Brennan, Xin Chen, Christopher A. Dickinson, Mark B. Neider, and Gregory J. Zelinsky. 2008. Coordinating cognition: the costs and benefits of shared gaze during collaborative search. Cognition 106, 3 (March 2008), 1465--1477. DOI:https://doi.org/10.1016/j.cognition.2007.05.012Google Scholar
Cross Ref
- Nicko R. Caluya, Alexander Plopski, Jayzon F. Ty, Christian Sandor, Takafumi Taketomi, and Hirokazu Kato. 2018. Transferability of Spatial Maps: Augmented Versus Virtual Reality Training. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 387--393. DOI:https://doi.org/10.1109/VR.2018.8447561Google Scholar
- Jeffrey W. Chastine, Kristine Nagel, Ying Zhu, and Luca Yearsovich. 2007. Understanding the Design Space of Referencing in Collaborative Augmented Reality Environments. In Proceedings of Graphics Interface 2007 (GI '07), 207--214. DOI:https://doi.org/10.1145/1268517.1268552Google Scholar
Digital Library
- Drago Datcu, Stephan G. Lukosch, and Heide K. Lukosch. 2016. Handheld Augmented Reality for Distributed Collaborative Crime Scene Investigation. In Proceedings of the 19th International Conference on Supporting Group Work (GROUP '16), 267--276. DOI:https://doi.org/10.1145/2957276.2957302Google Scholar
- Arturo Deza, Jeffrey R. Peters, Grant S. Taylor, Amit Surana, and Miguel P. Eckstein. 2017. Attention Allocation Aid for Visual Search. arXiv:1701.03968 [cs] (2017), 220--231. DOI:https://doi.org/10.1145/3025453.3025834Google Scholar
- Paul Dourish and Victoria Bellotti. 1992. Awareness and coordination in shared workspaces. In CSCW '92: Proceedings of the 1992 ACM conference on Computer-supported cooperative work, 107--114. DOI:http://doi.acm.org/10.1145/143457.143468Google Scholar
Digital Library
- Omid Fakourfar, Kevin Ta, Richard Tang, Scott Bateman, and Anthony Tang. 2016. Stabilized Annotations for Mobile Remote Assistance. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 1548--1560. DOI:https://doi.org/10.1145/2858036.2858171Google Scholar
Digital Library
- Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. World-stabilized Annotations and Virtual Scene Navigation for Remote Collaboration. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14), 449--459. DOI:https://doi.org/10.1145/2642918.2647372Google Scholar
Digital Library
- K. Gupta, G. A. Lee, and M. Billinghurst. 2016. Do You See What I See? The Effect of Gaze Tracking on Task Space Remote Collaboration. IEEE Transactions on Visualization and Computer Graphics 22, 11 (November 2016), 2413--2422. DOI:https://doi.org/10.1109/TVCG.2016.2593778Google Scholar
Digital Library
- Carl Gutwin, Mutasem Barjawi, and David Pinelle. 2016. The Emergence of High-Speed Interaction and Coordination in a (Formerly) Turn-based Groupware Game. In Proceedings of the 19th International Conference on Supporting Group Work, 277--286.Google Scholar
Digital Library
- Carl Gutwin and Saul Greenberg. 2002. A descriptive framework of workspace awareness for real-time groupware. Computer Supported Cooperative Work (CSCW) 11, 3--4 (2002), 411--446.Google Scholar
Digital Library
- J. Hagler, M. Lankes, and J. Diephuis. 2018. Animating Participants in Co-located Playful Mixed-Reality Installations. In 2018 IEEE 1st Workshop on Animation in Virtual and Augmented Environments (ANIVAE), 1--4. DOI:https://doi.org/10.1109/ANIVAE.2018.8587269Google Scholar
- Jon Hindmarsh, Mike Fraser, Christian Heath, Steve Benford, and Chris Greenhalgh. 1998. Fragmented Interaction: Establishing Mutual Orientation in Virtual Environments. In Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work (CSCW '98), 217--226. DOI:https://doi.org/10.1145/289444.289496Google Scholar
Digital Library
- Jon Hindmarsh, Mike Fraser, Christian Heath, Steve Benford, and Chris Greenhalgh. 1998. Fragmented interaction: establishing mutual orientation in virtual environments. In Proceedings of the 1998 ACM conference on Computer supported cooperative work, 217--226.Google Scholar
Digital Library
- Hossein Mousavi Hondori, Maryam Khademi, Lucy Dodakian, Steven C Cramer, and Cristina Videira Lopes. 2013. A Spatial Augmented Reality rehab system for post-stroke hand rehabilitation. In Medicine Meets Virtual Reality, 279--285.Google Scholar
- Ke Huo, Tianyi Wang, Luis Paredes, Ana M. Villanueva, Yuanzhi Cao, and Karthik Ramani. 2018. SynchronizAR: Instant Synchronization for Spontaneous and Spatial Collaborations in Augmented Reality. In Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology (UIST '18), 19--30. DOI:https://doi.org/10.1145/3242587.3242595Google Scholar
Digital Library
- Steven Johnson, Madeleine Gibson, and Bilge Mutlu. 2015. Handheld or Handsfree?: Remote Collaboration via Lightweight Head-Mounted Displays and Handheld Devices. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15), 1825--1836. DOI:https://doi.org/10.1145/2675133.2675176Google Scholar
Digital Library
- Kiyoshi Kiyokawa, Mark Billinghurst, Sohan E. Hayes, Anoop Gupta, Yuki Sannohe, and Hirokazu Kato. 2002. Communication behaviors of co-located users in collaborative AR interfaces. In Proceedings of the 1st International Symposium on Mixed and Augmented Reality, 139.Google Scholar
Digital Library
- Chris L. Kleinke. 1986. Gaze and eye contact: a research review. Psychological bulletin 100, 1 (1986), 78.Google Scholar
- Mikko Kytö, Barrett Ens, Thammathip Piumsomboon, Gun A. Lee, and Mark Billinghurst. 2018. Pinpointing: Precise Head- and Eye-Based Target Selection for Augmented Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18), 81:1--81:14. DOI:https://doi.org/10.1145/3173574.3173655Google Scholar
Digital Library
- Cha Lee, Gustavo A. Rincon, Greg Meyer, Tobias Höllerer, and Doug A. Bowman. 2013. The effects of visual realism on search tasks in mixed reality simulation. IEEE transactions on visualization and computer graphics 19, 4 (2013), 547--556.Google Scholar
- Jerry Li, Mia Manavalan, Sarah D'Angelo, and Darren Gergle. 2016. Designing Shared Gaze Awareness for Remote Collaboration. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (CSCW '16 Companion), 325--328. DOI:https://doi.org/10.1145/2818052.2869097Google Scholar
Digital Library
- Kristine Lund. 2007. The importance of gaze and gesture in interactive multimodal explanation. Language Resources and Evaluation 41, 3--4 (2007), 289--303.Google Scholar
Cross Ref
- Paul Milgram and Fumio Kishino. 1994. A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems 77, 12 (1994), 1321--1329.Google Scholar
- Robert J. Moore, Nicolas Ducheneaut, and Eric Nickell. 2007. Doing virtually nothing: Awareness and accountability in massively multiplayer online worlds. Computer Supported Cooperative Work (CSCW) 16, 3 (2007), 265--305.Google Scholar
Digital Library
- Hermann J. Müller and Joseph Krummenacher. 2006. Visual search and selective attention. Visual Cognition 14, 4--8 (August 2006), 389--410. DOI:https://doi.org/10.1080/13506280500527676Google Scholar
- Jens Müller, Roman Rädle, and Harald Reiterer. 2016. Virtual Objects As Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 1245--1249. DOI:https://doi.org/10.1145/2858036.2858043Google Scholar
Digital Library
- Romy Müller, Jens R. Helmert, Sebastian Pannasch, and Boris M. Velichkovsky. 2013. Gaze transfer in remote cooperation: is it always helpful to see what your partner is attending to? Q J Exp Psychol (Hove) 66, 7 (2013), 1302--1316. DOI:https://doi.org/10.1080/17470218.2012.737813Google Scholar
Cross Ref
- Alaeddin Nassani, Hyungon Kim, Gun Lee, Mark Billinghurst, Tobias Langlotz, and Robert W. Lindeman. 2016. Augmented Reality Annotation for Social Video Sharing. In SIGGRAPH ASIA 2016 Mobile Graphics and Interactive Applications (SA '16), 9:1--9:5. DOI:https://doi.org/10.1145/2999508.2999529Google Scholar
- Thammathip Piumsomboon, Arindam Dey, Barrett Ens, Gun Lee, and Mark Billinghurst. 2019. The Effects of Sharing Awareness Cues in Collaborative Mixed Reality. Front. Robot. AI 6, (2019). DOI:https://doi.org/10.3389/frobt.2019.00005Google Scholar
- Ronald Poelman, Oytun Akman, Stephan Lukosch, and Pieter Jonker. 2012. As if Being There: Mediated Reality for Crime Scene Investigation. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW '12), 1267--1276. DOI:https://doi.org/10.1145/2145204.2145394Google Scholar
Digital Library
- Marc Pomplun, Tyler W Garaas, and Marisa Carrasco. 2013. The effects of task difficulty on visual search strategy in virtual 3D displays. Journal of vision 13, 3 (2013), 24--24.Google Scholar
Cross Ref
- Dieter Schmalstieg, Anton Fuhrmann, Gerd Hesina, Zsolt Szalavári, L Miguel Encarnaçao, Michael Gervautz, and Werner Purgathofer. 2002. The studierstube augmented reality project. Presence: Teleoperators and Virtual Environments 11, 1 (2002), 33--54.Google Scholar
Digital Library
- Dieter Schmalstieg and Tobias Hollerer. 2016. Augmented Reality: Principles and Practice (01 ed.). Addison Wesley, Boston.Google Scholar
- Christian Schnier, Karola Pitsch, Angelika Dierker, and Thomas Hermann. 2011. Collaboration in Augmented Reality: How to establish coordination and joint attention? In ECSCW 2011: Proceedings of the 12th European Conference on Computer Supported Cooperative Work, 24--28 September 2011, Aarhus Denmark, 405--416.Google Scholar
Cross Ref
- Bari Serim, Ken Pfeuffer, Hans Gellersen, and Giulio Jacucci. 2018. Visual Attention-Based Access: Granting Access Based on Users' Joint Attention on Shared Workspaces. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 133.Google Scholar
Digital Library
- Harri Siirtola, Oleg pakov, Howell Istance, and Kari-Jouko Räihä. 2019. Shared Gaze in Collaborative Visual Search. International Journal of Human--Computer Interaction 0, 0 (February 2019), 1--13. DOI:https://doi.org/10.1080/10447318.2019.1565746Google Scholar
- Randy Stein and Susan E. Brennan. 2004. Another Person's Eye Gaze As a Cue in Solving Programming Problems. In Proceedings of the 6th International Conference on Multimodal Interfaces (ICMI '04), 9--15. DOI:https://doi.org/10.1145/1027933.1027936Google Scholar
- L. Suchman. 1996. Supporting articulation work. 407--423. Retrieved from http://books.google.com/books?hl=de&lr=&id=9wlN9eOomacC&oi=fnd&pg=PA407&dq=Supporting+articulation+work&ots=fgGvj8yL3W&sig=t1qgKME9J04GkzS7Haw1MbytSJ4#v=onepage&q=Supporting%20articulation%20work&f=falseGoogle Scholar
- Anne M. Treisman and Garry Gelade. 1980. A feature-integration theory of attention. Cognitive Psychology 12, 1 (January 1980), 97--136. DOI:https://doi.org/10.1016/0010-0285(80)90005--5Google Scholar
Digital Library
- Roel Vertegaal. 1999. The GAZE Groupware System: Mediating Joint Attention in Multiparty Communication and Collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99), 294--301. DOI:https://doi.org/10.1145/302979.303065Google Scholar
Digital Library
- Benjamin Volmer, James Baumeister, Stewart Von Itzstein, Ina Bornkessel-Schlesewsky, Matthias Schlesewsky, Mark Billinghurst, and Bruce H. Thomas. 2018. A Comparison of Predictive Spatial Augmented Reality Cues for Procedural Tasks. IEEE transactions on visualization and computer graphics (2018).Google Scholar
- Jeremy M. Wolfe. 2007. Guided search 4.0. In Integrated models of cognitive systems, W. Gray (ed.). Oxford, New York, 99--119. Retrieved March 7, 2017 from https://books.google.com/books?hl=en&lr=&id=eB9nDAAAQBAJ&oi=fnd&pg=PA99&dq=%22attention.+Search+tasks+could+be+divided%22+%22VISUAL+ATTENTION+AND%22+%22color+processes+could+guide+the%22+%22among+green),+attention+would+be+directed%22+%2220%E2%80%9330+ms/item+on+target-present+trials+and+a+bit%22+&ots=zSV_aFeqFI&sig=HXB0dHMmamx0mTE4dxDgOVdIkoEGoogle Scholar
- Nancy Yao, Jeff Brewer, Sarah D'Angelo, Mike Horn, and Darren Gergle. 2018. Visualizing Gaze Information from Multiple Students to Support Remote Instruction. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (CHI EA '18), LBW051:1--LBW051:6. DOI:https://doi.org/10.1145/3170427.3188453Google Scholar
Digital Library
- Yanxia Zhang, Ken Pfeuffer, Ming Ki Chong, Jason Alexander, Andreas Bulling, and Hans Gellersen. 2017. Look together: using gaze for assisting co-located collaborative search. Pers Ubiquit Comput 21, 1 (February 2017), 173--186. DOI:https://doi.org/10.1007/s00779-016-0969-xGoogle Scholar
Digital Library
- Egui Zhu, Arash Hadadgar, Italo Masiello, and Nabil Zary. 2014. Augmented reality in healthcare education: an integrative review. PeerJ 2, (2014), e469.Google Scholar
Index Terms
"I simply watched where she was looking at": Coordination in Short-term Synchronous Cooperative Mixed Reality
Recommendations
Knowledge Cartography and Social Network Representation: Application to Collaborative Platforms in Scientific Area
SITIS '10: Proceedings of the 2010 Sixth International Conference on Signal-Image Technology and Internet Based SystemsOur research revolves around collaborative platforms entirely dedicated to research activities for several scientific organizations. Here, researchers from different domains interact and exchange information using our platforms as the common ground ...
Overlay networks for task allocation and coordination in large-scale networks of cooperative agents
This paper proposes a novel method for scheduling and allocating atomic and complex tasks in large-scale networks of homogeneous or heterogeneous cooperative agents. Our method encapsulates the concepts of searching, task allocation and scheduling ...
Physicality As an Anchor for Coordination: Examining Collocated Collaboration in Physical and Mobile Augmented Reality Settings
CSCW2Design and co-creation activities around 3D artifacts often require close collocated coordination between multiple users. Augmented reality (AR) technology can support collocated work enabling users to flexibly work with digital objects while still ...






Comments