Abstract
Augmented Reality (AR) applications can enable geographically distant users to collaborate using shared video feeds or interactive 3D holograms, and may be particularly useful in the socially distant context of the Covid-19 pandemic. However, a good user experience is key for their success and could be negatively impacted by network impairments, which are an inevitable occurrence in today's best-effort Internet. In this paper, we present the findings of an empirical user study, aimed at understanding the effects of network outages, on user experience and behavior, in a collaborative AR task. We highlight how network outages affected users in different ways depending on their role in the collaborative task, and how giving users explicit information about poor network conditions helped them deal with some of these negative effects. Furthermore, we report the strategies that users themselves adopted, to deal with outages, such as batching instructions, or shifting to a different spatial referencing style when communicating with their partners. Lastly, based on our findings, we present some design implications for future remote-collaborative AR applications.
- [n. d.]. http://arsharing.cs.washington.edu/index.htmlGoogle Scholar
- [n. d.]. https://objecttheory.com/platform/Google Scholar
- 2020. 10 BEST Augmented Reality Glasses (Smart Glasses) In 2021. https://www.softwaretestinghelp.com/best-augmented-reality-glasses/Google Scholar
- Giuseppe Aceto, Alessio Botta, Pietro Marchetta, Valerio Persico, and Antonio Pescapé. 2018. A comprehensive survey on internet outages. Journal of Network and Computer Applications, Vol. 113 (2018), 36--63.Google Scholar
Cross Ref
- Sudhir Aggarwal, Hemant Banavar, Amit Khandelwal, Sarit Mukherjee, and Sampath Rangarajan. 2004. Accuracy in dead-reckoning based distributed multi-player games. In Proceedings of 3rd ACM SIGCOMM workshop on Network and system support for games. 161--165.Google Scholar
Digital Library
- Tooba Ahsen, Fahad R Dogar, and Aaron L Gardony. 2019. Exploring the Impact of Network Impairments on Remote Collaborative Augmented Reality Applications. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, LBW2619.Google Scholar
- Deepak Akkil and Poika Isokoski. 2016a. Accuracy of interpreting pointing gestures in egocentric view. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 262--273.Google Scholar
Digital Library
- Deepak Akkil and Poika Isokoski. 2016b. Gaze Augmentation in Egocentric Video Improves Awareness of Intention. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1573--1584.Google Scholar
Digital Library
- Deepak Akkil, Jobin Mathew James, Poika Isokoski, and Jari Kangas. 2016. GazeTorch: Enabling Gaze Awareness in Collaborative Physical Tasks. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1151--1158.Google Scholar
Digital Library
- Judith Amores, Xavier Benavides, and Pattie Maes. 2015. Showme: A remote collaboration system that supports immersive gestural communication. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. 1343--1348.Google Scholar
Digital Library
- Huidong Bai, Prasanth Sasikumar, Jing Yang, and Mark Billinghurst. 2020. A user study on mixed reality remote collaboration with eye gaze and hand gesture sharing. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1--13.Google Scholar
Digital Library
- Yahn W Bernier. 2001. Latency compensating methods in client/server in-game protocol design and optimization. In Game Developers Conference, Vol. 98033.Google Scholar
- Henrik Brun, Robin Anton Birkeland Bugge, LKR Suther, Sigurd Birkeland, Rahul Kumar, Egidijus Pelanis, and Ole Jacob Elle. 2019. Mixed reality holograms for heart surgery planning: first user experience in congenital heart disease. European Heart Journal-Cardiovascular Imaging, Vol. 20, 8 (2019), 883--888.Google Scholar
Cross Ref
- Wolfgang Büschel. [n. d.]. Challenges in Collaborative Immersive Visualization. ([n. d.]).Google Scholar
- Simon Butscher, Sebastian Hubenschmid, Jens Müller, Johannes Fuchs, and Harald Reiterer. 2018. Clusters, trends, and outliers: How immersive technologies can facilitate the collaborative analysis of multidimensional data. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--12.Google Scholar
Digital Library
- Tolga K Capin and Igor S Pandzic. 1997. A dead-reckoning algorithm for virtual human figures. In Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality. IEEE, 161--169.Google Scholar
Digital Library
- Armando L Caro, Janardhan R Iyengar, Paul D Amer, Sourabh Ladha, Gerard J Heinz, and Keyur C Shah. 2003. SCTP: a proposed standard for robust internet data transport. Computer, Vol. 36, 11 (2003), 56--63.Google Scholar
Digital Library
- Kuan-Ta Chen, Chun-Ying Huang, Polly Huang, and Chin-Laung Lei. 2006. Quantifying Skype user satisfaction. ACM SIGCOMM Computer Communication Review, Vol. 36, 4 (2006), 399--410.Google Scholar
Digital Library
- Marshini Chetty, Srikanth Sundaresan, Sachit Muckaden, Nick Feamster, and Enrico Calandro. 2013. Measuring broadband performance in South Africa. In Proceedings of the 4th Annual Symposium on Computing for Development. ACM, 1.Google Scholar
Digital Library
- Herbert H. Clark. 1996. Common ground .Cambridge University Press, 92--122. https://doi.org/10.1017/CBO9780511620539.005Google Scholar
- Herbert H Clark and Susan E Brennan. 1991. Grounding in communication. (1991).Google Scholar
- Catherine Durnell Cramton. 2001. The mutual knowledge problem and its consequences for dispersed collaboration. Organization science, Vol. 12, 3 (2001), 346--371.Google Scholar
Digital Library
- Alberto Dainotti, Claudio Squarcella, Emile Aben, Kimberly C Claffy, Marco Chiesa, Michele Russo, and Antonio Pescapé. 2011. Analysis of country-wide internet outages caused by censorship. In Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference. ACM, 1--18.Google Scholar
Digital Library
- Barrett Ens, Sarah Goodwin, Arnaud Prouzeau, Fraser Anderson, Florence Y Wang, Samuel Gratzl, Zac Lucarelli, Brendan Moyle, Jim Smiley, and Tim Dwyer. 2020. Uplift: A Tangible and Immersive Tabletop System for Casual Collaborative Visual Analytics. IEEE Transactions on Visualization and Computer Graphics (2020).Google Scholar
- Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, and Mark Billinghurst. 2019. Revisiting collaboration through mixed reality: The evolution of groupware. International Journal of Human-Computer Studies, Vol. 131 (2019), 81--98.Google Scholar
Digital Library
- Brian Eriksson, Ramakrishnan Durairajan, and Paul Barford. 2013. Riskroute: a framework for mitigating network outage threats. In Proceedings of the ninth ACM conference on Emerging networking experiments and technologies. ACM, 405--416.Google Scholar
Digital Library
- P Esteban, JE Jaramillo, N Álvarez, J Restrepo, and H Trefftz. 2004. Augmented Reality: a space for the understanding of multi-variate Calculus. In Proceedings of the IADATE conference. Spain.Google Scholar
- Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014a. In touch with the remote world: Remote collaboration with augmented reality drawings and virtual navigation. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology. 197--205.Google Scholar
Digital Library
- Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014b. World-stabilized annotations and virtual scene navigation for remote collaboration. In Proceedings of the 27th annual ACM symposium on User interface software and technology. 449--459.Google Scholar
Digital Library
- Aaron M Genest, Carl Gutwin, Anthony Tang, Michael Kalyn, and Zenja Ivkovic. 2013. KinectArms: a toolkit for capturing and displaying arm embodiments in distributed tabletop groupware. In Proceedings of the 2013 conference on Computer supported cooperative work. 157--166.Google Scholar
Digital Library
- Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004 a. Action as language in a shared visual space. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. ACM, 487--496.Google Scholar
Digital Library
- Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004 b. Language efficiency and visual technology: Minimizing collaborative effort with visual information. Journal of language and social psychology, Vol. 23, 4 (2004), 491--517.Google Scholar
Cross Ref
- Darren Gergle, Robert E Kraut, and Susan R Fussell. 2013. Using visual information for grounding and awareness in collaborative tasks. Human-Computer Interaction, Vol. 28, 1 (2013), 1--39.Google Scholar
- Raphaël Grasset, Philip Lamb, and Mark Billinghurst. 2005. Evaluation of mixed-space collaboration. In Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 90--99.Google Scholar
Digital Library
- Sebastian Günther, Sven Kratz, Daniel Avrahami, and Max Mühlh"auser. 2018. Exploring Audio, Visual, and Tactile Cues for Synchronous Remote Assistance. In Proceedings of the 11th Pervasive Technologies Related to Assistive Environments Conference. 339--344.Google Scholar
Digital Library
- Felix G Hamza-Lup, Jannick P Rolland, and Charles Hughes. 2018. A distributed augmented reality system for medical training and simulation. arXiv preprint arXiv:1811.12815 (2018).Google Scholar
- Osama Haq and Fahad R Dogar. 2015. Leveraging the power of cloud for reliable wide area communication. In Proc. ACM HotNets.Google Scholar
Digital Library
- Tomoki Itamiya, Toshinori Iwai, and Tsuyoshi Kaneko. 2018. The Holographic Human for surgical navigation using Microsoft HoloLens. EPiC Series in Engineering, Vol. 1 (2018), 26--30.Google Scholar
Cross Ref
- Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz. 2018. Super slomo: High quality estimation of multiple intermediate frames for video interpolation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9000--9008.Google Scholar
Cross Ref
- Junchen Jiang, Rajdeep Das, Ganesh Ananthanarayanan, Philip A Chou, Venkata Padmanabhan, Vyas Sekar, Esbjorn Dominique, Marcin Goliszewski, Dalibor Kukoleca, Renat Vafin, et al. 2016. Via: Improving internet telephony call quality using predictive relay selection. In Proc. SIGCOMM'16. ACM.Google Scholar
Digital Library
- Wenyu Jiang and Henning Schulzrinne. 2003. Assessment of voip service availability in the current internet. In Proceedings of the 4th International Workshop on Passive and Active Network Measurement (PAM 2003).Google Scholar
- Laura Casado Joey Hadden. 2020. 21 major companies that have announced employees can work remotely long-term. https://www.businessinsider.com/companies-asking-employees-to-work-from-home-due-to-coronavirus-2020Google Scholar
- Steven Johnson, Madeleine Gibson, and Bilge Mutlu. 2015. Handheld or handsfree?: Remote collaboration via lightweight head-mounted displays and handheld devices. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1825--1836.Google Scholar
Digital Library
- Ethan Katz-Bassett, Colin Scott, David R Choffnes, Ítalo Cunha, Vytautas Valancius, Nick Feamster, Harsha V Madhyastha, Thomas Anderson, and Arvind Krishnamurthy. 2012. LIFEGUARD: Practical repair of persistent route failures. In Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication. ACM, 395--406.Google Scholar
Digital Library
- Hannes Kaufmann. 2002. Construct3D: an augmented reality application for mathematics and geometry education. In Proceedings of the tenth ACM international conference on Multimedia. 656--657.Google Scholar
Digital Library
- Seungwon Kim, Allison Jing, Hanhoon Park, Gun A Lee, Weidong Huang, and Mark Billinghurst. 2020. Hand-in-Air (HiA) and Hand-on-Target (HoT) Style Gesture Cues for Mixed Reality Collaboration. IEEE Access, Vol. 8 (2020), 224145--224161.Google Scholar
- Kiyoshi Kiyokawa, Mark Billinghurst, Sohan E Hayes, Anoop Gupta, Yuki Sannohe, and Hirokazu Kato. 2002. Communication behaviors of co-located users in collaborative AR interfaces. In Proceedings of the 1st International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 139.Google Scholar
Digital Library
- Adam Langley, Alistair Riddoch, Alyssa Wilk, Antonio Vicente, Charles Krasic, Dan Zhang, Fan Yang, Fedor Kouranov, Ian Swett, Janardhan Iyengar, et al. 2017. The quic transport protocol: Design and internet-scale deployment. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 183--196.Google Scholar
Digital Library
- Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Chapter 11 - Analyzing qualitative data. In Research Methods in Human Computer Interaction (Second Edition) second edition ed.), Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser (Eds.). Morgan Kaufmann, Boston, 299 -- 327. https://doi.org/10.1016/B978-0--12--805390--4.00011-XGoogle Scholar
Digital Library
- Jun Li and Scott Brooks. 2011. I-seismograph: Observing and measuring Internet earthquakes. In INFOCOM, 2011 Proceedings IEEE. IEEE, 2624--2632.Google Scholar
Cross Ref
- Stephan Lukosch, Heide Lukosch, Dragos Datcu, and Marina Cidota. 2015. On the spot information in augmented reality for teams in the security domain. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 983--988.Google Scholar
Digital Library
- Tahir Mahmood, Willis Fulmer, Neelesh Mungoli, Jian Huang, and Aidong Lu. 2019. Improving information sharing and collaborative analysis for remote geospatial visualization using mixed reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 236--247.Google Scholar
Cross Ref
- Shawn McKee, Mike O'Connor, Soichi Hayashi, Marian Babik, Brian Tierney, Ilija Vukotic, Les Cottrell, and Henryk Giemza. 2016. ICFA SCIC Network Monitoring Report. Technical Report.Google Scholar
- Microsoft. [n. d.] a. Microsoft HoloLens. https://www.microsoft.com/en-us/hololens [Online; accessed 21-September-2018].Google Scholar
- Microsoft. [n. d.] b. Mixed Reality Toolkit. https://github.com/Microsoft/MixedRealityToolkit-Unity [Online; accessed 21-September-2018].Google Scholar
- Jens Müller, Roman R"adle, and Harald Reiterer. 2016. Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1245--1249.Google Scholar
Digital Library
- Jens Müller, Roman R"adle, and Harald Reiterer. 2017. Remote collaboration with mixed reality displays: how shared virtual landmarks facilitate spatial referencing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 6481--6486.Google Scholar
Digital Library
- Nahal Norouzi*, Austin Erickson*, Kangsoo Kim, Ryan Schubert, Joseph LaViola, Gerd Bruder, and Greg Welch. 2019. Effects of shared gaze parameters on visual target identification task performance in augmented reality. In Symposium on Spatial User Interaction. 1--11.Google Scholar
- Lyn Pemberton and Marcus Winter. 2009. Collaborative augmented reality in schools. (2009).Google Scholar
- Thammathip Piumsomboon, Arindam Dey, Barrett Ens, Gun Lee, and Mark Billinghurst. 2019 a. The effects of sharing awareness cues in collaborative mixed reality. Frontiers in Robotics and AI, Vol. 6 (2019), 5.Google Scholar
Cross Ref
- Thammathip Piumsomboon, Gun A Lee, Jonathon D Hart, Barrett Ens, Robert W Lindeman, Bruce H Thomas, and Mark Billinghurst. 2018. Mini-me: An adaptive avatar for mixed reality remote collaboration. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1--13.Google Scholar
Digital Library
- Thammathip Piumsomboon, Gun A Lee, Andrew Irlitti, Barrett Ens, Bruce H Thomas, and Mark Billinghurst. 2019 b. On the shoulder of the giant: A multi-scale mixed reality collaboration with 360 video sharing and tangible interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1--17.Google Scholar
Digital Library
- Thammathip Piumsomboon, Youngho Lee, Gun Lee, and Mark Billinghurst. 2017. CoVAR: a collaborative virtual and augmented reality system for remote collaboration. In SIGGRAPH Asia 2017 Emerging Technologies. 1--2.Google Scholar
- Ronald Poelman, Oytun Akman, Stephan Lukosch, and Pieter Jonker. 2012. As if being there: mediated reality for crime scene investigation. In Proceedings of the ACM 2012 conference on computer supported cooperative work. 1267--1276.Google Scholar
Digital Library
- Lin Quan, John Heidemann, and Yuri Pradkin. 2013. Trinocular: Understanding internet reliability through adaptive probing. ACM SIGCOMM Computer Communication Review, Vol. 43, 4 (2013), 255--266.Google Scholar
Digital Library
- Fraser JM Reid and Susan E Reed. 2007. Conversational grounding and visual access in collaborative design. CoDesign, Vol. 3, 2 (2007), 111--122.Google Scholar
Cross Ref
- Kimberly Ruth, Tadayoshi Kohno, and Franziska Roesner. 2019. Secure multi-user content sharing for augmented reality applications. In 28th USENIX Security Symposium ($$USENIX$$ Security 19). 141--158.Google Scholar
- Harvey Sacks, Emanuel A Schegloff, and Gail Jefferson. 1978. A simplest systematics for the organization of turn taking for conversation. In Studies in the organization of conversational interaction. Elsevier, 7--55.Google Scholar
- M.F. Schober. 1995. Speakers, addressees, and frames of reference: Whose effort is minimized in conversations about locations? Discourse Processes, Vol. 20, 2 (1995), 219--247.Google Scholar
Cross Ref
- Ronell Sicat, Jiabao Li, JunYoung Choi, Maxime Cordeil, Won-Ki Jeong, Benjamin Bach, and Hanspeter Pfister. 2018. Dxr: A toolkit for building immersive data visualizations. IEEE transactions on visualization and computer graphics, Vol. 25, 1 (2018), 715--725.Google Scholar
Digital Library
- Rajinder S Sodhi, Brett R Jones, David Forsyth, Brian P Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 179--188.Google Scholar
Digital Library
- Theophilus Teo, Louise Lawrence, Gun A Lee, Mark Billinghurst, and Matt Adcock. 2019. Mixed reality remote collaboration combining 360 video and 3d reconstruction. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1--14.Google Scholar
Digital Library
- Trint. [n. d.]. Trint. https://trint.com/Google Scholar
- Unity. [n. d.]. Unity. https://unity3d.com [Online; accessed 21-September-2018].Google Scholar
- Yang Xu, Chenguang Yu, Jingjiang Li, and Yong Liu. 2012. Video telephony for end-consumers: measurement study of Google+, iChat, and Skype. In Proceedings of the 2012 Internet Measurement Conference. 371--384.Google Scholar
Digital Library
- Naomi Yamashita, Katsuhiko Kaji, Hideaki Kuzuoka, and Keiji Hirata. 2011. Improving visibility of remote gestures in distributed tabletop collaboration. In Proceedings of the ACM 2011 conference on Computer supported cooperative work. 95--104.Google Scholar
Digital Library
- Jeffrey M Zacks and Pascale Michelon. 2005. Transformations of visuospatial images. Behavioral and Cognitive Neuroscience Reviews, Vol. 4, 2 (2005), 96--118.Google Scholar
Cross Ref
Index Terms
The Effects of Network Outages on User Experience in Augmented Reality Based Remote Collaboration - An Empirical Study
Recommendations
Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsWe present Mini-Me, an adaptive avatar for enhancing Mixed Reality (MR) remote collaboration between a local Augmented Reality (AR) user and a remote Virtual Reality (VR) user. The Mini-Me avatar represents the VR user's gaze direction and body gestures ...
Merging environments for shared spaces in mixed reality
VRST '18: Proceedings of the 24th ACM Symposium on Virtual Reality Software and TechnologyIn virtual reality a real walking interface limits the extent of a virtual environment to our local walkable space. As local spaces are specific to each user, sharing a virtual environment with others for collaborative work or games becomes complicated. ...
A critical analysis on remote collaboration mediated by Augmented Reality: Making a case for improved characterization and evaluation of the collaborative process
AbstractRemote Collaboration mediated by Mixed and Augmented Reality (MR/AR) shows great potential in scenarios where physically distributed collaborators need to establish a common ground to achieve a shared goal. So far, most research ...
Graphical abstractDisplay Omitted
Highlights- How evaluation of the collaborative process is conducted in remote scenarios using AR.






Comments