skip to main content
research-article
Open Access

The Effects of Network Outages on User Experience in Augmented Reality Based Remote Collaboration - An Empirical Study

Authors Info & Claims
Published:18 October 2021Publication History
Skip Abstract Section

Abstract

Augmented Reality (AR) applications can enable geographically distant users to collaborate using shared video feeds or interactive 3D holograms, and may be particularly useful in the socially distant context of the Covid-19 pandemic. However, a good user experience is key for their success and could be negatively impacted by network impairments, which are an inevitable occurrence in today's best-effort Internet. In this paper, we present the findings of an empirical user study, aimed at understanding the effects of network outages, on user experience and behavior, in a collaborative AR task. We highlight how network outages affected users in different ways depending on their role in the collaborative task, and how giving users explicit information about poor network conditions helped them deal with some of these negative effects. Furthermore, we report the strategies that users themselves adopted, to deal with outages, such as batching instructions, or shifting to a different spatial referencing style when communicating with their partners. Lastly, based on our findings, we present some design implications for future remote-collaborative AR applications.

References

  1. [n. d.]. http://arsharing.cs.washington.edu/index.htmlGoogle ScholarGoogle Scholar
  2. [n. d.]. https://objecttheory.com/platform/Google ScholarGoogle Scholar
  3. 2020. 10 BEST Augmented Reality Glasses (Smart Glasses) In 2021. https://www.softwaretestinghelp.com/best-augmented-reality-glasses/Google ScholarGoogle Scholar
  4. Giuseppe Aceto, Alessio Botta, Pietro Marchetta, Valerio Persico, and Antonio Pescapé. 2018. A comprehensive survey on internet outages. Journal of Network and Computer Applications, Vol. 113 (2018), 36--63.Google ScholarGoogle ScholarCross RefCross Ref
  5. Sudhir Aggarwal, Hemant Banavar, Amit Khandelwal, Sarit Mukherjee, and Sampath Rangarajan. 2004. Accuracy in dead-reckoning based distributed multi-player games. In Proceedings of 3rd ACM SIGCOMM workshop on Network and system support for games. 161--165.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Tooba Ahsen, Fahad R Dogar, and Aaron L Gardony. 2019. Exploring the Impact of Network Impairments on Remote Collaborative Augmented Reality Applications. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, LBW2619.Google ScholarGoogle Scholar
  7. Deepak Akkil and Poika Isokoski. 2016a. Accuracy of interpreting pointing gestures in egocentric view. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 262--273.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Deepak Akkil and Poika Isokoski. 2016b. Gaze Augmentation in Egocentric Video Improves Awareness of Intention. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1573--1584.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Deepak Akkil, Jobin Mathew James, Poika Isokoski, and Jari Kangas. 2016. GazeTorch: Enabling Gaze Awareness in Collaborative Physical Tasks. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 1151--1158.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Judith Amores, Xavier Benavides, and Pattie Maes. 2015. Showme: A remote collaboration system that supports immersive gestural communication. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. 1343--1348.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Huidong Bai, Prasanth Sasikumar, Jing Yang, and Mark Billinghurst. 2020. A user study on mixed reality remote collaboration with eye gaze and hand gesture sharing. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1--13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Yahn W Bernier. 2001. Latency compensating methods in client/server in-game protocol design and optimization. In Game Developers Conference, Vol. 98033.Google ScholarGoogle Scholar
  13. Henrik Brun, Robin Anton Birkeland Bugge, LKR Suther, Sigurd Birkeland, Rahul Kumar, Egidijus Pelanis, and Ole Jacob Elle. 2019. Mixed reality holograms for heart surgery planning: first user experience in congenital heart disease. European Heart Journal-Cardiovascular Imaging, Vol. 20, 8 (2019), 883--888.Google ScholarGoogle ScholarCross RefCross Ref
  14. Wolfgang Büschel. [n. d.]. Challenges in Collaborative Immersive Visualization. ([n. d.]).Google ScholarGoogle Scholar
  15. Simon Butscher, Sebastian Hubenschmid, Jens Müller, Johannes Fuchs, and Harald Reiterer. 2018. Clusters, trends, and outliers: How immersive technologies can facilitate the collaborative analysis of multidimensional data. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--12.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Tolga K Capin and Igor S Pandzic. 1997. A dead-reckoning algorithm for virtual human figures. In Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality. IEEE, 161--169.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Armando L Caro, Janardhan R Iyengar, Paul D Amer, Sourabh Ladha, Gerard J Heinz, and Keyur C Shah. 2003. SCTP: a proposed standard for robust internet data transport. Computer, Vol. 36, 11 (2003), 56--63.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kuan-Ta Chen, Chun-Ying Huang, Polly Huang, and Chin-Laung Lei. 2006. Quantifying Skype user satisfaction. ACM SIGCOMM Computer Communication Review, Vol. 36, 4 (2006), 399--410.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Marshini Chetty, Srikanth Sundaresan, Sachit Muckaden, Nick Feamster, and Enrico Calandro. 2013. Measuring broadband performance in South Africa. In Proceedings of the 4th Annual Symposium on Computing for Development. ACM, 1.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Herbert H. Clark. 1996. Common ground .Cambridge University Press, 92--122. https://doi.org/10.1017/CBO9780511620539.005Google ScholarGoogle Scholar
  21. Herbert H Clark and Susan E Brennan. 1991. Grounding in communication. (1991).Google ScholarGoogle Scholar
  22. Catherine Durnell Cramton. 2001. The mutual knowledge problem and its consequences for dispersed collaboration. Organization science, Vol. 12, 3 (2001), 346--371.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Alberto Dainotti, Claudio Squarcella, Emile Aben, Kimberly C Claffy, Marco Chiesa, Michele Russo, and Antonio Pescapé. 2011. Analysis of country-wide internet outages caused by censorship. In Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conference. ACM, 1--18.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Barrett Ens, Sarah Goodwin, Arnaud Prouzeau, Fraser Anderson, Florence Y Wang, Samuel Gratzl, Zac Lucarelli, Brendan Moyle, Jim Smiley, and Tim Dwyer. 2020. Uplift: A Tangible and Immersive Tabletop System for Casual Collaborative Visual Analytics. IEEE Transactions on Visualization and Computer Graphics (2020).Google ScholarGoogle Scholar
  25. Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, and Mark Billinghurst. 2019. Revisiting collaboration through mixed reality: The evolution of groupware. International Journal of Human-Computer Studies, Vol. 131 (2019), 81--98.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Brian Eriksson, Ramakrishnan Durairajan, and Paul Barford. 2013. Riskroute: a framework for mitigating network outage threats. In Proceedings of the ninth ACM conference on Emerging networking experiments and technologies. ACM, 405--416.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. P Esteban, JE Jaramillo, N Álvarez, J Restrepo, and H Trefftz. 2004. Augmented Reality: a space for the understanding of multi-variate Calculus. In Proceedings of the IADATE conference. Spain.Google ScholarGoogle Scholar
  28. Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014a. In touch with the remote world: Remote collaboration with augmented reality drawings and virtual navigation. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology. 197--205.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014b. World-stabilized annotations and virtual scene navigation for remote collaboration. In Proceedings of the 27th annual ACM symposium on User interface software and technology. 449--459.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Aaron M Genest, Carl Gutwin, Anthony Tang, Michael Kalyn, and Zenja Ivkovic. 2013. KinectArms: a toolkit for capturing and displaying arm embodiments in distributed tabletop groupware. In Proceedings of the 2013 conference on Computer supported cooperative work. 157--166.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004 a. Action as language in a shared visual space. In Proceedings of the 2004 ACM conference on Computer supported cooperative work. ACM, 487--496.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Darren Gergle, Robert E Kraut, and Susan R Fussell. 2004 b. Language efficiency and visual technology: Minimizing collaborative effort with visual information. Journal of language and social psychology, Vol. 23, 4 (2004), 491--517.Google ScholarGoogle ScholarCross RefCross Ref
  33. Darren Gergle, Robert E Kraut, and Susan R Fussell. 2013. Using visual information for grounding and awareness in collaborative tasks. Human-Computer Interaction, Vol. 28, 1 (2013), 1--39.Google ScholarGoogle Scholar
  34. Raphaël Grasset, Philip Lamb, and Mark Billinghurst. 2005. Evaluation of mixed-space collaboration. In Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 90--99.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Sebastian Günther, Sven Kratz, Daniel Avrahami, and Max Mühlh"auser. 2018. Exploring Audio, Visual, and Tactile Cues for Synchronous Remote Assistance. In Proceedings of the 11th Pervasive Technologies Related to Assistive Environments Conference. 339--344.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Felix G Hamza-Lup, Jannick P Rolland, and Charles Hughes. 2018. A distributed augmented reality system for medical training and simulation. arXiv preprint arXiv:1811.12815 (2018).Google ScholarGoogle Scholar
  37. Osama Haq and Fahad R Dogar. 2015. Leveraging the power of cloud for reliable wide area communication. In Proc. ACM HotNets.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Tomoki Itamiya, Toshinori Iwai, and Tsuyoshi Kaneko. 2018. The Holographic Human for surgical navigation using Microsoft HoloLens. EPiC Series in Engineering, Vol. 1 (2018), 26--30.Google ScholarGoogle ScholarCross RefCross Ref
  39. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz. 2018. Super slomo: High quality estimation of multiple intermediate frames for video interpolation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9000--9008.Google ScholarGoogle ScholarCross RefCross Ref
  40. Junchen Jiang, Rajdeep Das, Ganesh Ananthanarayanan, Philip A Chou, Venkata Padmanabhan, Vyas Sekar, Esbjorn Dominique, Marcin Goliszewski, Dalibor Kukoleca, Renat Vafin, et al. 2016. Via: Improving internet telephony call quality using predictive relay selection. In Proc. SIGCOMM'16. ACM.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Wenyu Jiang and Henning Schulzrinne. 2003. Assessment of voip service availability in the current internet. In Proceedings of the 4th International Workshop on Passive and Active Network Measurement (PAM 2003).Google ScholarGoogle Scholar
  42. Laura Casado Joey Hadden. 2020. 21 major companies that have announced employees can work remotely long-term. https://www.businessinsider.com/companies-asking-employees-to-work-from-home-due-to-coronavirus-2020Google ScholarGoogle Scholar
  43. Steven Johnson, Madeleine Gibson, and Bilge Mutlu. 2015. Handheld or handsfree?: Remote collaboration via lightweight head-mounted displays and handheld devices. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1825--1836.Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Ethan Katz-Bassett, Colin Scott, David R Choffnes, Ítalo Cunha, Vytautas Valancius, Nick Feamster, Harsha V Madhyastha, Thomas Anderson, and Arvind Krishnamurthy. 2012. LIFEGUARD: Practical repair of persistent route failures. In Proceedings of the ACM SIGCOMM 2012 conference on Applications, technologies, architectures, and protocols for computer communication. ACM, 395--406.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Hannes Kaufmann. 2002. Construct3D: an augmented reality application for mathematics and geometry education. In Proceedings of the tenth ACM international conference on Multimedia. 656--657.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Seungwon Kim, Allison Jing, Hanhoon Park, Gun A Lee, Weidong Huang, and Mark Billinghurst. 2020. Hand-in-Air (HiA) and Hand-on-Target (HoT) Style Gesture Cues for Mixed Reality Collaboration. IEEE Access, Vol. 8 (2020), 224145--224161.Google ScholarGoogle Scholar
  47. Kiyoshi Kiyokawa, Mark Billinghurst, Sohan E Hayes, Anoop Gupta, Yuki Sannohe, and Hirokazu Kato. 2002. Communication behaviors of co-located users in collaborative AR interfaces. In Proceedings of the 1st International Symposium on Mixed and Augmented Reality. IEEE Computer Society, 139.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Adam Langley, Alistair Riddoch, Alyssa Wilk, Antonio Vicente, Charles Krasic, Dan Zhang, Fan Yang, Fedor Kouranov, Ian Swett, Janardhan Iyengar, et al. 2017. The quic transport protocol: Design and internet-scale deployment. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication. 183--196.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Chapter 11 - Analyzing qualitative data. In Research Methods in Human Computer Interaction (Second Edition) second edition ed.), Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser (Eds.). Morgan Kaufmann, Boston, 299 -- 327. https://doi.org/10.1016/B978-0--12--805390--4.00011-XGoogle ScholarGoogle ScholarDigital LibraryDigital Library
  50. Jun Li and Scott Brooks. 2011. I-seismograph: Observing and measuring Internet earthquakes. In INFOCOM, 2011 Proceedings IEEE. IEEE, 2624--2632.Google ScholarGoogle ScholarCross RefCross Ref
  51. Stephan Lukosch, Heide Lukosch, Dragos Datcu, and Marina Cidota. 2015. On the spot information in augmented reality for teams in the security domain. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. ACM, 983--988.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Tahir Mahmood, Willis Fulmer, Neelesh Mungoli, Jian Huang, and Aidong Lu. 2019. Improving information sharing and collaborative analysis for remote geospatial visualization using mixed reality. In 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 236--247.Google ScholarGoogle ScholarCross RefCross Ref
  53. Shawn McKee, Mike O'Connor, Soichi Hayashi, Marian Babik, Brian Tierney, Ilija Vukotic, Les Cottrell, and Henryk Giemza. 2016. ICFA SCIC Network Monitoring Report. Technical Report.Google ScholarGoogle Scholar
  54. Microsoft. [n. d.] a. Microsoft HoloLens. https://www.microsoft.com/en-us/hololens [Online; accessed 21-September-2018].Google ScholarGoogle Scholar
  55. Microsoft. [n. d.] b. Mixed Reality Toolkit. https://github.com/Microsoft/MixedRealityToolkit-Unity [Online; accessed 21-September-2018].Google ScholarGoogle Scholar
  56. Jens Müller, Roman R"adle, and Harald Reiterer. 2016. Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1245--1249.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Jens Müller, Roman R"adle, and Harald Reiterer. 2017. Remote collaboration with mixed reality displays: how shared virtual landmarks facilitate spatial referencing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 6481--6486.Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Nahal Norouzi*, Austin Erickson*, Kangsoo Kim, Ryan Schubert, Joseph LaViola, Gerd Bruder, and Greg Welch. 2019. Effects of shared gaze parameters on visual target identification task performance in augmented reality. In Symposium on Spatial User Interaction. 1--11.Google ScholarGoogle Scholar
  59. Lyn Pemberton and Marcus Winter. 2009. Collaborative augmented reality in schools. (2009).Google ScholarGoogle Scholar
  60. Thammathip Piumsomboon, Arindam Dey, Barrett Ens, Gun Lee, and Mark Billinghurst. 2019 a. The effects of sharing awareness cues in collaborative mixed reality. Frontiers in Robotics and AI, Vol. 6 (2019), 5.Google ScholarGoogle ScholarCross RefCross Ref
  61. Thammathip Piumsomboon, Gun A Lee, Jonathon D Hart, Barrett Ens, Robert W Lindeman, Bruce H Thomas, and Mark Billinghurst. 2018. Mini-me: An adaptive avatar for mixed reality remote collaboration. In Proceedings of the 2018 CHI conference on human factors in computing systems. 1--13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Thammathip Piumsomboon, Gun A Lee, Andrew Irlitti, Barrett Ens, Bruce H Thomas, and Mark Billinghurst. 2019 b. On the shoulder of the giant: A multi-scale mixed reality collaboration with 360 video sharing and tangible interaction. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1--17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Thammathip Piumsomboon, Youngho Lee, Gun Lee, and Mark Billinghurst. 2017. CoVAR: a collaborative virtual and augmented reality system for remote collaboration. In SIGGRAPH Asia 2017 Emerging Technologies. 1--2.Google ScholarGoogle Scholar
  64. Ronald Poelman, Oytun Akman, Stephan Lukosch, and Pieter Jonker. 2012. As if being there: mediated reality for crime scene investigation. In Proceedings of the ACM 2012 conference on computer supported cooperative work. 1267--1276.Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. Lin Quan, John Heidemann, and Yuri Pradkin. 2013. Trinocular: Understanding internet reliability through adaptive probing. ACM SIGCOMM Computer Communication Review, Vol. 43, 4 (2013), 255--266.Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Fraser JM Reid and Susan E Reed. 2007. Conversational grounding and visual access in collaborative design. CoDesign, Vol. 3, 2 (2007), 111--122.Google ScholarGoogle ScholarCross RefCross Ref
  67. Kimberly Ruth, Tadayoshi Kohno, and Franziska Roesner. 2019. Secure multi-user content sharing for augmented reality applications. In 28th USENIX Security Symposium ($$USENIX$$ Security 19). 141--158.Google ScholarGoogle Scholar
  68. Harvey Sacks, Emanuel A Schegloff, and Gail Jefferson. 1978. A simplest systematics for the organization of turn taking for conversation. In Studies in the organization of conversational interaction. Elsevier, 7--55.Google ScholarGoogle Scholar
  69. M.F. Schober. 1995. Speakers, addressees, and frames of reference: Whose effort is minimized in conversations about locations? Discourse Processes, Vol. 20, 2 (1995), 219--247.Google ScholarGoogle ScholarCross RefCross Ref
  70. Ronell Sicat, Jiabao Li, JunYoung Choi, Maxime Cordeil, Won-Ki Jeong, Benjamin Bach, and Hanspeter Pfister. 2018. Dxr: A toolkit for building immersive data visualizations. IEEE transactions on visualization and computer graphics, Vol. 25, 1 (2018), 715--725.Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Rajinder S Sodhi, Brett R Jones, David Forsyth, Brian P Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 179--188.Google ScholarGoogle ScholarDigital LibraryDigital Library
  72. Theophilus Teo, Louise Lawrence, Gun A Lee, Mark Billinghurst, and Matt Adcock. 2019. Mixed reality remote collaboration combining 360 video and 3d reconstruction. In Proceedings of the 2019 CHI conference on human factors in computing systems. 1--14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  73. Trint. [n. d.]. Trint. https://trint.com/Google ScholarGoogle Scholar
  74. Unity. [n. d.]. Unity. https://unity3d.com [Online; accessed 21-September-2018].Google ScholarGoogle Scholar
  75. Yang Xu, Chenguang Yu, Jingjiang Li, and Yong Liu. 2012. Video telephony for end-consumers: measurement study of Google+, iChat, and Skype. In Proceedings of the 2012 Internet Measurement Conference. 371--384.Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Naomi Yamashita, Katsuhiko Kaji, Hideaki Kuzuoka, and Keiji Hirata. 2011. Improving visibility of remote gestures in distributed tabletop collaboration. In Proceedings of the ACM 2011 conference on Computer supported cooperative work. 95--104.Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. Jeffrey M Zacks and Pascale Michelon. 2005. Transformations of visuospatial images. Behavioral and Cognitive Neuroscience Reviews, Vol. 4, 2 (2005), 96--118.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. The Effects of Network Outages on User Experience in Augmented Reality Based Remote Collaboration - An Empirical Study

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!