skip to main content
research-article
Public Access

XD-AR: Challenges and Opportunities in Cross-Device Augmented Reality Application Development

Published:19 June 2018Publication History
Skip Abstract Section

Abstract

Augmented Reality (AR) developers face a proliferation of new platforms, devices, and frameworks. This often leads to applications being limited to a single platform and makes it hard to support collaborative AR scenarios involving multiple different devices. This paper presents XD-AR, a cross-device AR application development framework designed to unify input and output across hand-held, head-worn, and projective AR displays. XD-AR's design was informed by challenging scenarios for AR applications, a technical review of existing AR platforms, and a survey of 30 AR designers, developers, and users. Based on the results, we developed a taxonomy of AR system components and identified key challenges and opportunities in making them work together. We discuss how our taxonomy can guide the design of future AR platforms and applications and how cross-device interaction challenges could be addressed. We illustrate this when using XD-AR to implement two challenging AR applications from the literature in a device-agnostic way.

References

  1. R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre . 2001. Recent advances in augmented reality. IEEE Computer Graphics and Applications Vol. 21, 6 (Nov . 2001), 34--47. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Ronald T Azuma . 1997. A survey of augmented reality. Presence: Teleoperators and virtual environments Vol. 6, 4 (1997), 355--385. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Martin Bauer, Bernd Bruegge, Gudrun Klinker, Asa MacWilliams, Thomas Reicher, Stefan Riss, Christian Sandor, and Martin Wagner . 2001. Design of a component-based augmented reality framework Proc. IEEE and ACM International Symposium on Augmented Reality. 45--54. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Oliver Baus and Stéphane Bouchard . 2014. Moving from Virtual Reality Exposure-Based Therapy to Augmented Reality Exposure-Based Therapy: A Review. Front. Hum. Neurosci. Vol. 8, March (2014), 1--15.Google ScholarGoogle Scholar
  5. Mark Billinghurst, Adrian Clark, and Gun Lee . 2015. A Survey of Augmented Reality. Foundations and Trends in Human-Computer Interaction Vol. 8, 2--3 (2015), 73--272. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Julie Carmigniani, Borko Furht, Marco Anisetti, Paolo Ceravolo, Ernesto Damiani, and Misa Ivkovic . 2011. Augmented reality technologies, systems and applications. Multimedia Tools and Applications Vol. 51, 1 (Jan . 2011), 341--377. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Pei-Yu (Peggy) Chi and Yang Li . 2015. Weave: Scripting Cross-Device Wearable Interaction Proc. CHI. 3923--3932. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Francesco Clemente, Strahinja Dosen, Luca Lonini, Marko Markovic, Dario Farina, and Christian Cipriani . 2017. Humans can integrate augmented reality feedback in their sensorimotor control of a robotic hand. IEEE Transactions on Human-Machine Systems Vol. 47, 4 (2017), 583--589.Google ScholarGoogle ScholarCross RefCross Ref
  9. David Dearman and Jeffrey S. Pierce . 2008. “It's on my other Computer!”: Computing with Multiple Devices Proc. CHI. 767--776. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. David Drascic and Paul Milgram . 1996. Perceptual issues in augmented reality. In SPIE The International Society For Optical Engineering. 123--134.Google ScholarGoogle Scholar
  11. Mark Fiala . 2004. ARTag Revision 1. A Fiducial Marker System Using Digital Techniques. National Research Council Publication Vol. 47419 (2004), 1--47.Google ScholarGoogle Scholar
  12. Luca Frosini and Fabio Paternò . 2014. User Interface Distribution in Multi-Device and Multi-User Environments with Dynamically Migrating Engines. In Proc. EICS. 55--64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Peter Hamilton and Daniel J. Wigdor . 2014. Conductor: Enabling and Understanding Cross-Device Interaction Proc. CHI. 2773--2782. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Thuong Hoang, Martin Reinoso, Zaher Joukhadar, Frank Vetere, and David Kelly . 2017. Augmented Studio: Projection Mapping on Moving Body for Physiotherapy Education Proc. CHI. 1419--1430. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Steven Houben and Nicolai Marquardt . 2015. WatchConnect: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications. In Proc. CHI. 1247--1256. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Tero Jokela, Jarno Ojala, and Thomas Olsson . 2015. A Diary Study on Combining Multiple Information Devices in Everyday Activities and Tasks. In Proc. CHI. 3903--3912. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Brett Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira . 2014. RoomAlive: Magical Experiences Enabled by Scalable, Adaptive Projector-camera Units Proc. UIST. 637--644. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Seokmin Jung and Woontack Woo . 2004. UbiTrack: Infrared-based user Tracking System for indoor environment. International Conference on Artificial Reality and Telexisitence (ICAT04) (2004), 1345--1278.Google ScholarGoogle Scholar
  19. Oliver Beren Kaul and Michael Rohs . 2017. HapticHead: A Spherical Vibrotactile Grid Around the Head for 3D Guidance in Virtual and Augmented Reality. In Proc. CHI. 3729--3740. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. A. Livingston . 2005. Evaluating human factors in augmented reality systems. IEEE Computer Graphics and Applications Vol. 25, 6 (Nov . 2005), 6--9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter . 2004. DART: a toolkit for rapid design exploration of augmented reality experiences Proceedings of the 17th annual ACM symposium on User interface software and technology. ACM, 197--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Matt Miesnieks . 2017 a. How is ARCore better than ARKit? (2017). https://medium.com/super-ventures-blog/how-is-arcore-better-than-arkit-5223e6b3e79dGoogle ScholarGoogle Scholar
  23. Matt Miesnieks . 2017 b. Why is ARKit better than the alternatives? (2017). https://medium.com/super-ventures-blog/why-is-arkit-better-than-the-alternatives-af8871889d6aGoogle ScholarGoogle Scholar
  24. Paul Milgram and Fumio Kishino . 1994. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems Vol. 77, 12 (1994), 1321--1329.Google ScholarGoogle Scholar
  25. Michael Nebeling and Anind K. Dey . 2016. XDBrowser: User-Defined Cross-Device Web Page Designs Proc. CHI. 5494--5505. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Michael Nebeling, Theano Mintsi, Maria Husmann, and Moira C. Norrie . 2014 a. Interactive Development of Cross-Device User Interfaces Proc. CHI. 2793--2802. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Michael Nebeling, Elena Teunissen, Maria Husmann, and Moira C. Norrie . 2014 b. XDKinect: Development Framework for Cross-Device Interaction using Kinect Proc. EICS. 2793--2802. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Antti Oulasvirta and Lauri Sumari . 2007. Mobile Kits and Laptop Trays: Managing Multiple Devices in Mobile Information Work. In Proc. CHI. 1127--1136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Jeffrey S. Pierce and Jeffrey Nichols . 2008. An infrastructure for extending applications' user experiences across multiple personal devices. In Proc. UIST. 101--110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Roman R"adle, Hans-Christian Jetter, Mario Schreiner, Zhihao Lu, Harald Reiterer, and Yvonne Rogers . 2015. Spatially-aware or Spatially-agnostic?: Elicitation and Evaluation of User-Defined Cross-Device Interactions. In Proc. CHI. 3913--3922. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Gerhard Reitmayr and Dieter Schmalstieg . 2005. OpenTracker: A flexible software design for three-dimensional interaction. Virtual reality Vol. 9, 1 (2005), 79--92. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Johnny Salda na . 2015. The coding manual for qualitative researchers. Sage.Google ScholarGoogle Scholar
  33. Stephanie Santosa and Daniel Wigdor . 2013. A Field Study of Multi-Device Workflows in Distributed Workspaces Proc. UbiComp. 63--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Jie Song, Gábor Sörös, Fabrizio Pece, Sean Ryan Fanello, Shahram Izadi, Cem Keskin, and Otmar Hilliges . 2014. In-air gestures around unmodified mobile devices. In Proc. UIST. 319--329. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Sebastian Thrun and John J Leonard . 2008. Simultaneous localization and mapping. In Springer handbook of robotics. Springer, 871--889.Google ScholarGoogle Scholar
  36. DWF Van Krevelen and Ronald Poelman . 2010. A survey of augmented reality technologies, applications and limitations. International Journal of Virtual Reality Vol. 9, 2 (2010), 1--20.Google ScholarGoogle ScholarCross RefCross Ref
  37. Maja Wrzesien, Cristina Botella, Juana Bretón-López, Eva del R'ıo González, Jean-Marie Burkhardt, Mariano Alca niz, and Mar'ıa Ángeles Pérez-Ara . 2015. Treating small animal phobias using a projective-augmented reality system: A single-case study. Computers in Human Behavior Vol. 49 (2015), 343--353. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Jishuo Yang and Daniel Wigdor . 2014. Panelrama: Enabling Easy Specification of Cross-Device Web Applications Proc. CHI. 2783--2792. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Feng Zhou, Henry Been-Lirn Duh, and Mark Billinghurst . 2008. Trends in Augmented Reality Tracking, Interaction and Display: A Review of Ten Years of ISMAR. In Proc. ISMAR. 193--202. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. XD-AR: Challenges and Opportunities in Cross-Device Augmented Reality Application Development

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!