Abstract
Augmented Reality (AR) developers face a proliferation of new platforms, devices, and frameworks. This often leads to applications being limited to a single platform and makes it hard to support collaborative AR scenarios involving multiple different devices. This paper presents XD-AR, a cross-device AR application development framework designed to unify input and output across hand-held, head-worn, and projective AR displays. XD-AR's design was informed by challenging scenarios for AR applications, a technical review of existing AR platforms, and a survey of 30 AR designers, developers, and users. Based on the results, we developed a taxonomy of AR system components and identified key challenges and opportunities in making them work together. We discuss how our taxonomy can guide the design of future AR platforms and applications and how cross-device interaction challenges could be addressed. We illustrate this when using XD-AR to implement two challenging AR applications from the literature in a device-agnostic way.
- R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre . 2001. Recent advances in augmented reality. IEEE Computer Graphics and Applications Vol. 21, 6 (Nov . 2001), 34--47. Google Scholar
Digital Library
- Ronald T Azuma . 1997. A survey of augmented reality. Presence: Teleoperators and virtual environments Vol. 6, 4 (1997), 355--385. Google Scholar
Digital Library
- Martin Bauer, Bernd Bruegge, Gudrun Klinker, Asa MacWilliams, Thomas Reicher, Stefan Riss, Christian Sandor, and Martin Wagner . 2001. Design of a component-based augmented reality framework Proc. IEEE and ACM International Symposium on Augmented Reality. 45--54. Google Scholar
Digital Library
- Oliver Baus and Stéphane Bouchard . 2014. Moving from Virtual Reality Exposure-Based Therapy to Augmented Reality Exposure-Based Therapy: A Review. Front. Hum. Neurosci. Vol. 8, March (2014), 1--15.Google Scholar
- Mark Billinghurst, Adrian Clark, and Gun Lee . 2015. A Survey of Augmented Reality. Foundations and Trends in Human-Computer Interaction Vol. 8, 2--3 (2015), 73--272. Google Scholar
Digital Library
- Julie Carmigniani, Borko Furht, Marco Anisetti, Paolo Ceravolo, Ernesto Damiani, and Misa Ivkovic . 2011. Augmented reality technologies, systems and applications. Multimedia Tools and Applications Vol. 51, 1 (Jan . 2011), 341--377. Google Scholar
Digital Library
- Pei-Yu (Peggy) Chi and Yang Li . 2015. Weave: Scripting Cross-Device Wearable Interaction Proc. CHI. 3923--3932. Google Scholar
Digital Library
- Francesco Clemente, Strahinja Dosen, Luca Lonini, Marko Markovic, Dario Farina, and Christian Cipriani . 2017. Humans can integrate augmented reality feedback in their sensorimotor control of a robotic hand. IEEE Transactions on Human-Machine Systems Vol. 47, 4 (2017), 583--589.Google Scholar
Cross Ref
- David Dearman and Jeffrey S. Pierce . 2008. “It's on my other Computer!”: Computing with Multiple Devices Proc. CHI. 767--776. Google Scholar
Digital Library
- David Drascic and Paul Milgram . 1996. Perceptual issues in augmented reality. In SPIE The International Society For Optical Engineering. 123--134.Google Scholar
- Mark Fiala . 2004. ARTag Revision 1. A Fiducial Marker System Using Digital Techniques. National Research Council Publication Vol. 47419 (2004), 1--47.Google Scholar
- Luca Frosini and Fabio Paternò . 2014. User Interface Distribution in Multi-Device and Multi-User Environments with Dynamically Migrating Engines. In Proc. EICS. 55--64. Google Scholar
Digital Library
- Peter Hamilton and Daniel J. Wigdor . 2014. Conductor: Enabling and Understanding Cross-Device Interaction Proc. CHI. 2773--2782. Google Scholar
Digital Library
- Thuong Hoang, Martin Reinoso, Zaher Joukhadar, Frank Vetere, and David Kelly . 2017. Augmented Studio: Projection Mapping on Moving Body for Physiotherapy Education Proc. CHI. 1419--1430. Google Scholar
Digital Library
- Steven Houben and Nicolai Marquardt . 2015. WatchConnect: A Toolkit for Prototyping Smartwatch-Centric Cross-Device Applications. In Proc. CHI. 1247--1256. Google Scholar
Digital Library
- Tero Jokela, Jarno Ojala, and Thomas Olsson . 2015. A Diary Study on Combining Multiple Information Devices in Everyday Activities and Tasks. In Proc. CHI. 3903--3912. Google Scholar
Digital Library
- Brett Jones, Rajinder Sodhi, Michael Murdock, Ravish Mehra, Hrvoje Benko, Andrew Wilson, Eyal Ofek, Blair MacIntyre, Nikunj Raghuvanshi, and Lior Shapira . 2014. RoomAlive: Magical Experiences Enabled by Scalable, Adaptive Projector-camera Units Proc. UIST. 637--644. Google Scholar
Digital Library
- Seokmin Jung and Woontack Woo . 2004. UbiTrack: Infrared-based user Tracking System for indoor environment. International Conference on Artificial Reality and Telexisitence (ICAT04) (2004), 1345--1278.Google Scholar
- Oliver Beren Kaul and Michael Rohs . 2017. HapticHead: A Spherical Vibrotactile Grid Around the Head for 3D Guidance in Virtual and Augmented Reality. In Proc. CHI. 3729--3740. Google Scholar
Digital Library
- M. A. Livingston . 2005. Evaluating human factors in augmented reality systems. IEEE Computer Graphics and Applications Vol. 25, 6 (Nov . 2005), 6--9. Google Scholar
Digital Library
- Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter . 2004. DART: a toolkit for rapid design exploration of augmented reality experiences Proceedings of the 17th annual ACM symposium on User interface software and technology. ACM, 197--206. Google Scholar
Digital Library
- Matt Miesnieks . 2017 a. How is ARCore better than ARKit? (2017). https://medium.com/super-ventures-blog/how-is-arcore-better-than-arkit-5223e6b3e79dGoogle Scholar
- Matt Miesnieks . 2017 b. Why is ARKit better than the alternatives? (2017). https://medium.com/super-ventures-blog/why-is-arkit-better-than-the-alternatives-af8871889d6aGoogle Scholar
- Paul Milgram and Fumio Kishino . 1994. A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems Vol. 77, 12 (1994), 1321--1329.Google Scholar
- Michael Nebeling and Anind K. Dey . 2016. XDBrowser: User-Defined Cross-Device Web Page Designs Proc. CHI. 5494--5505. Google Scholar
Digital Library
- Michael Nebeling, Theano Mintsi, Maria Husmann, and Moira C. Norrie . 2014 a. Interactive Development of Cross-Device User Interfaces Proc. CHI. 2793--2802. Google Scholar
Digital Library
- Michael Nebeling, Elena Teunissen, Maria Husmann, and Moira C. Norrie . 2014 b. XDKinect: Development Framework for Cross-Device Interaction using Kinect Proc. EICS. 2793--2802. Google Scholar
Digital Library
- Antti Oulasvirta and Lauri Sumari . 2007. Mobile Kits and Laptop Trays: Managing Multiple Devices in Mobile Information Work. In Proc. CHI. 1127--1136. Google Scholar
Digital Library
- Jeffrey S. Pierce and Jeffrey Nichols . 2008. An infrastructure for extending applications' user experiences across multiple personal devices. In Proc. UIST. 101--110. Google Scholar
Digital Library
- Roman R"adle, Hans-Christian Jetter, Mario Schreiner, Zhihao Lu, Harald Reiterer, and Yvonne Rogers . 2015. Spatially-aware or Spatially-agnostic?: Elicitation and Evaluation of User-Defined Cross-Device Interactions. In Proc. CHI. 3913--3922. Google Scholar
Digital Library
- Gerhard Reitmayr and Dieter Schmalstieg . 2005. OpenTracker: A flexible software design for three-dimensional interaction. Virtual reality Vol. 9, 1 (2005), 79--92. Google Scholar
Digital Library
- Johnny Salda na . 2015. The coding manual for qualitative researchers. Sage.Google Scholar
- Stephanie Santosa and Daniel Wigdor . 2013. A Field Study of Multi-Device Workflows in Distributed Workspaces Proc. UbiComp. 63--72. Google Scholar
Digital Library
- Jie Song, Gábor Sörös, Fabrizio Pece, Sean Ryan Fanello, Shahram Izadi, Cem Keskin, and Otmar Hilliges . 2014. In-air gestures around unmodified mobile devices. In Proc. UIST. 319--329. Google Scholar
Digital Library
- Sebastian Thrun and John J Leonard . 2008. Simultaneous localization and mapping. In Springer handbook of robotics. Springer, 871--889.Google Scholar
- DWF Van Krevelen and Ronald Poelman . 2010. A survey of augmented reality technologies, applications and limitations. International Journal of Virtual Reality Vol. 9, 2 (2010), 1--20.Google Scholar
Cross Ref
- Maja Wrzesien, Cristina Botella, Juana Bretón-López, Eva del R'ıo González, Jean-Marie Burkhardt, Mariano Alca niz, and Mar'ıa Ángeles Pérez-Ara . 2015. Treating small animal phobias using a projective-augmented reality system: A single-case study. Computers in Human Behavior Vol. 49 (2015), 343--353. Google Scholar
Digital Library
- Jishuo Yang and Daniel Wigdor . 2014. Panelrama: Enabling Easy Specification of Cross-Device Web Applications Proc. CHI. 2783--2792. Google Scholar
Digital Library
- Feng Zhou, Henry Been-Lirn Duh, and Mark Billinghurst . 2008. Trends in Augmented Reality Tracking, Interaction and Display: A Review of Ten Years of ISMAR. In Proc. ISMAR. 193--202. Google Scholar
Digital Library
Index Terms
XD-AR: Challenges and Opportunities in Cross-Device Augmented Reality Application Development
Recommendations
Survey on Collaborative AR for Multi-user in Urban Studies and Planning
Edutainment '09: Proceedings of the 4th International Conference on E-Learning and Games: Learning by Playing. Game-based Education System Design and DevelopmentThis paper describes an augmented reality (AR) environment that allows multiple participants or multi-user to interact with 2D and 3D data. AR simply can provide a collaborative interactive AR environment for urban planning, where users can interact ...
Dual Phone AR: Using a Second Phone as a Controller for Mobile Augmented Reality
UIST '20 Adjunct: Adjunct Proceedings of the 33rd Annual ACM Symposium on User Interface Software and TechnologyMobile Augmented Reality applications have become increasingly popular, however the possible interactions with AR content are largely limited to on-screen gestures and spatial movement. There has been a renewed interest in designing interaction methods ...
Exhibition approach using an AR and VR pillar
SA '17: SIGGRAPH Asia 2017 Mobile Graphics & Interactive ApplicationsThis demonstration presents a development of an Augmented Reality (AR) and Virtual Reality (AR) pillar, a novel approach for showing AR and VR content in a public setting. A pillar in a public exhibition venue was converted to a four-sided AR and VR ...






Comments