skip to main content
research-article

Design, Large-Scale Usage Testing, and Important Metrics for Augmented Reality Gaming Applications

Authors Info & Claims
Published:05 June 2019Publication History
Skip Abstract Section

Abstract

Augmented Reality (AR) offers the possibility to enrich the real world with digital mediated content, increasing in this way the quality of many everyday experiences. While in some research areas such as cultural heritage, tourism, or medicine there is a strong technological investment, AR for game purposes struggles to become a widespread commercial application. In this article, a novel framework for AR kid games is proposed, already developed by the authors for other AR applications such as Cultural Heritage and Arts. In particular, the framework includes different layers such as the development of a series of AR kid puzzle games in an intermediate structure which can be used as a standard for different applications development, the development of a smart configuration tool, together with general guidelines and long-life usage tests and metrics. The proposed application is designed for augmenting the puzzle experience, but can be easily extended to other AR gaming applications. Once the user has assembled the real puzzle, AR functionality within the mobile application can be unlocked, bringing to life puzzle characters, creating a seamless game that merges AR interactions with the puzzle reality. The main goals and benefits of this framework can be seen in the development of a novel set of AR tests and metrics in the pre-release phase (in order to help the commercial launch and developers), and in the release phase by introducing the measures for long-life app optimization, usage tests and hint on final users together with a measure to design policy, providing a method for automatic testing of quality and popularity improvements. Moreover, smart configuration tools, as part of the general framework, enabling multi-app and eventually also multi-user development, have been proposed, facilitating the serialization of the applications. Results were obtained from a large-scale user test with about 4 million users on a set of eight gaming applications, providing the scientific community a workflow for implicit quantitative analysis in AR gaming. Different data analytics developed on the data collected by the framework prove that the proposed approach is affordable and reliable for long-life testing and optimization.

References

  1. Junho Ahn, James Williamson, Mike Gartrell, Richard Han, Qin Lv, and Shivakant Mishra. 2015. Supporting healthy grocery shopping via mobile augmented reality. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 1s (Oct. 2015), Article 16, 24 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Sonia M. Arteaga, Víctor M. González, Sri Kurniawan, and Raquel A. Benavides. 2012. Mobile games and design requirements to increase teenagers’ physical activity. Pervasive and Mobile Computing 8, 6 (2012), 900--908. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Mafkereseb Kassahun Bekele, Roberto Pierdicca, Emanuele Frontoni, Eva Savina Malinverni, and James Gain. 2018. A survey of augmented, virtual, and mixed reality for cultural heritage. Journal on Computing and Cultural Heritage (JOCCH) 11, 2 (2018), 7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mark Billinghurst. 2002. Augmented reality in education. New Horizons for Learning 12 (2002). Retrieved January 24, 2019 from http://www.it.civil.aau.dk/it/education/reports/ar_edu.pdf.Google ScholarGoogle Scholar
  5. Paolo Clini, Emanuele Frontoni, Ramona Quattrini, and Roberto Pierdicca. 2014. Augmented reality experience: From high-resolution acquisition to real time augmented contents. Advances in Multimedia 2014 (2014), 18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. N. I. De Leon, S. K. Bhatt, and Adel Al-Jumaily. 2014. Augmented reality game based multi-usage rehabilitation therapist for stroke patients. International Journal on Smart Sensing 8 Intelligent Systems 7, 3 (2014).Google ScholarGoogle ScholarCross RefCross Ref
  7. Emanuele Frontoni, Roberto Pierdicca, Ramona Quattrini, and Paolo Clini. 2016. Improving the development of AR application for artwork collections with standard data layer. In International Conference on Augmented Reality, Virtual Reality and Computer Graphics. Springer, 435--443.Google ScholarGoogle ScholarCross RefCross Ref
  8. Iryna Gordon and David G. Lowe. 2006. What and where: 3D object recognition with accurate pose. In Toward Category-Level Object Recognition. Springer, 67--82.Google ScholarGoogle Scholar
  9. Wei Guan, Suya You, and Ulrich Newmann. 2012. Efficient matchings and mobile augmented reality. ACM Transactions on Multimedia Computing, Commununications, and Applications 8, 3s, Article 47 (Oct. 2012), 15 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Taejin Ha, Youngho Lee, and Woontack Woo. 2011. Digilog book for temple bell tolling experience based on interactive augmented reality. Virtual Reality 15, 4 (2011), 295--309.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Taejin Ha, Woontack Woo, Youngho Lee, Junhun Lee, Jeha Ryu, Hankyun Choi, and Kwanheng Lee. 2010. ARtalet: Tangible user interface based immersive augmented reality authoring tool for Digilog book. In International Symposium on Ubiquitous Virtual Reality (ISUVR’10). IEEE, 40--43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. A.-C. Haugstvedt and John Krogstie. 2012. Mobile augmented reality for cultural heritage: A technology acceptance study. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR’12). IEEE, 247--255. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Anders Henrysson, Mark Billinghurst, and Mark Ollila. 2006. AR tennis. In ACM SIGGRAPH 2006 Sketches. ACM, 13. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Hyung-Keun Jee, Sukhyun Lim, Jinyoung Youn, and Junsuk Lee. 2014. An augmented reality-based authoring tool for E-learning applications. Multimedia Tools and Applications 68, 2 (2014), 225--235. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Lucinda Kerawalla, Rosemary Luckin, Simon Seljeflot, and Adrian Woolard. 2006. “Making it real”: Exploring the potential of augmented reality for teaching primary school science. Virtual Reality 10, 3--4 (2006), 163--174. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Minoru Kojima, Maki Sugimoto, Akihiro Nakamura, Masahiro Tomita, Masahiko Inami, and N. I. I. Hideaki. 2006. Augmented coliseum: An augmented game environment with small vehicles. In null. IEEE, 3--8.Google ScholarGoogle Scholar
  17. Panos Kourouthanassis, Costas Boletsis, Cleopatra Bardaki, and Dimitra Chasanidou. 2015. Tourists responses to mobile augmented reality travel guides: The role of emotions on adoption behavior. Pervasive and Mobile Computing 18 (2015), 71--87. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Youngho Lee. 2015. AR coloring jigsaw puzzles with texture extraction and auto-UV mapping algorithm. In Distributed, Ambient, and Pervasive Interactions, Norbert Streitz and Panos Markopoulos (Eds.). Lecture Notes in Computer Science, Vol. 9189. Springer International Publishing, 182--187. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Kaikai Liu and Xiaolin Li. 2015. Enabling context-aware indoor augmented reality via smartphone sensing and vision tracking. ACM Transactions on Multimedia Computing, Communications, and Applications 12, 1s (Oct. 2015), Article 15, 23 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Su-Ju Lu and Ying-Chieh Liu. 2015. Integrating augmented reality technology to enhance children’s learning in marine education. Environmental Education Research 21, 4 (2015), 525--541.Google ScholarGoogle ScholarCross RefCross Ref
  21. Alessandro Mulloni, Daniel Wagner, and Dieter Schmalstieg. 2008. Mobility and social interaction as core gameplay elements in multi-player augmented reality. In Proceedings of the 3rd International Conference on Digital Interactive Media in Entertainment and Arts (DIMEA’08). ACM, New York, 472--478. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Simona Naspetti, Roberto Pierdicca, Serena Mandolesi, Marina Paolanti, Emanuele Frontoni, and Raffaele Zanoli. 2016. Automatic analysis of eye-tracking data for augmented reality applications: A prospective outlook. In International Conference on Augmented Reality, Virtual Reality and Computer Graphics. Springer, 217--230.Google ScholarGoogle ScholarCross RefCross Ref
  23. Trond Nilsen, Steven Linton, and Julian Looser. 2004. Motivations for augmented reality gaming. Proceedings of FUSE 4 (2004), 86--93.Google ScholarGoogle Scholar
  24. Jong-Seung Park. 2011. AR-Room: A rapid prototyping framework for augmented reality applications. Multimedia Tools and Applications 55, 3 (2011), 725--746. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Wayne Piekarski and Bruce H. Thomas. 2001. Tinmith-evo5--An architecture for supporting mobile augmented reality environments. In ISAR. IEEE, 177.Google ScholarGoogle Scholar
  26. Roberto Pierdicca, Emanuele Frontoni, Primo Zingaretti, Eva Savina Malinverni, Francesca Colosi, and Roberto Orazi. 2015. Making visible the invisible. Augmented reality visualization for 3D reconstructions of archaeological sites. In International Conference on Augmented and Virtual Reality. Springer, 25--37. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Roberto Pierdicca, Emanuele Frontoni, Primo Zingaretti, Adriano Mancini, Eva Savina Malinverni, Anna Nora Tassetti, Ernesto Marcheggiani, and Andrea Galli. 2016. Smart maintenance of riverbanks using a standard data layer and augmented reality. Computers 8 Geosciences 95 (2016), 67--74. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Roberto Pierdicca, Emanuele Frontoni, Primo Zingaretti, Mirco Sturari, Paolo Clini, and Ramona Quattrini. 2015. Advanced interaction with paintings by augmented reality and high resolution visualization: A real case exhibition. In Augmented and Virtual Reality. Springer, 38--50. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Klen Čopič Pucihar and Paul Coulton. 2014. Exploring the evolution of mobile augmented reality for future entertainment systems. Computers in Entertainment (CIE) 11, 2 (2014), 1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Ho-Sub Ryu and Hanhoon Park. 2016. A system for supporting paper-based augmented reality. Multimedia Tools and Applications 75, 6 (2016), 3375--3390. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Katie Seaborn and Deborah I. Fels. 2015. Gamification in theory and action: A survey. International Journal of Human-Computer Studies 74 (2015), 14--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Vinicius Emanuel Silva, Caio Lins, Andre Silva, Rafael Roberto, Cristiano Araujo, and Veronica Teichrieb. 2015. Voxar puzzle: An innovative hardware/software computer vision game for children development. In XVII Symposium on Virtual and Augmented Reality (SVR’15). IEEE, 147--153. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Nobuko Taketa, Kenichi Hayashi, Hirokazu Kato, and Shogo Noshida. 2007. Virtual pop-up book based on augmented reality. In Human Interface and the Management of Information. Interacting in Information Environments. Springer, 475--484. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Markus Tatzgern, Raphael Grasset, Eduardo Veas, Denis Kalkofen, Hartmut Seichter, and Dieter Schmalstieg. 2015. Exploring real world points of interest: Design and evaluation of object-centric exploration techniques for augmented reality. Pervasive and Mobile Computing 18 (2015), 55--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Bruce H. Thomas. 2012. A survey of visual, mixed, and augmented reality gaming. Computers in Entertainment (CIE) 10, 3 (2012), 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Igoris Trimailovas. 2013. MagMAR. Collaborative, Mobile Augmented Reality Supported Game with Player Generated Content. Institutt for datateknikk og informasjonsvitenskap. Retrieved January 24, 2019 from https://brage.bibsys.no/xmlui/handle/11250/253555.Google ScholarGoogle Scholar
  37. Daniel Wagner, Thomas Pintaric, and Dieter Schmalstieg. 2004. The invisible train: A collaborative handheld augmented reality demonstrator. In ACM SIGGRAPH 2004 Emerging Technologies. ACM, 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Tetsuo Yamabe and Tatsuo Nakajima. 2013. Playful training with augmented reality games: Case studies towards reality-oriented system design. Multimedia Tools and Applications 62, 1 (2013), 259--286.Google ScholarGoogle ScholarCross RefCross Ref
  39. S. Yuen, Gallayanee Yaoyuneyong, and Erik Johnson. 2011. Augmented reality: An overview and five directions for AR in education. Journal of Educational Technology Development and Exchange 4, 1 (2011), 119--140.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Design, Large-Scale Usage Testing, and Important Metrics for Augmented Reality Gaming Applications

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 15, Issue 2
            May 2019
            375 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/3339884
            Issue’s Table of Contents

            Copyright © 2019 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 5 June 2019
            • Revised: 1 January 2019
            • Accepted: 1 January 2019
            • Received: 1 June 2018
            Published in tomm Volume 15, Issue 2

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!