skip to main content
research-article

An Augmented Reality Online Assistance Platform for Repair Tasks

Authors Info & Claims
Published:11 May 2021Publication History
Skip Abstract Section

Abstract

Our augmented reality online assistance platform enables an expert to specify 6DoF movements of a component and apply the geometrical and physical constraints in real-time. We track the real components on the expert’s side to monitor the operations of an expert. We leverage a remote rendering technique that we proposed previously to relieve the rendering burden of the augmented reality end devices. By conducting a user study, we show that the proposed method outperforms conventional instructional videos and sketches. The answers to the questionnaires show that the proposed method receives higher recommendation than sketching, and, compared to conventional instructional videos, is outstanding in terms of instruction clarity, preference, recommendation, and confidence of task completion. Moreover, as to the overall user experience, the proposed method has an advantage over the video method.

References

  1. Matt Adcock, Dulitha Ranatunga, Ross Smith, and Bruce H. Thomas. 2014. Object-based touch manipulation for remote guidance of physical tasks. In Proceedings of the 2nd ACM Symposium on Spatial User Interaction. 113--122.Google ScholarGoogle Scholar
  2. Paul J. Besl and Neil D. McKay. 1992. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 14, 2 (1992), 239--256.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Bhaskar Bhattacharya and Eliot Winer. 2015. A method for real-time generation of augmented reality work instructions via expert movements. In Proceedings of the Engineering Reality of Virtual Reality, Vol. 9392. 109--121.Google ScholarGoogle Scholar
  4. Ócar Blanco-Novoa, Tiago M. FernáNdez-Caramés, Paula Fraga-Lamas, and Miguel A. Vilar-Montesinos. 2018. A practical evaluation of commercial industrial augmented reality systems in an industry 4.0 shipyard. IEEE Access 6 (2018), 8201--8218.Google ScholarGoogle ScholarCross RefCross Ref
  5. Silvia Blanco-Pons, Berta Carrión-Ruiz, and José Lerma. 2018. Augmented reality application assessment for disseminating rock art. Multimedia Tools and Applications 78 (2018), 10265--10286.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Pierre Boulanger. 2004. Application of augmented reality to industrial tele-training. In Proceedings of the 1st Canadian Conference on Computer and Robot Vision. 320--328.Google ScholarGoogle ScholarCross RefCross Ref
  7. Thomas P. Caudell and David W. Mizell. 1992. Augmented reality: An application of heads-up display technology to manual manufacturing processes. In Proceedings of the 25th Hawaii International Conference on System Sciences, Vol. II. 659--669.Google ScholarGoogle Scholar
  8. Francesca De Crescenzio, Massimiliano Fantini, Franco Persiani, Luigi Di Stefano, Pietro Azzari, and Samuele Salti. 2011. Augmented reality for aircraft maintenance training and operations support. IEEE Computer Graphics and Applications 31, 1 (2011), 96--101.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Michele Gattullo, Giulia Wally Scurati, Michele Fiorentino, Antonio Emmanuele Uva, Francesco Ferrise, and Monica Bordegoni. 2019. Towards augmented reality manuals for industry 4.0: A methodology. Robotics and Computer-Integrated Manufacturing 56 (2019), 276--286.Google ScholarGoogle ScholarCross RefCross Ref
  10. Nirit Gavish, Teresa Gutiérrez, Sabine Webel, Jorge Rodríguez, Matteo Peveri, Uli Bockholt, and Franco Tecchia. 2015. Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks. Interactive Learning Environments 23, 6 (2015), 778--798.Google ScholarGoogle ScholarCross RefCross Ref
  11. Google LLC. 2020. Build New Augmented Reality Experiences that Seamlessly Blend the Digital and Physical Worlds. Retrieved October 4, 2020 from https://developers.google.com/ar.Google ScholarGoogle Scholar
  12. Google LLC. 2020. Download Android Studio and SDK tools | Android Developers. Retrieved July 14, 2020 from https://developer.android.com/studio.Google ScholarGoogle Scholar
  13. Pavel Gurevich, Joel Lanir, Benjamin Cohen, and Ran Stone. 2012. TeleAdvisor: A versatile augmented reality tool for remote assistance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 619--622.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Matthias Haringer and Holger Regenbrecht. 2002. A pragmatic approach to augmented reality authoring. In Proceedings of the International Symposium on Mixed and Augmented Reality. 237--245.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Roger Harrabin. 2019. EU brings in ‘right to repair’ rules for appliances—BBC News. Retrieved April 30, 2020 from https://www.bbc.com/news/business-49884827.Google ScholarGoogle Scholar
  16. Steven J. Henderson and Steven K. Feiner. 2009. Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In Proceedings of the 8th IEEE International Symposium on Mixed and Augmented Reality. 135--144.Google ScholarGoogle Scholar
  17. Steven J. Henderson and Steven K. Feiner. 2011. Augmented reality in the psychomotor phase of a procedural task. In Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality. 191--200.Google ScholarGoogle Scholar
  18. Xueshi Hou, Yao Lu, and Sujit Dey. 2017. Wireless VR/AR with edge/cloud computing. In Proceedings of the 26th International Conference on Computer Communication and Networks. 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  19. jMonkeyEngine. 2020. jMonkeyEngine. Retrieved August 16, 2020 from https://jmonkeyengine.org.Google ScholarGoogle Scholar
  20. Juhwan Lee, Sangwon Hwang, Jisun Lee, and Seungwoo Kang. 2020. Comparative performance characterization of mobile AR frameworks in the context of AR-based grocery shopping applications. Applied Sciences 10, 4 (2020), 1547--1561.Google ScholarGoogle ScholarCross RefCross Ref
  21. MAXST Co., Ltd. 2019. MAXST | Technology company specialized in AR to make a better augmented reality world.Retrieved October 4, 2020 from http://maxst.com.Google ScholarGoogle Scholar
  22. Antonija Mitrovic, Brent Martin, Pramuditha Suraweera, Konstantin Zakharov, Nancy Milik, Holl, Jay, and Nicholas Mcguigan. 2009. ASPIRE: An authoring system and deployment environment for constraint-based tutors. International Journal of Artificial Intelligence in Education 19, 2 (2009), 155--188.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Ohan Oda, Carmine Elvezio, Mengu Sukan, Steven Feiner, and Barbara Tversky. 2015. Virtual replicas for remote assistance in virtual and augmented reality. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. 405--415.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Riccardo Palmarini, John Ahmet Erkoyuncu, Rajkumar Roy, and Hosein Torabmostaedi. 2018. A systematic review of augmented reality applications in maintenance. Robotics and Computer-Integrated Manufacturing 49 (2018), 215--228.Google ScholarGoogle ScholarCross RefCross Ref
  25. Philip Lamb. 2020. ARToolKit Home Page. Retrieved August 16, 2020 from http://www.hitl.washington.edu/artoolkit/.Google ScholarGoogle Scholar
  26. PTC. 2020. Vuforia—engine. Retrieved April 30, 2020 from https://engine.vuforia.com/engine.Google ScholarGoogle Scholar
  27. Dulitha Ranatunga, Matt Adcock, David Feng, and Bruce Thomas. 2013. Towards object based manipulation in remote guidance. In IEEE International Symposium on Mixed and Augmented Reality. 1--6.Google ScholarGoogle ScholarCross RefCross Ref
  28. RE’FLEKT GmbH. 2019. Augmented reality platform for maintenance operations and training. Retrieved April 30, 2020 from https://www.re-flekt.com/reflekt-one.Google ScholarGoogle Scholar
  29. Maximilian Speicher, Kristina Tenhaft, Simon Heinen, and Harry Handorf. 2015. Enabling industry 4.0 with holobuilder. In INFORMATIK. 1561--1575.Google ScholarGoogle Scholar
  30. Hongling Sun, Yue Liu, Zhenliang Zhang, Xiaoxu Liu, and Yongtian Wang. 2018. Employing different viewpoints for remote guidance in a collaborative augmented environment. In Proceedings of the 6th International Symposium of Chinese CHI. 64--70.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Lu Sun, Hussein Al Osman, and Jochen Lang. 2019. A hybrid remote rendering method for mobile applications. Multimedia Tools and Applications 79, 5 (2019), 1--26.Google ScholarGoogle Scholar
  32. Ivan E. Sutherland. 1968. A head-mounted three dimensional display. In Proceedings of the Fall Joint Computer Conference, Part I. 757--764.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Anna Syberfeldt, Oscar Danielsson, and Patrik Gustavsson. 2017. Augmented reality smart glasses in the smart factory: Product evaluation guidelines and review of available products. IEEE Access 5 (2017), 9118--9130.Google ScholarGoogle ScholarCross RefCross Ref
  34. John W. Tukey. 1949. Comparing individual means in the analysis of variance. Biometrics 5, 2 (1949), 99--114.Google ScholarGoogle Scholar
  35. Ming-Jen Wang, Chien-Hao Tseng, and Cherng-Yeu Shen. 2010. An easy to use augmented reality authoring tool for use in examination purpose. In Human-Computer Interaction. 285--288.Google ScholarGoogle Scholar
  36. Xiangyu Wang, Sohkhim K. Ong, and Andrew Y. C. Nee. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing 4 (2016), 1--22.Google ScholarGoogle Scholar
  37. Sabine Webel, Uli Bockholt, Timo Engelke, Nirit Gavish, Manuel Olbrich, and Carsten Preusche. 2013. An augmented reality training platform for assembly and maintenance skills. Robotics and Autonomous Systems 61, 4 (2013), 398--403.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. An Augmented Reality Online Assistance Platform for Repair Tasks

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 2
            May 2021
            410 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/3461621
            Issue’s Table of Contents

            Copyright © 2021 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 11 May 2021
            • Revised: 1 October 2020
            • Accepted: 1 October 2020
            • Received: 1 May 2020
            Published in tomm Volume 17, Issue 2

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
            • Research
            • Refereed

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!