skip to main content
research-article

Interaction with Virtual Content using Augmented Reality: A User Study in Assembly Procedures

Published:04 November 2020Publication History
Skip Abstract Section

Abstract

Assembly procedures are a common task in several domains of application. Augmented Reality (AR) has been considered as having great potential in assisting users while performing such tasks. However, poor interaction design and lack of studies, often results in complex and hard to use AR systems. This paper considers three different interaction methods for assembly procedures (Touch gestures in a mobile device; Mobile Device movements; 3D Controllers and See-through HMD). It also describes a controlled experiment aimed at comparing acceptance and usability between these methods in an assembly task using Lego blocks. The main conclusions are that participants were faster using the 3D controllers and Video see-through HMD. Participants also preferred the HMD condition, even though some reported light symptoms of nausea, sickness and/or disorientation, probably due to limited resolution of the HMD cameras used in the video see-through setting and some latency issues. In addition, although some research claims that manipulation of virtual objects with movements of the mobile device can be considered as natural, this condition was the least preferred by the participants.

Skip Supplemental Material Section

Supplemental Material

V4issA196.mp4

Supplemental video

References

  1. Jo a o Alves, Bernardo Marques, Miguel Neves, Rafael Maio, André Santos, Inê s Justo, Raquel Rainho, Dany Costa, Beatriz Sousa Santos, and Paulo Dias. 2019. Augmented Reality Platform to foster Collaborative Manipulation of 3D virtual objects: A Prototype. In International Conference on Graphics and Interaction, ICGI 2019.Google ScholarGoogle Scholar
  2. J. Alves, B. Marques , M. Oliveira, T. Araújo , P. Dias, and B. S. Santos. 2019. Comparing Spatial and Mobile Augmented Reality for Guiding Assembling Procedures with Task Validation. In 2019 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC). 1--6.Google ScholarGoogle Scholar
  3. Ronald T. Azuma. 1997. A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments, Vol. 6, 4 (1997).Google ScholarGoogle Scholar
  4. R. T. Azuma. 2016. The Most Important Challenge Facing Augmented Reality. Presence, Vol. 25, 3 (Dec 2016), 234--238.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Mafkereseb Kassahun Bekele, Roberto Pierdicca, Emanuele Frontoni, Eva Savina Malinverni, and James Gain. 2018. A Survey of Augmented, Virtual, and Mixed Reality for Cultural Heritage. J. Comput. Cult. Herit. , Vol. 11, 2 (March 2018), 7:1--7:36.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality . Foundations and Trends in Human--Computer Interaction, Vol. 8, 2--3 (2015), 73--272.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Mark Billinghurst and Hirokazu Kato. 2002. Collaborative Augmented reality. In Communications of the ACM .Google ScholarGoogle Scholar
  8. Mark Billinghurst, Hiroyuki Kato, and Ivan Poupyrev. 2008. Tangible augmented reality. In SIGGRAPH 2008.Google ScholarGoogle ScholarCross RefCross Ref
  9. Volkert Buchmann, Stephen Violich, Mark Billinghurst, and Andy Cockburn. 2004. FingARtips: Gesture Based Direct Manipulation in Augmented Reality. In Proceedings of the 2Nd International Conference on Computer Graphics and Interactive Techniques in Australasia and South East Asia (Singapore) (GRAPHITE '04). ACM, New York, NY, USA, 212--221.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Sebastian Büttner, Markus Funk, Oliver Sand, and Carsten Röcker. 2016. Using Head-Mounted Displays and In-Situ Projection for Assistive Systems: A Comparison. In Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments (Corfu, Island, Greece) (PETRA '16). ACM, New York, NY, USA, Article 44, bibinfonumpages8 pages.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Fabio M. Caputo, Marco Emporio, and Andrea Giachetti. 2018. The Smart Pin: An effective tool for object manipulation in immersive virtual reality environments. Computers & Graphics, Vol. 74 (2018), 225 -- 233.Google ScholarGoogle ScholarCross RefCross Ref
  12. D. Chatzopoulos, C. Bermejo, Z. Huang, and P. Hui. 2017. Mobile Augmented Reality Survey: From Where We Are to Where We Go. IEEE Access , Vol. 5 (2017), 6917--6950.Google ScholarGoogle ScholarCross RefCross Ref
  13. Arindam Dey, Mark Billinghurst, Robert W Lindeman, and J Edward Swan Ii. 2018. A Systematic Review of 10 Years of Augmented Reality Usability Studies: 2005 to 2014 . Frontiers in Robotics and AI , Vol. 5 (2018).Google ScholarGoogle ScholarCross RefCross Ref
  14. Paul Dizio and James Lackner. 2000. Motion Sickness Side Effects and Aftereffects of Immersive Virtual Environments Created with Helmet-Mounted Visual Displays. (11 2000), 4.Google ScholarGoogle Scholar
  15. Andreas Dü nser and Mark Billinghurst. 2011. Evaluating Augmented Reality Systems . In Handbook of Augmented Reality . 289--307.Google ScholarGoogle Scholar
  16. I n igo Ferná ndez Del Amo , John Ahmet Erkoyuncu, Rajkumar Roy, Riccardo Palmarini, and Demetrius Onoufriou. 2018. A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications . Computers in Industry , Vol. 103 (2018), 47--71.Google ScholarGoogle ScholarCross RefCross Ref
  17. Lawrence H. Frank, John G. Casali, and Walter W. Wierwille. 1988. Effects of Visual Display and Motion System Delays on Operator Performance and Uneasiness in a Driving Simulator. Human Factors , Vol. 30, 2 (1988), 201--217.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Markus Funk, Andreas B"achler, Liane B"achler, Thomas Kosch, Thomas Heidenreich, and Albrecht Schmidt. 2017. Working with Augmented Reality? A Long-Term Analysis of In-Situ Instructions at the Assembly Workplace. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '17). Association for Computing Machinery, New York, NY, USA, 222--229.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Jean Dickinson Gibbons and Subhabrata Chakraborti. 2011. Nonparametric Statistical Inference .Springer Berlin Heidelberg, Berlin, Heidelberg, 977--979.Google ScholarGoogle Scholar
  20. E. S. Goh, M. S. Sunar, and A. W. Ismail. 2019. 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review. IEEE Access , Vol. 7 (2019), 40581--40601.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Jens Grubert and Stefanie Zollmann. 2017. Towards Pervasive Augmented Reality: Context- Awareness in Augmented Reality . IEEE Transactions on Visualization and Computer Graphics , Vol. 23, 6 (2017), 1706--1724.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Devamardeep Hayatpur, Seongkook Heo, Haijun Xia, Wolfgang Stuerzlinger, and Daniel Wigdor. 2019. Plane, Ray, and Point: Enabling Precise Spatial Manipulations with Shape Constraints. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (UIST '19). Association for Computing Machinery, New York, NY, USA, 1185--1195.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Anders Henrysson, Mark Billinghurst, and Mark Ollila. 2005. Face to face collaborative AR on mobile phones . IEEE a International Symposium on Symposium on Mixed and Augmented Reality, ISMAR 2005 (2005), 80--89.Google ScholarGoogle Scholar
  24. David C. Hoaglin, Frederick Mosteller, and John W. Tukey (Editor). 2000. Understanding Robust and Exploratory Data Analysis 1 ed.). Wiley-Interscience.Google ScholarGoogle Scholar
  25. Jé rme Jetter, Jö rgen Eimecke, and Alexandra Rese. 2018. Augmented reality tools for industrial applications: What are potential key performance indicators and who benefits? Computers in Human Behavior , Vol. 87 (2018), 18--33.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. B. M. Khuong, K. Kiyokawa, A. Miller, J. J. La Viola, T. Mashita, and H. Takemura. 2014. The effectiveness of an AR-based context-aware assembly support system in object assembly. In 2014 IEEE Virtual Reality (VR) . 57--62.Google ScholarGoogle Scholar
  27. K. Kim, M. Billinghurst, G. Bruder, H. B. Duh, and G. F. Welch. 2018. Revisiting Trends in Augmented Reality Research: A Review of the 2nd Decade of ISMAR (2008--2017). IEEE Transactions on Visualization and Computer Graphics , Vol. 24, 11 (2018), 2947--2962.Google ScholarGoogle ScholarCross RefCross Ref
  28. D W F Van Krevelen and R Poelman. 2010. A Survey of Augmented Reality Technologies, Applications and Limitations . The International Journal of Virtual Reality , Vol. 9 (2010).Google ScholarGoogle ScholarCross RefCross Ref
  29. M. Krichenbauer, G. Yamamoto, T. Taketom, C. Sandor , and H. Kato. 2018. Augmented Reality versus Virtual Reality for 3D Object Manipulation. IEEE Transactions on Visualization and Computer Graphics , Vol. 24, 2 (Feb 2018), 1038--1048.Google ScholarGoogle ScholarCross RefCross Ref
  30. Ze-Hao Lai, Wenjin Tao, Ming C. Leu, and Zhaozheng Yin. 2020. Smart augmented reality instructional system for mechanical assembly towards worker-centered intelligent manufacturing. Journal of Manufacturing Systems , Vol. 55 (2020), 69 -- 81.Google ScholarGoogle ScholarCross RefCross Ref
  31. Joseph J. LaViola Jr., Ernst Kruijff, Doug Bowman, Ivan P. Poupyrev, and Ryan P. McMahan. 2017. 3D User Interfaces: Theory and Practice (second edition) .Google ScholarGoogle Scholar
  32. Wenkai Li, A. Nee, and S. Ong. 2017. A State-of-the-Art Review of Augmented Reality in Engineering Analysis and Simulation . Multimodal Technologies and Interaction , Vol. 1, 3 (2017).Google ScholarGoogle Scholar
  33. Jingbo Liu, Oscar Kin-Chung Au, Hongbo Fu, and Chiew-Lan Tai. 2012. Two-Finger Gestures for 6DOF Manipulation of 3D Objects. Comput. Graph. Forum , Vol. 31, 7pt1 (Sept. 2012), 2047--2055.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. F. Loch, F. Quint , and I. Brishtel. 2016. Comparing Video and Augmented Reality Assistance in Manual Assembly. In 2016 12th International Conference on Intelligent Environments (IE). 147--150.Google ScholarGoogle Scholar
  35. Annette Mossel, Benjamin Venditti, and Hannes Kaufmann. 2013. 3DTouch and HOMER-S: Intuitive Manipulation Techniques for One-Handed Handheld Augmented Reality. In Proceedings of the Virtual Reality International Conference: Laval Virtual. Association for Computing Machinery, New York, NY, USA, 10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. K. Murakami, R. Kiyama, T. Narumi, T. Tanikawa , and M. Hirose. 2013. Poster: A wearable augmented reality system with haptic feedback and its performance in virtual assembly tasks. In 2013 IEEE Symposium on 3D User Interfaces (3DUI) . 161--162.Google ScholarGoogle Scholar
  37. A. Nishihara and J. Okamoto. 2015. Object recognition in assembly assisted by augmented reality system. In 2015 SAI Intelligent Systems Conference (IntelliSys). 400--407.Google ScholarGoogle Scholar
  38. Michael E. Porter and James E. Heppelmann. 2017. A Manager's Guide to Augmented Reality. Harvard Business Review. Technical Report.Google ScholarGoogle Scholar
  39. Moritz Quandt, Benjamin Knoke, Christian Gorldt, Michael Freitag, and Klaus Dieter Thoben. 2018. General Requirements for Industrial Augmented Reality Applications . Procedia CIRP , Vol. 72 (2018), 1130--1135.Google ScholarGoogle ScholarCross RefCross Ref
  40. Rafael Radkowski and Christian Stritzke. 2012. Interactive Hand Gesture-based Assembly for Augmented Reality Applications. , bibinfonumpages5 pages.Google ScholarGoogle Scholar
  41. Christian Sandor, Martin Fuchs, Alvaro Cassinelli, Hao Li, Richard Newcombe, Goshiro Yamamoto, and Steven Feiner. 2015. Breaking the Barriers to True Augmented Reality . ArXiv e-prints , Vol. 1512.05471 (2015), 1--13.Google ScholarGoogle Scholar
  42. C.Y. Siew, S.K. Ong, and A.Y.C. Nee. 2019. A practical augmented reality-assisted maintenance system framework for adaptive user support. Robotics and Computer-Integrated Manufacturing , Vol. 59 (2019), 115 -- 129.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. B.G. Tabachnick and L.S. Fidell. 2007. Experimental Designs Using ANOVA .Thomson/Brooks/Cole.Google ScholarGoogle Scholar
  44. Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative Effectiveness of Augmented Reality in Object Assembly. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA) (CHI '03). ACM, New York, NY, USA, 73--80.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Matthew Turk. 2014. Multimodal interaction: A review. Pattern Recognition Letters , Vol. 36 (2014), 189 -- 195.Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Peng Wang, Shusheng Zhang, Mark Billinghurst, Xiaoliang Bai, Weiping He, Shuxia Wang, Mengmeng Sun, and Xu Zhang. 2019. A comprehensive survey of AR/MR-based co-design in manufacturing . Engineering with Computers (2019), 1--24.Google ScholarGoogle Scholar
  47. Xiangyu Wang, Soh K Ong, and Andrew YC Nee. 2016. A comprehensive survey of augmented reality assembly research. Advances in Manufacturing , Vol. 4, 1 (2016), 1--22.Google ScholarGoogle Scholar
  48. Sabine Webel, Uli Bockholt, Timo Engelke, Nirit Gavish, Manuel Olbrich, and Carsten Preusche. 2013. An augmented reality training platform for assembly and maintenance skills. Robotics and Autonomous Systems , Vol. 61, 4 (2013), 398 -- 403. Models and Technologies for Multi-modal Skill Training.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. M. L. Yuan, S. K. Ong, and A. Y. C. Nee. 2008. Augmented reality for assembly guidance using a virtual interactive tool. International Journal of Production Research , Vol. 46, 7 (2008), 1745--1767.Google ScholarGoogle ScholarCross RefCross Ref
  50. Feng Zhou, Henry Duh, and Mark Billinghurst. 2008. Trends in Augmented Reality Tracking, Interaction and Display: A Review of Ten Years of ISMAR . IEEE International Symposium on Symposium on Mixed and Augmented Reality, ISMAR 2008 (2008), 193--202.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Interaction with Virtual Content using Augmented Reality: A User Study in Assembly Procedures

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!