skip to main content
research-article

Evaluating Multimodal Feedback for Assembly Tasks in a Virtual Environment

Published:13 June 2019Publication History
Skip Abstract Section

Abstract

Operating power tools over extended periods of time can pose significant risks to humans, due to the strong forces and vibrations they impart to the limbs. Telemanipulation systems can be employed to minimize these risks, but may impede effective task performance due to the reduced sensory cues they typically convey. To address this shortcoming, we explore the benefits of augmenting these cues with the addition of audition, vibration, and force feedback, and evaluate them on users' performance in a VR mechanical assembly task employing a simulated impact wrench. Our research focuses on the utility of vibrotactile feedback, rendered as a simplified and attenuated version of the vibrations experienced while operating an actual impact wrench. We investigate whether such feedback can serve to enhance the operator's awareness of the state of the tool, as well as a proxy for the forces experienced during collisions and coupling, while operating the tool an actual impact wrench. Results from our user study comparing feedback modalities confirm that the introduction of vibrotactile, in addition to auditory feedback can significantly improve user performance as assessed by completion time. However, the addition of force feedback to these two modalities did not further improve performance.

References

  1. Abdulaziz Alshaer, Holger Regenbrecht, and David O'Hare. 2015. Investigating visual dominance with a virtual driving task. In 2015 IEEE Virtual Reality (VR). IEEE, 145--146.Google ScholarGoogle Scholar
  2. Amrish O. Chourasia, Mary E. Sesto, Walter F. Block, and Robert G. Radwin. 2009. Prolonged mechanical and physiological changes in the upper extremity following short-term simulated power hand tool use. Ergonomics, Vol. 52, 1 (2009), 15--24.Google ScholarGoogle ScholarCross RefCross Ref
  3. J.V. Draper, W.E. Moore, J.N. Herndon, B.S., and Weil. 1997. Effects of Force Reflection on Servomanipulator Task Performance. In Proc. IEEE International Topical Meeting Remote Systems and Robots in Hostile Environments .Google ScholarGoogle Scholar
  4. B. Hannaford, L. Wood, D.A. McAffee, and H. Zak. 1991. Performance Evaluation of a Six-Axis Generalized Force-Reflecting Teleoperator., Vol. 21 (May/June 1991), 620--633.Google ScholarGoogle Scholar
  5. Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology, Vol. 52 (1988), 139 -- 183.Google ScholarGoogle Scholar
  6. Jun Okamoto Jr and Anderson Nishihara. 2016. Assembly Assisted by Augmented Reality (A3R). In Intelligent Systems and Applications. Springer-Verlag, 281--300.Google ScholarGoogle Scholar
  7. Roberta L. Klatzky, Susan Lederman, and Catherine Reed. 1987. There's More to Touch than Meets the Eye: The Salience of Object Attributes for Haptics with and without Vision. Journal of Experimental Psychology: General, Vol. 116 (1987), 356--369.Google ScholarGoogle ScholarCross RefCross Ref
  8. Raymond M Klein. 1977. Attention and visual dominance: a chronometric analysis. Journal of Experimental Psychology: Human Perception and Performance, Vol. 3, 3 (1977), 365.Google ScholarGoogle ScholarCross RefCross Ref
  9. Jia-Hua Lin, Raymond W. McGorry, and Jacob J. Banks. 2010. Exposures and physiological responses in power tool operations: fastening vs. unfastening threaded hardware. Journal of occupational and environmental hygiene, Vol. 7, 5 (2010), 290--297.Google ScholarGoogle ScholarCross RefCross Ref
  10. Robert W Lindeman, James N Templeman, John L Sibert, and Justin R Cutler. 2002. Handling of virtual contact in immersive virtual environments: beyond visuals. Virtual Reality, Vol. 6, 3 (2002), 130--139.Google ScholarGoogle ScholarCross RefCross Ref
  11. Sotiris Makris, Panagiotis Karagiannis, Spyridon Koukas, and Aleksandros-Stereos Matthaiakis. 2016. Augmented reality system for operator support in human-robot collaborative assembly. CIRP Annals - Manufacturing Technology, Vol. 65, 1 (2016), 61 -- 64.Google ScholarGoogle ScholarCross RefCross Ref
  12. M. Massimo and T. Sheridan. 1989. Variable Force and Visual Feedback Effects on Teleoperator Man-Machine Performance. In Proceedings of the NASA Conference on Space Telerobotics, Vol. 1. 89--98.Google ScholarGoogle Scholar
  13. Gilberto Osorio-Gómez, Roberto Viganò, and Juan Carlos Arbeláez. 2016. An augmented reality tool to validate the assembly sequence of a discrete product. International Journal of Computer Aided Engineering and Technology, Vol. 8, 1--2 (2016), 164--178.Google ScholarGoogle Scholar
  14. Claudio Pacchierotti, Domenico Prattichizzo, and Katherine J Kuchenbecker. 2016. Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery. IEEE Transactions on Biomedical Engineering, Vol. 63, 2 (2016), 278--287.Google ScholarGoogle ScholarCross RefCross Ref
  15. Bernd Petzold, Michael F Zaeh, Berthold Faerber, Barbara Deml, Hans Egermeier, Johannes Schilp, and Stella Clarke. 2004. A study on visual, auditory, and haptic feedback for assembly tasks. Presence: teleoperators and virtual environments, Vol. 13, 1 (2004), 16--21.Google ScholarGoogle Scholar
  16. M. D. Pottenger and P. Benhaim. 2004. Vibration absorbing brace for study of work-related upper extremity musculoskeletal disorders. In Engineering in Medicine and Biology Society, 2004. IEMBS'04. 26th Annual International Conference of the IEEE, Vol. 1. IEEE, 2466--2469.Google ScholarGoogle Scholar
  17. M Poyade, M Kargas, and V Portela. 2014. Haptic plug-in for unity. Digital Design Studio (DDS), Glasgow School of Art, Glasgow, United Kingdom (2014).Google ScholarGoogle Scholar
  18. John Rosecrance, Dan Anton, Thomas Cook, and Linda Merlino. 2005. Effect of pneumatic power tool use on nerve conduction velocity across the wrist. Human Factors and Ergonomics in Manufacturing & Service Industries, Vol. 15, 4 (2005), 339--352. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. C. H. Roux, F. Guillemin, S. Boini, F. Longuetaud, N. Arnault, S. Hercberg, and S. Briancc on. 2005. Impact of musculoskeletal disorders on quality of life: An inception cohort study. Annals of the Rheumatic Diseases, Vol. 64, 4 (2005), 606--611.Google ScholarGoogle ScholarCross RefCross Ref
  20. Mikel Sagardia and Thomas Hulin. 2017. Multimodal evaluation of the differences between real and virtual assemblies. IEEE transactions on haptics (2017).Google ScholarGoogle Scholar
  21. ZB Wang, LX Ng, SK Ong, and AYC Nee. 2013. Assembly planning and evaluation in an augmented reality environment. International Journal of Production Research, Vol. 51, 23--24 (2013), 7388--7404.Google ScholarGoogle ScholarCross RefCross Ref
  22. Sabine Webel, Uli Bockholt, Timo Engelke, Nirit Gavish, Manuel Olbrich, and Carsten Preusche. 2013. An augmented reality training platform for assembly and maintenance skills. Robotics and Autonomous Systems, Vol. 61, 4 (2013), 398--403. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Bernhard Weber, Mikel Sagardia, Thomas Hulin, and Carsten Preusche. 2013. Visual, vibrotactile, and force feedback of collisions in virtual environments: effects on performance, mental workload and spatial orientation. In International Conference on Virtual, Augmented and Mixed Reality. Springer, 241--250.Google ScholarGoogle ScholarCross RefCross Ref
  24. Jeroen G.W. Wildenbeest, David A. Abbink, Cock J.M. Heemskerk, Frans C.T. van der Helm, and Henri Boessenkool. 2013. The Impact of Haptic Feedback Quality on the Performance of Teleoperated Assembly Tasks., Vol. 6 (April/June 2013), 242--252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. M.L. Yuan, S.K. Ong, and A.Y.C. Nee. 2008. Augmented reality for assembly guidance using a virtual interactive tool. International Journal of Production Research, Vol. 46 (2008), 1745--1767.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Evaluating Multimodal Feedback for Assembly Tasks in a Virtual Environment

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!