Abstract
Operating power tools over extended periods of time can pose significant risks to humans, due to the strong forces and vibrations they impart to the limbs. Telemanipulation systems can be employed to minimize these risks, but may impede effective task performance due to the reduced sensory cues they typically convey. To address this shortcoming, we explore the benefits of augmenting these cues with the addition of audition, vibration, and force feedback, and evaluate them on users' performance in a VR mechanical assembly task employing a simulated impact wrench. Our research focuses on the utility of vibrotactile feedback, rendered as a simplified and attenuated version of the vibrations experienced while operating an actual impact wrench. We investigate whether such feedback can serve to enhance the operator's awareness of the state of the tool, as well as a proxy for the forces experienced during collisions and coupling, while operating the tool an actual impact wrench. Results from our user study comparing feedback modalities confirm that the introduction of vibrotactile, in addition to auditory feedback can significantly improve user performance as assessed by completion time. However, the addition of force feedback to these two modalities did not further improve performance.
- Abdulaziz Alshaer, Holger Regenbrecht, and David O'Hare. 2015. Investigating visual dominance with a virtual driving task. In 2015 IEEE Virtual Reality (VR). IEEE, 145--146.Google Scholar
- Amrish O. Chourasia, Mary E. Sesto, Walter F. Block, and Robert G. Radwin. 2009. Prolonged mechanical and physiological changes in the upper extremity following short-term simulated power hand tool use. Ergonomics, Vol. 52, 1 (2009), 15--24.Google Scholar
Cross Ref
- J.V. Draper, W.E. Moore, J.N. Herndon, B.S., and Weil. 1997. Effects of Force Reflection on Servomanipulator Task Performance. In Proc. IEEE International Topical Meeting Remote Systems and Robots in Hostile Environments .Google Scholar
- B. Hannaford, L. Wood, D.A. McAffee, and H. Zak. 1991. Performance Evaluation of a Six-Axis Generalized Force-Reflecting Teleoperator., Vol. 21 (May/June 1991), 620--633.Google Scholar
- Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology, Vol. 52 (1988), 139 -- 183.Google Scholar
- Jun Okamoto Jr and Anderson Nishihara. 2016. Assembly Assisted by Augmented Reality (A3R). In Intelligent Systems and Applications. Springer-Verlag, 281--300.Google Scholar
- Roberta L. Klatzky, Susan Lederman, and Catherine Reed. 1987. There's More to Touch than Meets the Eye: The Salience of Object Attributes for Haptics with and without Vision. Journal of Experimental Psychology: General, Vol. 116 (1987), 356--369.Google Scholar
Cross Ref
- Raymond M Klein. 1977. Attention and visual dominance: a chronometric analysis. Journal of Experimental Psychology: Human Perception and Performance, Vol. 3, 3 (1977), 365.Google Scholar
Cross Ref
- Jia-Hua Lin, Raymond W. McGorry, and Jacob J. Banks. 2010. Exposures and physiological responses in power tool operations: fastening vs. unfastening threaded hardware. Journal of occupational and environmental hygiene, Vol. 7, 5 (2010), 290--297.Google Scholar
Cross Ref
- Robert W Lindeman, James N Templeman, John L Sibert, and Justin R Cutler. 2002. Handling of virtual contact in immersive virtual environments: beyond visuals. Virtual Reality, Vol. 6, 3 (2002), 130--139.Google Scholar
Cross Ref
- Sotiris Makris, Panagiotis Karagiannis, Spyridon Koukas, and Aleksandros-Stereos Matthaiakis. 2016. Augmented reality system for operator support in human-robot collaborative assembly. CIRP Annals - Manufacturing Technology, Vol. 65, 1 (2016), 61 -- 64.Google Scholar
Cross Ref
- M. Massimo and T. Sheridan. 1989. Variable Force and Visual Feedback Effects on Teleoperator Man-Machine Performance. In Proceedings of the NASA Conference on Space Telerobotics, Vol. 1. 89--98.Google Scholar
- Gilberto Osorio-Gómez, Roberto Viganò, and Juan Carlos Arbeláez. 2016. An augmented reality tool to validate the assembly sequence of a discrete product. International Journal of Computer Aided Engineering and Technology, Vol. 8, 1--2 (2016), 164--178.Google Scholar
- Claudio Pacchierotti, Domenico Prattichizzo, and Katherine J Kuchenbecker. 2016. Cutaneous feedback of fingertip deformation and vibration for palpation in robotic surgery. IEEE Transactions on Biomedical Engineering, Vol. 63, 2 (2016), 278--287.Google Scholar
Cross Ref
- Bernd Petzold, Michael F Zaeh, Berthold Faerber, Barbara Deml, Hans Egermeier, Johannes Schilp, and Stella Clarke. 2004. A study on visual, auditory, and haptic feedback for assembly tasks. Presence: teleoperators and virtual environments, Vol. 13, 1 (2004), 16--21.Google Scholar
- M. D. Pottenger and P. Benhaim. 2004. Vibration absorbing brace for study of work-related upper extremity musculoskeletal disorders. In Engineering in Medicine and Biology Society, 2004. IEMBS'04. 26th Annual International Conference of the IEEE, Vol. 1. IEEE, 2466--2469.Google Scholar
- M Poyade, M Kargas, and V Portela. 2014. Haptic plug-in for unity. Digital Design Studio (DDS), Glasgow School of Art, Glasgow, United Kingdom (2014).Google Scholar
- John Rosecrance, Dan Anton, Thomas Cook, and Linda Merlino. 2005. Effect of pneumatic power tool use on nerve conduction velocity across the wrist. Human Factors and Ergonomics in Manufacturing & Service Industries, Vol. 15, 4 (2005), 339--352. Google Scholar
Digital Library
- C. H. Roux, F. Guillemin, S. Boini, F. Longuetaud, N. Arnault, S. Hercberg, and S. Briancc on. 2005. Impact of musculoskeletal disorders on quality of life: An inception cohort study. Annals of the Rheumatic Diseases, Vol. 64, 4 (2005), 606--611.Google Scholar
Cross Ref
- Mikel Sagardia and Thomas Hulin. 2017. Multimodal evaluation of the differences between real and virtual assemblies. IEEE transactions on haptics (2017).Google Scholar
- ZB Wang, LX Ng, SK Ong, and AYC Nee. 2013. Assembly planning and evaluation in an augmented reality environment. International Journal of Production Research, Vol. 51, 23--24 (2013), 7388--7404.Google Scholar
Cross Ref
- Sabine Webel, Uli Bockholt, Timo Engelke, Nirit Gavish, Manuel Olbrich, and Carsten Preusche. 2013. An augmented reality training platform for assembly and maintenance skills. Robotics and Autonomous Systems, Vol. 61, 4 (2013), 398--403. Google Scholar
Digital Library
- Bernhard Weber, Mikel Sagardia, Thomas Hulin, and Carsten Preusche. 2013. Visual, vibrotactile, and force feedback of collisions in virtual environments: effects on performance, mental workload and spatial orientation. In International Conference on Virtual, Augmented and Mixed Reality. Springer, 241--250.Google Scholar
Cross Ref
- Jeroen G.W. Wildenbeest, David A. Abbink, Cock J.M. Heemskerk, Frans C.T. van der Helm, and Henri Boessenkool. 2013. The Impact of Haptic Feedback Quality on the Performance of Teleoperated Assembly Tasks., Vol. 6 (April/June 2013), 242--252. Google Scholar
Digital Library
- M.L. Yuan, S.K. Ong, and A.Y.C. Nee. 2008. Augmented reality for assembly guidance using a virtual interactive tool. International Journal of Production Research, Vol. 46 (2008), 1745--1767.Google Scholar
Cross Ref
Index Terms
Evaluating Multimodal Feedback for Assembly Tasks in a Virtual Environment
Recommendations
The Impact of Haptic Feedback Quality on the Performance of Teleoperated Assembly Tasks
In teleoperation, haptic feedback allows the human operator to touch the remote environment. Yet, it is only partially understood to what extent the quality of haptic feedback contributes to human-in-the-loop task performance. This paper presents a ...
Tangible virtual reality in a multi-user environment
AutomotiveUI '19: Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct ProceedingsDue to more and more recent technologies, better simulations can be developed, which can create a greater fusion between the real and virtual world and thus, also generate a higher degree of immersion. Higher levels of immersion can provide meaningful ...
X-Wing: Propeller-Based Force Feedback to Head in a Virtual Environment
SA '21 XR: SIGGRAPH Asia 2021 XRIn this demonstration, we present X-Wing, a force feedback device using ducted fans attached to a Virtual Reality (VR) Head Mounted Display (HMD). Applying haptic technologies to VR has become a standard solution to enhance the immersive experience. It ...






Comments