skip to main content
research-article

Intensifying Emotional Reactions via Tactile Gestures in Immersive Films

Published:28 June 2017Publication History
Skip Abstract Section

Abstract

The film industry continuously strives to make visitors’ movie experience more immersive and thus, more captivating. This is realized through larger screens, sophisticated speaker systems, and high quality 2D and 3D content. Moreover, a recent trend in the film industry is to incorporate multiple interaction modalities, such as 4D film, to simulate rain, wind, vibration, and heat, in order to intensify viewers’ emotional reactions. In this context, humans’ sense of touch possesses significant potential for intensifying emotional reactions for the film experience beyond audio-visual sensory modalities. This article presents a framework for authoring tactile cues (tactile gestures as used in this article) and enabling automatic rendering of said gestures to intensify emotional reactions in an immersive film experience. To validate the proposed framework, we conducted an experimental study where tactile gestures are designed and evaluated for the ability to intensify four emotional reactions: high valence-high arousal, high valence-low arousal, low valence-high arousal, and low valence-low arousal. Using a haptic jacket, participants felt tactile gestures that are synchronized with the audio-visual contents of a film. Results demonstrated that (1) any tactile feedback generated a positive user experience; (2) the tactile feedback intensifies emotional reactions when the audio-visual stimuli elicit clear emotional responses, except for low arousal emotional response since tactile gestures seem to always generate excitement; (3) purposed tactile gestures do not seem to significantly outperform randomized tactile gesture for intensifying specific emotional reactions; and (4) using a haptic jacket is not distracting for the users.

References

  1. William Albert and Thomas Tullis. 2013. Measuring the User Experience: Collecting, Analyzing, and Presenting Usability Metrics. Elsevier. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Faisal Arafsha, Kazi Masudul Alam, and Abdulmotaleb El Saddik. 2015. Design and development of a user centric affective haptic jacket. Multimedia Tools and Applications 74, 9 (2015), 3035--3052. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Yoann Baveye, Emmanuel Dellandrea, Christel Chamaret, and Liming Chen. 2015. LIRIS-ACCEDE: A video database for affective content analysis. IEEE Transactions on Affective Computing 6, 1 (2015), 43--55.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Leonardo Bonanni, Cati Vaucelle, Jeff Lieberman, and Orit Zuckerman. 2006. TapTap: A haptic wearable for asynchronous distributed touch therapy. In CHI’06 Extended Abstracts on Human Factors in Computing Systems. ACM, 580--585. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Margaret M. Bradley and Peter J. Lang. 1994. Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25, 1 (1994), 49--59.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Jongeun Cha, Mohamad Eid, Ahmad Barghout, A. S. M. Rahman, and Abdulmotaleb El Saddik. 2009. HugMe: Synchronous haptic teleconferencing. In Proceedings of the 17th ACM International Conference on Multimedia. ACM, 1135--1136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Fabien Danieau, Anatole Lécuyer, Philippe Guillotel, Julien Fleureau, Nicolas Mollet, and Marc Christie. 2013. Enhancing audiovisual experience with haptic feedback: A survey on HAV. IEEE Transactions on Haptics 6, 2 (2013), 193--205. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Mohamad A. Eid and Hussein Al Osman. 2016. Affective haptics: Current research and future directions. IEEE Access 4 (2016), 26--40.Google ScholarGoogle ScholarCross RefCross Ref
  9. Paul Ekman and Wallace V. Friesen. 1971. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology 17, 2 (1971), 124.Google ScholarGoogle ScholarCross RefCross Ref
  10. Abdelwahab Hamam, Mohamad Eid, and Abdulmotaleb El Saddik. 2013. Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications. Multimedia Tools and Applications 2, 67 (2013), 455--472. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Immersion Corporation. 2016. Immersion Corporation Website. Retrieved from http://ir.immersion.com.Google ScholarGoogle Scholar
  12. Ali Israr and Ivan Poupyrev. 2011. Control space of apparent haptic motion. In World Haptics Conference (WHC), 2011 IEEE. IEEE, 457--462.Google ScholarGoogle ScholarCross RefCross Ref
  13. Ali Israr, Siyan Zhao, Kaitlyn Schwalje, Roberta Klatzky, and Jill Lehman. 2014. Feel effects: Enriching storytelling with haptic feedback. ACM Transactions on Applied Perception (TAP) 11, 3 (2014), 11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Sreekar Krishna, Shantanu Bala, Troy McDaniel, Stephen McGuire, and Sethuraman Panchanathan. 2010. VibroGlove: An assistive technology aid for conveying facial expressions. In CHI’10 Extended Abstracts on Human Factors in Computing Systems, April 10--15, 2010. 3637--3642. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Victor V. Kryssanov, Eric W. Cooper, Hitoshi Ogawa, and I. Kurose. 2009. A computational model to relay emotions with tactile stimuli. In Proceedings of the 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops. IEEE, 1--6.Google ScholarGoogle Scholar
  16. Paul Lemmens, Floris Crompvoets, Dirk Brokken, Jack Van Den Eerenbeemd, and Gert-Jan de Vries. 2009. A body-conforming tactile jacket to enrich movie viewing. In Proceedings of the EuroHaptics Conference, 2009 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics 2009. Third Joint. IEEE, 7--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Rodrigo Lentini, Beatrice Ionascu, Friederike A. Eyssel, Scandar Copti, and Mohamad Eid. 2016. Authoring tactile gestures: Case study for emotion stimulation. World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering 10, 11 (2016), 1862--1867.Google ScholarGoogle Scholar
  18. Antonella Mazzoni and Nick Bryan-Kinns. 2015. How does it feel like? An exploratory study of a prototype system to convey emotion through haptic wearable devices. In Proceedings of the 2015 7th International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN’15). IEEE, 64--68. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Lauri Nummenmaa, Enrico Glerean, Riitta Hari, and Jari K. Hietanen. 2014. Bodily maps of emotions. Proceedings of the National Academy of Sciences 111, 2 (2014), 646--651.Google ScholarGoogle ScholarCross RefCross Ref
  20. Jonathan Posner, James A. Russell, and Bradley S. Peterson. 2005. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and Psychopathology 17, 03 (2005), 715--734.Google ScholarGoogle ScholarCross RefCross Ref
  21. Munaf Rashid, S. A. R. Abu-Bakar, and Musa Mokji. 2013. Human emotion recognition from videos using spatio-temporal and audio features. The Visual Computer 29, 12 (2013), 1269--1275. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Miriam Reiner. 2004. The role of haptics in immersive telecommunication environments. IEEE Transactions on Circuits and Systems for Video Technology 14, 3 (2004), 392--401. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Katri Salminen, Veikko Surakka, Jani Lylykangas, Jukka Raisamo, Rami Saarinen, Roope Raisamo, Jussi Rantala, and Grigori Evreinov. 2008. Emotional and behavioral responses to haptic stimulation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1555--1562. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Alexandre Schaefer, Frédéric Nils, Xavier Sanchez, and Pierre Philippot. 2010. Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers. Cognition and Emotion 24, 7 (2010), 1153--1172.Google ScholarGoogle ScholarCross RefCross Ref
  25. Kai Sun, Junqing Yu, Yue Huang, and Xiaoqiang Hu. 2009. An improved valence-arousal emotion space for video affective content representation and recognition. In Proceedings of the 2009 IEEE International Conference on Multimedia and Expo. IEEE, 566--569. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Amaya Becvar Weddle and Hua Yu. 2013. How does audio-haptic enhancement influence emotional response to mobile media? In Proceedings of the 2013 5th International Workshop on Quality of Multimedia Experience (QoMEX). IEEE, 158--163.Google ScholarGoogle ScholarCross RefCross Ref
  27. Bob G. Witmer and Michael J. Singer. 1998. Measuring presence in virtual environments: A presence questionnaire. Presence: Teleoperators and Virtual Environments 7, 3 (1998), 225--240. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Intensifying Emotional Reactions via Tactile Gestures in Immersive Films

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!