Abstract
Today, ubiquitous digital communication systems do not have an intuitive, natural way of communicating emotion, which, in turn, affects the degree to which humans can emotionally connect and interact with one another. To address this problem, a more natural, intuitive, and implicit emotion communication system was designed and created that employs asymmetry-based EEG emotion classification for detecting the emotional state of the sender and haptic feedback (in the form of tactile gestures) for displaying emotions for a receiver. Emotions are modeled in terms of valence (positive/negative emotions) and arousal (intensity of the emotion). Performance analysis shows that the proposed EEG subject-dependent emotion classification model with Free Asymmetry features allows for more flexible feature-generation schemes than other existing algorithms and attains an average accuracy of 92.5% for valence and 96.5% for arousal, outperforming previous-generation schemes in high feature space. As for the haptic feedback, a tactile gesture authoring tool and a haptic jacket were developed to design tactile gestures that can intensify emotional reactions in terms of valence and arousal. Experimental study demonstrated that subject-independent emotion transmission through tactile gestures is effective for the arousal dimension of an emotion but is less effective for valence. Consistency in subject-dependent responses for both valence and arousal suggests that personalized tactile gestures would be more effective.
- Omar AlZoubi, Rafael A. Calvo, and Ronald H. Stevens. 2009. Classification of EEG for affect recognition: An adaptive approach. In Australasian Joint Conference on Artificial Intelligence. Springer, 52--61. Google Scholar
Digital Library
- Bradley M. Appelhans and Linda J. Luecken. 2006. Heart rate variability as an index of regulated emotional responding. Review of General Psychology 10, 3, 229.Google Scholar
Cross Ref
- Faisal Arafsha, Kazi Masudul Alam, and Abdulmotaleb El Saddik. 2015. Design and development of a user centric affective haptic jacket. Multimedia Tools and Applications 74, 9, 3035--3052. Google Scholar
Digital Library
- Jeremy N. Bailenson, Nick Yee, Scott Brave, Dan Merget, and David Koslow. 2007. Virtual interpersonal touch: Expressing and recognizing emotions through haptic devices. Human--Computer Interaction 22, 3, 325--353. Google Scholar
Digital Library
- Tonio Ball, Markus Kern, Isabella Mutschler, Ad Aertsen, and Andreas Schulze-Bonhage. 2009. Signal quality of simultaneously recorded invasive and non-invasive EEG. Neuroimage 46, 3, 708--716.Google Scholar
Cross Ref
- Margaret M. Bradley and Peter J. Lang. 1994. Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25, 1, 49--59.Google Scholar
Digital Library
- Lindsay Brown, Bernard Grundlehner, and Julien Penders. 2011. Towards wireless emotional valence detection from EEG. In Engineering in Medicine and Biology Society (EMBC’11) Annual International Conference of the IEEE. IEEE, 2188--2191.Google Scholar
- Ross Buck. 1994. The neuropsychology of communication: Spontaneous and symbolic aspects. Journal of Pragmatics 22, 3-4, 265--278.Google Scholar
Cross Ref
- Jongeun Cha, Mohamad Eid, Ahmad Barghout, ASM Rahman, and Abdulmotaleb El Saddik. 2009. HugMe: Synchronous haptic teleconferencing. In Proceedings of the 17th ACM International Conference on Multimedia. ACM, 1135--1136. Google Scholar
Digital Library
- Monisha Chakraborty and Shreya Das. 2012. Determination of signal to noise ratio of electrocardiograms filtered by band pass and Savitzky-Golay filters. Procedia Technology 4, 830--833.Google Scholar
Cross Ref
- Manfred Clynes. 1977. Sentics: The Touch of Emotions. Anchor Press, New York, NY.Google Scholar
- Ruo-Nan Duan, Jia-Yi Zhu, and Bao-Liang Lu. 2013. Differential entropy feature for EEG-based emotion classification. In Neural Engineering (NER), 2013 6th International IEEE/EMBS Conference on. IEEE, 81--84.Google Scholar
Cross Ref
- Elia Gatti, Giandomenico Caruso, Monica Bordegoni, and Charles Spence. 2013. Can the feel of the haptic interaction modify a user’s emotional state?. In World Haptics Conference (WHC’13). IEEE, 247--252.Google Scholar
Cross Ref
- Antal Haans and Wijnand I. Jsselsteijn. 2006. Mediated social touch: A review of current research and future directions. Virtual Reality 9, 2-3, 149--159. Google Scholar
Digital Library
- Ali Israr and Ivan Poupyrev. 2011. Control space of apparent haptic motion. In World Haptics Conference (WHC’11). IEEE, 457--462.Google Scholar
Cross Ref
- Tanimu Ahmed Jibril and Mardziah Hayati Abdullah. 2013. Relevance of emoticons in computer-mediated communication contexts: An overview. Asian Social Science 9, 4, 201.Google Scholar
Cross Ref
- Min-Ki Kim, Miyoung Kim, Eunmi Oh, and Sung-Phil Kim. 2013. A review on the computational methods for emotional state estimation from the human EEG. Computational and Mathematical Methods in Medicine, vol. 2013, 13 pages.Google Scholar
Cross Ref
- Franklin B. Krohn. 2004. A generational approach to using emoticons as nonverbal communication. Journal of Technical Writing and Communication 34, 4, 321--328.Google Scholar
Cross Ref
- Peter J. Lang, Margaret M. Bradley, and Bruce N. Cuthbert. 2008. International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical Report A-8. Gainesville, FL: University of Florida, USA.Google Scholar
- Paul Lemmens, Floris Crompvoets, Dirk Brokken, Jack Van Den Eerenbeemd, and Gert-Jan de Vries. 2009. A body-conforming tactile jacket to enrich movie viewing. In 3rd Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (World Haptics’09). IEEE, 7--12. Google Scholar
Digital Library
- Rodrigo Lentini, Beatrice Ionascu, Friederike A. Eyssel, Scandar Copti, and Mohamad Eid. 2016. Authoring tactile gestures: Case study for emotion stimulation. World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering 10, 11, 1862--1867.Google Scholar
- Antonella Mazzoni and Nick Bryan-Kinns. 2015a. How does it feel like? An exploratory study of a prototype system to convey emotion through haptic wearable devices. In 7th International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN’15). IEEE, 64--68. Google Scholar
Digital Library
- Troy McDaniel, Shantanu Bala, Jacob Rosenthal, Ramin Tadayon, Arash Tadayon, and Sethuraman Panchanathan. 2014. Affective haptics for enhancing access to social interactions for individuals who are blind. In International Conference on Universal Access in Human-Computer Interaction. Springer, 419--429.Google Scholar
Cross Ref
- Neurobehavioral Systems Inc. 2017. Presentation. Retrieved November 14, 2017 from https://www.neurobs.com/.Google Scholar
- Lauri Nummenmaa, Enrico Glerean, Riitta Hari, and Jari K. Hietanen. 2014. Bodily maps of emotions. Proceedings of the National Academy of Sciences 111, 2, 646--651.Google Scholar
Cross Ref
- Panagiotis C. Petrantonakis and Leontios J. Hadjileontiadis. 2011. A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition. IEEE Transactions on Information Technology in Biomedicine 15, 5, 737--746. Google Scholar
Digital Library
- Abu Saleh Md Mahfujur Rahman, S. K. Alamgir Hossain, and Abdulmotaleb El Saddik. 2010. Bridging the gap between virtual and real world by bringing an interpersonal haptic communication system in second life. In IEEE International Symposium on Multimedia (ISM’10). IEEE, 228--235. Google Scholar
Digital Library
- Jukka Raisamo, Roope Raisamo, and Veikko Surakka. 2013. Comparison of saltation, amplitude modulation, and a hybrid method of vibrotactile stimulation. IEEE Transactions on Haptics 6, 4, 517--521. Google Scholar
Digital Library
- Debra L. Roter, Richard M. Frankel, Judith A. Hall, and David Sluyter. 2006. The expression of emotion through nonverbal behavior in medical visits. Journal of General Internal Medicine 21, S1, S28--S34.Google Scholar
Cross Ref
- Katri Salminen, Veikko Surakka, Jani Lylykangas, Jukka Raisamo, Rami Saarinen, Roope Raisamo, Jussi Rantala, and Grigori Evreinov. 2008. Emotional and behavioral responses to haptic stimulation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1555--1562. Google Scholar
Digital Library
- Louis A. Schmidt and Laurel J. Trainor. 2001. Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cognition 8 Emotion 15, 4, 487--500.Google Scholar
- Mohammad Soleymani, Maja Pantic, and Thierry Pun. 2012. Multimodal emotion recognition in response to videos. IEEE Transactions on Affective Computing 3, 2, 211--223. Google Scholar
Digital Library
- Kai Sun, Junqing Yu, Yue Huang, and Xiaoqiang Hu. 2009. An improved valence-arousal emotion space for video affective content representation and recognition. In IEEE International Conference on Multimedia and Expo (ICME’09). IEEE, 566--569. Google Scholar
Digital Library
- Dzmitry Tsetserukou, Alena Neviarouskaya, Helmut Prendinger, Mitsuru Ishizuka, and Susumu Tachi. 2010. iFeel_IM: Innovative real-time communication system with rich emotional and haptic channels. In CHI’10 Extended Abstracts on Human Factors in Computing Systems. ACM, 3031--3036. Google Scholar
Digital Library
- Xiao-Wei Wang, Dan Nie, and Bao-Liang Lu. 2014. Emotional state classification from EEG data using machine learning approach. Neurocomputing 129, 94--106. Google Scholar
Digital Library
- Joyce H. D. M. Westerink, Egon L. Van Den Broek, Marleen H. Schut, Jan Van Herk, and Kees Tuinenbreijer. 2008. Computing emotion awareness through galvanic skin response and facial electromyography. In Probing Experience. Springer, 149--162.Google Scholar
- Yongjae Yoo, Taekbeom Yoo, Jihyun Kong, and Seungmoon Choi. 2015. Emotional responses of tactile icons: Effects of amplitude, frequency, duration, and envelope. In IEEE World Haptics Conference (WHC’15). IEEE, 235--240.Google Scholar
- Xu Zhe and A. C. Boucouvalas. 2002. Text-to-emotion engine for real time Internet communication. In Proceedings of International Symposium on Communication Systems, Networks and DSPs. 164--168.Google Scholar
- Wei-Long Zheng, Jia-Yi Zhu, and Bao-Liang Lu. 2016. Identifying stable patterns over time for emotion recognition from EEG. Arxiv Preprint Arxiv:1601.02197.Google Scholar
Index Terms
Implicit Emotion Communication: EEG Classification and Haptic Feedback
Recommendations
Emotion Actuator: Embodied Emotional Feedback through Electroencephalography and Electrical Muscle Stimulation
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing SystemsThe human body reveals emotional and bodily states through measurable signals, such as body language and electroencephalography. However, such manifestations are difficult to communicate to others remotely. We propose EmotionActuator, a proof-of-concept ...
Intensifying Emotional Reactions via Tactile Gestures in Immersive Films
The film industry continuously strives to make visitors’ movie experience more immersive and thus, more captivating. This is realized through larger screens, sophisticated speaker systems, and high quality 2D and 3D content. Moreover, a recent trend in ...
iFeel_IM! Emotion Enhancing Garment for Communication in Affect Sensitive Instant Messenger
Proceedings of the Symposium on Human Interface 2009 on ConferenceUniversal Access in Human-Computer Interaction. Part I: Held as Part of HCI International 2009In the paper we are proposing a conceptually novel approach to reinforcing (intensifying) own feelings and reproducing (simulating) the emotions felt by the partner during online communication through specially designed system, iFeel_IM!. The core ...






Comments