skip to main content
research-article

Emo2Vec: Learning Emotional Embeddings via Multi-Emotion Category

Authors Info & Claims
Published:17 April 2020Publication History
Skip Abstract Section

Abstract

Sentiment analysis or opinion mining for subject information extraction from the text has become more and more dependent on natural language processing, especially for business and healthcare, since the online products and service reviews affect the consuming behaviors. Word embeddings that can map the words to low-dimensional vector representations have been widely used in natural language processing tasks. But the word embeddings based on context such as Word2Vec and GloVe fail to capture the sentiment information. Most of existing sentiment analysis methods incorporate emotional polarity (positive and negative) to improve the sentiment embeddings for the emotion classification. This article takes advantage of an emotional psychology model to learn the emotional embeddings in Chinese first. In order to combine the semantic space and an emotional space, we present two different purifying models from local (LPM) and global (GPM) perspectives based on Plutchik's wheel of emotions to add the emotional information into word vectors. The two models aim to improve the word vectors so that not only the semantically similar words but also the sentimentally similar words can be closer than before. The Plutchik's wheel of emotions model can give eight-dimensional vector for one word in emotional space that can capture more sentiment information than the binary polarity labels. The obvious advantage of the local purifying model is that it can be fit for any pretrained word embeddings. For the global purifying model, we can get the final emotional embeddings at once. These models have been extended to handle English texts. The experimental results on Chinese and English datasets show that our purifying model can improve the conventional word embeddings and some proposed sentiment embeddings for sentiment classification and multi-emotion classification.

References

  1. R. Plutchik. 2001. The nature of emotions. American Scientist, 89(4).Google ScholarGoogle Scholar
  2. Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research 3 (2003), 1137--1155.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. R. Collobert and J. Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multi task learning. In Proc. Int. Conf. Machine Learning. 160--167.Google ScholarGoogle Scholar
  4. A. Mnih and G. E. Hinton 2008. A scalable hierarchical distributed language model. Neural Information Processing Systems. 1081--1088.Google ScholarGoogle Scholar
  5. T. Mikolov, K. Chen, G. Corrado, and J. Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations. 1--12.Google ScholarGoogle Scholar
  6. O. Levy and Y. Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Baltimore, Maryland, 302--308.Google ScholarGoogle Scholar
  7. Y. Liu, Z. Liu, T.-S. Chua, and M. Sun. 2015. Topical word embeddings. Association for the Advancement of Artificial Intelligence. 2418--2424.Google ScholarGoogle Scholar
  8. A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Ng, and C. Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, 142--150.Google ScholarGoogle Scholar
  9. I. Labutov and H. Lipson. 2013. Re-embedding words. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Sofia, Bulgaria 489--493.Google ScholarGoogle Scholar
  10. D. Tang, F. Wei, B. Qin, N. Yang, T. Liu, and M. Zhou. 2016. Sentiment embeddings with applications to sentiment analysis. IEEE Transactions on Knowledge and Data Engineering 28, 2 (2016), 496--509.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Y. Ren, Y. Zhang, M. Zhang, and D. Ji. 2016. Improving twitter sentiment classification using topic-enriched multi-prototype word embeddings. In Proceedings of the Association for the Advancement of Artificial Intelligence. 3038--3044.Google ScholarGoogle Scholar
  12. M. Lan, Z. Zhang, Y. Lu, and J. Wu. 2016. Three convolutional neural network based models for learning sentiment word vectors towards sentiment analysis. In Proceedings of the International Joint Conference on Neural Networks. 3172--3179.Google ScholarGoogle Scholar
  13. Saif M. Mohammad and Peter D. Turney. 2013. NRC emotion lexicon. NRC Technical Report.Google ScholarGoogle Scholar
  14. Carlo Strapparava and Alessandro Valitutti. 2004. Wordnet affect: An affective extension of wordnet. In Proceedings of the International Conference on Language Resources and Evaluation.Google ScholarGoogle Scholar
  15. Soujanya Poria, Alexander Gelbukh, Erik Cambria, Amir Hussain, and Guang-Bin Huang. 2014. Emosenticspace: A novel framework for affective common-sense reasoning. Knowledge-Based Systems, 69:108--123.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Ameeta Agrawal and Aijun An. 2012. Unsupervised emotion detection from text using semantic and syntactic relations. In Proceedings of the International Conference on Web Intelligence and Intelligent Agent Technology.Google ScholarGoogle ScholarCross RefCross Ref
  17. Isidoros Perikos and Ioannis Hatzilygeroudis. 2016. Recognizing emotions in text using ensemble of classifiers. Engineering Applications of Artificial Intelligence, 51:191--201.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Munmun De Choudhury, Michael Gamon, and Scott Counts. 2012. Happy, nervous or surprised? Classification of human affective states in social media. In Proceedings of the International Conference on Web and Social Media.Google ScholarGoogle Scholar
  19. Min-Ling Zhang and Zhi-Hua Zhou. 2014. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering 26, 8 (2014), 1819--1837.Google ScholarGoogle ScholarCross RefCross Ref
  20. Xin Geng, Chao Yin, and Zhi-Hua Zhou. 2013 Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 10 (2013), 2401--2412.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.Google ScholarGoogle ScholarCross RefCross Ref
  22. Qiao Qian, Minlie Huang, and Xiaoyan Zhu. 2017. Linguistically regularized LSTMs for sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.Google ScholarGoogle ScholarCross RefCross Ref
  23. Carlo Strapparava and Rada Mihalcea. 2007. Semeval-2007 task 14: Affective text. In Proceedings of International Workshop on Semantic Evaluations.Google ScholarGoogle ScholarCross RefCross Ref
  24. R. Plutchik. 2005. Emotions and life: Perspectives from psychology, biology, and evolution. Psychology 8 Marketing. 97--101.Google ScholarGoogle Scholar
  25. R. Plutchik. 2002. Emotions and life. Perspectives from Psychology Biology 8 Evolution.Google ScholarGoogle Scholar
  26. S. Wang and X. Meng. 2018. Multi-emotion category improving embedding for sentiment classification. In Proceedings of the Conference on Information and Knowledge Management. 1719--1722.Google ScholarGoogle Scholar
  27. J. Pennington, R. Socher, and C. D. Manning. 2014. GloVe: Global vectors forword representation. In Proceedings of the Empirical Methods of Natural Language Processing. 1532--1543.Google ScholarGoogle Scholar
  28. G. A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM 38, 11 (1995), 39--41.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. C. F. Baker, C. J. Fillmore, and J. B. Lowe. 1998. The Berkeley framenet project. In Proceedings of the Association for Computational Linguistics. 86--90.Google ScholarGoogle Scholar
  30. J. Ganitkevitch, B. Van Durme, and C. Callison-Burch. 2013. PPDB: The paraphrase database. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics. 758--764.Google ScholarGoogle Scholar
  31. E. Cambria. 2016. Affective computing and sentiment analysis. IEEE Intelligent Systems 31, 2 (2016), 102--107.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016. Attention-based LSTM for aspect-level sentiment classification. In Empirical Methods Natural Language Processing. 606--615Google ScholarGoogle Scholar
  33. Y. Li, Q. Pan, T. Yang, et al. 2017. Learning word representation for sentiment analysis. Cognitive Computation. 9, 6 (2017), 843--851.Google ScholarGoogle ScholarCross RefCross Ref
  34. M. Munezero, C. S. Montero, E. Sutinen, and J. Pajunen. 2014. Are they different? affect, feeling, emotion, sentiment, and opinion detection in text. IEEE Transactions on Affective Computing 5, 2 (2014), 101--111.Google ScholarGoogle ScholarCross RefCross Ref
  35. M. B. Arnold. 1950. An excitatory theory of emotion. In Feelings and Emotions: The Mooseheart Symposium, M. L. Reymert (Ed.). McGraw-Hill, 11--33.Google ScholarGoogle Scholar
  36. P. Ekman. 1992. An argument for basic emotions. Cognition 8 Emotion 6 (3–4), 169--200.Google ScholarGoogle Scholar
  37. C. E. Izard. 1991. The Psychology of Emotions. Plenum, New York.Google ScholarGoogle Scholar
  38. W. MacDougall. 1921. An Introduction to Social Psychology. John W. Luce, Boston, MA.Google ScholarGoogle Scholar
  39. Saif M. Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. Semeval-2018 Task 1: Affect in tweets. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2018), New Orleans, LA.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Emo2Vec: Learning Emotional Embeddings via Multi-Emotion Category

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!