skip to main content
note

Fusion Pre-trained Emoji Feature Enhancement for Sentiment Analysis

Authors Info & Claims
Published:25 March 2023Publication History
Skip Abstract Section

Abstract

Emoji are often used in social media to enrich users’ emotions, and they play an important role in the task of social media sentiment analysis. In practice, researchers are more likely to consider emoji as special symbols and treat them separately from the text. Some existing methods use emoji as a dictionary for matching or converting emoji into text. However, these methods disregard the relationship between emoji and context, blue and they do not reflect the emotions that users are expected to express. It is challenging to incorporate the original emotions of emoji in social media sentiment analysis. In this article, we propose the EPE model: Emoji Pre-trained feature Enhanced sentiment analysis. Specifically, we collected 8 million tweets and selected 5 million tweets with pre-trained emoji with context using the BERT model. We labeled 20,000 tweets as a three-category dataset and used Bi-LSTM with an attention layer to extract text features. Emoji were retained as key emotion information and combined with text features in the final layer as a connected vector for final prediction. Experimental results with our dataset showed that the proposed EPE model achieved better performance than other baseline models.

REFERENCES

  1. [1] Al-Azani Sadam and El-Alfy El-Sayed. 2018. Emojis-based sentiment classification of arabic microblogs using deep recurrent neural networks. In Proceedings of the International Conference on Computing Sciences and Engineering. 16.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Barbieri Francesco, Anke Luis Espinosa, Camacho-Collados José, Schockaert Steven, and Saggion Horacio. 2018. Interpretable emoji prediction via label-wise attention LSTMs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 47664771.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Baron Naomi S.. 2009. The myth of impoverished signal: Dispelling the spoken language fallacy for emoticons in online communication. In Proceedings of the Electronic Emotion: The Mediation of Emotion via Information and Communication Technologies. 107135.Google ScholarGoogle Scholar
  4. [4] Chen Peng, Sun Zhongqian, Bing Lidong, and Yang Wei. 2017. Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 452461.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Chen Yuxiao, Yuan Jianbo, You Quanzeng, and Luo Jiebo. 2018. Twitter sentiment analysis via bi-sense emoji embedding and attention-based LSTM. In Proceedings of the 2018 ACM Multimedia Conference on Multimedia Conference. ACM, 117125.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Chen Yuxiao, Yuan Jianbo, You Quanzeng, and Luo Jiebo. 2018. Twitter sentiment analysis via bisense emoji embedding and attention-based LSTM. In Proceedings of the 2018 ACM Multimedia Conference on Multimedia Conference. 117125.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Chen Zhenpeng, Shen Sheng, Hu Ziniu, Lu Xuan, Mei Qiaozhu, and Liu Xuanzhe. 2019. Emoji-powered representation learning for cross-lingual sentiment classification. In Proceedings of the World Wide Web Conference, WWW 2019. ACM, 251262.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Davidov Dmitry, Tsur Oren, and Rappoport Ari. 2010. Enhanced sentiment learning using twitter hashtags and smileys. In Proceedings of the COLING 2010, 23rd International Conference on Computational Linguistics. Chinese Information Processing Society of China, 241249.Google ScholarGoogle Scholar
  9. [9] Devlin Jacob, Chang Ming-Wei, Lee Kenton, and Toutanova Kristina. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019. Association for Computational Linguistics, 41714186.Google ScholarGoogle Scholar
  10. [10] Felbo Bjarke, Mislove Alan, Søgaard Anders, Rahwan Iyad, and Lehmann Sune. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017. Association for Computational Linguistics, 16151625.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Hu Xia, Tang Jiliang, Gao Huiji, and Liu Huan. 2013. Unsupervised sentiment analysis with emotional signals. In Proceedings of the 22nd International World Wide Web Conference. International World Wide Web Conferences Steering Committee/ACM, 607618.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Joshi Aditya, Bhattacharyya Pushpak, and Carman Mark James. 2017. Automatic sarcasm detection: A survey. ACM Computing Surveys 50, 5 (2017), 73:1–73:22.Google ScholarGoogle Scholar
  13. [13] Kalchbrenner Nal, Grefenstette Edward, and Blunsom Phil. 2014. A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014. The Association for Computer Linguistics, 655665.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Kiritchenko Svetlana, Zhu Xiaodan, and Mohammad Saif M.. 2014. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research 50, 2014 (2014), 723762. Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Le Tuan Anh, Moeljadi David, Miura Yasuhide, and Ohkuma Tomoko. 2016. Sentiment analysis for low resource languages: A study on informal indonesian tweets. In Proceedings of the 12th Workshop on Asian Language Resources.Hasida Kôiti, Wong Kam-Fai, Calzorari Nicoletta, and Choi Key-Sun (Eds.), The COLING 2016 Organizing Committee, 123131.Google ScholarGoogle Scholar
  16. [16] Li Da, Rzepka Rafal, Ptaszynski Michal, and Araki Kenji. 2018. Emoticon-aware recurrent neural network model for chinese sentiment analysis. In Proceedings of the 9th International Conference on Awareness Science and Technology. IEEE, 161166.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Li Da, Rzepka Rafal, Ptaszynski Michal, and Araki Kenji. 2019. Emoji-aware attention-based bi-directional GRU network model for chinese sentiment analysis. In Joint Proceedings of the Workshops on Linguistic and Cognitive Approaches to Dialog Agents (LaCATODA 2019) and on Bridging the Gap Between Human and Automated Reasoning (BtG 2019) co-located with 28th International Joint Conference on Artificial Intelligence (IJCAI 2019).CEUR-WS.org, 1118.Google ScholarGoogle Scholar
  18. [18] Li Da, Rzepka Rafal, Ptaszynski Michal, and Araki Kenji. 2019. A novel machine learning-based sentiment analysis method for chinese social media considering chinese slang lexicon and emoticons. In Proceedings of the 2nd Workshop on Affective Content Analysis (AffCon 2019) Co-located with 33rd AAAI Conference on Artificial Intelligence . CEUR-WS.org, 88103.Google ScholarGoogle Scholar
  19. [19] Liu Bing. 2012. Sentiment Analysis and Opinion Mining. Morgan and Claypool Publishers.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Lou Yinxia, Zhang Yue, Li Fei, Qian Tao, and Ji Donghong. 2020. Emoji-based sentiment analysis using attention networks. ACM Transactions on Asian and Low-Resource Language Information Processing 19, 5 (2020), 64:1–64:13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. [21] Mohammad Saif M., Kiritchenko Svetlana, and Zhu Xiaodan. 2013. NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. In Proceedings of the 7th International Workshop on Semantic Evaluation. The Association for Computer Linguistics, 321327.Google ScholarGoogle Scholar
  22. [22] Moschini Ilaria. 2016. The “face with tears of joy” emoji. a socio-semiotic and multimodal insight into a japan-america mash-up. HERMES-Journal of Language and Communication in Business 55, 2016 (2016), 1125.Google ScholarGoogle Scholar
  23. [23] Narr Sascha, Hulfenhaus Michael, and Albayrak Sahin. 2012. Language-independent twitter sentiment analysis. In Proceedings of the Learning, Knowledge, and Adaption Conference. 1214.Google ScholarGoogle Scholar
  24. [24] Pang Bo, Lee Lillian, and Vaithyanathan Shivakumar. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing. 7986.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Ren Yafeng, Ji Donghong, and Ren Han. 2018. Context-augmented convolutional neural networks for twitter sarcasm detection. Neurocomputing 308, 2018 (2018), 17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Subramanian Jayashree, Sridharan Varun, Shu Kai, and Liu Huan. 2019. Exploiting emojis for sarcasm detection. In Proceedings of the Social, Cultural, and Behavioral Modeling—12th International Conference, SBP-BRiMS 2019. Springer, 7080.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Thelwall Mike, Buckley Kevan, and Paltoglou Georgios. 2012. Sentiment strength detection for the social web. Journal of the Association for Information Science and Technology 63, 1 (2012), 163173.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Yang Zichao, Yang Diyi, Dyer Chris, He Xiaodong, Smola Alexander J., and Hovy Eduard H.. 2016. Hierarchical attention networks for document classification. In Proceedings of the NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. The Association for Computational Linguistics, 14801489.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Yih Wen-tau, He Xiaodong, and Meek Christopher. 2014. Semantic parsing for single-relation question answering. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics, 643648.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Zhou Peng, Shi Wei, Tian Jun, Qi Zhenyu, Li Bingchen, Hao Hongwei, and Xu Bo. 2016. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Fusion Pre-trained Emoji Feature Enhancement for Sentiment Analysis

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 4
      April 2023
      682 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3588902
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 25 March 2023
      • Online AM: 29 December 2022
      • Accepted: 19 December 2022
      • Revised: 21 September 2022
      • Received: 1 September 2021
      Published in tallip Volume 22, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • note
    • Article Metrics

      • Downloads (Last 12 months)164
      • Downloads (Last 6 weeks)19

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!