skip to main content
research-article

Exploring Multi-lingual, Multi-task, and Adversarial Learning for Low-resource Sentiment Analysis

Published:23 September 2022Publication History
Skip Abstract Section

Abstract

Deep learning has become most prominent in solving various Natural Language Processing (NLP) tasks including sentiment analysis. However, these techniques require a considerably large amount of annotated corpus, which is not easy to obtain for most of the languages, especially under the scenario of low-resource settings. In this article, we propose a deep multi-task multi-lingual adversarial framework to solve the resource-scarcity problem of sentiment analysis by leveraging the useful and relevant knowledge from a high-resource language. To transfer the knowledge between the different languages, both the languages are mapped to the shared semantic space using cross-lingual word embeddings. We evaluate our proposed architecture on a low-resource language, Hindi, using English as the high-resource language. Experiments show that our proposed model achieves an accuracy of 60.09% for the movie review dataset and 72.14% for the product review dataset. The effectiveness of our proposed approach is demonstrated with significant performance gains over the state-of-the-art systems and translation-based baselines.

REFERENCES

  1. [1] Mamta, Ekbal Asif, Bhattacharyya Pushpak, Srivastava Shikha, Kumar Alka, and Saha Tista. 2020. Multi-domain tweet corpora for sentiment analysis: Resource creation and evaluation. In Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association, Marseille, France, 50465054. Google ScholarGoogle Scholar
  2. [2] Abadi Martín, Barham Paul, Chen Jianmin, Chen Zhifeng, Davis Andy, Dean Jeffrey, Devin Matthieu, Ghemawat Sanjay, Irving Geoffrey, Isard Michael, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI’16). 265283.Google ScholarGoogle Scholar
  3. [3] Ahmad Zishan, Jindal Raghav, Ekbal Asif, and Bhattachharyya Pushpak. 2020. Borrow from rich cousin: Transfer learning for emotion detection using cross lingual embedding. Expert Systems with Applications 139 (2020), 112851.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Akhtar Md Shad, Ekbal Asif, and Cambria Erik. 2020. How intense are you? Predicting intensities of emotions and sentiments using stacked ensemble. IEEE Computational Intelligence Magazine 15, 1 (2020), 6475.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Akhtar Md Shad, Gupta Deepak, Ekbal Asif, and Bhattacharyya Pushpak. 2017. Feature selection and ensemble construction: A two-step method for aspect based sentiment analysis. Knowledge-based Systems 125 (2017), 116135.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Akhtar Md Shad, Kumar Ayush, Ekbal Asif, and Bhattacharyya Pushpak. 2016. A hybrid deep learning architecture for sentiment analysis. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING’16). 482493.Google ScholarGoogle Scholar
  7. [7] Akhtar Md Shad, Kumar Abhishek, Ghosal Deepanway, Ekbal Asif, and Bhattacharyya Pushpak. 2017. A multilayer perceptron based ensemble technique for fine-grained financial sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 540546.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Akhtar Md Shad, Sawant Palaash, Sen Sukanta, Ekbal Asif, and Bhattacharyya Pushpak. 2018. Solving data sparsity for aspect based sentiment analysis using cross-linguality and multi-linguality. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, 572582. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Araque Oscar, Corcuera-Platas Ignacio, Sánchez-Rada J. Fernando, and Iglesias Carlos A.. 2017. Enhancing deep learning sentiment analysis with ensemble techniques in social applications. Expert Systems with Applications 77 (2017), 236246.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Attia Mohammed, Samih Younes, Elkahky Ali, and Kallmeyer Laura. 2018. Multilingual multi-class sentiment classification using convolutional neural networks. In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC’18).Google ScholarGoogle Scholar
  11. [11] Bakliwal Akshat, Foster Jennifer, Puil Jennifer van der, O’Brien Ron, Tounsi Lamia, and Hughes Mark. 2013. Sentiment Analysis of Political Tweets: Towards an Accurate Classifier. Association for Computational Linguistics.Google ScholarGoogle Scholar
  12. [12] Balamurali A. R., Joshi Aditya, and Bhattacharyya Pushpak. 2012. Cross-lingual sentiment analysis for Indian languages using linked wordnets. In Proceedings of COLING 2012: Posters, Martin Kay and Christian Boitet (Eds.). Indian Institute of Technology Bombay, 73–82. https://aclanthology.org/C12-2008/.Google ScholarGoogle Scholar
  13. [13] Bojanowski Piotr, Grave Edouard, Joulin Armand, and Mikolov Tomas. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics 5 (2017), 135146.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Can Ethem F., Ezen-Can Aysu, and Can Fazli. 2018. Multilingual sentiment analysis: An RNN-based framework for limited data. arXiv preprint arXiv:1806.04511 (2018).Google ScholarGoogle Scholar
  15. [15] Chaturvedi Iti, Cambria Erik, Welsch Roy E., and Herrera Francisco. 2018. Distinguishing between facts and opinions for sentiment analysis: Survey and challenges. Information Fusion 44 (2018), 6577.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Chen Xilun, Sun Yu, Athiwaratkun Ben, Cardie Claire, and Weinberger Kilian. 2018. Adversarial deep averaging networks for cross-lingual sentiment classification. Transactions of the Association for Computational Linguistics 6 (2018), 557570.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Chikersal Prerna, Poria Soujanya, and Cambria Erik. 2015. SeNTU: Sentiment analysis of tweets by combining a rule-based classifier with supervised learning. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval’15). 647651.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Chung Junyoung, Gulcehre Caglar, Cho Kyunghyun, and Bengio Yoshua. 2015. Gated feedback recurrent neural networks. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, R. Bach Francis and M. Blei David (Eds.). JMLR.org, 2067–2075. http://proceedings.mlr.press/v37/chung15.html.Google ScholarGoogle Scholar
  19. [19] Conneau Alexis, Lample Guillaume, Ranzato Marc’Aurelio, Denoyer Ludovic, and Jégou Hervé. 2017. Word translation without parallel data. arXiv preprint arXiv:1710.04087 (2017).Google ScholarGoogle Scholar
  20. [20] Dashtipour Kia, Poria Soujanya, Hussain Amir, Cambria Erik, Hawalah Ahmad Y. A., Gelbukh Alexander, and Zhou Qiang. 2016. Multilingual sentiment analysis: State of the art and independent comparison of techniques. Cognitive Computation 8, 4 (2016), 757771.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Dey Atanu, Jenamani Mamata, and Thakkar Jitesh J.. 2018. Senti-N-Gram: An n-gram lexicon for sentiment analysis. Expert Systems with Applications 103 (2018), 92105.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Ghiassi Manoochehr and Lee S.. 2018. A domain transferable lexicon set for Twitter sentiment analysis using a supervised machine learning approach. Expert Systems with Applications 106 (2018), 197216.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Goodfellow Ian, Pouget-Abadie Jean, Mirza Mehdi, Xu Bing, Warde-Farley David, Ozair Sherjil, Courville Aaron, and Bengio Yoshua. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems. 26722680.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Gulli Antonio and Pal Sujit. 2017. Deep Learning with Keras. Packt Publishing Ltd.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Hawkins Douglas M.. 2004. The problem of overfitting. Journal of Chemical Information and Computer Sciences 44, 1 (2004), 112.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Hochreiter Sepp and Schmidhuber Jürgen. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 17351780.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Hu Junjie, Ruder Sebastian, Siddhant Aditya, Neubig Graham, Firat Orhan, and Johnson Melvin. 2020. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event. PMLR, 4411–4421. http://proceedings.mlr.press/v119/hu20b.html.Google ScholarGoogle Scholar
  28. [28] Joshi Aditya, Balamurali A. R., and Bhattacharyya Pushpak. 2010. A fall-back strategy for sentiment analysis in hindi: A case study. Proceedings of the 8th ICON.Google ScholarGoogle Scholar
  29. [29] Kingma Diederik P. and Ba Jimmy. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google ScholarGoogle Scholar
  30. [30] Kumar Sudhanshu, Yadava Mahendra, and Roy Partha Pratim. 2019. Fusion of EEG response and sentiment analysis of products review to predict customer satisfaction. Information Fusion 52 (2019), 4152.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Lin Ying, Yang Shengqi, Stoyanov Veselin, and Ji Heng. 2018. A multi-lingual multi-task architecture for low-resource sequence labeling. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 799809.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Liu Pengfei, Qiu Xipeng, and Huang Xuanjing. 2017. Adversarial multi-task learning for text classification. arXiv preprint arXiv:1704.05742 (2017).Google ScholarGoogle Scholar
  33. [33] Liu Yang, Bi Jian-Wu, and Fan Zhi-Ping. 2017. Ranking products through online reviews: A method based on sentiment analysis technique and intuitionistic fuzzy set theory. Information Fusion 36 (2017), 149161.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Lou Yinxia, Zhang Yue, Li Fei, Qian Tao, and Ji Donghong. 2020. Emoji-based sentiment analysis using attention networks. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 19, 5 (2020), 113.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Masumura Ryo, Tanaka Tomohiro, Higashinaka Ryuichiro, Masataki Hirokazu, and Aono Yushi. 2018. Multi-task and multi-lingual joint learning of neural lexical utterance classification based on partially-shared modeling. In Proceedings of the 27th International Conference on Computational Linguistics, Emily M. Bender, Leon Derczynski and, Pierre Isabelle (Eds.). Association for Computational Linguistics, 35863596. https://aclanthology.org/C18-1304/.Google ScholarGoogle Scholar
  36. [36] Mishra Sudhanshu, Prasad Shivangi, and Mishra Shubhanshu. 2021. Exploring multi-task multi-lingual learning of transformer models for hate speech and offensive speech identification in social media. SN Computer Science 2, 2 (2021), 119.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Mohammad Saif M., Sobhani Parinaz, and Kiritchenko Svetlana. 2017. Stance and sentiment in tweets. ACM Transactions on Internet Technology (TOIT) 17, 3 (2017), 123.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] O’Hare Neil, Davy Michael, Bermingham Adam, Ferguson Paul, Sheridan Páraic, Gurrin Cathal, and Smeaton Alan F.. 2009. Topic-dependent sentiment analysis of financial blogs. In Proceedings of the 1st International CIKM Workshop on Topic-sentiment Analysis for Mass Opinion. 916.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Pang Bo, Lee Lillian, and Vaithyanathan Shivakumar. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. Association for Computational Linguistics, 7986.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Pennington Jeffrey, Socher Richard, and Manning Christopher D.. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 15321543.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Poria Soujanya, Cambria Erik, Bajpai Rajiv, and Hussain Amir. 2017. A review of affective computing: From unimodal analysis to multimodal fusion. Information Fusion 37 (2017), 98125.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Shuang Kai, Yang Qianqian, Loo Jonathan, Li Rui, and Gu Mengyu. 2020. Feature distillation network for aspect-based sentiment analysis. Information Fusion 61 (2020), 13–23.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Singhal Prerana and Bhattacharyya Pushpak. 2016. Borrow a little from your rich cousin: Using embeddings and polarities of English words for multilingual sentiment classification. In Proceedings of the 26th International Conference on Computational Linguistics (COLING’16): Technical Papers. 30533062.Google ScholarGoogle Scholar
  44. [44] Srivastava Nitish, Hinton Geoffrey, Krizhevsky Alex, Sutskever Ilya, and Salakhutdinov Ruslan. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1 (2014), 19291958.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Thakkar Gaurish, Preradovic Nives Mikelic, and Tadic Marko. 2021. Multi-task learning for cross-lingual sentiment analysis. In Proceedings of the 2nd International Workshop on Cross-lingual Event-centric Open Analytics co-located with the 30th The Web Conference (WWW’21), Ljubljana, Slovenia, April 12, 2021 (online event due to COVID-19 outbreak), Vol. 2829. CEUR-WS.org, 76–84. http://ceur-ws.org/Vol-2829/short1.pdf.Google ScholarGoogle Scholar
  46. [46] Valdivia Ana, Luzón M. Victoria, Cambria Erik, and Herrera Francisco. 2018. Consensus vote models for detecting and filtering neutrality in sentiment analysis. Information Fusion 44 (2018), 126135.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Kauter Marjan Van de, Breesch Diane, and Hoste Véronique. 2015. Fine-grained analysis of explicit and implicit sentiment in financial news articles. Expert Systems with Applications 42, 11 (2015), 49995010.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Wu Chuhan, Wu Fangzhao, Liu Junxin, Huang Yongfeng, and Xie Xing. 2019. Sentiment lexicon enhanced neural sentiment classification. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Wenwu Zhu, Dacheng Tao, Xueqi Cheng, Peng Cui, Elke A. Rundensteiner, David Carmel, Qi He, and Jeffrey Xu Yu (Eds.). ACM, 1091–1100. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Wu Fangzhao, Huang Yongfeng, and Yuan Zhigang. 2017. Domain-specific sentiment classification via fusing sentiment knowledge from multiple sources. Information Fusion 35 (2017), 2637.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Yadav Shweta, Ekbal Asif, Saha Sriparna, and Bhattacharyya Pushpak. 2019. A unified multi-task adversarial learning framework for pharmacovigilance mining. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 52345245.Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Ye Xin, Dai Hongxia, Dong Lu-an, and Wang Xinyue. 2021. Multi-view ensemble learning method for microblog sentiment classification. Expert Systems with Applications 166 (2021), 113987.Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Zhou Xinjie, Wan Xiaojun, and Xiao Jianguo. 2016. Attention-based LSTM network for cross-lingual sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 247256.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Exploring Multi-lingual, Multi-task, and Adversarial Learning for Low-resource Sentiment Analysis

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Asian and Low-Resource Language Information Processing
          ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 21, Issue 5
          September 2022
          486 pages
          ISSN:2375-4699
          EISSN:2375-4702
          DOI:10.1145/3533669
          Issue’s Table of Contents

          ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 23 September 2022
          • Online AM: 22 August 2022
          • Accepted: 30 January 2022
          • Revised: 20 December 2021
          • Received: 9 February 2021
          Published in tallip Volume 21, Issue 5

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        View Full Text

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!