skip to main content
research-article

A Comprehensive Guideline for Bengali Sentiment Annotation

Published:31 October 2021Publication History
Skip Abstract Section

Abstract

Sentiment Analysis (SA) is a Natural Language Processing (NLP) and an Information Extraction (IE) task that primarily aims to obtain the writer’s feelings expressed in positive or negative by analyzing a large number of documents. SA is also widely studied in the fields of data mining, web mining, text mining, and information retrieval. The fundamental task in sentiment analysis is to classify the polarity of a given content as Positive, Negative, or Neutral. Although extensive research has been conducted in this area of computational linguistics, most of the research work has been carried out in the context of English language. However, Bengali sentiment expression has varying degree of sentiment labels, which can be plausibly distinct from English language. Therefore, sentiment assessment of Bengali language is undeniably important to be developed and executed properly. In sentiment analysis, the prediction potential of an automatic modeling is completely dependent on the quality of dataset annotation. Bengali sentiment annotation is a challenging task due to diversified structures (syntax) of the language and its different degrees of innate sentiments (i.e., weakly and strongly positive/negative sentiments). Thus, in this article, we propose a novel and precise guideline for the researchers, linguistic experts, and referees to annotate Bengali sentences immaculately with a view to building effective datasets for automatic sentiment prediction efficiently.

REFERENCES

  1. [1] Alam Firoj, Sajjad Hassan, Imran Muhammad, and Ofli Ferda. 2020. Standardizing and benchmarking crisis-related social media datasets for humanitarian information processing. arXiv, abs/2004.06774 (2020), 117.Google ScholarGoogle Scholar
  2. [2] Alam Firoj, Shaar Shaden, Nikolov Alex, Mubarak Hamdy, Da San Martino Giovanni, Abdelali Ahmed, Dalvi Fahim, Durrani Nadir, Sajjad Hassan, Darwish Kareem, and Nakov Preslav. 2020. Fighting the COVID-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policymakers, and the society. arXiv, abs/2005.00033 (2020), 139.Google ScholarGoogle Scholar
  3. [3] Aman Saima and Szpakowicz Stan. 2007. Identifying expressions of emotion in text. In Proceedings of the 10th International Conference onText, Speech and Dialogue. Springer-Verlag, Berlin, 196205. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Anik Md. Saidul Hoque, Islam Md. Adnanul, and AlIslam A. B. M. Alim. 2018. An approach towards multilingual translation bysemantic-based verb identification and root word analysis. In Proceedings of the 2018 5th International Conference on Networking, Systems and Security. IEEE, 19. DOI: https://doi.org/10.1109/NSysS.2018.8631383Google ScholarGoogle Scholar
  5. [5] Bakshi Rushlene Kaur, Kaur Navneet, Kaur Ravneet, and Kaur Gurpreet. 2016. Opinion mining and sentiment analysis. In Proceedings of the 2016 3rd International Conference on Computing for Sustainable Global Development. IEEE, 452455.Google ScholarGoogle Scholar
  6. [6] Banik Nayan, Chakraborty Shima, Seddiqui Hanif, Azim Muhammad, and Rahman Md. 2019. Survey on text-based sentiment analysis of bengali language. In Proceedings of the 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology. IEEE, 16.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Barnes Jeremy, Klinger Roman, and Walde Sabine Schulte im. 2017. Assessing state-of-the-art sentiment models on state-of-the-art sentiment datasets. In Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. Association for Computational Linguistics, 212.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Bosco Cristina, Patti Viviana, and Bolioli Andrea. 2015. Developing corpora for sentiment analysis: The Case of Irony and Senti-TUT. In Proceedings of the 24th International Conference on Artificial Intelligence. AAAI Press, 41584162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Chowdhury S. and Chowdhury W.. 2014. Performing sentiment analysis in Bangla microblog posts. In Proceedings of the 2014 International Conference on Informatics, Electronics Vision. IEEE, 16.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Devlin Jacob, Chang Ming-Wei, Lee Kenton, and KristinaToutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 41714186. DOI: https://doi.org/10.18653/v1/N19-1423Google ScholarGoogle Scholar
  11. [11] Castilho Richard Eckart de, Mújdricza-Maydt Éva, MuhieYimam Seid, Hartmann Silvana, Gurevych Iryna, Frank Anette, and Biemann Chris. 2016. A web-based tool for the integrated annotation of semantic and syntactic structures. In Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities. The COLING 2016 Organizing Committee, 7684.Google ScholarGoogle Scholar
  12. [12] Emon Ismail Siddiqi, Ahmed Sabiha Sunjida, Milu Sharmin Akter, and Mahtab S. S.. 2019. Sentiment analysis of bengali online reviews written with english letter using machine learning approaches. In Proceedings of the 6th International Conference on Networking, Systems and Security. Association for Computing Machinery, 109115. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Esuli Andrea and Sebastiani Fabrizio. 2006. SENTIWORDNET: A publicly available lexical resource for opinion mining. In Proceedings of the 5th International Conference on Language Resources and Evaluation. European Language Resources Association.Google ScholarGoogle Scholar
  14. [14] Francisca Judith, Mia Md Mamun, and Rahman S. M. Monzurur. 2011. Adapting rule based machine translation from english to bangla. Indian Journal of Computer Science and Engineering 2, 3 (2011), 334342.Google ScholarGoogle Scholar
  15. [15] Rexach Javier Gutiérrez. 1998. Rhetorical questions, relevance and scales. Revista Alicantina De Estudios Ingleses, 11, 11 (1998), 139155.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Hasan H. M. M. and Islam M. A.. 2020. Emotion recognition from bengali speech using RNN modulation-based categorization. In Proceedings of the 2020 3rd International Conference on Smart Systems and Inventive Technology. IEEE, 11311136.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Hendrycks Dan and Gimpel Kevin. 2016. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. arXiv, abs/1606.08415 (2016), 17.Google ScholarGoogle Scholar
  18. [18] Hu Minqing and Liu Bing. 2004. Mining and summarizing customer reviews. InProceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 168177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Hu Minqing and Liu Bing. 2004. Mining and Summarizing Customer Reviews. InProceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 168177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Islam Md. Adnanul and Islam A. B. M. Alim Al. 2016. Polygot: Going beyond database driven and syntax-based translation. Association for Computing Machinery, New York, NY, Article 28, 4 pages.Google ScholarGoogle Scholar
  21. [21] Islam Md. Adnanul, Islam A. B. M. Alim Al, and Anik Md. Saidul Hoque. 2021. Towards achieving a delicate blending between rule-based translator and neural machine translator. Neural Computing and Applications 33, 18 (2021), 127.Google ScholarGoogle Scholar
  22. [22] Kaur Amandeep and Gupta Vishal. 2013. A survey on sentiment analysis and opinion mining techniques. Journal of Emerging Technologies in Web Intelligence 5, 4 (2013), 367371.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Khan Euna Mehnaz, Mukta Md Saddam Hossain, Ali Mohammed Eunus, and Mahmud Jalal. 2020. Predicting users’ movie preference and rating behavior from personality and values. ACM Transactions on Interactive Intelligent Systems 10, 3 (2020), 125. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Kiritchenko Svetlana and Mohammad Saif M.. 2016. Capturing reliable fine-grained sentiment associations by crowdsourcing and best–worst scaling. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 811817.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Liu Bing. 2012. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies 5, 1 (2012), 1167.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Liu Bing and Zhang Lei. 2012. A Survey of opinion mining and sentiment analysis. Mining Text Data. Springer, 415463.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Liu Yinhan, Ott Myle, Goyal Naman, Du Jingfei, Joshi Mandar, Chen Danqi, Levy Omer, Lewis Mike, Zettlemoyer Luke, and Stoyanov Veselin. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv, abs/1907.11692 (2019), 1–13.Google ScholarGoogle Scholar
  28. [28] Loshchilov Ilya and Hutter Frank. 2017. Fixing weight decay regularization in adam. arXiv, abs/1711.05101 (2017), 1–14.Google ScholarGoogle Scholar
  29. [29] Maas Andrew L., Daly Raymond E., Pham Peter T., Huang Dan, Ng Andrew Y., and Potts Christopher. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 142150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Mæhlum Petter, Barnes Jeremy Claude, Øvrelid Lilja,and Velldal Erik. 2019. Annotating evaluative sentences for sentiment analysis: A dataset for Norwegian. In Proceedings of the 22nd Nordic Conference on Computational Linguistics Linköping Electronic Conference. Linköping University Electronic Press, 121130.Google ScholarGoogle Scholar
  31. [31] Manning Christopher, Surdeanu Mihai, Bauer John, Finkel Jenny, Bethard Steven, and Closky David Mc. 2014. The stanford core NLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 5560.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] McHugh Mary L.. 2012. Interrater reliability: The kappa statistic. Biochemia Medica 22, 3 (2012), 276282.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Mohammad Saif. 2016. A practical guide to sentiment annotation: Challenges and solutions. In Proceedings of the7th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. Association for Computational Linguistics, 174179.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Mohammad Saif M. and Turney Peter D.. 2013. Crowdsourcing a word-emotion association lexicon. Computational Intelligence 29, 3 (2013), 436465.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Mohammad Saif M., Zhu Xiaodan, Kiritchenko Svetlana, and Martin Joel. 2015. Sentiment, emotion, purpose, and style in electoral tweets. Information Processing and Management 51, 4 (July 2015), 480499. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Mozetic Igor, Grcar Miha, and Smailovic Jasmina. 2016. Multilingual twitter sentiment classification: The role of human annotators. PLOS ONE 11, 5 (2016), 1–26.Google ScholarGoogle Scholar
  37. [37] Mukta Md Saddam Hossain, Ali Mohammed Eunus, and Mahmud Jalal. 2016. Identifying and validating personality traits-based homophilies for an egocentric network. Social Network Analysis and Mining 6, 1 (2016), 116.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Nadeau David and Sekine Satoshi. 2007. A survey of named entity recognition and classification. Lingvisticae Investigationes 30, 1 (2007), 326.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Pak Alexander and Paroubek Patrick. 2010. Twitter as a corpus for sentiment analysis and opinion mining. In Proceedings of the 7th International Conference on Language Resources and Evaluation. European Language Resources Association.Google ScholarGoogle Scholar
  40. [40] Pires Telmo, Schlinger Eva, and Garrette Dan. 2019. How multilingual is Multilingual BERT? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Rosenthal Sara, Farra Noura, and Nakov Preslav. 2017. SemEval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation. Association for Computational Linguistics, 502518.Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Salinca Andreea. 2015. Business reviews classification using sentiment analysis. In Proceedings of the 2015 17th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing. IEEE, 247250. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Sarker Sagor. 2020. BanglaBERT: Bengali mask language model for bengali language understading. Retrieved 26 April, 2021 from https://github.com/sagorbrur/bangla-bert.Google ScholarGoogle Scholar
  44. [44] Saura Jose Ramon, Palos-Sanchez Pedro, and Grilo Antonio. 2019. Detecting indicators for startup business success: Sentiment analysis using text data mining. Sustainability 11, 3 (2019), 917.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Shen Chenlin, Sun Changlong, Wang Jingjing, Kang Yangyang, Li Shoushan, Liu Xiaozhong, Si Luo, Zhang Min, and Zhou Guodong. 2018. Sentiment classification towards question-answering with hierarchical matching network. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 36543663.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Speriosu Michael, Sudan Nikita, Upadhyay Sid, and Baldridge Jason. 2011. Twitter polarity classification with label propagation over lexical links and the follower graph. In Proceedings of the 1st Workshop on Unsupervised Learning in NLP. Association for Computational Linguistics, 5363. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] Stone Philip J. and Hunt Earl B.. 1963. A computer approach to content analysis: Studies using the general inquirer system. In Proceedings of the May 21–23, 1963, Spring Joint Computer Conference. Association for Computing Machinery, 241256. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Tang Duyu, Wei Furu, Qin Bing, Zhou Ming, and Liu Ting. 2014. Building large-scale twitter-specific sentiment lexicon: A representation learning approach. In Proceedings of the COLING 2014, 25th International Conference on Computational Linguistics. Dublin City University and Association for Computational Linguistics, 172182.Google ScholarGoogle Scholar
  49. [49] Toprak Cigdem, Jakob Niklas, and Gurevych Iryna. 2010. Sentence and expression level annotation of opinions in user-generated discourse. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 575584. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Tuhin R. A., Paul B. K., Nawrine F., Akter M., and Das A. K.. 2019. An automated system of sentiment analysis from Bangla text using supervised learning techniques. In Proceedings of the 2019 IEEE 4th International Conference on Computerand Communication Systems. IEEE, 360364.Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Kauter Marjan Van de, Desmet Bart, and Hoste Véronique. 2015. The good, the bad and the implicit: A comprehensive approach to annotating explicit and implicit sentiment. Language Resources and Evaluation 49, 3 (2015), 685720. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. [52] Vinodhini G. and Chandrasekaran R. M.. 2012. Sentiment analysis and opinion mining: A survey. International Journal of Advanced Research in Computer Science and Software Engineering 2, 6 (2012), 282292.Google ScholarGoogle Scholar
  53. [53] Wang Xinyu, Zhang Chunhong, Ji Yang, Sun Li, Wu Leijia, and Bao Zhana. 2013. A depression detection model based on sentiment analysis in micro-blog social network. In Proceedings of the Pacific-Asia Conference on Knowledge Discovery and Data Mining: Trends and Applications in Knowledge Discovery and Data Mining, Vol. 7867. Lecture Notes in Computer Science, Springer, 201213. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. [54] Wilson Theresa, Wiebe Janyce, and Hoffmann Paul. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 347354. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. [55] Zapf Antonia, Castell Stefanie, Morawietz Lars, and Karch André. 2016. Measuring inter-rater reliability for nominal data–which coefficients and confidence intervals are appropriate?BMC Medical Research Methodology 16, 1 (2016), 110.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. A Comprehensive Guideline for Bengali Sentiment Annotation

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Asian and Low-Resource Language Information Processing
        ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 21, Issue 2
        March 2022
        413 pages
        ISSN:2375-4699
        EISSN:2375-4702
        DOI:10.1145/3494070
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 31 October 2021
        • Accepted: 1 July 2021
        • Revised: 1 May 2021
        • Received: 1 November 2020
        Published in tallip Volume 21, Issue 2

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!