skip to main content
research-article

Abstractive Summarization of Text Document in Malayalam Language: Enhancing Attention Model Using POS Tagging Feature

Authors Info & Claims
Published:23 March 2023Publication History
Skip Editorial Notes Section

Editorial Notes

The authors have requested minor, non-substantive changes to the VoR and, in accordance with ACM policies, a Corrected Version of Record was published on May 18, 2023. For reference purposes, the VoR may still be accessed via the Supplemental Material section on this citation page.

Skip Abstract Section

Abstract

Over the past few years, researchers are showing huge interest in sentiment analysis and summarization of documents. The primary reason being that huge volumes of information are available in textual format, and this data has proven helpful for real-world applications and challenges. The sentiment analysis of a document will help the user comprehend the content’s emotional intent. Abstractive summarization algorithms generate a condensed version of the text, which can then be used to determine the emotion represented in the text using sentiment analysis. Recent research in abstractive summarization concentrates on neural network-based models, rather than conjunctions-based approaches, which might improve the overall efficiency. Neural network models like attention mechanism are tried out to handle complex works with promising results. The proposed work aims to present a novel framework that incorporates the part of speech tagging feature to the word embedding layer, which is then used as the input to the attention mechanism. With POS feature being part of the input layer, this framework is capable of dealing with words containing contextual and morphological information. The relevance of POS tagging here is due to its strong reliance on the language’s syntactic, contextual, and morphological information. The three main elements in the work are pre-processing, POS tagging feature in the embedding phase, and the incorporation of it into the attention mechanism. The word embedding provides the semantic concept about the word, while the POS tags give an idea about how significant the words are in the context of the content, which corresponds to the syntactic information. The proposed work was carried out in Malayalam, one of the prominent Indian languages. A widely used and accepted dataset from the English language was translated to Malayalam for conducting the experiments. The proposed framework gives a ROUGE score of 28, which outperformed the baseline models.

Skip Supplemental Material Section

Supplemental Material

REFERENCES

  1. [1] Ajees A. P. and Idicula Sumam Mary. 2018. A POS tagger for Malayalam using conditional random fields. Int. J. Appl. Eng. Res. 13, 3 (2018).Google ScholarGoogle Scholar
  2. [2] Angeli Gabor, Premkumar Melvin Jose Johnson, and Manning Christopher D.. 2015. Leveraging linguistic structure for open domain information extraction. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 344354. DOI:. arxiv:1603.07252.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014).Google ScholarGoogle Scholar
  4. [4] Lidong Bing, Piji Li, Yi Liao, Wai Lam, Weiwei Guo, and Rebecca J. Passonneau. 2015. Abstractive multi-document summarization via phrase selection and merging. arXiv preprint arXiv:1506.01597 (2015).Google ScholarGoogle Scholar
  5. [5] Celikyilmaz Asli, Bosselut Antoine, He Xiaodong, and Choi Yejin. 2018. Deep communicating agents for abstractive summarization. arXiv preprint arXiv:1803.10357 (2018).Google ScholarGoogle Scholar
  6. [6] Devlin Jacob, Chang Ming-Wei, Lee Kenton, and Toutanova Kristina. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).Google ScholarGoogle Scholar
  7. [7] Deepali K. Gaikwad and C. Namrata Mahender. 2016. A review paper on text summarization. International Journal of Advanced Research in Computer and Communication Engineering 5, 3 (2016), 154–160.Google ScholarGoogle Scholar
  8. [8] Idicula Sumam Mary and David Peter S.. 2007. A morphological processor for malayalam language. South Asia Res. 27, 2 (2007), 173186.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Jaafar Younes and Bouzoubaa Karim. 2018. Towards a new hybrid approach for abstractive summarization. Procedia Comput. Sci. 142 (2018), 286293.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Khan Atif and Salim Naomie. 2014. A review on abstractive summarization methods. J. Theoret. Appl. Inf. Technol. 59, 1 (2014), 6472. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Khan Atif, Salim Naomie, and Kumar Yogan Jaya. 2015. A framework for multi-document abstractive summarization based on semantic role labelling. Appl. Soft Comput. J. 30 (2015), 737747. DOI:Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. [12] Khandelwal Renu. 2020. Attention: Sequence 2 Sequence model with Attention Mechanism. Retrieved from https://towardsdatascience.com/sequence-2-sequence-model-with-attention-mechanism-9e9ca2a613a.Google ScholarGoogle Scholar
  13. [13] Lewis Mike, Liu Yinhan, Goyal Naman, Ghazvininejad Marjan, Mohamed Abdelrahman, Levy Omer, Stoyanov Ves, and Zettlemoyer Luke. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019).Google ScholarGoogle Scholar
  14. [14] Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out. 74–81.Google ScholarGoogle Scholar
  15. [15] Liu Yang. 2019. Fine-tune BERT for extractive summarization. arXiv preprint arXiv:1903.10318 (2019).Google ScholarGoogle Scholar
  16. [16] Luhn Hans Peter. 1958. The automatic creation of literature abstracts. IBM J. Res. Devel. 2, 2 (1958), 159165.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Mikolov Tomas, Chen Kai, Corrado Greg, and Dean Jeffrey. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).Google ScholarGoogle Scholar
  18. [18] Moawad Ibrahim F. and Aref Mostafa. 2012. Semantic graph reduction approach for abstractive text summarization. Proceedings of the International Conference on Computer Engineering and Systems. 132138. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Mohanan Karvanuur P.. 1997. Grammatical relations and clause structure in Malayalam.Google ScholarGoogle Scholar
  20. [20] Nambiar Sindhya K., Idicula Sumam Mary, et al. 2021. Attention based abstractive summarization of Malayalam document. Procedia Comput. Sci. 189 (2021), 250257.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Nambiar Sindhya K., Peter S. David, and Idicula Sumam Mary. 2021. Abstractive summarization of Malayalam document using sequence to sequence model. In Proceedings of the 7th International Conference on Advanced Computing and Communication Systems (ICACCS’21). IEEE, 347352.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Paulus Romain, Xiong Caiming, and Socher Richard. 2018. A deep reinforced model for abstractive summarization. In Proceedings of the 6th International Conference on Learning Representations. 112. arxiv:1705.04304.Google ScholarGoogle Scholar
  23. [23] Sebastian Mary Priya, Kurian K. Sheena, and Kumar G. Santhosh. 2010. A classification of Sandhi rules for suffix separation in Malayalam. In Proceedings of the 38th All India Conference of Dravidian Linguists.Google ScholarGoogle Scholar
  24. [24] See Abigail, Liu Peter J., and Manning Christopher D.. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368 (2017).Google ScholarGoogle Scholar
  25. [25] Strub Florian, Vries Harm De, Mary Jeremie, Piot Bilal, Courvile Aaron, and Pietquin Olivier. 2017. End-to-end optimization of goal-driven and visually grounded dialogue systems. In Proceedings of the IJCAI International Joint Conference on Artificial Intelligence. 27652771. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] George Tsatsaronis, Iraklis Varlamis, and Kjetil Nørvåg. 2010. SemanticRank: ranking keywords and sentences using semantic graphs. In Proceedings of the 23rd International Conference on Computational Linguistics (Coling’10). 1074–1082.Google ScholarGoogle Scholar
  27. [27] Zhang Jingqing, Zhao Yao, Saleh Mohammad, and Liu Peter. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the International Conference on Machine Learning. PMLR, 1132811339.Google ScholarGoogle Scholar
  28. [28] Zhou Qingyu, Yang Nan, Wei Furu, Huang Shaohan, Zhou Ming, and Zhao Tiejun. 2018. Neural document summarization by jointly learning to score and select sentences. arXiv preprint arXiv:1807.02305 (2018).Google ScholarGoogle Scholar

Index Terms

  1. Abstractive Summarization of Text Document in Malayalam Language: Enhancing Attention Model Using POS Tagging Feature

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 2
      February 2023
      624 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3572719
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 23 March 2023
      • Online AM: 10 September 2022
      • Accepted: 29 August 2022
      • Revised: 21 July 2022
      • Received: 27 April 2022
      Published in tallip Volume 22, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
    • Article Metrics

      • Downloads (Last 12 months)153
      • Downloads (Last 6 weeks)22

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!