skip to main content
research-article

Clustering-based Sequence to Sequence Model for Generative Question Answering in a Low-resource Language

Published:27 December 2022Publication History
Skip Abstract Section

Abstract

Despite the impressive success of sequence to sequence models for generative question answering, they need a vast amount of question-answer pairs during training, which is hard and expensive to obtain, especially for low-resource languages. In this article, we present a framework that exploits the semantic clusters among the question-answer pairs to compensate for the lack of enough training data. In the training phase, the question-answer pairs are clustered, and a cluster predictor is trained to identify the cluster each question belongs to. Then, a sequence to sequence model is trained, where there is a different generator for each cluster in the decoder component. During the test phase, the cluster of the input question is first identified using the trained cluster predictor, and the appropriate decoder is exploited. Our experiments on a Persian religious dataset show that the proposed method outperforms the standard sequence to sequence model by a large margin in terms of ROUGE and BLEU scores. This is traced back to the lower number of words in each cluster, leading to a reduction in the number of effective parameters each generator needs to learn, which help the model learn from fewer training data with less overfitting.

REFERENCES

  1. [1] Abdi Asad, Hasan Shafaatunnur, Arshi Mohammad, Shamsuddin Siti Mariyam, and Idris Norisma. 2020. A question answering system in Hadith using linguistic knowledge. Comput. Speech Lang. 60 (2020), 101023.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Amirkhani Hossein, Jafari Mohammad Azari, Amirak Azadeh, Pourjafari Zohreh, Jahromi Soroush Faridan, and Kouhkan Zeinab. 2020. FarsTail: A Persian natural language inference dataset. arXiv preprint arXiv:2009.08820 (2020).Google ScholarGoogle Scholar
  3. [3] Baranova-Bolotova Valeriia. 2020. Multi-document answer generation for non-factoid questions. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 24772477.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Bi Bin, Wu Chen, Yan Ming, Wang Wei, Xia Jiangnan, and Li Chenliang. 2019. Incorporating external knowledge into machine reading for generative question answering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 25212530.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Blei David M., Ng Andrew Y., and Jordan Michael I.. 2003. Latent Dirichlet allocation. J. Mach. Learn. Res. 3, Jan. (2003), 9931022.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Boyd Adriane. 2018. Using Wikipedia edits in low resource grammatical error correction. In Proceedings of the EMNLP Workshop W-NUT: The 4th Workshop on Noisy User-generated Text. 7984.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Cho Kyunghyun, Merriënboer Bart Van, Gulcehre Caglar, Bahdanau Dzmitry, Bougares Fethi, Schwenk Holger, and Bengio Yoshua. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078 (2014).Google ScholarGoogle Scholar
  8. [8] Dabre Raj, Fujita Atsushi, and Chu Chenhui. 2019. Exploiting multilingualism through multistage fine-tuning for low-resource neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 14101416.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Daniel Jeanne E., Brink Willie, Eloff Ryan, and Copley Charles. 2019. Towards automating healthcare question answering in a noisy multilingual low-resource setting. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 948953.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Das Rajarshi, Zaheer Manzil, Reddy Siva, and McCallum Andrew. 2017. Question answering on knowledge bases and text using universal schema and memory networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 358365.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Deepak Padmanabhan. 2016. MixKMeans: Clustering question-answer archives. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 15761585.Google ScholarGoogle Scholar
  12. [12] Devlin Jacob, Chang Ming-Wei, Lee Kenton, and Toutanova Kristina. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 41714186.Google ScholarGoogle Scholar
  13. [13] Dhingra Bhuwan, Danish Danish, and Rajagopal Dheeraj. 2018. Simple and effective semi-supervised question answering. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 582587.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Dong Xin Luna and Melo Gerard de. 2019. A robust self-learning framework for cross-lingual text classification. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 63076311.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Fajcik Martin, Josef Jon, and Pavel Smrz. 2021. Rethinking the objectives of extractive question answering. In Proceedings of the 3rd Workshop on Machine Reading for Question Answering.Google ScholarGoogle Scholar
  16. [16] Fu Yao and Feng Yansong. 2018. Natural answer generation with heterogeneous memory. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 185195.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Gupta Deepak, Ekbal Asif, and Bhattacharyya Pushpak. 2019. A deep neural network framework for English Hindi question answering. ACM Trans. Asian Low-resour. Lang. Inf. Process. 19, 2 (2019), 122.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] He Shizhu, Liu Cao, Liu Kang, and Zhao Jun. 2017. Generating natural answers by incorporating copying and retrieving mechanisms in sequence-to-sequence learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 199208.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Huang Yanzhou and Zhong Tao. 2018. Multitask learning for neural generative question answering. Mach. Vis. Applic. 29, 6 (2018), 10091017.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. [20] Karpagam K. and Saradha A.. 2019. A framework for intelligent question answering system using semantic context-specific document clustering and WordNet. Sādhanā 44, 3 (2019), 110.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Khalil Talaat, Kiełczewski Kornel, Chouliaras Georgios Christos, Keldibek Amina, and Versteegh Maarten. 2019. Cross-lingual intent classification in a low resource industrial setting. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 64206425.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Lang Qi, Liu Xiaodong, and Deng Yingjie. 2021. Multi-level retrieval with semantic Axiomatic Fuzzy Set clustering for question answering. Appl. Soft Comput. 111 (2021), 107858.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Lee Kyungjae, Park Sunghyun, Han Hojae, Yeo Jinyoung, Hwang Seung-won, and Lee Juho. 2019. Learning with limited data for multilingual reading comprehension. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 28332843.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Lewis Mike and Fan Angela. 2018. Generative question answering: Learning to answer the whole question. In Proceedings of the International Conference on Learning Representations.Google ScholarGoogle Scholar
  25. [25] Lewis Patrick, Denoyer Ludovic, and Riedel Sebastian. 2019. Unsupervised question answering by cloze translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 48964910.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Lin Chin-Yew. 2004. Rouge: a package for automatic evaluation of summaries. In ACL Workshop on Text Summarization Branches Out, Association for Computational Linguistics.Google ScholarGoogle Scholar
  27. [27] Liu Cao, He Shizhu, Liu Kang, and Zhao Jun. 2018. Curriculum learning for natural answer generation. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence Organization, 42234229. DOI:Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Louvan Samuel and Magnini Bernardo. 2018. Exploring named entity recognition as an auxiliary task for slot filling in conversational language understanding. In Proceedings of the EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI. 7480.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Maaten Laurens van der and Hinton Geoffrey. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, Nov. (2008), 25792605.Google ScholarGoogle Scholar
  30. [30] Mishra Amit and Jain Sanjay Kumar. 2016. A survey on question answering systems with classification. J. King Saud Univ.-Comput. Inf. Sci. 28, 3 (2016), 345361.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Momtazi Saeedeh and Klakow Dietrich. 2009. A word clustering approach for language model-based sentence retrieval in question answering systems. In Proceedings of the 18th ACM Conference on Information and Knowledge Management. 19111914.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Nakatsuji Makoto and Okui Sohei. 2020. Conclusion-supplement answer generation for non-factoid questions. In Proceedings of the AAAI Conference on Artificial Intelligence. 85208527.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Otegi Arantxa, Agirre Aitor, Campos Jon Ander, Soroa Aitor, and Agirre Eneko. 2020. Conversational question answering in low resource scenarios: A dataset and case study for Basque. In Proceedings of the 12th Language Resources and Evaluation Conference. 436442.Google ScholarGoogle Scholar
  34. [34] Papineni Kishore, Roukos Salim, Ward Todd, and Zhu Wei-Jing. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 311318.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Parida Shantipriya and Motlicek Petr. 2019. Abstract text summarization: A low resource challenge. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 59966000.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Perera Rivindu. 2012. IPedagogy: Question answering system based on web information clustering. In Proceedings of the IEEE 4th International Conference on Technology for Education. IEEE, 245246.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Rajpurkar Pranav, Jia Robin, and Liang Percy. 2018. Know what you don’t know: Unanswerable questions for SQuAD. (2018). arxiv:cs.CL/1806.03822.Google ScholarGoogle Scholar
  38. [38] Rajpurkar Pranav, Zhang Jian, Lopyrev Konstantin, and Liang Percy. 2016. SQuAD: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016).Google ScholarGoogle Scholar
  39. [39] Sondhi Parikshit, Raj Purushottam, Kumar V. Vinod, and Mittal Ankush. 2007. Question processing and clustering in INDOC: A biomedical question answering system. EURASIP J. Bioinf. Syst. Biol. (2007), 17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Sutskever Ilya, Vinyals Oriol, and Le Quoc V.. 2014. Sequence to sequence learning with neural networks. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 31043112.Google ScholarGoogle Scholar
  41. [41] Tan Chuanqi, Wei Furu, Yang Nan, Du Bowen, Lv Weifeng, and Zhou Ming. 2018. S-Net: From answer extraction to answer synthesis for machine reading comprehension. In Proceedings of the AAAI Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Łukasz, and Polosukhin Illia. 2017. Attention is all you need. In Proceedings of the International Conference on Advances in Neural Information Processing Systems. 59986008.Google ScholarGoogle Scholar
  43. [43] Wang Sinong, Khabsa Madian, and Ma Hao. 2020. To pretrain or not to pretrain: Examining the benefits of pretraining on resource rich tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 22092213.Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Wang Shuohang, Zhou Luowei, Gan Zhe, Chen Yen-Chun, Fang Yuwei, Sun Siqi, Cheng Yu, and Liu Jingjing. 2021. Cluster-former: Clustering-based sparse transformer for question answering. In Findings of the Association for Computational Linguistics (ACL-IJCNLP’21). Online Event, 39583968.Google ScholarGoogle Scholar
  45. [45] Wei Mengxi and Zhang Yang. 2019. Natural answer generation with attention over instances. IEEE Access 7 (2019), 6100861017.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Wu Lijun, Zhu Jinhua, He Di, Gao Fei, Tao Qin, Lai Jianhuang, and Liu Tie-Yan. 2019. Machine translation with weakly paired documents. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 43664375.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Xu Wei, Liu Xin, and Gong Yihong. 2003. Document clustering based on non-negative matrix factorization. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Informaion Retrieval. 267273.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Yang Jingying, Fu Xianghua, Wang Shuxin, and Xie Wenhao. 2020. Lexicon-enhanced transformer with pointing for domains specific generative question answering. In Proceedings of the International Conference on Algorithms and Architectures for Parallel Processing. Springer, 340354.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Yin Jun, Jiang Xin, Lu Zhengdong, Shang Lifeng, Li Hang, and Li Xiaoming. 2016. Neural generative question answering. In Proceedings of the International Joint Conference on Artificial Intelligence.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Yoon Seunghyun, Dernoncourt Franck, Kim Doo Soon, Bui Trung, and Jung Kyomin. 2019. A compare-aggregate model with latent clustering for answer selection. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 20932096.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Yoon Seunghyun, Shin Joongbo, and Jung Kyomin. 2017. Learning to rank question-answer pairs using hierarchical recurrent encoder with latent topic clustering. arXiv preprint arXiv:1710.03430 (2017).Google ScholarGoogle Scholar
  52. [52] Zhang Meishan, Zhang Yue, and Fu Guohong. 2019. Cross-lingual dependency parsing using code-mixed TreeBank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 9961005.Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Zuin Gianlucca, Chaimowicz Luiz, and Veloso Adriano. 2018. Learning transferable features for open-domain question answering. In Proceedings of the International Joint Conference on Neural Networks (IJCNN’18). IEEE, 18.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Clustering-based Sequence to Sequence Model for Generative Question Answering in a Low-resource Language

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 2
      February 2023
      624 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3572719
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 27 December 2022
      • Online AM: 15 September 2022
      • Accepted: 29 August 2022
      • Revised: 24 July 2022
      • Received: 21 March 2021
      Published in tallip Volume 22, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed
    • Article Metrics

      • Downloads (Last 12 months)134
      • Downloads (Last 6 weeks)4

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!