skip to main content
research-article

Generating Factoid Questions with Question Type Enhanced Representation and Attention-based Copy Mechanism

Authors Info & Claims
Published:28 January 2022Publication History
Skip Abstract Section

Abstract

Question generation over knowledge bases is an important research topic. How to deal with rare and low-frequency words in traditional generation models is a key challenge for question generation. Although the copy mechanism provides significant performance improvements, the original copy mechanism weakens the focus on aspect generation in the overall representations. In this article, we present a novel method to improve question generation with a question type enhanced representation and attention-based copy mechanism. The proposed method exploits the advantages of the generate mode in the copy mechanism and replaces objects in the factual triples with question types, which attempts to improve the output quality in the generate mode and effectively generate questions with proper interrogative words. We evaluate the proposed method on two standard benchmark datasets. The experimental results demonstrate that our proposed method can produce higher-quality questions than these of the Encoder-Decoder-based and CopyNet-based methods.

REFERENCES

  1. Abadi Martín, Barham Paul, Chen Jianmin, Chen Zhifeng, Davis Andy, Dean Jeffrey, Devin Matthieu, Ghemawat Sanjay, Irving Geoffrey, Isard Michael, Kudlur Manjunath, Levenberg Josh, Monga Rajat, Moore Sherry, Murray Derek Gordon, Steiner Benoit, Tucker Paul A., Vasudevan Vijay, Warden Pete, Wicke Martin, Yu Yuan, and Zheng Xiaoqiang. 2016. TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of 12th USENIX Symposium on OSDI. 265283. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bahdanau Dzmitry, Cho Kyunghyun, and Bengio Yoshua. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations (ICLR’15) 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings, Bengio Yoshua and LeCun Yann (Eds.). http://arxiv.org/abs/1409.0473.Google ScholarGoogle Scholar
  3. Bird Steven, Klein Ewan, and Loper Edward. 2009. Natural Language Processing with Python. O’Reilly. http://www.oreilly.de/catalog/9780596516499/index.html.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bollacker Kurt D., Evans Colin, Paritosh Praveen, Sturge Tim, and Taylor Jamie. 2008. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge. In Proceedings of the ACM SIGMOD. 12471250. DOI: DOI: https://doi.org/10.1145/1376616.1376746Google ScholarGoogle ScholarCross RefCross Ref
  5. Bordes Antoine, Usunier Nicolas, Chopra Sumit, and Weston Jason. 2015. Large-scale Simple Question Answering with Memory Networks. CoRR abs/1506.02075 (2015). arxiv:1506.02075 http://arxiv.org/abs/1506.02075.Google ScholarGoogle Scholar
  6. Chen Wei, Aist Gregory, and Mostow Jack. 2009. Generating Questions Automatically from Informational Text. In Proceedings of the 2nd Workshop on Question Generation, 1724.Google ScholarGoogle Scholar
  7. Cho Kyunghyun, Merrienboer Bart van, Gulcehre Caglar, Bahdanau Dzmitry, Bougares Fethi, Schwenk Holger, and Bengio Yoshua. 2014. Learning Phrase Representations Using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (Proceedings of the EMNLP). 17241734. DOI: DOI: https://doi.org/10.3115/v1/D14-1179Google ScholarGoogle ScholarCross RefCross Ref
  8. Curto Sérgio, Mendes Ana Cristina, and Coheur Luísa. 2012. Question Generation Based on Lexico-Syntactic Patterns Learned from the Web. Dialogue and Discourse 3, 2 (2012), 147175. http://dad.uni-bielefeld.de/index.php/dad/article/view/1469.Google ScholarGoogle ScholarCross RefCross Ref
  9. Denkowski Michael J. and Lavie Alon. 2014. Meteor Universal: Language Specific Translation Evaluation for Any Target Language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, [email protected] 2014, June 26–27, 2014, Baltimore, Maryland, USA. 376380. https://www.aclweb.org/anthology/W14-3348/.Google ScholarGoogle ScholarCross RefCross Ref
  10. Dong Li, Mallinson Jonathan, Reddy Siva, and Lapata Mirella. 2017. Learning to Paraphrase for Question Answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), Copenhagen, Denmark, September 9–11, 2017. 875886. https://www.aclweb.org/anthology/D17-1091/.Google ScholarGoogle ScholarCross RefCross Ref
  11. Duma Daniel and Klein Ewan. 2013. Generating Natural Language from Linked Data: Unsupervised Template Extraction. In Proceedings of the 10th International Conference on Computational Semantics (IWCS’13) – Long Papers. 8394. http://aclweb.org/anthology/W13-0108.Google ScholarGoogle Scholar
  12. Elsahar Hady, Gravier Christophe, and Laforest Frédérique. 2018. Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), New Orleans, Louisiana, USA, June 1–6, 2018, Volume 1 (Long Papers). 218228. https://www.aclweb.org/anthology/N18-1020/.Google ScholarGoogle ScholarCross RefCross Ref
  13. Gu Jiatao, Lu Zhengdong, Li Hang, and Li Victor O. K.. 2016. Incorporating Copying Mechanism in Sequence-to-Sequence Learning. In Proceedings the ACL (Volume 1: Long Papers). 16311640. https://doi.org/10.18653/v1/P16-1154Google ScholarGoogle ScholarCross RefCross Ref
  14. Guo Daya, Sun Yibo, Tang Duyu, Duan Nan, Yin Jian, Chi Hong, Cao James, Chen Peng, and Zhou Ming. 2018. Question Generation from SQL Queries Improves Neural Semantic Parsing. In Proceedings of the EMNLP. 15971607. http://aclweb.org/anthology/D18-1188.Google ScholarGoogle ScholarCross RefCross Ref
  15. He Shizhu, Liu Cao, Liu Kang, and Zhao Jun. 2017. Generating Natural Answers by Incorporating Copying and Retrieving Mechanisms in Sequence-to-Sequence Learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, Barzilay Regina and Kan Min-Yen (Eds.). Association for Computational Linguistics, 199208. DOI: DOI: https://doi.org/10.18653/v1/P17-1019Google ScholarGoogle ScholarCross RefCross Ref
  16. Heilman Michael and Smith Noah A.. 2010. Good Question! Statistical Ranking for Question Generation. In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 2–4, 2010, Los Angeles, California, USA. 609617. https://www.aclweb.org/anthology/N10-1086/. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Hochreiter Sepp and Schmidhuber Jürgen. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 17351780. DOI: DOI: https://doi.org/10.1162/neco.1997.9.8.1735 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Hu Zhiting, Yang Zichao, Salakhutdinov Ruslan R., Qin Lianhui, Liang Xiaodan, Dong Haoye, and Xing Eric P.. 2018. Deep Generative Models with Learnable Knowledge Constraints. In Proceedings of the NeurIPS. 1052210533. http://papers.nips.cc/paper/8250-deep-generative-models-with-learnable-knowledge-constraints. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Kalady Saidalavi, Elikkottil Ajeesh, and Das Rajarshi. 2010. Natural Language Question Generation Using Syntax and Keywords. In Proceedings of QG2010: The Third Workshop on Question Generation, Vol. 2.Google ScholarGoogle Scholar
  20. Khapra Mitesh M., Raghu Dinesh, Joshi Sachindra, and Reddy Sathish. 2017. Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’17), Valencia, Spain, April 3–7, 2017, Volume 1: Long Papers. 376385. https://www.aclweb.org/anthology/E17-1036/.Google ScholarGoogle Scholar
  21. Kingma Diederik P. and Ba Jimmy. 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014). arxiv:1412.6980 http://arxiv.org/abs/1412.6980.Google ScholarGoogle Scholar
  22. Koncel-Kedziorski Rik, Bekal Dhanush, Luan Yi, Lapata Mirella, and Hajishirzi Hannaneh. 2019. Text Generation from Knowledge Graphs with Graph Transformers. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers). 22842293. https://www.aclweb.org/anthology/N19-1238/.Google ScholarGoogle Scholar
  23. Lehmann Jens, Isele Robert, Jakob Max, Jentzsch Anja, Kontokostas Dimitris, Mendes Pablo N., Hellmann Sebastian, Morsey Mohamed, Kleef Patrick van, Auer Sören, and Bizer Christian. 2015. DBpedia - A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia. Semantic Web 6, 2 (2015), 167195. DOI: DOI: https://doi.org/10.3233/SW-140134Google ScholarGoogle ScholarCross RefCross Ref
  24. Lin Chin Yew. 2004. ROUGE: A Package for Automatic Evaluation of summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS’04).Google ScholarGoogle Scholar
  25. Liu Bang, Zhao Mingjun, Niu Di, Lai Kunfeng, He Yancheng, Wei Haojie, and Xu Yu. 2019. Learning to Generate Questions by Learning What Not to Generate. In The World Wide Web Conference. Association for Computing Machinery, New York, NY, USA, 11061118. DOI: DOI: https://doi.org/10.1145/3308558.3313737 Google ScholarGoogle ScholarCross RefCross Ref
  26. Liu Cao, He Shizhu, Liu Kang, and Zhao Jun. 2018. Curriculum Learning for Natural Answer Generation. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI’18), July 13–19, 2018, Stockholm, Sweden, Lang Jérôme (Ed.). ijcai.org, 42234229. DOI: DOI: https://doi.org/10.24963/ijcai.2018/587 Google ScholarGoogle ScholarCross RefCross Ref
  27. Liu Cao, Liu Kang, He Shizhu, Nie Zaiqing, and Zhao Jun. 2019. Generating Questions for Knowledge Bases via Incorporating Diversified Contexts and Answer-Aware Loss. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Hong Kong, China, November 3–7, 2019, Inui Kentaro, Jiang Jing, Ng Vincent, and Wan Xiaojun (Eds.). Association for Computational Linguistics, 24312441. DOI: DOI: https://doi.org/10.18653/v1/D19-1247Google ScholarGoogle ScholarCross RefCross Ref
  28. Liu Tianyu, Wei Bingzhen, Chang Baobao, and Sui Zhifang. 2017. Large-Scale Simple Question Generation by Template-Based Seq2seq Learning. In Natural Language Processing and Chinese Computing - 6th CCF International Conference (NLPCC’17), Dalian, China, November 8–12, 2017, Proceedings (Lecture Notes in Computer Science), Vol. 10619. Springer, 7587. DOI: DOI: https://doi.org/10.1007/978-3-319-73618-1_7Google ScholarGoogle Scholar
  29. Luong Thang, Pham Hieu, and Manning Christopher D.. 2015a. Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of the EMNLP. 14121421. http://aclweb.org/anthology/D/D15/D15-1166.pdf.Google ScholarGoogle ScholarCross RefCross Ref
  30. Luong Thang, Sutskever Ilya, Le Quoc V., Vinyals Oriol, and Zaremba Wojciech. 2015b. Addressing the Rare Word Problem in Neural Machine Translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26–31, 2015, Beijing, China, Volume 1: Long Papers. 1119. https://www.aclweb.org/anthology/P15-1002/.Google ScholarGoogle Scholar
  31. McConnell Claire Cooper, Mannem Prashanth, Prasad Rashmi, and Joshi Aravind K.. 2011. A New Approach to Ranking Over-Generated Questions. In AAAI Fall Symposium: Question Generation (AAAI Technical Report), Vol. FS-11-04. AAAI.Google ScholarGoogle Scholar
  32. Mostafazadeh Nasrin, Misra Ishan, Devlin Jacob, Mitchell Margaret, He Xiaodong, and Vanderwende Lucy. 2016. Generating Natural Questions about an Image. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1170/.Google ScholarGoogle ScholarCross RefCross Ref
  33. Papineni Kishore, Roukos Salim, Ward Todd, and Zhu Wei-Jing. 2002. Bleu: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the ACL. http://aclweb.org/anthology/P02-1040. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Peng Yehong, Fang Yizhen, Xie Zhiwen, and Zhou Guangyou. 2019. Topic-enhanced Emotional Conversation Generation with Attention Mechanism. Knowl.-Based Syst. 163 (2019), 429437. DOI: DOI: https://doi.org/10.1016/j.knosys.2018.09.006Google ScholarGoogle ScholarCross RefCross Ref
  35. Pennington Jeffrey, Socher Richard, and Manning Christopher D.. 2014. Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25–29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL. 15321543. https://www.aclweb.org/anthology/D14-1162/.Google ScholarGoogle ScholarCross RefCross Ref
  36. Qiu Jiazuo and Xiong Deyi. 2019. Generating Highly Relevant Questions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Hong Kong, China, November 3–7, 2019. Association for Computational Linguistics, 59825986.Google ScholarGoogle ScholarCross RefCross Ref
  37. Serban Iulian Vlad, García-Durán Alberto, Gülçehre Çaglar, Ahn Sungjin, Chandar Sarath, Courville Aaron C., and Bengio Yoshua. 2016. Generating Factoid Questions with Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1056/.Google ScholarGoogle ScholarCross RefCross Ref
  38. Seyler Dominic, Yahya Mohamed, and Berberich Klaus. 2015. Generating Quiz Questions from Knowledge Graphs. In Proceedings of the 24th International Conference on World Wide Web Companion (WWW’15), Florence, Italy, May 18–22, 2015 - Companion Volume. 113114. DOI: DOI: https://doi.org/10.1145/2740908.2742722 Google ScholarGoogle ScholarCross RefCross Ref
  39. Seyler Dominic, Yahya Mohamed, and Berberich Klaus. 2017. Knowledge Questions from Knowledge Graphs. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR’17), Amsterdam, The Netherlands, October 1–4, 2017. 1118. DOI: DOI: https://doi.org/10.1145/3121050.3121073 Google ScholarGoogle ScholarCross RefCross Ref
  40. Shang Lifeng, Lu Zhengdong, and Li Hang. 2015. Neural Responding Machine for Short-Text Conversation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26–31, 2015, Beijing, China, Volume 1: Long Papers. 15771586. https://www.aclweb.org/anthology/P15-1152/.Google ScholarGoogle Scholar
  41. Song Linfeng, Wang Zhiguo, Hamza Wael, Zhang Yue, and Gildea Daniel. 2018. Leveraging Context Information for Natural Question Generation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1–6, 2018, Volume 2 (Short Papers). 569574. https://www.aclweb.org/anthology/N18-2090/.Google ScholarGoogle ScholarCross RefCross Ref
  42. Sutskever Ilya, Vinyals Oriol, and Le Quoc V.. 2014. Sequence to Sequence Learning with Neural Networks. In Proceedings of the NeurIPS. 31043112. http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, and Polosukhin Illia. 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4–9 December 2017, Long Beach, CA, USA. 59986008. http://papers.nips.cc/paper/7181-attention-is-all-you-need.Google ScholarGoogle Scholar
  44. Wang Hao, Zhang Xiaodong, and Wang Houfeng. 2018. A Neural Question Generation System Based on Knowledge Base. In Proceedings of the NLPCC. 133142. DOI: DOI: https://doi.org/10.1007/978-3-319-99495-6_12Google ScholarGoogle ScholarCross RefCross Ref
  45. Weissenborn Dirk, Wiese Georg, and Seiffe Laura. 2017. FastQA: A Simple and Efficient Neural Architecture for Question Answering. CoRR abs/1703.04816 (2017).Google ScholarGoogle Scholar
  46. Wu Xiuyu, Jiang Nan, and Wu Yunfang. 2020. A Question Type Driven and Copy Loss Enhanced Framework for Answer-Agnostic Neural Question Generation. In Proceedings of the Fourth Workshop on Neural Generation and Translation. Association for Computational Linguistics, 6978. DOI: DOI: https://doi.org/10.18653/v1/2020.ngt-1.8Google ScholarGoogle ScholarCross RefCross Ref
  47. Xie Zhiwen, Zeng Zhao, Zhou Guangyou, and Wang Weijun. 2017. Topic enhanced deep structured semantic models for knowledge base question answering. SCIENCE CHINA Information Sciences 60, 11 (2017), 110103:1–110103:15. DOI: DOI: https://doi.org/10.1007/s11432-017-9136-xGoogle ScholarGoogle ScholarCross RefCross Ref
  48. Zhang Shijie, Qu Lizhen, You Shaodi, Yang Zhenglu, and Zhang Jiawan. 2017. Automatic Generation of Grounded Visual Questions. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19–25, 2017. 42354243. DOI: DOI: https://doi.org/10.24963/ijcai.2017/592 Google ScholarGoogle ScholarCross RefCross Ref
  49. Zhou Guangyou, Fang Yizhen, Peng Yehong, and Lu Jiaheng. 2020. Neural Conversation Generation with Auxiliary Emotional Supervised Models. ACM Trans. Asian Low Resour. Lang. Inf. Process. 19, 2 (2020), 19:1–19:17. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Zhou Guangyou and Huang Jimmy Xiangji. 2017. Modeling and Learning Distributed Word Representation with Metadata for Question Retrieval. IEEE Trans. Knowl. Data Eng. 29, 6 (2017), 12261239. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Zhou Guangyou, Xie Zhiwen, Huang Jimmy Xiangji, and He Tingting. 2016. Bi-Transferring Deep Neural Networks for Domain Adaptation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1031/.Google ScholarGoogle ScholarCross RefCross Ref
  52. Zhou Wenjie, Zhang Minghua, and Wu Yunfang. 2019a. Multi-Task Learning with Language Modeling for Question Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 33923397.Google ScholarGoogle ScholarCross RefCross Ref
  53. Zhou Wenjie, Zhang Minghua, and Wu Yunfang. 2019b. Question-type Driven Question Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 60316036.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Generating Factoid Questions with Question Type Enhanced Representation and Attention-based Copy Mechanism

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 21, Issue 2
      March 2022
      413 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3494070
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 28 January 2022
      • Accepted: 1 July 2021
      • Revised: 1 February 2021
      • Received: 1 March 2020
      Published in tallip Volume 21, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed
    • Article Metrics

      • Downloads (Last 12 months)186
      • Downloads (Last 6 weeks)7

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!