Abstract
Question generation over knowledge bases is an important research topic. How to deal with rare and low-frequency words in traditional generation models is a key challenge for question generation. Although the copy mechanism provides significant performance improvements, the original copy mechanism weakens the focus on aspect generation in the overall representations. In this article, we present a novel method to improve question generation with a question type enhanced representation and attention-based copy mechanism. The proposed method exploits the advantages of the generate mode in the copy mechanism and replaces objects in the factual triples with question types, which attempts to improve the output quality in the generate mode and effectively generate questions with proper interrogative words. We evaluate the proposed method on two standard benchmark datasets. The experimental results demonstrate that our proposed method can produce higher-quality questions than these of the Encoder-Decoder-based and CopyNet-based methods.
- . 2016. TensorFlow: A System for Large-Scale Machine Learning. In Proceedings of 12th USENIX Symposium on OSDI. 265–283. https://www.usenix.org/conference/osdi16/technical-sessions/presentation/abadi. Google Scholar
Digital Library
- . 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In 3rd International Conference on Learning Representations (ICLR’15) 2015, San Diego, CA, USA, May 7–9, 2015, Conference Track Proceedings, and (Eds.). http://arxiv.org/abs/1409.0473.Google Scholar
- . 2009. Natural Language Processing with Python. O’Reilly. http://www.oreilly.de/catalog/9780596516499/index.html.Google Scholar
Digital Library
- . 2008. Freebase: A Collaboratively Created Graph Database for Structuring Human Knowledge. In Proceedings of the ACM SIGMOD. 1247–1250.
DOI: DOI: https://doi.org/10.1145/1376616.1376746Google ScholarCross Ref
- . 2015. Large-scale Simple Question Answering with Memory Networks. CoRR abs/1506.02075 (2015).
arxiv:1506.02075 http://arxiv.org/abs/1506.02075.Google Scholar - . 2009. Generating Questions Automatically from Informational Text. In Proceedings of the 2nd Workshop on Question Generation, 17–24.Google Scholar
- . 2014. Learning Phrase Representations Using RNN Encoder–Decoder for Statistical Machine Translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (Proceedings of the EMNLP). 1724–1734.
DOI: DOI: https://doi.org/10.3115/v1/D14-1179Google ScholarCross Ref
- . 2012. Question Generation Based on Lexico-Syntactic Patterns Learned from the Web. Dialogue and Discourse 3, 2 (2012), 147–175. http://dad.uni-bielefeld.de/index.php/dad/article/view/1469.Google Scholar
Cross Ref
- . 2014. Meteor Universal: Language Specific Translation Evaluation for Any Target Language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, [email protected] 2014, June 26–27, 2014, Baltimore, Maryland, USA. 376–380. https://www.aclweb.org/anthology/W14-3348/.Google Scholar
Cross Ref
- . 2017. Learning to Paraphrase for Question Answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), Copenhagen, Denmark, September 9–11, 2017. 875–886. https://www.aclweb.org/anthology/D17-1091/.Google Scholar
Cross Ref
- . 2013. Generating Natural Language from Linked Data: Unsupervised Template Extraction. In Proceedings of the 10th International Conference on Computational Semantics (IWCS’13) – Long Papers. 83–94. http://aclweb.org/anthology/W13-0108.Google Scholar
- . 2018. Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), New Orleans, Louisiana, USA, June 1–6, 2018, Volume 1 (Long Papers). 218–228. https://www.aclweb.org/anthology/N18-1020/.Google Scholar
Cross Ref
- . 2016. Incorporating Copying Mechanism in Sequence-to-Sequence Learning. In Proceedings the ACL (Volume 1: Long Papers). 1631–1640. https://doi.org/10.18653/v1/P16-1154Google Scholar
Cross Ref
- . 2018. Question Generation from SQL Queries Improves Neural Semantic Parsing. In Proceedings of the EMNLP. 1597–1607. http://aclweb.org/anthology/D18-1188.Google Scholar
Cross Ref
- . 2017. Generating Natural Answers by Incorporating Copying and Retrieving Mechanisms in Sequence-to-Sequence Learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, and (Eds.). Association for Computational Linguistics, 199–208.
DOI: DOI: https://doi.org/10.18653/v1/P17-1019Google ScholarCross Ref
- . 2010. Good Question! Statistical Ranking for Question Generation. In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 2–4, 2010, Los Angeles, California, USA. 609–617. https://www.aclweb.org/anthology/N10-1086/. Google Scholar
Digital Library
- . 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735–1780.
DOI: DOI: https://doi.org/10.1162/neco.1997.9.8.1735 Google ScholarDigital Library
- . 2018. Deep Generative Models with Learnable Knowledge Constraints. In Proceedings of the NeurIPS. 10522–10533. http://papers.nips.cc/paper/8250-deep-generative-models-with-learnable-knowledge-constraints. Google Scholar
Digital Library
- . 2010. Natural Language Question Generation Using Syntax and Keywords. In Proceedings of QG2010: The Third Workshop on Question Generation, Vol. 2.Google Scholar
- . 2017. Generating Natural Language Question-Answer Pairs from a Knowledge Graph Using a RNN Based Question Generation Model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’17), Valencia, Spain, April 3–7, 2017, Volume 1: Long Papers. 376–385. https://www.aclweb.org/anthology/E17-1036/.Google Scholar
- . 2014. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2014).
arxiv:1412.6980 http://arxiv.org/abs/1412.6980.Google Scholar - . 2019. Text Generation from Knowledge Graphs with Graph Transformers. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2–7, 2019, Volume 1 (Long and Short Papers). 2284–2293. https://www.aclweb.org/anthology/N19-1238/.Google Scholar
- . 2015. DBpedia - A Large-scale, Multilingual Knowledge Base Extracted from Wikipedia. Semantic Web 6, 2 (2015), 167–195.
DOI: DOI: https://doi.org/10.3233/SW-140134Google ScholarCross Ref
- . 2004. ROUGE: A Package for Automatic Evaluation of summaries. In Proceedings of the Workshop on Text Summarization Branches Out (WAS’04).Google Scholar
- . 2019. Learning to Generate Questions by Learning What Not to Generate. In The World Wide Web Conference. Association for Computing Machinery, New York, NY, USA, 1106–1118.
DOI: DOI: https://doi.org/10.1145/3308558.3313737 Google ScholarCross Ref
- . 2018. Curriculum Learning for Natural Answer Generation. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI’18), July 13–19, 2018, Stockholm, Sweden, (Ed.). ijcai.org, 4223–4229.
DOI: DOI: https://doi.org/10.24963/ijcai.2018/587 Google ScholarCross Ref
- . 2019. Generating Questions for Knowledge Bases via Incorporating Diversified Contexts and Answer-Aware Loss. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Hong Kong, China, November 3–7, 2019, , , , and (Eds.). Association for Computational Linguistics, 2431–2441.
DOI: DOI: https://doi.org/10.18653/v1/D19-1247Google ScholarCross Ref
- . 2017. Large-Scale Simple Question Generation by Template-Based Seq2seq Learning. In Natural Language Processing and Chinese Computing - 6th CCF International Conference (NLPCC’17), Dalian, China, November 8–12, 2017, Proceedings (Lecture Notes in Computer Science), Vol. 10619. Springer, 75–87.
DOI: DOI: https://doi.org/10.1007/978-3-319-73618-1_7Google Scholar - . 2015a. Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of the EMNLP. 1412–1421. http://aclweb.org/anthology/D/D15/D15-1166.pdf.Google Scholar
Cross Ref
- . 2015b. Addressing the Rare Word Problem in Neural Machine Translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26–31, 2015, Beijing, China, Volume 1: Long Papers. 11–19. https://www.aclweb.org/anthology/P15-1002/.Google Scholar
- . 2011. A New Approach to Ranking Over-Generated Questions. In AAAI Fall Symposium: Question Generation (AAAI Technical Report), Vol. FS-11-04. AAAI.Google Scholar
- . 2016. Generating Natural Questions about an Image. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1170/.Google Scholar
Cross Ref
- . 2002. Bleu: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the ACL. http://aclweb.org/anthology/P02-1040. Google Scholar
Digital Library
- . 2019. Topic-enhanced Emotional Conversation Generation with Attention Mechanism. Knowl.-Based Syst. 163 (2019), 429–437.
DOI: DOI: https://doi.org/10.1016/j.knosys.2018.09.006Google ScholarCross Ref
- . 2014. Glove: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25–29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL. 1532–1543. https://www.aclweb.org/anthology/D14-1162/.Google Scholar
Cross Ref
- . 2019. Generating Highly Relevant Questions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19), Hong Kong, China, November 3–7, 2019. Association for Computational Linguistics, 5982–5986.Google Scholar
Cross Ref
- . 2016. Generating Factoid Questions with Recurrent Neural Networks: The 30M Factoid Question-Answer Corpus. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1056/.Google Scholar
Cross Ref
- . 2015. Generating Quiz Questions from Knowledge Graphs. In Proceedings of the 24th International Conference on World Wide Web Companion (WWW’15), Florence, Italy, May 18–22, 2015 - Companion Volume. 113–114.
DOI: DOI: https://doi.org/10.1145/2740908.2742722 Google ScholarCross Ref
- . 2017. Knowledge Questions from Knowledge Graphs. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR’17), Amsterdam, The Netherlands, October 1–4, 2017. 11–18.
DOI: DOI: https://doi.org/10.1145/3121050.3121073 Google ScholarCross Ref
- . 2015. Neural Responding Machine for Short-Text Conversation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26–31, 2015, Beijing, China, Volume 1: Long Papers. 1577–1586. https://www.aclweb.org/anthology/P15-1152/.Google Scholar
- . 2018. Leveraging Context Information for Natural Question Generation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana, USA, June 1–6, 2018, Volume 2 (Short Papers). 569–574. https://www.aclweb.org/anthology/N18-2090/.Google Scholar
Cross Ref
- . 2014. Sequence to Sequence Learning with Neural Networks. In Proceedings of the NeurIPS. 3104–3112. http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks. Google Scholar
Digital Library
- . 2017. Attention Is All You Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, 4–9 December 2017, Long Beach, CA, USA. 5998–6008. http://papers.nips.cc/paper/7181-attention-is-all-you-need.Google Scholar
- . 2018. A Neural Question Generation System Based on Knowledge Base. In Proceedings of the NLPCC. 133–142.
DOI: DOI: https://doi.org/10.1007/978-3-319-99495-6_12Google ScholarCross Ref
- . 2017. FastQA: A Simple and Efficient Neural Architecture for Question Answering. CoRR abs/1703.04816 (2017).Google Scholar
- . 2020. A Question Type Driven and Copy Loss Enhanced Framework for Answer-Agnostic Neural Question Generation. In Proceedings of the Fourth Workshop on Neural Generation and Translation. Association for Computational Linguistics, 69–78.
DOI: DOI: https://doi.org/10.18653/v1/2020.ngt-1.8Google ScholarCross Ref
- . 2017. Topic enhanced deep structured semantic models for knowledge base question answering. SCIENCE CHINA Information Sciences 60, 11 (2017), 110103:1–110103:15.
DOI: DOI: https://doi.org/10.1007/s11432-017-9136-xGoogle ScholarCross Ref
- . 2017. Automatic Generation of Grounded Visual Questions. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19–25, 2017. 4235–4243.
DOI: DOI: https://doi.org/10.24963/ijcai.2017/592 Google ScholarCross Ref
- . 2020. Neural Conversation Generation with Auxiliary Emotional Supervised Models. ACM Trans. Asian Low Resour. Lang. Inf. Process. 19, 2 (2020), 19:1–19:17. Google Scholar
Digital Library
- . 2017. Modeling and Learning Distributed Word Representation with Metadata for Question Retrieval. IEEE Trans. Knowl. Data Eng. 29, 6 (2017), 1226–1239. Google Scholar
Digital Library
- . 2016. Bi-Transferring Deep Neural Networks for Domain Adaptation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7–12, 2016, Berlin, Germany, Volume 1: Long Papers. https://www.aclweb.org/anthology/P16-1031/.Google Scholar
Cross Ref
- . 2019a. Multi-Task Learning with Language Modeling for Question Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 3392–3397.Google Scholar
Cross Ref
- . 2019b. Question-type Driven Question Generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 6031–6036.Google Scholar
Cross Ref
Index Terms
Generating Factoid Questions with Question Type Enhanced Representation and Attention-based Copy Mechanism
Recommendations
Asking Questions the Human Way: Scalable Question-Answer Generation from Text Corpus
WWW '20: Proceedings of The Web Conference 2020The ability to ask questions is important in both human and machine intelligence. Learning to ask questions helps knowledge acquisition, improves question-answering and machine reading comprehension tasks, and helps a chatbot to keep the conversation ...
QDG: A unified model for automatic question-distractor pairs generation
AbstractGenerating high-quality complete question sets (for example, the question, answer and distractors) in reading comprehension tasks is challenging and rewarding. This paper proposes a question-distractor joint generation framework (QDG). The ...
A Survey of Evaluation Metrics Used for NLG Systems
In the last few years, a large number of automatic evaluation metrics have been proposed for evaluating Natural Language Generation (NLG) systems. The rapid development and adoption of such automatic evaluation metrics in a relatively short time has ...






Comments