Abstract
Building a human-computer conversational system that can communicate with humans is a research hotspot in the field of artificial intelligence. Traditional dialogue systems tend to produce irrelevant and non-information responses, which reduce people’s interest in engaging in a conversation. This often leads to boring conversations. To alleviate this problem, many researchers use external knowledge to assist conversation generation. The accuracy of knowledge selection is the prerequisite to ensure the quality of knowledge conversation. This approach has worked positively to a certain extent, but generally only searches knowledge information based on entity words themselves, without considering the specific conversation context. Therefore, if irrelevant knowledge is retrieved, the quality of conversation generation will be reduced. Motivated by this, we propose a novel neural knowledge-based conversation generation model, named Siamese Network based Posterior Knowledge Selection Model for Knowledge Driven Conversation Generation (SPK-CG). We have designed a novel knowledge selection mechanism to obtain knowledge information that is highly relevant to the context of the conversation. Specifically, the posterior knowledge distribution is used as a soft label to make the prior distribution consistent with the posterior distribution in the training process. At the same time, in order to narrow the gap between prior and posterior distributions and improve the accuracy of knowledge selection, we leverage siamese network and design multi-granularity matching module for knowledge selection. Compared with previous knowledge-based models, our method can select more appropriate knowledge and use the selected knowledge to generate responses that are more relevant to the conversation context. Extensive automatic and human evaluations demonstrate that our model has advantages over previous baselines.
- [1] . 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems 33 (2020), 1877–1901.Google Scholar
- [2] . 2020. Bridging the gap between prior and posterior knowledge selection for knowledge-grounded dialogue generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16–20, 2020. Association for Computational Linguistics, 3426–3437.Google Scholar
Cross Ref
- [3] . 2016. Fast and accurate deep network learning by exponential linear units (ELUs). In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2–4, 2016, Conference Track Proceedings.Google Scholar
- [4] . 2019. Wizard of Wikipedia: Knowledge-powered conversational agents. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net.Google Scholar
- [5] . 2019. A discrete CVAE for response generation on short-text conversation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, 1898–1908.Google Scholar
Cross Ref
- [6] . 2018. A knowledge-grounded neural conversation model. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. AAAI Press, 5110–5117.Google Scholar
Cross Ref
- [7] . 2020. Utterance-to-utterance interactive matching network for multi-turn response selection in retrieval-based chatbots. IEEE ACM Trans. Audio Speech Lang. Process. 28 (2020), 369–379.Google Scholar
Digital Library
- [8] . 2021. Sentence similarity evaluation using Sent2Vec and Siamese neural network with parallel structure. J. Intell. Fuzzy Syst. 40, 4 (2021), 7735–7744.Google Scholar
Digital Library
- [9] . 2020. Challenges in building intelligent open-domain dialog systems. ACM Trans. Inf. Syst. 38, 3 (2020), 21:1–21:32.Google Scholar
Digital Library
- [10] . 2020. Knowledge augmented dialogue generation with divergent facts selection. Knowledge-Based Systems 210 (2020), 106479.Google Scholar
Cross Ref
- [11] . 2020. Sequential latent knowledge selection for knowledge-grounded dialogue. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2226–2237.Google Scholar
- [12] . 2021. Comparing Kullback-Leibler divergence and mean squared error loss in knowledge distillation. ijcai.org, 2628–2635.Google Scholar
- [13] . 2021. Chinese emotional dialogue response generation via reinforcement learning. ACM Trans. Internet Techn. 21, 4 (2021), 94:1–94:17.Google Scholar
Digital Library
- [14] . 2021. Knowledge-driven answer generation for conversational search. arXiv preprint arXiv:2104.06892 (2021).Google Scholar
- [15] . 2021. Topic-level knowledge sub-graphs for multi-turn dialogue generation. Knowledge-Based Systems 234 (2021), 107499.Google Scholar
Digital Library
- [16] . 2021. Medical term and status generation from Chinese clinical dialogue with multi-granularity transformer. IEEE ACM Trans. Audio Speech Lang. Process. 29 (2021), 3362–3374.Google Scholar
Digital Library
- [17] . 2019. Learning to select knowledge for response generation in dialog systems. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. IJCAI, 5081–5087.Google Scholar
Cross Ref
- [18] . 2021. Context-controlled topic-aware neural response generation for open-domain dialog systems. Information Processing & Management 58, 1 (2021), 102392.Google Scholar
Cross Ref
- [19] . 2019. Knowledge aware conversation generation with explainable reasoning over augmented graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019. Association for Computational Linguistics, 1782–1792.Google Scholar
Cross Ref
- [20] . 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. Association for Computational Linguistics, 1412–1421.Google Scholar
Cross Ref
- [21] . 2021. A hybrid Chinese conversation model based on retrieval and generation. Future Generation Computer Systems 114 (2021), 481–490.Google Scholar
Cross Ref
- [22] . 2020. RefNet: A reference-aware network for background based conversation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. AAAI Press, 8496–8503.Google Scholar
Cross Ref
- [23] . 2018. Towards exploiting background knowledge for building conversation systems. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. Association for Computational Linguistics, 2322–2332.Google Scholar
Cross Ref
- [24] . 2021. An intelligent knowledge-based chatbot for customer service. Electronic Commerce Research and Applications (2021), 101098.Google Scholar
Digital Library
- [25] . 2017. Gated multimodal units for information fusion. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings. OpenReview.net.Google Scholar
- [26] . 2020. Condition-transforming variational autoencoder for generating diverse short text conversations. ACM Trans. Asian Low Resour. Lang. Inf. Process. 19, 6 (2020), 79:1–79:13.Google Scholar
Digital Library
- [27] . 2020. A constrained optimization algorithm for learning GloVe embeddings with semantic lexicons. Knowledge-Based Systems 195 (2020), 105628.Google Scholar
Cross Ref
- [28] . 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers. Association for Computational Linguistics, 1073–1083.Google Scholar
Cross Ref
- [29] . 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada. 3104–3112.Google Scholar
- [30] . 2020. Cluster-based beam search for pointer-generator chatbot grounded by knowledge. Computer Speech & Language 64 (2020), 101094.Google Scholar
Cross Ref
- [31] . 2020. Seq2Seq models for recommending short text conversations. Expert Systems with Applications 150 (2020), 113270.Google Scholar
Cross Ref
- [32] . 2019. Gating mechanism based natural language generation for spoken dialogue systems. Neurocomputing 325 (2019), 48–58.Google Scholar
Cross Ref
- [33] . 2021. Towards information-rich, logical dialogue systems with knowledge-enhanced neural models. Neurocomputing (2021).Google Scholar
Digital Library
- [34] . 2020. Diverse and informative dialogue generation with context-specific commonsense knowledge awareness. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 5811–5820.Google Scholar
Cross Ref
- [35] . 2019. Proactive human-machine conversation with explicit conversation goals. arXiv preprint arXiv:1906.05572 (2019).Google Scholar
- [36] . 2019. Neural response generation with meta-words. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 5416–5426.Google Scholar
Cross Ref
- [37] . 2020. Knowledge-grounded response generation with deep attentional latent-variable model. Computer Speech & Language 63 (2020), 101069.Google Scholar
Cross Ref
- [38] . 2021. CoLV: A collaborative latent variable model for knowledge-grounded dialogue generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2250–2261.Google Scholar
Cross Ref
- [39] . 2020. Neural machine translation with GRU-gated attention model. IEEE Trans. Neural Networks Learn. Syst. 31, 11 (2020), 4688–4698.Google Scholar
Cross Ref
- [40] . 2019. Bridging the gap between training and inference for neural machine translation. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 4334–4343.Google Scholar
Cross Ref
- [41] . 2019. Neural conversation generation with auxiliary emotional supervised models. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 19, 2 (2019), 1–17.Google Scholar
- [42] . 2018. Commonsense knowledge aware conversation generation with graph attention. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. ijcai.org, 4623–4629.Google Scholar
Cross Ref
- [43] . 2021. Dual-copying mechanism and dynamic emotion dictionary for generating emotional responses. Neurocomputing 454 (2021), 303–312.Google Scholar
Cross Ref
Index Terms
SPK-CG: Siamese Network based Posterior Knowledge Selection Model for Knowledge Driven Conversation Generation
Recommendations
Prediction, selection, and generation: a knowledge-driven conversation system
AbstractIn conversational systems, we can use external knowledge to generate more diverse sentences and make these sentences contain actual knowledge. Leveraging knowledge for conversation system is important but challenging. Firstly, the conversation ...
Deliberation Selector for Knowledge-Grounded Conversation Generation
PRICAI 2022: Trends in Artificial IntelligenceAbstractThe integration of external knowledge is an important aspect for developing multi-turn conversation generation. However, most existing knowledge-background conversation generation models ignore the characteristics of the human brain to select ...
Improving knowledge-based dialogue generation through two-stage knowledge selection and knowledge selection-guided pointer network
AbstractExisting End-to-End neural models for dialogue generation tend to generate generic and uninformative responses. Recently, knowledge-based dialogue models have been developed to generate more informative responses by leveraging external knowledge. ...






Comments