skip to main content
research-article

SPK-CG: Siamese Network based Posterior Knowledge Selection Model for Knowledge Driven Conversation Generation

Published:10 March 2023Publication History
Skip Abstract Section

Abstract

Building a human-computer conversational system that can communicate with humans is a research hotspot in the field of artificial intelligence. Traditional dialogue systems tend to produce irrelevant and non-information responses, which reduce people’s interest in engaging in a conversation. This often leads to boring conversations. To alleviate this problem, many researchers use external knowledge to assist conversation generation. The accuracy of knowledge selection is the prerequisite to ensure the quality of knowledge conversation. This approach has worked positively to a certain extent, but generally only searches knowledge information based on entity words themselves, without considering the specific conversation context. Therefore, if irrelevant knowledge is retrieved, the quality of conversation generation will be reduced. Motivated by this, we propose a novel neural knowledge-based conversation generation model, named Siamese Network based Posterior Knowledge Selection Model for Knowledge Driven Conversation Generation (SPK-CG). We have designed a novel knowledge selection mechanism to obtain knowledge information that is highly relevant to the context of the conversation. Specifically, the posterior knowledge distribution is used as a soft label to make the prior distribution consistent with the posterior distribution in the training process. At the same time, in order to narrow the gap between prior and posterior distributions and improve the accuracy of knowledge selection, we leverage siamese network and design multi-granularity matching module for knowledge selection. Compared with previous knowledge-based models, our method can select more appropriate knowledge and use the selected knowledge to generate responses that are more relevant to the conversation context. Extensive automatic and human evaluations demonstrate that our model has advantages over previous baselines.

REFERENCES

  1. [1] Brown Tom, Mann Benjamin, Ryder Nick, Subbiah Melanie, Kaplan Jared D., Dhariwal Prafulla, Neelakantan Arvind, Shyam Pranav, Sastry Girish, Askell Amanda, et al. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems 33 (2020), 18771901.Google ScholarGoogle Scholar
  2. [2] Chen Xiuyi, Meng Fandong, Li Peng, Chen Feilong, Xu Shuang, Xu Bo, and Zhou Jie. 2020. Bridging the gap between prior and posterior knowledge selection for knowledge-grounded dialogue generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16–20, 2020. Association for Computational Linguistics, 34263437.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Clevert Djork-Arné, Unterthiner Thomas, and Hochreiter Sepp. 2016. Fast and accurate deep network learning by exponential linear units (ELUs). In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2–4, 2016, Conference Track Proceedings.Google ScholarGoogle Scholar
  4. [4] Dinan Emily, Roller Stephen, Shuster Kurt, Fan Angela, Auli Michael, and Weston Jason. 2019. Wizard of Wikipedia: Knowledge-powered conversational agents. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6–9, 2019. OpenReview.net.Google ScholarGoogle Scholar
  5. [5] Gao Jun, Bi Wei, Liu Xiaojiang, Li Junhui, Zhou Guodong, and Shi Shuming. 2019. A discrete CVAE for response generation on short-text conversation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, 18981908.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Ghazvininejad Marjan, Brockett Chris, Chang Ming-Wei, Dolan Bill, Gao Jianfeng, Yih Wen-tau, and Galley Michel. 2018. A knowledge-grounded neural conversation model. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. AAAI Press, 51105117.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Gu Jia-Chen, Ling Zhen-Hua, and Liu Quan. 2020. Utterance-to-utterance interactive matching network for multi-turn response selection in retrieval-based chatbots. IEEE ACM Trans. Audio Speech Lang. Process. 28 (2020), 369379.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Heo Tak-Sung, Kim Jong-Dae, Park Chan-Young, and Kim Yu-Seop. 2021. Sentence similarity evaluation using Sent2Vec and Siamese neural network with parallel structure. J. Intell. Fuzzy Syst. 40, 4 (2021), 77357744.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Huang Minlie, Zhu Xiaoyan, and Gao Jianfeng. 2020. Challenges in building intelligent open-domain dialog systems. ACM Trans. Inf. Syst. 38, 3 (2020), 21:1–21:32.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Jiang Bin, Yang Jingxu, Yang Chao, Zhou Wanyue, Pang Liang, and Zhou Xiaokang. 2020. Knowledge augmented dialogue generation with divergent facts selection. Knowledge-Based Systems 210 (2020), 106479.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Kim Byeongchang, Ahn Jaewoo, and Kim Gunhee. 2020. Sequential latent knowledge selection for knowledge-grounded dialogue. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 22262237.Google ScholarGoogle Scholar
  12. [12] Kim Taehyeon, Oh Jaehoon, Kim Nakyil, Cho Sangwook, and Yun Se-Young. 2021. Comparing Kullback-Leibler divergence and mean squared error loss in knowledge distillation. ijcai.org, 26282635.Google ScholarGoogle Scholar
  13. [13] Lan Rushi, Wang Jing, Huang Wenming, Deng Zhenrong, Sun Xiyan, Chen Zhuo, and Luo Xiaonan. 2021. Chinese emotional dialogue response generation via reinforcement learning. ACM Trans. Internet Techn. 21, 4 (2021), 94:1–94:17.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. [14] Leite Mariana, Ferreira Rafael, Semedo David, and Magalhães João. 2021. Knowledge-driven answer generation for conversational search. arXiv preprint arXiv:2104.06892 (2021).Google ScholarGoogle Scholar
  15. [15] Li Jing, Huang Qingbao, Cai Yi, Liu Yongkang, Fu Mingyi, and Li Qing. 2021. Topic-level knowledge sub-graphs for multi-turn dialogue generation. Knowledge-Based Systems 234 (2021), 107499.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Li Mei, Xiang Lu, Kang Xiaomian, Zhao Yang, Zhou Yu, and Zong Chengqing. 2021. Medical term and status generation from Chinese clinical dialogue with multi-granularity transformer. IEEE ACM Trans. Audio Speech Lang. Process. 29 (2021), 33623374.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. [17] Lian Rongzhong, Xie Min, Wang Fan, Peng Jinhua, and Wu Hua. 2019. Learning to select knowledge for response generation in dialog systems. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. IJCAI, 50815087.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Ling Yanxiang, Cai Fei, Hu Xuejun, Liu Jun, Chen Wanyu, and Chen Honghui. 2021. Context-controlled topic-aware neural response generation for open-domain dialog systems. Information Processing & Management 58, 1 (2021), 102392.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Liu Zhibin, Niu Zheng-Yu, Wu Hua, and Wang Haifeng. 2019. Knowledge aware conversation generation with explainable reasoning over augmented graphs. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019. Association for Computational Linguistics, 17821792.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Luong Thang, Pham Hieu, and Manning Christopher D.. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. Association for Computational Linguistics, 14121421.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Ma Tinghuai, Yang Huimin, Tian Qing, Tian Yuan, and Al-Nabhan Najla. 2021. A hybrid Chinese conversation model based on retrieval and generation. Future Generation Computer Systems 114 (2021), 481490.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Meng Chuan, Ren Pengjie, Chen Zhumin, Monz Christof, Ma Jun, and Rijke Maarten de. 2020. RefNet: A reference-aware network for background based conversation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. AAAI Press, 84968503.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Moghe Nikita, Arora Siddhartha, Banerjee Suman, and Khapra Mitesh M.. 2018. Towards exploiting background knowledge for building conversation systems. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. Association for Computational Linguistics, 23222332.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Ngai Eric W. T., Lee Maggie C. M., Luo Mei, Chan Patrick S. L., and Liang Tenglu. 2021. An intelligent knowledge-based chatbot for customer service. Electronic Commerce Research and Applications (2021), 101098.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Ovalle John Edison Arevalo, Solorio Thamar, Montes-y-Gómez Manuel, and González Fabio A.. 2017. Gated multimodal units for information fusion. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings. OpenReview.net.Google ScholarGoogle Scholar
  26. [26] Ruan Yu-Ping, Ling Zhen-Hua, and Zhu Xiaodan. 2020. Condition-transforming variational autoencoder for generating diverse short text conversations. ACM Trans. Asian Low Resour. Lang. Inf. Process. 19, 6 (2020), 79:1–79:13.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Sakketou Flora and Ampazis Nicholas. 2020. A constrained optimization algorithm for learning GloVe embeddings with semantic lexicons. Knowledge-Based Systems 195 (2020), 105628.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] See Abigail, Liu Peter J., and Manning Christopher D.. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers. Association for Computational Linguistics, 10731083.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Sutskever Ilya, Vinyals Oriol, and Le Quoc V.. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada. 31043112.Google ScholarGoogle Scholar
  30. [30] Tam Yik-Cheung. 2020. Cluster-based beam search for pointer-generator chatbot grounded by knowledge. Computer Speech & Language 64 (2020), 101094.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Torres Johnny, Vaca Carmen, Terán Luis, and Abad Cristina L.. 2020. Seq2Seq models for recommending short text conversations. Expert Systems with Applications 150 (2020), 113270.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Tran Van-Khanh and Nguyen Le-Minh. 2019. Gating mechanism based natural language generation for spoken dialogue systems. Neurocomputing 325 (2019), 4858.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Wang Hao, Guo Bin, Wu Wei, Liu Sicong, and Yu Zhiwen. 2021. Towards information-rich, logical dialogue systems with knowledge-enhanced neural models. Neurocomputing (2021).Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Wu Sixing, Li Ying, Zhang Dawei, Zhou Yang, and Wu Zhonghai. 2020. Diverse and informative dialogue generation with context-specific commonsense knowledge awareness. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 58115820.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Wu Wenquan, Guo Zhen, Zhou Xiangyang, Wu Hua, Zhang Xiyuan, Lian Rongzhong, and Wang Haifeng. 2019. Proactive human-machine conversation with explicit conversation goals. arXiv preprint arXiv:1906.05572 (2019).Google ScholarGoogle Scholar
  36. [36] Xu Can, Wu Wei, Tao Chongyang, Hu Huang, Schuerman Matt, and Wang Ying. 2019. Neural response generation with meta-words. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 54165426.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Ye Hao-Tong, Lo Kai-Lin, Su Shang-Yu, and Chen Yun-Nung. 2020. Knowledge-grounded response generation with deep attentional latent-variable model. Computer Speech & Language 63 (2020), 101069.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Zhan Haolan, Shen Lei, Chen Hongshen, and Zhang Hainan. 2021. CoLV: A collaborative latent variable model for knowledge-grounded dialogue generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 22502261.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Zhang Biao, Xiong Deyi, Xie Jun, and Su Jinsong. 2020. Neural machine translation with GRU-gated attention model. IEEE Trans. Neural Networks Learn. Syst. 31, 11 (2020), 46884698.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Zhang Wen, Feng Yang, Meng Fandong, You Di, and Liu Qun. 2019. Bridging the gap between training and inference for neural machine translation. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 43344343.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Zhou Guangyou, Fang Yizhen, Peng Yehong, and Lu Jiaheng. 2019. Neural conversation generation with auxiliary emotional supervised models. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 19, 2 (2019), 117.Google ScholarGoogle Scholar
  42. [42] Zhou Hao, Young Tom, Huang Minlie, Zhao Haizhou, Xu Jingfang, and Zhu Xiaoyan. 2018. Commonsense knowledge aware conversation generation with graph attention. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. ijcai.org, 46234629.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Zhou Qiji, Ji Donghong, Ren Yafeng, and Tang Hao. 2021. Dual-copying mechanism and dynamic emotion dictionary for generating emotional responses. Neurocomputing 454 (2021), 303312.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. SPK-CG: Siamese Network based Posterior Knowledge Selection Model for Knowledge Driven Conversation Generation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 3
      March 2023
      570 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3579816
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 10 March 2023
      • Online AM: 14 December 2022
      • Accepted: 11 October 2022
      • Revised: 19 August 2022
      • Received: 22 April 2022
      Published in tallip Volume 22, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
    • Article Metrics

      • Downloads (Last 12 months)93
      • Downloads (Last 6 weeks)6

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!