Abstract
End-to-end neural modeling with the encoder-decoder architecture has shown great promise in response generation. However, it often generates dull and generic responses due to its failure to effectively perceive various kinds of act, sentiment, and topic information. To address these challenges, we propose a response-generation model with structure-aware constraints to capture the structure of dialog and generate a better response with various constraints of the act, sentiment, and topic. In particular, given an utterance sequence, we first learn the representation of each utterance in the encoding stage. We then learn the turn, speaker, and dialog representation from the utterance representations and construct the structure of dialog. Third, we employ an attention mechanism to extract the constraints of act, sentiment, and topic based on the structure of the dialog. Finally, we utilize these structure-aware constraints to control the response-generation process in decoding stage. Extensive experimental results validate the superiority of our proposed model against the state-of-the-art baselines. In addition, the results also show that the proposed model can generate responses with more appropriate content based on the structure-aware constraints.
- [1] . 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations. 1–15.Google Scholar
- [2] . 2020. PLATO: Pre-trained dialogue generation model with discrete latent variable. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, , , , and (Eds.). Association for Computational Linguistics, 85–96.Google Scholar
Cross Ref
- [3] . 2018. MultiWOZ—A large-scale multi-domain Wizard-of-Oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, , , , and (Eds.). Association for Computational Linguistics, 5016–5026.Google Scholar
Cross Ref
- [4] . 2018. Hierarchical variational memory network for dialogue generation. In Proceedings of the 2018 World Wide Web Conference on World Wide Web. 1653–1662.Google Scholar
Digital Library
- [5] . 2019. Semantically conditioned dialog response generation via hierarchical disentangled self-attention. In Proceedings of the 57th Conference of the Association for Computational Linguistics, , , and (Eds.). Association for Computational Linguistics, 3696–3709.Google Scholar
Cross Ref
- [6] . 2018. XL-NBT: A cross-lingual neural belief tracking framework. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, , , , and (Eds.). Association for Computational Linguistics, 414–424.Google Scholar
Cross Ref
- [7] . 2019. A working memory model for task-oriented dialog response generation. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 2687–2693.Google Scholar
Cross Ref
- [8] . 2020. F-softmax: Diversifying neural text generation via frequency factorized softmax. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 9167–9182.Google Scholar
Cross Ref
- [9] . 2020. A corpus of controlled opinionated and knowledgeable movie discussions for training neural conversation models. In Proceedings of the 12th Language Resources and Evaluation Conference. 565–573.Google Scholar
- [10] . 2021. Space efficient context encoding for non-task-oriented dialogue generation with graph attention transformer. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 7028–7041.Google Scholar
Cross Ref
- [11] . 2020. Paraphrase augmented task-oriented dialog generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, , , , and (Eds.). Association for Computational Linguistics, 639–649.Google Scholar
Cross Ref
- [12] . 2018. Slot-gated modeling for joint slot filling and intent prediction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, , , and (Eds.). Association for Computational Linguistics, 753–757.Google Scholar
Cross Ref
- [13] . 2020. Controlling dialogue generation with semantic exemplars. arXiv preprint arXiv:2008.09075 (2020).Google Scholar
- [14] . 2017. Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 1766–1776.Google Scholar
Cross Ref
- [15] . 2020. The curious case of neural text degeneration. In 8th International Conference on Learning Representations.Google Scholar
- [16] . 2020. Challenges in building intelligent open-domain dialog systems. ACM Transactions on Information Systems 38, 3 (2020), 21:1–21:32.Google Scholar
Digital Library
- [17] . 2015. Adam: A method for stochastic optimization. In 3rd International Conference on Learning Representations.Google Scholar
- [18] . 2018. Sequicity: Simplifying task-oriented dialogue systems with single sequence-to-sequence architectures. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, and (Eds.). Association for Computational Linguistics, 1437–1447.Google Scholar
Cross Ref
- [19] . 2016. A diversity-promoting objective function for neural conversation models. In the 2016 Conference of the North American Chapter of the Association for Computational Linguistics, , , and (Eds.). Association for Computational Linguistics, 110–119.Google Scholar
Cross Ref
- [20] . 2017. Adversarial learning for neural dialogue generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2157–2169.Google Scholar
Cross Ref
- [21] . 2021. Deep context modeling for multi-turn response selection in dialogue systems. Information Processing and Management 58, 1 (2021), 102415.Google Scholar
Cross Ref
- [22] . 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the 8th International Joint Conference on Natural Language Processing. 986–995.Google Scholar
- [23] . 2020. A hierarchical structured multi-head attention network for multi-turn response generation. IEEE Access 8 (2020), 46802–46810.Google Scholar
Cross Ref
- [24] . 2020. The world is not binary: Learning to rank with grayscale data for dialogue response selection. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 9220–9229.Google Scholar
Cross Ref
- [25] . 2021. Context-controlled topic-aware neural response generation for open-domain dialog systems. Information Processing and Management 58, 1 (2021), 102392.Google Scholar
Cross Ref
- [26] . 2020. Mitigating gender bias for neural dialogue generation with adversarial learning. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, , , , and (Eds.). Association for Computational Linguistics, 893–903.Google Scholar
Cross Ref
- [27] . 2020. You impress me: Dialogue generation via mutual persona perception. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, , , , and (Eds.). Association for Computational Linguistics, 1417–1427.Google Scholar
Cross Ref
- [28] . 2019. RefNet: A reference-aware network for background based conversation. arXiv preprint arXiv:1908.06449 (2019).Google Scholar
- [29] . 2017. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, and (Eds.). Association for Computational Linguistics, 1777–1788.Google Scholar
- [30] . 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 311–318.Google Scholar
Digital Library
- [31] . 2011. Data-driven response generation in social media. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 583–593.Google Scholar
Digital Library
- [32] . 2021. Recipes for building an open-domain chatbot. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics. 300–325.Google Scholar
Cross Ref
- [33] . 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 3295–3301.Google Scholar
Cross Ref
- [34] . 2015. Neural responding machine for short-text conversation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1577–1586.Google Scholar
Cross Ref
- [35] . 2020. Counseling-style reflection generation using generative pretrained transformers with augmented context. In Proceedings of the 21st Annual Meeting of the Special Interest Group on Discourse and Dialogue. 10–20.Google Scholar
- [36] . 2016. Deep LSTM based feature mapping for query classification. In 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, , , and (Eds.). Association for Computational Linguistics, 1501–1511.Google Scholar
Cross Ref
- [37] . 2021. BoB: BERT over BERT for training persona-based dialogue models from limited personalized data. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 167–177.Google Scholar
Cross Ref
- [38] . 2020. Generate, delete and rewrite: A three-stage framework for improving persona consistency of dialogue generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 5821–5831.Google Scholar
Cross Ref
- [39] . 2021. Dialogue response selection with hierarchical curriculum learning. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 1740–1751.Google Scholar
Cross Ref
- [40] . 2017. Attention is all you need. In Proceedings of NIPS. 5998–6008.Google Scholar
- [41] . 2017. Latent intention dialogue models. In Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research), and (Eds.), Vol. 70. PMLR, 3732–3741.Google Scholar
- [42] . 2017. A network-based end-to-end trainable task-oriented dialogue system. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, , , and (Eds.). Association for Computational Linguistics, 438–449.Google Scholar
Cross Ref
- [43] . 2016. The dialog state tracking challenge series: A review. D&D 7, 3 (2016), 4–33.Google Scholar
Cross Ref
- [44] . 2013. The dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference. Association for Computer Linguistics, 404–413.Google Scholar
- [45] . 2020. TOD-BERT: Pre-trained natural language understanding for task-oriented dialogue. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, , , , and (Eds.). Association for Computational Linguistics, 917–929.Google Scholar
Cross Ref
- [46] . 2018. Hierarchical recurrent attention network for response generation. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, and (Eds.). AAAI Press, 5610–5617.Google Scholar
Cross Ref
- [47] . 2017. Neural response generation via GAN with an approximate embedding layer. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 617–626.Google Scholar
Cross Ref
- [48] . 2020. StyleDGPT: Stylized response generation with pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 1548–1559.Google Scholar
Cross Ref
- [49] . 2019. Improving abstractive document summarization with salient information modeling. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 2132–2141.Google Scholar
Cross Ref
- [50] . 2019. Dirichlet latent variable hierarchical recurrent encoder-decoder in dialogue generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. 1267–1272.Google Scholar
Cross Ref
- [51] . 2019. Hierarchy response learning for neural conversation generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. 1772–1781.Google Scholar
Cross Ref
- [52] . 2019. ReCoSa: Detecting the relevant contexts with self-attention for multi-turn dialogue generation. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 3721–3730.Google Scholar
Cross Ref
- [53] . 2020. Multi-turn dialogue generation in e-commerce platform with the context of historical dialogue. In EMNLP 2020, , , and (Eds.). 1981–1990.Google Scholar
- [54] . 2020. DIALOGPT: Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 270–278.Google Scholar
Cross Ref
- [55] . 2020. Low-resource knowledge-grounded dialogue generation. arXiv preprint arXiv:2002.10348 (2020).Google Scholar
- [56] . 2021. Neural stylistic response generation with disentangled latent variables. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 4391–4401.Google Scholar
Cross Ref
Index Terms
Response Generation via Structure-Aware Constraints
Recommendations
Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generation
SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information RetrievalRecently, pre-training methods have shown remarkable success in task-oriented dialog (TOD) systems. However, most existing pre-trained models for TOD focus on either dialog understanding or dialog generation, but not both. In this paper, we propose ...
Topic aware neural response generation
AAAI'17: Proceedings of the Thirty-First AAAI Conference on Artificial IntelligenceWe consider incorporating topic information into a sequence-to-sequence framework to generate informative and interesting responses for chatbots. To this end, we propose a topic aware sequence-to-sequence (TA-Seq2Seq) model. The model utilizes topics to ...
CTRLStruct: Dialogue Structure Learning for Open-Domain Response Generation
WWW '23: Proceedings of the ACM Web Conference 2023Dialogue structure discovery is essential in dialogue generation. Well-structured topic flow can leverage background information and predict future topics to help generate controllable and explainable responses. However, most previous work focused on ...






Comments