Abstract
Thanks to the advancements in deep learning, chatbots are widely used in messaging applications. Undoubtedly, a chatbot is a new way of interaction between humans and machines. However, most of the chatbots act as a simple question answering system that responds with formulated answers. Traditional conversational chatbots usually adopt a retrieval-based model that requires a large amount of conversational data for retrieving various intents. Hence, training a chatbot model that uses low-resource conversational data to generate more diverse dialogues is desirable. We propose a method to build a task-oriented chatbot using a sentence generation model that generates sequences based on the generative adversarial network. The architecture of our model contains a generator that generates a diverse sentence and a discriminator that judges the sentences by comparing the generated and the ground-truth sentences. In the generator, we combine the attention model with the sequence-to-sequence model using hierarchical long short-term memory to extract sentence information. For the discriminator, our reward mechanism assigns low rewards for repeated sentences and high rewards for diverse sentences. Extensive experiments are presented to demonstrate the utility of our model that generates more diverse and information-rich sentences than those of the existing approaches.
- [1] 2015. LTP-Cloud. Retrieved from https://www.ltp-cloud.com.Google Scholar
- [2] 2017. THULAC. Retrieved from http://thulac.thunlp.org/.Google Scholar
- [3] 2021. Build Natural and Rich Conversational Experiences. Retrieved from https://dialogflow.com/.Google Scholar
- [4] 2021. Emotibot. Retrieved from http://www.emotibot.com/zh-tw/story.html?n=75.Google Scholar
- [5] 2021. ICTCLAS. Retrieved from http://ictclas.nlpir.org/.Google Scholar
- [6] 2021. Jieba. Retrieved from https://github.com/fxsjy/jieb/.Google Scholar
- [7] 2021. Language Understanding (LUIS). Retrieved from https://www.luis.ai/.Google Scholar
- [8] 2021. SIRI. Retrieved from https://www.apple.com/tw/siri/.Google Scholar
- [9] 2021. Wit.ai. Retrieved from https://wit.ai/.Google Scholar
- [10] . 2017. Wasserstein gan. arXiv:1701.07875. Retrieved from https://arxiv.org/abs/1701.07875.Google Scholar
- [11] . 2017. A retrieval-based dialogue system utilizing utterance and context embeddings. In Proceedings of the 16th IEEE International Conference on Machine Learning and Applications (ICMLA’17). 1120–1125.Google Scholar
Cross Ref
- [12] . 1979. Dialogue management for rulebased tutorials. In Proceedings of the 6th International Joint Conference on Artificial Intelligence - Volume 1 (IJCAI’79). Morgan Kaufmann, San Francisco, CA, 155–161. Google Scholar
Digital Library
- [13] . 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, 4171–4186. Google Scholar
Cross Ref
- [14] . 2010. Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 61 (2010), 2121–2159.Google Scholar
Digital Library
- [15] . 2005. The second international chinese word segmentation bakeoff. In Proceedings of the 4th SIGHAN Workshop on Chinese Language Processing.Google Scholar
- [16] . 2017. A copy-augmented sequence-to-sequence architecture gives good performance on task-oriented dialogue. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. Association for Computational Linguistics, 468–473.Google Scholar
Cross Ref
- [17] . 2017. Learning policies for adaptive tracking with deep feature cascades. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17), 105–114.Google Scholar
- [18] . 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL’17), System Demonstrations. Association for Computational Linguistics, 67–72.Google Scholar
Cross Ref
- [19] . 2007. Meteor: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the 2nd Workshop on Statistical Machine Translation (StatMT’07). Association for Computational Linguistics, 228–231.Google Scholar
Cross Ref
- [20] . 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 110–119. Google Scholar
Cross Ref
- [21] . 2016. Deep reinforcement learning for dialogue generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1192–1202. Google Scholar
Cross Ref
- [22] . 2017. Adversarial learning for neural dialogue generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2157–2169. Google Scholar
Cross Ref
- [23] . 2017. End-to-end task-completion neural dialogue systems. arXiv:1703.01008. Retrieved from https://arxiv.org/abs/1703.01008.Google Scholar
- [24] . 2004. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the ACL Workshop: Text Summarization Branches Out, 10.Google Scholar
- [25] . 2018. End-to-end learning of task-oriented dialogs. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. Association for Computational Linguistics, 67–73.Google Scholar
Cross Ref
- [26] . 2017. End-to-end optimization of task-oriented dialogue model with deep reinforcement learning. arXiv:1711.10712. Retrieved from http://arxiv.org/abs/1711.10712.Google Scholar
- [27] . 2011. Discovering spatio-temporal causal interactions in traffic data streams. In Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1010–1018.Google Scholar
Digital Library
- [28] . 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1412–1421. Google Scholar
Cross Ref
- [29] . 2012. Machine Learning: A Probabilistic Perspective. Cambridge, MA.Google Scholar
Digital Library
- [30] . 2018. Deep contextualized word representations. arXiv:1802.05365. Retrieved from https://arxiv.org/abs/1802.05365.Google Scholar
- [31] . 2004. Understanding inverse document frequency: On theoretical arguments for IDF. J Doc. 60 (
10 2004), 503–520. Google ScholarCross Ref
- [32] . 2013. The Cross-entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning. Springer Science & Business Media.Google Scholar
- [33] . 1997. Bidirectional recurrent neural networks. IEEE Trans. Sign. Process. 45, 11 (1997), 2673–2681.Google Scholar
Digital Library
- [34] . 2015. Hierarchical neural network generative models for movie dialogues. arXiv:1507.04808. Retrieved from https://arxiv.org/abs/1507.04808.Google Scholar
- [35] . 2018. Bootstrapping a neural conversational agent with dialogue self-play, crowdsourcing and on-line reinforcement learning. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers). Association for Computational Linguistics, 41–51. Google Scholar
Cross Ref
- [36] . 2019. Flexibly-structured model for task-oriented dialogues. arXiv:1908.02402. Retrieved from http://arxiv.org/abs/1908.02402.Google Scholar
- [37] . 2014. Sequence to sequence learning with neural networks. In Proceedings of Advances in Neural Information Processing Systems, Vol. 2. 3104–3112.Google Scholar
- [38] . 2011. Reinforcement learning: An introduction. MIT Press, Cambridge, Massachusetts.Google Scholar
- [39] . 2015. CIDEr: Consensus-based image description evaluation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15), 4566–4575.Google Scholar
- [40] . 2017. A network-based end-to-end trainable task-oriented dialogue system. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Association for Computational Linguistics, 438–449.Google Scholar
Cross Ref
- [41] . 2017. Hybrid code networks: Practical and efficient end-to-end dialog control with supervised and reinforcement learning. arXiv:1702.03274. Retrieved from https://arxiv.org/abs/1702.03274.Google Scholar
- [42] . 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 3 (1992), 229–256.Google Scholar
Digital Library
- [43] . 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv:1609.08144. Retrieved from https://arxiv.org/abs/1609.08144.Google Scholar
- [44] . 2018. DP-GAN: Diversity-promoting generative adversarial network for generating informative and diversified text. arXiv:1802.01345. Retrieved from https://arxiv.org/abs/1802.01345.Google Scholar
- [45] . 2018. Crafting a toolchain for image restoration by deep reinforcement learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2443–2452.Google Scholar
Cross Ref
- [46] . 2017. SeqGAN: Sequence generative adversarial nets with policy gradient. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2852–2858.Google Scholar
Cross Ref
- [47] . 2018. Deep reinforcement learning for unsupervised video summarization with diversity-representativeness reward. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, and (Eds.). AAAI Press, 7582–7589.Google Scholar
Cross Ref
- [48] . 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 2223–2232.Google Scholar
Cross Ref
Index Terms
A Task-oriented Chatbot Based on LSTM and Reinforcement Learning
Recommendations
A Task-oriented Chatbot Based on LSTM and Reinforcement Learning
NLPIR '19: Proceedings of the 2019 3rd International Conference on Natural Language Processing and Information RetrievalTraditional conversational chatbots usually adopt a retrieved-based model. Developers have to provide a large amount of conversational data and classify those data to different intents. To avoid cumbersome development processes, we propose a method to ...
Apply Natural Language Processing-Chatbot on Industry 4.0
Social Computing and Social MediaAbstractNLP, or natural language processing, is an area of artificial intelligence that has been studied for more than 50 years and allows computers to comprehend human language. NLP interprets and makes sense of spoken or written natural language inputs ...
Recent advances in deep learning based dialogue systems: a systematic survey
AbstractDialogue systems are a popular natural language processing (NLP) task as it is promising in real-life applications. It is also a complicated task since many NLP tasks deserving study are involved. As a result, a multitude of novel works on this ...






Comments