research-article

Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System

Authors:

Hua WuAuthors Info & Claims

SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

Pages 55 - 64

https://doi.org/10.1145/2911451.2911542

Published: 07 July 2016 Publication History

Abstract

To establish an automatic conversation system between humans and computers is regarded as one of the most hardcore problems in computer science, which involves interdisciplinary techniques in information retrieval, natural language processing, artificial intelligence, etc. The challenges lie in how to respond so as to maintain a relevant and continuous conversation with humans. Along with the prosperity of Web 2.0, we are now able to collect extremely massive conversational data, which are publicly available. It casts a great opportunity to launch automatic conversation systems. Owing to the diversity of Web resources, a retrieval-based conversation system will be able to find at least some responses from the massive repository for any user inputs. Given a human issued message, i.e., query, our system would provide a reply after adequate training and learning of how to respond. In this paper, we propose a retrieval-based conversation system with the deep learning-to-respond schema through a deep neural network framework driven by web data. The proposed model is general and unified for different conversation scenarios in open domain. We incorporate the impact of multiple data inputs, and formulate various features and factors with optimization into the deep learning framework. In the experiments, we investigate the effectiveness of the proposed deep neural network structures with better combinations of all different evidence. We demonstrate significant performance improvement against a series of standard and state-of-art baselines in terms of p@1, MAP, nDCG, and MRR for conversational purposes.

References

[1]

Y. Bengio. Learning deep architectures for AI. Foundations and Trends in Machine Learning, 2(1):1--127, 2009.

Digital Library

[2]

F. Bessho, T. Harada, and Y. Kuniyoshi. Dialog system using real-time crowdsourcing and Twitter large-scale corpus. In SIGDIAL, pages 227--231, 2012.

Digital Library

[3]

G. Cong, L. Wang, C.-Y. Lin, Y.-I. Song, and Y. Sun. Finding question-answer pairs from online forums. In SIGIR, pages 467--474.

Digital Library

[4]

A. Graves, A.-r. Mohamed, and G. Hinton. Speech recognition with deep recurrent neural networks. In Proc. Acoustics, Speech and Signal Processing, pages 6645--6649, 2013.

[5]

H. He, K. Gimpel, and J. Lin. Multi-perspective sentence similarity modeling with convolutional neural networks. In EMNLP, pages 1576--1586, 2015.

[6]

R. Higashinaka, K. Imamura, T. Meguro, C. Miyazaki, N. Kobayashi, H. Sugiyama, T. Hirano, T. Makino, and Y. Matsuo. Towards an open domain conversational system fully based on natural language processing. In COLING, 2014.

[7]

B. Hu, Z. Lu, H. Li, and Q. Chen. Convolutional neural network architectures for matching natural language sentences. In NIPS, pages 2042--2050, 2014.

Digital Library

[8]

K. Järvelin and J. Kek\"al\"ainen. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst., 20(4):422--446, 2002.

Digital Library

[9]

Z. Ji, Z. Lu, and H. Li. An information retrieval approach to short text conversation. CoRR, abs/1408.6988, 2014.

[10]

N. Kalchbrenner, E. Grefenstette, and P. Blunsom. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188, 2014.

[11]

C.-J. Lee, Q. Ai, W. B. Croft, and D. Sheldon. An optimization framework for merging multiple result lists. In CIKM '15, pages 303--312, 2015.

Digital Library

[12]

A. Leuski, R. Patel, D. Traum, and B. Kennedy. Building effective question answering characters. In SIGDIAL, pages 18--27, 2009.

Digital Library

[13]

A. Leuski and D. Traum. NPCEditor: Creating virtual human dialogue using information retrieval techniques. AI Magazine, 32(2):42--56, 2011.

Digital Library

[14]

H. Li and J. Xu. Semantic matching in search. Foundations and Trends in Information Retrieval, 8:89, 2014.

Digital Library

[15]

J. Li, M. Galley, C. Brockett, J. Gao, and B. Dolan. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055, 2015.

[16]

X. Li, L. Mou, R. Yan, and M. Zhang. Stalematebreaker: A proactive content-introducing approach to automatic human-computer conversation. In IJCAI, 2016.

[17]

Z. Lu and H. Li. A deep architecture for matching short texts. In NIPS, pages 1367--1375, 2013.

Digital Library

[18]

C. D. Manning, P. Raghavan, and H. Schütze. Introduction to Information Retrieval. Cambridge University Press, 2008.

[19]

T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. arXiv:1301.3781, 2013.

[20]

L. Mou, G. Li, L. Zhang, T. Wang, and Z. Jin. Convolutional neural networks over tree structures for programming language processing. In AAAI, pages 1287--1292, 2016.

Digital Library

[21]

L. Mou, H. Peng, G. Li, Y. Xu, L. Zhang, and Z. Jin. Discriminative neural sentence modeling by tree-based convolution. In EMNLP, pages 2315--2325, 2015.

[22]

L. Mou, M. Rui, G. Li, Y. Xu, L. Zhang, R. Yan, and Z. Jin. Recognizing entailment and contradiction by tree-based convolution. arXiv preprint arXiv:1512.08422, 2015.

[23]

M. Nakano, N. Miyazaki, N. Yasuda, A. Sugiyama, J.-i. Hirasawa, K. Dohsaka, and K. Aikawa. WIT: A toolkit for building robust and real-time spoken dialogue systems. In SIGDIAL, pages 150--159.

Digital Library

[24]

E. Nouri, R. Artstein, A. Leuski, and D. R. Traum. Augmenting conversational characters with generated question-answer pairs. In AAAI Fall Symposium: Question Generation, 2011.

[25]

H. Palangi, L. Deng, Y. Shen, J. Gao, X. He, J. Chen, X. Song, and R. Ward. Deep sentence embedding using the long short term memory network: Analysis and application to information retrieval. arXiv preprint arXiv:1502.06922, 2015.

Digital Library

[26]

A. Ritter, C. Cherry, and W. B. Dolan. Data-driven response generation in social media. In EMNLP, pages 583--593, 2011.

Digital Library

[27]

T. Rocktäschel, E. Grefenstette, K. M. Hermann, T. Kočiskỳ, and P. Blunsom. Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.06664, 2015.

[28]

A. Severyn and A. Moschitti. Learning to rank short text pairs with convolutional deep neural networks. In SIGIR '15, pages 373--382.

Digital Library

[29]

L. Shang, Z. Lu, and H. Li. Neural responding machine for short-text conversation. In ACL-IJCNLP, pages 1577--1586, 2015.

[30]

R. Socher, J. Pennington, E. H. Huang, A. Y. Ng, and C. D. Manning. Semi-supervised recursive autoencoders for predicting sentiment distributions. In EMNLP, pages 151--161, 2011.

Digital Library

[31]

H. Sugiyama, T. Meguro, R. Higashinaka, and Y. Minami. Open-domain utterance generation for conversational dialogue systems using Web-scale dependency structures. In SIGDIAL, pages 334--338, 2013.

[32]

I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In NIPS, pages 3104--3112, 2014.

Digital Library

[33]

M. A. Walker, R. Passonneau, and J. E. Boland. Quantitative and qualitative evaluation of darpa communicator spoken dialogue systems. In ACL, pages 515--522, 2001.

Digital Library

[34]

R. S. Wallace. The Anatomy of ALICE. Springer, 2009.

[35]

H. Wang, Z. Lu, H. Li, and E. Chen. A dataset for research on short-text conversations. In EMNLP, pages 935--945, 2013.

[36]

J. Williams, A. Raux, D. Ramachandran, and A. Black. The dialog state tracking challenge. In SIGDIAL, pages 404--413, 2013.

[37]

Y. Xu, R. Jia, L. Mou, G. Li, Y. Chen, Y. Lu, and Z. Jin. Improved relation classification by deep recurrent neural networks with data augmentation. arXiv preprint arXiv:1601.03651, 2016.

[38]

Y. Xu, L. Mou, G. Li, Y. Chen, H. Peng, and Z. Jin. Classifying relations via long short term memory networks along shortest dependency paths. In EMNLP, 2015.

[39]

R. Yan. i, poet: Automatic poetry composition through recurrent neural networks with iterative polishing schema. In IJCAI, 2016.

[40]

R. Yan, M. Lapata, and X. Li. Tweet recommendation with graph co-ranking. In ACL, pages 516--525, 2012.

Digital Library

[41]

R. Yan, C.-T. Li, H.-P. Hsieh, P. Hu, X. Hu, and T. He. Socialized language model smoothing via bi-directional influence propagation on social networks. In WWW '16, pages 1395--1405, 2016.

Digital Library

[42]

R. Yan, X. Wan, J. Otterbacher, L. Kong, X. Li, and Y. Zhang. Evolutionary timeline summarization: A balanced optimization framework via iterative substitution. In SIGIR '11, pages 745--754, 2011.

Digital Library

[43]

R. Yan, I. E. Yen, C.-T. Li, S. Zhao, and X. Hu. Tackling the achilles heel of social networks: Influence propagation based language model smoothing. In WWW '15, pages 1318--1328, 2015.

Digital Library

[44]

K. Zhai and D. J. Williams. Discovering latent structure in task-oriented dialogues. In ACL, pages 36--46, 2014.

[45]

B. Zhang, J. Su, D. Xiong, Y. Lu, H. Duan, and J. Yao. Shallow convolutional neural network for implicit discourse relation recognition. In EMNLP, pages 2230--2235, 2015.

Cited By

Sekulić IAlinannejadi MCrestani F(2024)Analysing Utterances in LLM-Based User Simulation for Conversational SearchACM Transactions on Intelligent Systems and Technology10.1145/365004115:3(1-22)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3650041
Zhang ZZhao HLiu L(2024)Channel-Aware Decoupling Network for Multiturn Dialog ComprehensionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.322004735:6(7685-7696)Online publication date: Jun-2024
https://doi.org/10.1109/TNNLS.2022.3220047
Ghosh SGhosh SShah CKlein MBen-David AJäschke RKelly M(2024)Toward Connecting Speech Acts and Search Actions in Conversational Search TasksProceedings of the 2023 ACM/IEEE Joint Conference on Digital Libraries10.1109/JCDL57899.2023.00027(119-131)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1109/JCDL57899.2023.00027
Show More Cited By

Index Terms

Learning to Respond with Deep Neural Networks for Retrieval-Based Human-Computer Conversation System
1. Information systems
  1. Information retrieval
    1. Document representation
    2. Retrieval models and ranking
      1. Learning to rank
  2. World Wide Web
    1. Web applications
      1. Internet communications tools
        Chat

Recommendations

Joint Learning of Response Ranking and Next Utterance Suggestion in Human-Computer Conversation System
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Conversation systems are of growing importance since they enable an easy interaction interface between humans and computers: using natural languages. To build a conversation system with adequate intelligence is challenging, and requires abundant ...
Multimodal and Crossmodal Representation Learning from Textual and Visual Features with Bidirectional Deep Neural Networks for Video Hyperlinking
iV&L-MM '16: Proceedings of the 2016 ACM workshop on Vision and Language Integration Meets Multimedia Fusion

Video hyperlinking represents a classical example of multimodal problems. Common approaches to such problems are early fusion of the initial modalities and crossmodal translation from one modality to the other. Recently, deep neural networks, especially ...
A Dialogue Contextual Flow Model for Utterance Intent Recognition in Multi-turn Online Conversation
Knowledge Science, Engineering and Management
Abstract
There are many intents of dialogues that cannot be recognized due to the contextual features of conversation, resulting in service failure for online chatting robots. Current methods leverage memory networks or machine reading comprehension (MRC) ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '16: Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval

July 2016

1296 pages

ISBN:9781450340694

DOI:10.1145/2911451

General Chairs:
Raffaele Perego
ISTI-CNR, Italy
,
Fabrizio Sebastiani
Qatar Computing Research Institute, HBKU, Qatar
,
Program Chairs:
Javed Aslam
Northeastern University, US
,
Ian Ruthven
University of Strathclyde, UK
,
Justin Zobel
University of Melbourne, Australia

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Basic Research Program of China

Conference

SIGIR '16

Sponsor:

SIGIR

SIGIR '16: The 39th International ACM SIGIR conference on research and development in Information Retrieval

July 17 - 21, 2016

Pisa, Italy

Acceptance Rates

SIGIR '16 Paper Acceptance Rate 62 of 341 submissions, 18%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

171
Total Citations
View Citations
1,948
Total Downloads

Downloads (Last 12 months)70
Downloads (Last 6 weeks)7

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sekulić IAlinannejadi MCrestani F(2024)Analysing Utterances in LLM-Based User Simulation for Conversational SearchACM Transactions on Intelligent Systems and Technology10.1145/365004115:3(1-22)Online publication date: 5-Mar-2024
https://dl.acm.org/doi/10.1145/3650041
Zhang ZZhao HLiu L(2024)Channel-Aware Decoupling Network for Multiturn Dialog ComprehensionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.322004735:6(7685-7696)Online publication date: Jun-2024
https://doi.org/10.1109/TNNLS.2022.3220047
Ghosh SGhosh SShah CKlein MBen-David AJäschke RKelly M(2024)Toward Connecting Speech Acts and Search Actions in Conversational Search TasksProceedings of the 2023 ACM/IEEE Joint Conference on Digital Libraries10.1109/JCDL57899.2023.00027(119-131)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1109/JCDL57899.2023.00027
Bouafoud CZine-Dine KMadani A(2024)The Evolution of Transformers in Education: A Literature Review2024 International Conference on Circuit, Systems and Communication (ICCSC)10.1109/ICCSC62074.2024.10617128(1-7)Online publication date: 28-Jun-2024
https://doi.org/10.1109/ICCSC62074.2024.10617128
Ma XXu YZhao HZhang Z(2024)Multi-turn dialogue comprehension from a topic-aware perspectiveNeurocomputing10.1016/j.neucom.2024.127385578(127385)Online publication date: Apr-2024
https://doi.org/10.1016/j.neucom.2024.127385
Varshney DEkbal ACambria E(2024)Emotion-and-knowledge grounded response generation in an open-domain dialogue settingKnowledge-Based Systems10.1016/j.knosys.2023.111173284(111173)Online publication date: Jan-2024
https://doi.org/10.1016/j.knosys.2023.111173
Mazumder SLiu BMazumder SLiu B(2024)IntroductionLifelong and Continual Learning Dialogue Systems10.1007/978-3-031-48189-5_1(1-19)Online publication date: 9-Jan-2024
https://doi.org/10.1007/978-3-031-48189-5_1
Nimal KNair VJegdeep RNehru J(2023)Artificial Intelligence Based Chatbot for Healthcare ApplicationsIoT, Cloud and Data Science10.4028/p-atr6jg(370-377)Online publication date: 27-Feb-2023
https://doi.org/10.4028/p-atr6jg
Hu KXiao N(2023)Research on Applying the “Shift” Concept to Deep Attention MatchingApplied Sciences10.3390/app1306393413:6(3934)Online publication date: 20-Mar-2023
https://doi.org/10.3390/app13063934
Khan IJaved JSajid AShahnoor Tabassum I(2023)Comparative Analysis of Classical and Neural Networks based ChatBot’s TechniquesSir Syed University Research Journal of Engineering & Technology10.33317/ssurj.50813:1(61-73)Online publication date: 28-Jun-2023
https://doi.org/10.33317/ssurj.508
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents