skip to main content
short-paper

Detecting Entities of Works for Chinese Chatbot

Authors Info & Claims
Published:27 September 2020Publication History
Skip Abstract Section

Abstract

Chatbots such as Xiaoice have gained huge popularity in recent years. Users frequently mention their favorite works such as songs and movies in conversations with chatbots. Detecting these entities can help design better chat strategies and improve user experience. Existing named entity recognition methods are mainly designed for formal texts, and their performance on the informal chatbot conversation texts may not be optimal. In addition, these methods rely on massive manually annotated data for model training. In this article, we propose a neural approach to detect entities of works for Chinese chatbot. Our approach is based on a language model (LM) long-short term memory (LSTM) convolutional neural network (CNN) conditional random value (CRF), or LM-LSTM-CNN-CRF, framework, which contains a language model to generate context-aware character embeddings, a Bi-LSTM network to learn contextual character representations from global contexts, a CNN to learn character representations from local contexts, and a CRF layer to jointly decode the character label sequence. In addition, we propose an automatic text annotation method via quote marks to reduce the effort of manual annotation. Besides, we propose an iterative data purification method to improve the quality of the automatically constructed labeled data. Massive experiments on a real-world dataset validate that our approach can achieve good performance on entity detection for Chinese chatbots.

References

  1. John Blitzer, Ryan McDonald, and Fernando Pereira. 2006. Domain adaptation with structural correspondence learning. In EMNLP. ACM, 120--128.Google ScholarGoogle Scholar
  2. Rich Caruana, Steve Lawrence, and C. Lee Giles. 2001. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In NIPS. 402--408.Google ScholarGoogle Scholar
  3. Aitao Chen, Fuchun Peng, Roy Shan, and Gordon Sun. 2006. Chinese named entity recognition with conditional probabilistic models. In Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing. 173--176.Google ScholarGoogle Scholar
  4. Jason P. C. Chiu and Eric Nichols. 2016. Named entity recognition with bidirectional LSTM-CNNs. TACL 4 (2016), 357--370.Google ScholarGoogle ScholarCross RefCross Ref
  5. Arjun Das, Debasis Ganguly, and Utpal Garain. 2017. Named entity recognition with word embeddings and Wikipedia categories for a low-resource language. TALLIP 16, 3 (2017), 18.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Yann Dauphin, Harm de Vries, and Yoshua Bengio. 2015. Equilibrated adaptive learning rates for non-convex optimization. In NIPS. 1504--1512.Google ScholarGoogle Scholar
  7. Leon Derczynski, Diana Maynard, Giuseppe Rizzo, Marieke van Erp, Genevieve Gorrell, Raphaël Troncy, Johann Petrak, and Kalina Bontcheva. 2015. Analysis of named entity recognition and linking for tweets. Information Processing 8 Management 51, 2 (2015), 32--49.Google ScholarGoogle Scholar
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT. 4171--4186.Google ScholarGoogle Scholar
  9. Chuanhai Dong, Huijia Wu, Jiajun Zhang, and Chengqing Zong. 2017. Multichannel LSTM-CRF for named entity recognition in Chinese social media. In CCL-NABD. Springer, 197--208.Google ScholarGoogle Scholar
  10. Chuanhai Dong, Jiajun Zhang, Chengqing Zong, Masanori Hattori, and Hui Di. 2016. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition. In Proceedings of the International Conference on Computer Processing of Oriental Languages. Springer, 239--250.Google ScholarGoogle ScholarCross RefCross Ref
  11. Cicero dos Santos and Victor Guimarães. 2015. Boosting named entity recognition with neural character embeddings. In Proceedings of the 5th Named Entity Workshop. 25--33.Google ScholarGoogle ScholarCross RefCross Ref
  12. Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2016. The rise of social bots. Communications of the ACM 59, 7 (2016), 96--104.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jianfeng Gao, Mu Li, Andi Wu, and Chang-Ning Huang. 2005. Chinese word segmentation and named entity recognition: A pragmatic approach. Computational Linguistics 31, 4 (2005), 531--574.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks 18, 5–6 (2005), 602--610.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Hangfeng He and Xu Sun. 2017. A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In AAAI. 3216--3222.Google ScholarGoogle Scholar
  16. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015).Google ScholarGoogle Scholar
  17. Safia Kanwal, Kamran Malik, Khurram Shahzad, Faisal Aslam, and Zubair Nawaz. 2019. Urdu named entity recognition: Corpus generation and deep learning applications. TALLIP 19, 1 (2019), 8.Google ScholarGoogle Scholar
  18. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML. 282--289.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In AAAI.Google ScholarGoogle Scholar
  20. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In NAACL. 260--270.Google ScholarGoogle Scholar
  21. Shuying Lin, Huosheng Xie, Liang-Chih Yu, and K. Robert Lai. 2017. SentiNLP at IJCNLP-2017 task 4: Customer feedback analysis using a Bi-LSTM-CNN model. In IJCNLP, Shared Tasks. 149--154.Google ScholarGoogle Scholar
  22. Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In ACL. 2124--2133.Google ScholarGoogle Scholar
  23. Zhangxun Liu, Conghui Zhu, and Tiejun Zhao. 2010. Chinese named entity recognition with a sequence labeling approach: Based on characters, or based on words? In Advanced Intelligent Computing Theories and Applications with Aspects of Artificial Intelligence. Springer, 634--640.Google ScholarGoogle Scholar
  24. Gang Luo, Xiaojiang Huang, Chin-Yew Lin, and Zaiqing Nie. 2015. Joint entity recognition and disambiguation. In EMNLP. 879--888.Google ScholarGoogle Scholar
  25. Wencan Luo and Fan Yang. 2016. An empirical study of automatic chinese word segmentation for spoken language understanding and named entity recognition. In NAACL. 238--248.Google ScholarGoogle Scholar
  26. Xuezhe Ma and Eduard H. Hovy. 2016. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In ACL. 1064--1074.Google ScholarGoogle Scholar
  27. Thien Huu Nguyen, Avirup Sil, Georgiana Dinu, and Radu Florian. 2016. Toward mention detection robustness with recurrent neural networks. arXiv preprint arXiv:1602.07749 (2016).Google ScholarGoogle Scholar
  28. Nanyun Peng and Mark Dredze. 2015. Named entity recognition for Chinese social media with jointly trained embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 548--554.Google ScholarGoogle ScholarCross RefCross Ref
  29. Nanyun Peng and Mark Dredze. 2016. Improving named entity recognition for Chinese social media with word segmentation representation learning. arXiv preprint arXiv:1603.00786 (2016).Google ScholarGoogle Scholar
  30. Matthew Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. In ACL. 1756--1765.Google ScholarGoogle Scholar
  31. Barbara Plank and Alessandro Moschitti. 2013. Embedding semantic similarity in tree kernels for domain adaptation of relation extraction. In ACL, Vol. 1. 1498--1507.Google ScholarGoogle Scholar
  32. Desh Raj, Sunil Sahu, and Ashish Anand. 2017. Learning local and global contexts using a convolutional recurrent network model for relation classification in biomedical text. In CoNLL. 311--321.Google ScholarGoogle Scholar
  33. Marc-Antoine Rondeau and Yi Su. 2016. LSTM-Based NeuroCRFs for named entity recognition. In INTERSPEECH. 665--669.Google ScholarGoogle Scholar
  34. Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1 (2014), 1929--1958.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Xiaojun Wan, Liang Zong, Xiaojiang Huang, Tengfei Ma, Houping Jia, Yuqian Wu, and Jianguo Xiao. 2011. Named entity recognition in Chinese news comments on the web. In IJCNLP. 856--864.Google ScholarGoogle Scholar
  36. Fangzhao Wu, Junxin Liu, Chuhan Wu, Yongfeng Huang, and Xing Xie. 2019. Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation. In WWW. 3342--3348.Google ScholarGoogle Scholar
  37. Yuejie Zhang, Zhiting Xu, and Tao Zhang. 2008. Fusion of multiple features for chinese named entity recognition based on CRF model. In Asia Information Retrieval Symposium. Springer, 95--106.Google ScholarGoogle ScholarCross RefCross Ref
  38. Yue Zhang and Jie Yang. 2018. Chinese NER using lattice LSTM. In ACL. 1554--1564.Google ScholarGoogle Scholar

Index Terms

  1. Detecting Entities of Works for Chinese Chatbot

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Asian and Low-Resource Language Information Processing
          ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 19, Issue 6
          November 2020
          277 pages
          ISSN:2375-4699
          EISSN:2375-4702
          DOI:10.1145/3426881
          Issue’s Table of Contents

          Copyright © 2020 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 27 September 2020
          • Accepted: 1 July 2020
          • Revised: 1 June 2020
          • Received: 1 July 2019
          Published in tallip Volume 19, Issue 6

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • short-paper
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!