skip to main content
research-article

Token Relation Aware Chinese Named Entity Recognition

Authors Info & Claims
Published:25 November 2022Publication History
Skip Abstract Section

Abstract

Due to the lack of natural delimiters, most Chinese Named Entity Recognition (NER) approaches are character-based and utilize an external lexicon to leverage the word-level information. Although they have achieved promising results, the latent words they introduced are still non-contextualized. In this paper, we investigate three relations, i.e., adjacent relation between characters, character co-occurrence relation between latent words, and dependency relation among tokens, to address this issue. Specifically, we first establish the local context for latent words and then propose a masked self-attention mechanism to incorporate such local contextual information. Besides, since introducing external knowledge such as lexicon and dependency relation inevitably brings in some noises, we propose a gated information controller to handle this problem. Extensive experimental results show that the proposed approach surpasses most similar methods on public datasets and demonstrates its promising potential.

REFERENCES

  1. [1] Berger Adam L. and Lafferty John D.. 2017. Information retrieval as statistical translation. SIGIR Forum 51, 2 (2017), 219226.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Blanco Roi, Ottaviano Giuseppe, and Meij Edgar. 2015. Fast and space-efficient entity linking for queries. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining. 179188.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Cai Xiaoling, Dong Shoubin, and Hu Jinlong. 2019. A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records. BMC Medical Informatics & Decision Making 19-S, 2 (2019), 101109.Google ScholarGoogle Scholar
  4. [4] Chen Miao, Lan Ganhui, Du Fang, and Lobanov Victor S.. 2020. Joint learning with pre-trained transformer on named entity recognition and relation extraction tasks for clinical analytics. In Proceedings of the 3rd Clinical Natural Language Processing Workshop. 234242.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Collobert Ronan, Weston Jason, Bottou Léon, Karlen Michael, Kavukcuoglu Koray, and Kuksa Pavel P.. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 24932537.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Cui Yiming, Che Wanxiang, Liu Ting, Qin Bing, Yang Ziqing, Wang Shijin, and Hu Guoping. 2019. Pre-training with whole word masking for Chinese BERT. CoRR abs/1906.08101 (2019).Google ScholarGoogle Scholar
  7. [7] Dai Zihang, Yang Zhilin, Yang Yiming, Carbonell Jaime G., Le Quoc Viet, and Salakhutdinov Ruslan. 2019. Transformer-XL: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 29782988.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Dang Thanh Hai, Le Hoang-Quynh, Nguyen Trang M., and Vu Sinh T.. 2018. D3NER: Biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information. Bioinformatics. 34, 20 (2018), 35393546.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Devlin Jacob, Chang Ming-Wei, Lee Kenton, and Toutanova Kristina. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 41714186.Google ScholarGoogle Scholar
  10. [10] Diao Shizhe, Bai Jiaxin, Song Yan, Zhang Tong, and Wang Yonggang. 2020. ZEN: Pre-training Chinese text encoder enhanced by N-gram representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. 47294740.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Ding Ruixue, Xie Pengjun, Zhang Xiaoyan, Lu Wei, Li Linlin, and Si Luo. 2019. A neural multi-digraph model for Chinese NER with gazetteers. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 14621467.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Santos Cícero Nogueira dos and Guimarães Victor. 2015. Boosting named entity recognition with neural character embeddings. In Proceedings of the 5th Named Entity Workshop. 2533.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Eddy Sean R.. 1996. Hidden Markov models. Current Opinion in Structural Biology 6, 3 (1996), 361365.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Gong Chen, Huang Saihao, Zhou Houquan, Li Zhenghua, Zhang Min, Wang Zhefeng, Huai Baoxing, and Yuan Nicholas Jing. 2021. An in-depth study on internal structure of Chinese words. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 58235833.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Gui Tao, Ma Ruotian, Zhang Qi, Zhao Lujun, Jiang Yu-Gang, and Huang Xuanjing. 2019. CNN-based Chinese NER with lexicon rethinking. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 49824988.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Gui Tao, Zou Yicheng, Zhang Qi, Peng Minlong, Fu Jinlan, Wei Zhongyu, and Huang Xuanjing. 2019. A lexicon-based graph neural network for Chinese NER. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 10401050.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Guo Qipeng, Qiu Xipeng, Liu Pengfei, Shao Yunfan, Xue Xiangyang, and Zhang Zheng. 2019. Star-transformer. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Human Language Technologies. 13151325.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Hu Dou and Wei Lingwei. 2020. SLK-NER: Exploiting second-order lexicon knowledge for Chinese NER. In Proceedings of the 32nd International Conference on Software Engineering and Knowledge Engineering. 413417.Google ScholarGoogle Scholar
  19. [19] Huang Zhiheng, Xu Wei, and Yu Kai. 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991 (2015).Google ScholarGoogle Scholar
  20. [20] Jie Zhanming and Lu Wei. 2019. Dependency-guided LSTM-CRF for named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 38603870.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Jin Yanliang, Xie Jinfei, Guo Weisi, Luo Can, Wu Dijia, and Wang Rui. 2019. LSTM-CRF neural network with gated self attention for Chinese NER. IEEE Access 7 (2019), 136694136703.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Kandasamy Saravanakumar and Cherukuri Aswani Kumar. 2020. Query expansion using named entity disambiguation for a question-answering system. Concurrency and Computation: Practice and Experience 32, 4 (2020).Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Lafferty John D., McCallum Andrew, and Pereira Fernando C. N.. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. 282289.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Lample Guillaume, Ballesteros Miguel, Subramanian Sandeep, Kawakami Kazuya, and Dyer Chris. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 260270.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Levow Gina-Anne. 2006. The third international Chinese language processing bakeoff: Word segmentation and named entity recognition. In Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing. 108117.Google ScholarGoogle Scholar
  26. [26] Li Jing, Sun Aixin, Han Jianglei, and Li Chenliang. 2018. A survey on deep learning for named entity recognition. CoRR abs/1812.09449 (2018).Google ScholarGoogle Scholar
  27. [27] Li Peng-Hsuan, Fu Tsu-Jui, and Ma Wei-Yun. 2020. Why attention? Analyze BiLSTM deficiency and its remedies in the case of NER. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 82368244.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Li Xiaonan, Yan Hang, Qiu Xipeng, and Huang Xuanjing. 2020. FLAT: Chinese NER using flat-lattice transformer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 68366842.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Li Yuan, Du Guodong, Xiang Yan, Li Shaozi, Ma Lei, Shao Dangguo, Wang Xiongbin, and Chen Haoyu. 2020. Towards Chinese clinical named entity recognition by dynamic embedding using domain-specific knowledge. Journal of Biomedical Informatics 106 (2020), 103435.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Lin Yankai, Shen Shiqi, Liu Zhiyuan, Luan Huanbo, and Sun Maosong. 2016. Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 21242133.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Liu Wei, Fu Xiyan, Zhang Yue, and Xiao Wenming. 2021. Lexicon enhanced Chinese sequence labeling using BERT adapter. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 58475858.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Liu Wei, Xu Tongge, Xu QingHua, Song Jiayu, and Zu Yueran. 2019. An encoding strategy based word-character LSTM for Chinese NER. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 23792389.Google ScholarGoogle Scholar
  33. [33] Liu Zhangxun, Zhu Conghui, and Zhao Tiejun. 2010. Chinese named entity recognition with a sequence labeling approach: Based on characters, or based on words? In Proceedings of 6th International Conference on Intelligent Computing. 634640.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Ma Ruotian, Peng Minlong, Zhang Qi, Wei Zhongyu, and Huang Xuanjing. 2020. Simplify the usage of lexicon in Chinese NER. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 59515960.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Ma Xindian, Zhang Peng, Zhang Shuai, Duan Nan, Hou Yuexian, Zhou Ming, and Song Dawei. 2019. A tensorized transformer for language modeling. In Proceedings of 2019 Annual Conference on Neural Information Processing Systems. 22292239.Google ScholarGoogle Scholar
  36. [36] Martins Pedro Henrique, Marinho Zita, and Martins André F. T.. 2019. Joint learning of named entity recognition and entity linking. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 190196.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Min Sewon, Zhong Victor, Socher Richard, and Xiong Caiming. 2018. Efficient and robust question answering from minimal context over documents. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 17251735.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Nie Yuyang, Tian Yuanhe, Song Yan, Ao Xiang, and Wan Xiang. 2020. Improving named entity recognition with attentive ensemble of syntactic information. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. 42314245.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Nie Yuyang, Tian Yuanhe, Wan Xiang, Song Yan, and Dai Bo. 2020. Named entity recognition for social media texts with semantic augmentation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 13831391.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Peng Nanyun and Dredze Mark. 2015. Named entity recognition for Chinese social media with jointly trained embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 548554.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Peters Matthew E., Neumann Mark, Iyyer Mohit, Gardner Matt, Clark Christopher, Lee Kenton, and Zettlemoyer Luke. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Human Language Technologies. 22272237.Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Qin Ying, Zhang Taozheng, and Wang Xiaojie. 2008. Chinese named entity recognition with new contextual features. In Proceedings of the 4th International Conference on Natural Language Processing and Knowledge Engineering. 16.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Sarawagi Sunita and Cohen William W.. 2004. Semi-Markov conditional random fields for information extraction. In Proceedings of 2004 Annual Conference on Neural Information Processing Systems. 11851192.Google ScholarGoogle Scholar
  44. [44] Song Cijian, Xiong Yan, Huang Wenchao, and Ma Lu. 2020. Joint self-attention and multi-embeddings for Chinese named entity recognition. In Proceedings of 6th International Conference on Big Data Computing and Communications. 7680.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Strubell Emma, Verga Patrick, Belanger David, and McCallum Andrew. 2017. Fast and accurate entity recognition with iterated dilated convolutions. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 26702680.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Sui Dianbo, Chen Yubo, Liu Kang, Zhao Jun, and Liu Shengping. 2019. Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 38283838.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, and Polosukhin Illia. 2017. Attention is all you need. In Proceedings of 2017 Annual Conference on Neural Information Processing Systems. 59986008.Google ScholarGoogle Scholar
  48. [48] Wang Caiyu, Wang Hong, Zhuang Hui, Li Wei, Han Shu, Zhang Hui, and Zhuang Luhe. 2020. Chinese medical named entity recognition based on multi-granularity semantic dictionary and multimodal tree. Journal of Biomedical Informatics 111 (2020), 103583.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Wang Qi, Zhou Yangming, Ruan Tong, Gao Daqi, Xia Yuhang, and He Ping. 2019. Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition. Journal of Biomedical Informatics 92 (2019).Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Wang Rui, Xin Xin, Chang Wei, Ming Kun, Li Biao, and Fan Xin. 2019. Chinese NER with height-limited constituent parsing. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 71607167.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Weischedel Ralph, Pradhan Sameer, Ramshaw Lance, Palmer Martha, Xue Nianwen, Marcus Mitchell, Taylor Ann, Greenberg Craig, Hovy Eduard, Belvin Robert, and Houston Ann. 2011. Ontonotes Release 4.0. https://catalog.ldc.upenn.edu/docs/LDC2011T03/OntoNotes-Release-4.0.pdf.Google ScholarGoogle Scholar
  52. [52] Wu Shuang, Song Xiaoning, and Feng Zhen-Hua. 2021. MECT: Multi-metadata embedding based cross-transformer for Chinese named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 15291539.Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Xue Mengge, Cai Weiming, Su Jinsong, Song Linfeng, Ge Yubin, Liu Yubao, and Wang Bin. 2019. Neural collective entity linking based on recurrent random walk network learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 53275333.Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Xue Mengge, Yu Bowen, Liu Tingwen, Zhang Yue, Meng Erli, and Wang Bin. 2020. Porous lattice transformer encoder for Chinese NER. In Proceedings of the 28th International Conference on Computational Linguistics. 38313841.Google ScholarGoogle Scholar
  55. [55] Yan Chengxi, Su Qi, and Wang Jun. 2020. MoGCN: Mixture of gated convolutional neural network for named entity recognition of Chinese historical texts. IEEE Access 8 (2020), 181629181639.Google ScholarGoogle ScholarCross RefCross Ref
  56. [56] Yan Hang, Deng Bocao, Li Xiaonan, and Qiu Xipeng. 2019. TENER: Adapting transformer encoder for named entity recognition. CoRR abs/1911.04474 (2019).Google ScholarGoogle Scholar
  57. [57] Yan Shuo, Chai Jianping, and Wu Liyun. 2020. Bidirectional GRU with multi-head attention for Chinese NER. In Proceedings of 5th IEEE Information Technology and Mechatronics Engineering Conference. 11601164.Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] Zhang Naixin, Li Feng, Xu Guangluan, Zhang Wenkai, and Yu Hongfeng. 2019. Chinese NER using dynamic meta-embeddings. IEEE Access 7 (2019), 6445064459.Google ScholarGoogle ScholarCross RefCross Ref
  59. [59] Zhang Naixin, Xu Guangluan, Zhang Zequn, and Li Feng. 2019. MIFM: Multi-granularity information fusion model for Chinese named entity recognition. IEEE Access 7 (2019), 181648181655.Google ScholarGoogle ScholarCross RefCross Ref
  60. [60] Zhang Yue and Yang Jie. 2018. Chinese NER using lattice LSTM. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 15541564.Google ScholarGoogle ScholarCross RefCross Ref
  61. [61] Zhao Dandan, Cao Jingxiang, Huang Degen, Meng Jiana, and Zhang Pan. 2021. Dual neural network fusion model for Chinese named entity recognition. International Journal of Computational Intelligence Systems 14, 1 (2021), 471481.Google ScholarGoogle ScholarCross RefCross Ref
  62. [62] Zhu Yuying and Wang Guoxin. 2019. CAN-NER: Convolutional attention network for Chinese named entity recognition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 33843393.Google ScholarGoogle Scholar

Index Terms

  1. Token Relation Aware Chinese Named Entity Recognition

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 1
      January 2023
      340 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3572718
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 25 November 2022
      • Online AM: 29 April 2022
      • Accepted: 3 April 2022
      • Revised: 25 November 2021
      • Received: 22 February 2021
      Published in tallip Volume 22, Issue 1

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed
    • Article Metrics

      • Downloads (Last 12 months)378
      • Downloads (Last 6 weeks)29

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!