Abstract
Chinese Named Entity Recognition (NER) is an essential task in natural language processing, and its performance directly impacts the downstream tasks. The main challenges in Chinese NER are the high dependence of named entities on context and the lack of word boundary information. Therefore, how to integrate relevant knowledge into the corresponding entity has become the primary task for Chinese NER. Both the lattice LSTM model and the WC-LSTM model did not make excellent use of contextual information. Additionally, the lattice LSTM model had a complex structure and did not exploit the word information well. To address the preceding problems, we propose a Chinese NER method based on the deep neural network with multiple ways of embedding fusion. First, we use a convolutional neural network to combine the contextual information of the input sequence and apply a self-attention mechanism to integrate lexicon knowledge, compensating for the lack of word boundaries. The word feature, context feature, bigram feature, and bigram context feature are obtained for each character. Second, four different features are used to fuse information at the embedding layer. As a result, four different word embeddings are obtained through cascading. Last, the fused feature information is input to the encoding and decoding layer. Experiments on three datasets show that our model can effectively improve the performance of Chinese NER.
- [1] . 2018. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 182–192.Google Scholar
Cross Ref
- [2] . 2015. Long short-term memory neural networks for Chinese word segmentation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1197–1206.Google Scholar
Cross Ref
- [3] . 2021. Lightner: A lightweight generative framework with prompt-guided attention for low-resource NER. arXiv preprint arXiv:2109.00720 (2021).Google Scholar
- [4] . 2015. Event extraction via dynamic multi-pooling convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 167–176.Google Scholar
Cross Ref
- [5] . 2016. Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics 4 (2016), 357–370.Google Scholar
Cross Ref
- [6] . 2021. Template-based named entity recognition using BART. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Association for Computational Linguistics, 1835–1845.Google Scholar
- [7] . 2018. Core techniques of question answering systems over knowledge bases: A survey. Knowledge and Information Systems 55, 3 (2018), 529–569.Google Scholar
Digital Library
- [8] . 2019. A neural multi-digraph model for Chinese NER with gazetteers. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1462–1467.Google Scholar
Cross Ref
- [9] . 2016. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition. In Natural Language Understanding and Intelligent Applications. Springer, 239–250.Google Scholar
Cross Ref
- [10] . 1973. The Viterbi algorithm. Proceedings of the IEEE 61, 3 (1973), 268–278.Google Scholar
Cross Ref
- [11] . 2022. TURNER: The uncertainty-based retrieval framework for Chinese NER. arXiv preprint arXiv:2202.09022 (2022).Google Scholar
- [12] . 2019. CNN-based Chinese NER with lexicon rethinking. In Proceedings of the 28th International Joint Conference on Artificial Intelligence: Main Track. 4982–4988.Google Scholar
- [13] . 2019. A lexicon-based graph neural network for Chinese NER. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 1040–1050.Google Scholar
Cross Ref
- [14] . 2016. F-score driven max margin neural network for named entity recognition in Chinese social media. arXiv preprint arXiv:1611.04234 (2016).Google Scholar
- [15] . 2017. A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In Proceedings of the 31st AAAI Conference on Artificial Intelligence.Google Scholar
Cross Ref
- [16] . 2015. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015).Google Scholar
- [17] . 2020. Entity enhanced BERT pre-training for Chinese NER. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP’20). 6384–6396.Google Scholar
Cross Ref
- [18] . 2022. Adaptive threshold selective self-attention for Chinese NER. In Proceedings of the 29th International Conference on Computational Linguistics, 1823–1833.Google Scholar
- [19] . 2020. Incorporating uncertain segmentation information into Chinese NER for social media text. arXiv preprint arXiv:2004.06384 (2020).Google Scholar
- [20] . 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’19). 4171–4186.Google Scholar
- [21] . 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (ICML’01).Google Scholar
- [22] . 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 260–270.Google Scholar
Cross Ref
- [23] . 2006. The Third International Chinese Language Processing Bakeoff: Word segmentation and named entity recognition. In Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing. 108–117.Google Scholar
- [24] . 2020. Character-level neural network model based on Nadam optimization and its application in clinical concept extraction. Neurocomputing 414 (2020), 182–190.Google Scholar
Cross Ref
- [25] . 2020. FLAT: Chinese NER using flat-lattice transformer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6836–6842.Google Scholar
Cross Ref
- [26] . 2020. Chinese clinical named entity recognition with variant neural structures based on BERT methods. Journal of Biomedical Informatics 107 (2020), 103422.Google Scholar
Cross Ref
- [27] . 2021. Noisy-labeled NER with confidence estimation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’21). 3437–3445.Google Scholar
Cross Ref
- [28] . 2022. Chinese named entity recognition: The state of the art. Neurocomputing 473 (2022), 37–53.Google Scholar
Digital Library
- [29] . 2019. An encoding strategy based word-character LSTM for Chinese NER. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2379–2389.Google Scholar
- [30] . 2016. Multi-prototype Chinese character embedding. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC’16). 855–859.Google Scholar
- [31] . 2019. Simplify the usage of lexicon in Chinese NER. arXiv preprint arXiv:1908.05969 (2019).Google Scholar
- [32] . 2016. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1064–1074.Google Scholar
Cross Ref
- [33] . 2020. Porous lattice transformer encoder for Chinese NER. In Proceedings of the 28th International Conference on Computational Linguistics. 3831–3841.Google Scholar
Cross Ref
- [34] . 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. arXiv preprint arXiv:1601.00770 (2016).Google Scholar
- [35] . 2015. Named entity recognition for Chinese social media with jointly trained embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 548–554.Google Scholar
Cross Ref
- [36] . 2016. Improving named entity recognition for Chinese social media with word segmentation representation learning. arXiv preprint arXiv:1603.00786 (2016).Google Scholar
- [37] . 2012. CoNLL-2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Proceedings of the Joint Conference on EMNLP and CoNLL-Shared Task (CoNLL’12). 1–40.Google Scholar
- [38] . 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL 2003. 142–147.Google Scholar
Digital Library
- [39] . 2010. Conditional random field based named entity recognition in geological text. International Journal of Computer Applications 1, 3 (2010), 143–147.Google Scholar
Cross Ref
- [40] . 2017. Fast and accurate entity recognition with iterated dilated convolutions. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2670–2680.Google Scholar
Cross Ref
- [41] . 2019. Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 3830–3840.Google Scholar
Cross Ref
- [42] . 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30.Google Scholar
- [43] . 2019. Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation. In Proceedings of the World Wide Web Conference. 3342–3348.Google Scholar
Digital Library
- [44] . 2022. Diverse dialogue generation by fusing mutual persona-aware and self-transferrer. Applied Intelligence 52, 5 (2022), 4744–4757.Google Scholar
Digital Library
- [45] . 2021. A unified generative framework for various NER subtasks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 5808–5822.Google Scholar
Cross Ref
- [46] . 2016. Combining discrete and neural features for sequence labeling. In Proceedings of the International Conference on Intelligent Text Processing and Computational Linguistics. 140–154.Google Scholar
- [47] . 2018. NCRF++: An open-source neural sequence labeling toolkit. In Proceedings of ACL 2018: System Demonstration.74.Google Scholar
Cross Ref
- [48] . 2018. Chinese NER using lattice LSTM. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1554–1564.Google Scholar
Cross Ref
- [49] . 2021. Dynamic modeling cross-and self-lattice attention network for Chinese NER. In Proceedings of the 35th AAAI Conference on Artificial Intelligence. 14515–14523.Google Scholar
Cross Ref
- [50] . 2002. Named entity recognition using an HMM-based chunk tagger. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 473–480.Google Scholar
Digital Library
- [51] . 2019. CAN-NER: Convolutional attention network for Chinese named entity recognition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 3384–3393.Google Scholar
Index Terms
Deep Neural Network with Embedding Fusion for Chinese Named Entity Recognition
Recommendations
Chinese Named Entity Recognition with CRFs: Two Levels
CIS '08: Proceedings of the 2008 International Conference on Computational Intelligence and Security - Volume 02Named Entity Recognition (NER) is one of the key techniques in natural language processing tasks such as information extraction, text summarization and so on. Chinese NER is more complicated and difficult than other languages because of its ...
TFM: A Triple Fusion Module for Integrating Lexicon Information in Chinese Named Entity Recognition
AbstractDue to the characteristics of the Chinese writing system, character-based Chinese named entity recognition models ignore the word information in sentences, which harms their performance. Recently, many works try to alleviate the problem by ...
Named entity recognition for Chinese marine text with knowledge-based self-attention
AbstractChinese named entity recognition has been widely used in many fields, such as species recognition in marine information, and so on. Compared with the standard named entity recognition (NER), the performance of the Chinese marine named entity ...






Comments