Abstract
Due to the lack of natural delimiters, most Chinese Named Entity Recognition (NER) approaches are character-based and utilize an external lexicon to leverage the word-level information. Although they have achieved promising results, the latent words they introduced are still non-contextualized. In this paper, we investigate three relations, i.e., adjacent relation between characters, character co-occurrence relation between latent words, and dependency relation among tokens, to address this issue. Specifically, we first establish the local context for latent words and then propose a masked self-attention mechanism to incorporate such local contextual information. Besides, since introducing external knowledge such as lexicon and dependency relation inevitably brings in some noises, we propose a gated information controller to handle this problem. Extensive experimental results show that the proposed approach surpasses most similar methods on public datasets and demonstrates its promising potential.
- [1] . 2017. Information retrieval as statistical translation. SIGIR Forum 51, 2 (2017), 219–226.Google Scholar
Digital Library
- [2] . 2015. Fast and space-efficient entity linking for queries. In Proceedings of the 8th ACM International Conference on Web Search and Data Mining. 179–188.Google Scholar
Digital Library
- [3] . 2019. A deep learning model incorporating part of speech and self-matching attention for named entity recognition of Chinese electronic medical records. BMC Medical Informatics & Decision Making 19-S, 2 (2019), 101–109.Google Scholar
- [4] . 2020. Joint learning with pre-trained transformer on named entity recognition and relation extraction tasks for clinical analytics. In Proceedings of the 3rd Clinical Natural Language Processing Workshop. 234–242.Google Scholar
Cross Ref
- [5] . 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 2493–2537.Google Scholar
Digital Library
- [6] . 2019. Pre-training with whole word masking for Chinese BERT. CoRR abs/1906.08101 (2019).Google Scholar
- [7] . 2019. Transformer-XL: Attentive language models beyond a fixed-length context. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 2978–2988.Google Scholar
Cross Ref
- [8] . 2018. D3NER: Biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information. Bioinformatics. 34, 20 (2018), 3539–3546.Google Scholar
Cross Ref
- [9] . 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171–4186.Google Scholar
- [10] . 2020. ZEN: Pre-training Chinese text encoder enhanced by N-gram representations. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. 4729–4740.Google Scholar
Cross Ref
- [11] . 2019. A neural multi-digraph model for Chinese NER with gazetteers. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 1462–1467.Google Scholar
Cross Ref
- [12] . 2015. Boosting named entity recognition with neural character embeddings. In Proceedings of the 5th Named Entity Workshop. 25–33.Google Scholar
Cross Ref
- [13] . 1996. Hidden Markov models. Current Opinion in Structural Biology 6, 3 (1996), 361–365.Google Scholar
Cross Ref
- [14] . 2021. An in-depth study on internal structure of Chinese words. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 5823–5833.Google Scholar
Cross Ref
- [15] . 2019. CNN-based Chinese NER with lexicon rethinking. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 4982–4988.Google Scholar
Cross Ref
- [16] . 2019. A lexicon-based graph neural network for Chinese NER. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 1040–1050.Google Scholar
Cross Ref
- [17] . 2019. Star-transformer. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Human Language Technologies. 1315–1325.Google Scholar
Cross Ref
- [18] . 2020. SLK-NER: Exploiting second-order lexicon knowledge for Chinese NER. In Proceedings of the 32nd International Conference on Software Engineering and Knowledge Engineering. 413–417.Google Scholar
- [19] . 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991 (2015).Google Scholar
- [20] . 2019. Dependency-guided LSTM-CRF for named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 3860–3870.Google Scholar
Cross Ref
- [21] . 2019. LSTM-CRF neural network with gated self attention for Chinese NER. IEEE Access 7 (2019), 136694–136703.Google Scholar
Cross Ref
- [22] . 2020. Query expansion using named entity disambiguation for a question-answering system. Concurrency and Computation: Practice and Experience 32, 4 (2020).Google Scholar
Cross Ref
- [23] . 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. 282–289.Google Scholar
Digital Library
- [24] . 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 260–270.Google Scholar
Cross Ref
- [25] . 2006. The third international Chinese language processing bakeoff: Word segmentation and named entity recognition. In Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing. 108–117.Google Scholar
- [26] . 2018. A survey on deep learning for named entity recognition. CoRR abs/1812.09449 (2018).Google Scholar
- [27] . 2020. Why attention? Analyze BiLSTM deficiency and its remedies in the case of NER. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. 8236–8244.Google Scholar
Cross Ref
- [28] . 2020. FLAT: Chinese NER using flat-lattice transformer. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6836–6842.Google Scholar
Cross Ref
- [29] . 2020. Towards Chinese clinical named entity recognition by dynamic embedding using domain-specific knowledge. Journal of Biomedical Informatics 106 (2020), 103435.Google Scholar
Cross Ref
- [30] . 2016. Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2124–2133.Google Scholar
Cross Ref
- [31] . 2021. Lexicon enhanced Chinese sequence labeling using BERT adapter. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 5847–5858.Google Scholar
Cross Ref
- [32] . 2019. An encoding strategy based word-character LSTM for Chinese NER. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2379–2389.Google Scholar
- [33] . 2010. Chinese named entity recognition with a sequence labeling approach: Based on characters, or based on words? In Proceedings of 6th International Conference on Intelligent Computing. 634–640.Google Scholar
Cross Ref
- [34] . 2020. Simplify the usage of lexicon in Chinese NER. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 5951–5960.Google Scholar
Cross Ref
- [35] . 2019. A tensorized transformer for language modeling. In Proceedings of 2019 Annual Conference on Neural Information Processing Systems. 2229–2239.Google Scholar
- [36] . 2019. Joint learning of named entity recognition and entity linking. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 190–196.Google Scholar
Cross Ref
- [37] . 2018. Efficient and robust question answering from minimal context over documents. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 1725–1735.Google Scholar
Cross Ref
- [38] . 2020. Improving named entity recognition with attentive ensemble of syntactic information. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. 4231–4245.Google Scholar
Cross Ref
- [39] . 2020. Named entity recognition for social media texts with semantic augmentation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. 1383–1391.Google Scholar
Cross Ref
- [40] . 2015. Named entity recognition for Chinese social media with jointly trained embeddings. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 548–554.Google Scholar
Cross Ref
- [41] . 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Human Language Technologies. 2227–2237.Google Scholar
Cross Ref
- [42] . 2008. Chinese named entity recognition with new contextual features. In Proceedings of the 4th International Conference on Natural Language Processing and Knowledge Engineering. 1–6.Google Scholar
Cross Ref
- [43] . 2004. Semi-Markov conditional random fields for information extraction. In Proceedings of 2004 Annual Conference on Neural Information Processing Systems. 1185–1192.Google Scholar
- [44] . 2020. Joint self-attention and multi-embeddings for Chinese named entity recognition. In Proceedings of 6th International Conference on Big Data Computing and Communications. 76–80.Google Scholar
Cross Ref
- [45] . 2017. Fast and accurate entity recognition with iterated dilated convolutions. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2670–2680.Google Scholar
Cross Ref
- [46] . 2019. Leverage lexical knowledge for Chinese named entity recognition via collaborative graph network. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 3828–3838.Google Scholar
Cross Ref
- [47] . 2017. Attention is all you need. In Proceedings of 2017 Annual Conference on Neural Information Processing Systems. 5998–6008.Google Scholar
- [48] . 2020. Chinese medical named entity recognition based on multi-granularity semantic dictionary and multimodal tree. Journal of Biomedical Informatics 111 (2020), 103583.Google Scholar
Digital Library
- [49] . 2019. Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition. Journal of Biomedical Informatics 92 (2019).Google Scholar
Digital Library
- [50] . 2019. Chinese NER with height-limited constituent parsing. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence. 7160–7167.Google Scholar
Digital Library
- [51] . 2011. Ontonotes Release 4.0. https://catalog.ldc.upenn.edu/docs/LDC2011T03/OntoNotes-Release-4.0.pdf.Google Scholar
- [52] . 2021. MECT: Multi-metadata embedding based cross-transformer for Chinese named entity recognition. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. 1529–1539.Google Scholar
Cross Ref
- [53] . 2019. Neural collective entity linking based on recurrent random walk network learning. In Proceedings of the 28th International Joint Conference on Artificial Intelligence. 5327–5333.Google Scholar
Cross Ref
- [54] . 2020. Porous lattice transformer encoder for Chinese NER. In Proceedings of the 28th International Conference on Computational Linguistics. 3831–3841.Google Scholar
- [55] . 2020. MoGCN: Mixture of gated convolutional neural network for named entity recognition of Chinese historical texts. IEEE Access 8 (2020), 181629–181639.Google Scholar
Cross Ref
- [56] . 2019. TENER: Adapting transformer encoder for named entity recognition. CoRR abs/1911.04474 (2019).Google Scholar
- [57] . 2020. Bidirectional GRU with multi-head attention for Chinese NER. In Proceedings of 5th IEEE Information Technology and Mechatronics Engineering Conference. 1160–1164.Google Scholar
Cross Ref
- [58] . 2019. Chinese NER using dynamic meta-embeddings. IEEE Access 7 (2019), 64450–64459.Google Scholar
Cross Ref
- [59] . 2019. MIFM: Multi-granularity information fusion model for Chinese named entity recognition. IEEE Access 7 (2019), 181648–181655.Google Scholar
Cross Ref
- [60] . 2018. Chinese NER using lattice LSTM. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 1554–1564.Google Scholar
Cross Ref
- [61] . 2021. Dual neural network fusion model for Chinese named entity recognition. International Journal of Computational Intelligence Systems 14, 1 (2021), 471–481.Google Scholar
Cross Ref
- [62] . 2019. CAN-NER: Convolutional attention network for Chinese named entity recognition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 3384–3393.Google Scholar
Index Terms
Token Relation Aware Chinese Named Entity Recognition
Recommendations
Attention in Character-Based BiLSTM-CRF for Chinese Named Entity Recognition
ICMAI 2019: Proceedings of the 2019 4th International Conference on Mathematics and Artificial IntelligenceNamed Entity Recognition (NER) is a crucial step in natural language processing (NLP). Recently, some researches work on enhancing the word representations by character-level extensions in English and have achieve excellent performance. The same method ...
Chinese named entity and relation identification system
COLING-ACL '06: Proceedings of the COLING/ACL on Interactive presentation sessionsIn this interactive presentation, a Chinese named entity and relation identification system is demonstrated. The domain-specific system has a three-stage pipeline architecture which includes word segmentation and part-of-speech (POS) tagging, named ...
Chinese Named Entity Recognition with Character-Word Mixed Embedding
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge ManagementNamed Entity Recognition (NER) is an important basis for the tasks in natural language processing such as relation extraction, entity linking and so on. The common method of existing Chinese NER systems is to use the character sequence as the input, and ...






Comments