Abstract
Attention mechanism is an increasingly important approach in the field of natural language processing (NLP). In the attention-based named entity recognition (NER) model, most attention mechanisms can calculate attention coefficient to express the importance of sentence semantic information but cannot adjust the position distribution of contextual feature vectors in the semantic space. To address this issue, a radial basis function attention (RBF-attention) layer is proposed to adaptively regulate the position distribution of sequence contextual feature vectors, which can minimize the relative distance of within-category named entities and maximize the relative distance of between-category named entities in the semantic space. The experimental results on CoNLL2003 English and MSRA Chinese NER datasets indicate that the proposed model performs better than other baseline approaches without relying on any external feature engineering.
- [1] . 2007. A survey of named entity recognition and classification. In Lingvisticae Investigationes. 30, 1 (2007), 3–26.Google Scholar
Cross Ref
- [2] . 2013. Textual inference and meaning representation in human robot interaction. In Joint Symposium on Semantic Processing Textual Inference and Structures in Corpora. (2013), 65–69.Google Scholar
- [3] . 2020. Quantifying the significance of cybersecurity text through semantic similarity and named entity recognition. In 6th International Conference on Information Systems Security and Privacy (ICISP’20). 325–332.Google Scholar
Cross Ref
- [4] . 2014. Information extraction over structured data: Question answering with freebase. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL’14). 956–966.Google Scholar
Cross Ref
- [5] , 2016. CharNER: Character-level named entity recognition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers (COLING’16). 911–921.Google Scholar
- [6] . 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 260–270.Google Scholar
Cross Ref
- [7] . 2018. Chinese name entity recognition using highway-LSTM-CRF. In Proceedings of the 2018 International Conference on Algorithms, Computing and Artificial Intelligence (ACAI’2018). Association for Computing Machinery, 1–5.
DOI: Google ScholarDigital Library
- [8] . 2016. Named entity recognition with bidirectional LSTM-CNNs. In Transactions of the Association for Computational Linguistics (2016), 4, 2016, 357–370.Google Scholar
Cross Ref
- [9] . 2018. Deep semantic role labeling with self-attention. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI'18). 4929–4936.Google Scholar
Cross Ref
- [10] . 2021. Position-aware self-attention based neural sequence labeling. Pattern Recognition 110, 1 (2021), 107636.Google Scholar
Cross Ref
- [11] . 2020. Joint model of entity recognition and relation extraction with self-attention mechanism. ACM Trans. Asian Low-Resour. Lang. (2020).
DOI: Google ScholarDigital Library
- [12] . 2020. Domain generalization for named entity boundary detection via meta-learning. In IEEE Transactions on Neural Networks and Learning Systems 32, 9 (2020), 3819–3830.Google Scholar
Cross Ref
- [13] . 2018. Module overlapping structure detection in PPI using an improved link similarity-based Markov clustering algorithm. In Neural Computing & Applications. 31, 5 (2019), 1481–1490.Google Scholar
Digital Library
- [14] . 2020. SVM ensembles for named entity disambiguation. In Computing. 102, 4 (2020), 1051–1076.Google Scholar
Cross Ref
- [15] . 2016. Multi-prototype Chinese character embedding. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16). 855–859.Google Scholar
- [16] . 2016. Long short-term memory-networks for machine reading. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP’16). 551–561.Google Scholar
Cross Ref
- [17] . 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17). 5999–6009.Google Scholar
- [18] . 2017. A structured self-attentive sentence embedding. In Proceedings of the 6th International Conference on Learning Representations (ICLR’17).Google Scholar
- [19] . 2018. DiSAN: Directional Self-attention network for RNN/CNN-free language understanding. In Proceeding of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18). 5446–5455.Google Scholar
Cross Ref
- [20] . 2017. NeuPL: Attention-based semantic matching and pair-linking for entity disambiguation. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management (CIKM’17). 1667–1676.Google Scholar
Digital Library
- [21] . 2018. Empower sequence labeling with task-aware neural language model. In Proceeding of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18).Google Scholar
Cross Ref
- [22] . 2017. Semi-supervised sequence tagging with bidirectional language models. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17).Google Scholar
Cross Ref
- [23] . 2020. ASTRAL: Adversarial trained LSTM-CNN for named entity recognition. Knowledge-Based Systems. (2020) 105842.Google Scholar
Cross Ref
- [24] . 2021. Sequence labeling with meta-learning. In IEEE Transactions on Knowledge and Data Engineering.Google Scholar
Cross Ref
- [25] . 2016. Character-based LSTM-CRF with radical-level features for Chinese named entity recognition. In Natural Language Understanding and Intelligent Applications (2016). 239–250.Google Scholar
Cross Ref
- [26] . 2019. Exploiting multiple embeddings for Chinese named entity recognition. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM’19). 2269–2272.Google Scholar
Digital Library
- [27] . 2020. Few-shot named entity recognition via meta-learning. In IEEE Transactions on Knowledge and Data Engineering.Google Scholar
Cross Ref
- [28] . 2019. A radical-based method for Chinese named entity recognition. In Proceedings of the 2nd International Conference on Big Data Technologies (ICBDT2019). Association for Computing Machinery, 125–130.
DOI: Google ScholarDigital Library
- [29] . 2020. RBF-Softmax: Learning deep representative prototypes with radial basis function softmax. In Vedaldi A., Bischof H., Brox T., Frahm J. M. (eds). Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science, vol 12371. Springer, Cham. Google Scholar
Digital Library
- [30] . 2018. Enhancement of the Word2vec class-based language modeling by optimizing the features vector using PCA. IEEE International Conference on Electro/Information Technology.Google Scholar
- [31] . 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (NIPS’13). 3111–3119.Google Scholar
- [32] . 2014. Face recognition using adaptive margin Fisher's criterion and linear discriminant analysis. International Arab Journal of Information Technology 11, 2 (2014), 1–11.Google Scholar
- [33] . 2019. Chinese named entity recognition in power domain based on Bi-LSTM-CRF. In Proceedings of the 2nd International Conference on Artificial Intelligence and Pattern Recognition (AIPR’19). Association for Computing Machinery, 176–180.
DOI: Google ScholarDigital Library
- [34] . 2016. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 1064–1074.Google Scholar
Cross Ref
- [35] . 2018. Deep affix features improve neural named entity recognizers. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics. 167–172.Google Scholar
Cross Ref
- [36] . 2020. Adversarial named entity recognition with POS label embedding. In 2020 International Joint Conference on Neural Networks (IJCNN'20). IEEE, 1–8.Google Scholar
Cross Ref
- [37] . 2020. Delayed combination of feature embedding in bidirectional LSTM CRF for NER. In Applied Sciences (2020).Google Scholar
Cross Ref
- [38] . 2020. Computationally efficient NER taggers with combined embeddings and constrained decoding. (2020). arXiv preprint arXiv:2001.01167v1.Google Scholar
- [39] . 2018. Adversarial transfer learning for Chinese named entity recognition with self-attention mechanism. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP’18). 182–192.Google Scholar
Cross Ref
- [40] . 2019. Chinese NER using dynamic meta-e mbeddings. In IEEE Access 7 (2019), 64450–64459.Google Scholar
Cross Ref
- [41] . 2017. Named entity recognition with gated convolutional neural networks. In Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. 110–121.Google Scholar
Cross Ref
- [42] . 2019. CAN-NER: Convolutional attention network for Chinese named entity recognition. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, (NAACL-HLT'19).Google Scholar
Index Terms
Radial Basis Function Attention for Named Entity Recognition
Recommendations
A named entity recognition method towards product reviews based on BiLSTM-attention-CRF
Named entity recognition (NER) towards product review intends to identify domain dependent named entities (e.g., organisation name, product name, etc.) from product reviews. Due to the fragmentation and non-construction of product reviews, traditional ...
A parallel computing-based Deep Attention model for named entity recognition
AbstractNamed entity recognition (NER) is an important task in natural language processing and has been widely studied. In recent years, end-to-end NER with bidirectional long short-term memory (BiLSTM) has received more and more attention. However, it ...
A Comparative Study of Dictionary-based and Machine Learning-based Named Entity Recognition in Pashto
NLPIR '20: Proceedings of the 4th International Conference on Natural Language Processing and Information RetrievalInformation Extraction (IE) is the process of extracting structured information from unstructured text using natural language processing (NLP). One important sub-task of IE is the extraction of names of persons, places, and organizations, called Named ...






Comments