skip to main content
research-article

Joined Type Length Encoding for Nested Named Entity Recognition

Authors Info & Claims
Published:13 December 2021Publication History
Skip Abstract Section

Abstract

In this article, we propose a new encoding scheme for named entity recognition (NER) called Joined Type-Length encoding (JoinedTL). Unlike most existing named entity encoding schemes, which focus on flat entities, JoinedTL can label nested named entities in a single sequence. JoinedTL uses a packed encoding to represent both type and span of a named entity, which not only results in less tagged tokens compared to existing encoding schemes, but also enables it to support nested NER. We evaluate the effectiveness of JoinedTL for nested NER on three nested NER datasets: GENIA in English, GermEval in German, and PerNest, our newly created nested NER dataset in Persian. We apply CharLSTM+WordLSTM+CRF, a three-layer sequence tagging model on three datasets encoded using JoinedTL and two existing nested NE encoding schemes, i.e., JoinedBIO and JoinedBILOU. Our experiment results show that CharLSTM+WordLSTM+CRF trained with JoinedTL encoded datasets can achieve competitive F1 scores as the ones trained with datasets encoded by two other encodings, but with 27%–48% less tagged tokens. To leverage the power of three different encodings, i.e., JoinedTL, JoinedBIO, and JoinedBILOU, we propose an encoding-based ensemble method for nested NER. Evaluation results show that the ensemble method achieves higher F1 scores on all datasets than the three models each trained using one of the three encodings. By using nested NE encodings including JoinedTL with CharLSTM+WordLSTM+CRF, we establish new state-of-the-art performance with an F1 score of 83.7 on PerNest, 74.9 on GENIA, and 70.5 on GermEval, surpassing two recent neural models specially designed for nested NER.

REFERENCES

  1. [1] Ahmad Muhammad Tayyab, Malik Muhammad Kamran, Shahzad Khurram, Aslam Faisal, Iqbal Asif, Nawaz Zubair, and Bukhari Faisal. 2020. Named entity recognition and classification for Punjabi shahmukhi. ACM Transactions on Asian and Low-Resource Language Information Processing 19, 4 (2020), 113. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Alex Beatrice, Haddow Barry, and Grover Claire. 2007. Recognising nested named entities in biomedical text. In Proceedings of the Biological, Translational, and Clinical Language Processing. Association for Computational Linguistics, 6572. Retrieved from https://www.aclweb.org/anthology/W07-1009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Benikova Darina, Biemann Chris, Kisselew Max, and Pado Sebastian. 2014. GermEval 2014 named entity recognition shared task: Companion paper. Retrieved 25 Nov., 2021. from https://hildok.bsz-bw.de/frontdoor/index/index/docId/283.Google ScholarGoogle Scholar
  4. [4] Benikova Darina, Biemann Chris, and Reznicek Marc. 2014. NoSta-D named entity annotation for German: Guidelines and dataset. In Proceedings of the 9th International Conference on Language Resources and Evaluation. European Language Resources Association, 25242531. Retrieved from http://www.lrec-conf.org/proceedings/lrec2014/pdf/276_Paper.pdf.Google ScholarGoogle Scholar
  5. [5] Bijankhan Mahmood, Sheykhzadegan Javad, Bahrani Mohammad, and Ghayoomi Masood. 2011. Lessons from building a Persian written corpus: Peykare. Language Resources and Evaluation 45, 2 (2011), 143164. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Borthwick Andrew Eliot. 1999. A Maximum Entropy Approach to Named Entity Recognition. Ph.D. Dissertation. New York University, New York, NY. UMI Order Number:AAI 9945252.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Chiu Jason P. C. and Nichols Eric. 2016. Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics 4 (2016), 357370. DOI: https://doi.org/10.1162/tacl_a_00104Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Das Arjun, Ganguly Debasis, and Garain Utpal. 2017. Named entity recognition with word embeddings and wikipedia categories for a low-resource language. ACM Transactions on Asian and Low-Resource Language Information Processing 16, 3 (2017), 119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Finkel Jenny Rose and Manning Christopher D.. 2009. Nested named entity recognition. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 141150. Retrieved from https://www.aclweb.org/anthology/D09-1015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Fisher Joseph and Vlachos Andreas. 2019. Merge and label: A novel neural network architecture for nested NER. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 58405850. DOI: https://doi.org/10.18653/v1/P19-1585.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] Hosseinnejad Shadi, Shekofteh Yasser, and Azadi Tahereh and Emami. 2017. A’laam corpus: A standard corpus of named entity for persian language. Signal and Data Processing 14, 3 (Dec 2017), 127142. DOI: https://doi.org/10.29252/jsdp.14.3.127arXiv:http://jsdp.rcisp.ac.ir/article-1-477-en.pdf.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Huang Zhiheng, Xu Wei, and Yu Kai. 2015. Bidirectional LSTM-CRF Models for Sequence Tagging. CoRR abs/1508.01991. http://arxiv.org/abs/1508.01991.Google ScholarGoogle Scholar
  13. [13] Ju Meizhi, Miwa Makoto, and Ananiadou Sophia. 2018. A neural layered model for nested named entity recognition. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1. Association for Computational Linguistics, 14461459. DOI: https://doi.org/10.18653/v1/N18-1131Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Kanwal Safia, Malik Kamran, Shahzad Khurram, Aslam Faisal, and Nawaz Zubair. 2019. Urdu named entity recognition: Corpus generation and deep learning applications. ACM Transactions on Asian and Low-Resource Language Information Processing 19, 1 (June 2019), Article 8, 13 pages. DOI: https://doi.org/10.1145/3329710 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Katiyar Arzoo and Cardie Claire. 2018. Nested named entity recognition revisited. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol. 1. Association for Computational Linguistics, 861871. DOI: https://doi.org/10.18653/v1/N18-1079Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Kim J.-D., Ohta T., Tateisi Y., and Tsujii J.. 2003. GENIA corpus’a semantically annotated corpus for bio-textmining. Bioinformatics 19, suppl_1 (July 2003), i180–i182. DOI: https://doi.org/10.1093/bioinformatics/btg1023arXiv:https://academic.oup.com/bioinformatics/article-pdf/19/suppl_1/i180/614820/btg1023.pdf.Google ScholarGoogle Scholar
  17. [17] Kudo Taku and Matsumoto Yuji. 2001. Chunking with support vector machines. In Proceedings of the 2nd Meeting of the North American Chapter of the Association for Computational Linguistics. Retrieved from https://www.aclweb.org/anthology/N01-1025. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Lample Guillaume, Ballesteros Miguel, Subramanian Sandeep, Kawakami Kazuya, and Dyer Chris. 2016. Neural architectures for named entity recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 260270. DOI: https://doi.org/10.18653/v1/N16-1030Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Li Jing, Sun Aixin, Han Jianglei, and Li Chenliang. 2020. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering PP, 99 (2020), 1–1.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Li Xiaoya, Feng Jingrong, Meng Yuxian, Han Qinghong, Wu Fei, and Li Jiwei. 2019. A unified MRC framework for named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 58495859.Google ScholarGoogle Scholar
  21. [21] Liu Liyuan, Shang Jingbo, Ren Xiang, Xu Frank Fangzheng, Gui Huan, Peng Jian, and Han Jiawei. 2018. Empower sequence labeling with task-aware neural language model. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Lu Wei and Roth Dan. 2015. Joint mention extraction and classification with mention hypergraphs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 857867.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Luo Ying and Zhao Hai. 2020. Bipartite flat-graph network for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Jurafsky Dan, Chai Joyce, Schluter Natalie, and Tetreault Joel R. (Eds.). Association for Computational Linguistics, 64086418. DOI: https://doi.org/10.18653/v1/2020.acl-main.571Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Malik Muhammad Kamran. 2017. Urdu named entity recognition and classification system using artificial neural network. ACM Transactions on Asian and Low-Resource Language Information Processing 17, 1 (2017), 113. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Martí Maria Antònia, Taulé Mariona, Bertran Manu, and Màrquez Lluís. 2007. Ancora: Multilingual and multilevel annotated corpora. MS, Universitat de Barcelona (2007).Google ScholarGoogle Scholar
  26. [26] McHugh Mary L. 2012. Interrater reliability: The kappa statistic. Biochemia Medica: Biochemia Medica 22, 3 (2012), 276282.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Mollá Diego, Zaanen Menno van, and Smith Daniel. 2006. Named entity recognition for question answering. In Proceedings of the Australasian Language Technology Workshop 2006, 5158. Retrieved from https://www.aclweb.org/anthology/U06-1009.Google ScholarGoogle Scholar
  28. [28] Muis Aldrian Obaja and Lu Wei. 2017. Labeling gaps between words: Recognizing overlapping mentions with mention separators. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 26082618. DOI: https://doi.org/10.18653/v1/D17-1276Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Nguyen Binh An, Nguyen Kiet Van, and Nguyen Ngan Luu-Thuy. 2019. Error Analysis for Vietnamese Named Entity Recognition on Deep Neural Network Models. CoRR abs/1911.07228. http://arxiv.org/abs/1911.07228.Google ScholarGoogle Scholar
  30. [30] Poostchi Hanieh, Borzeshi Ehsan Zare, and Piccardi Massimo. 2018. BiLSTM-CRF for persian named-entity recognition ArmanPersoNERCorpus: The first entity-annotated persian dataset. In Proceedings of the 11th International Conference on Language Resources and Evaluation. European Language Resources Association. Retrieved from https://www.aclweb.org/anthology/L18-1701.Google ScholarGoogle Scholar
  31. [31] Prokofyev Roman, Tonon Alberto, Luggen Michael, Vouilloz Loic, Difallah Djellel Eddine, and Cudré-Mauroux Philippe. 2015. SANAPHOR: Ontology-based coreference resolution. In Proceedings of the Semantic Web - ISWC 2015. Arenas Marcelo, Corcho Oscar, Simperl Elena, Strohmaier Markus, d’Aquin Mathieu, Srinivas Kavitha, Groth Paul, Dumontier Michel, Heflin Jeff, Thirunarayan Krishnaprasad, Thirunarayan Krishnaprasad, and Staab Steffen (Eds.). Springer International Publishing, Cham, 458473. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Ramshaw Lance and Marcus Mitch. 1995. Text chunking using transformation-based learning. In Proceedings of the 3rd Workshop on Very Large Corpora. Retrieved from https://www.aclweb.org/anthology/W95-0107.Google ScholarGoogle Scholar
  33. [33] Ratinov Lev and Roth Dan. 2009. Design challenges and misconceptions in named entity recognition. In Proceedings of the 13th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 147155. Retrieved from https://www.aclweb.org/anthology/W09-1119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Ratnaparkhi Adwait and Marcus Mitchell P.. 1998. Maximum Entropy Models for Natural Language Ambiguity Resolution. Ph.D. Dissertation. UMI Order Number: AAI 9840230. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Rei Marek. 2017. Semi-supervised multitask learning for sequence labeling. CoRR abs/1704.07156.Google ScholarGoogle Scholar
  36. [36] Sang Erik F. Tjong Kim and Veenstra Jorn. 1999. Representing text chunks. In Proceedings of the 9th Conference on European Chapter of the Association for Computational Linguistics . Association for Computational Linguistics, 173179. DOI: https://doi.org/10.3115/977035.977059 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Shahshahani Mahsa Sadat, Mohseni Mahdi, Shakery Azadeh, and Faili Heshaam. 2018. PEYMA: A tagged corpus for Persian named entities. CoRR.Google ScholarGoogle Scholar
  38. [38] Shen Hong and Sarkar Anoop. 2005. Voting between multiple data representations for text chunking. In Proceedings of the 18th Canadian Society Conference on Advances in Artificial Intelligence. Springer-Verlag, Berlin, 389400. DOI: https://doi.org/10.1007/11424918_40 Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Shibuya Takashi and Hovy Eduard. 2020. Nested named entity recognition via second-best sequence learning and decoding. Transactions of the Association for Computational Linguistics 8 (2020), 605620.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Sohrab Mohammad Golam and Miwa Makoto. 2018. Deep exhaustive model for nested named entity recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 28432849. DOI: https://doi.org/10.18653/v1/D18-1309Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Stenetorp Pontus, Pyysalo Sampo, Topić Goran, Ohta Tomoko, Ananiadou Sophia, and Tsujii Jun’ichi. 2012. BRAT: A web-based tool for NLP-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 102107. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. [42] Straková Jana, Straka Milan, and Hajic Jan. 2019. Neural architectures for nested NER through linearization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 53265331. DOI: https://doi.org/10.18653/v1/P19-1527Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Tang Buzhou, Wu Yonghui, Jiang M., Denny Joshua, and Xu H.. 2013. Recognizing and encoding disorder concepts in clinical text using machine learning and vector space model. CEUR Workshop Proceedings 1179 (Jan. 2013).Google ScholarGoogle Scholar
  44. [44] Sang Erik F. Tjong Kim and Meulder Fien De. 2003. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition. In Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL 2003. 142147. Retrieved from https://www.aclweb.org/anthology/W03-0419. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Wang Bailin and Lu Wei. 2018. Neural segmental hypergraphs for overlapping mention recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 204214. DOI: https://doi.org/10.18653/v1/D18-1019Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Wang Bailin, Lu Wei, Wang Yu, and Jin Hongxia. 2018. A neural transition-based model for nested mention recognition. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 10111017. DOI: https://doi.org/10.18653/v1/D18-1124Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Wang Jue, Shou Lidan, Chen Ke, and Chen Gang. 2020. Pyramid: A layered model for nested named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 59185928. DOI: https://doi.org/10.18653/v1/2020.acl-main.525Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Wang Yu, Li Yun, Tong Hanghang, and Zhu Ziye. 2020. HIT: Nested named entity recognition via head-tail pair and token interaction. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 60276036. DOI: https://doi.org/10.18653/v1/2020.emnlp-main.486Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] Weston L., Tshitoyan V., Dagdelen J., Kononova O., Trewartha A., Persson K. A., Ceder G., and Jain A.. 2019. Named entity recognition and normalization applied to large-scale information extraction from the materials science literature. Journal of Chemical Information and Modeling 59, 9 (2019), 36923702. DOI: https://doi.org/10.1021/acs.jcim.9b00470arXiv:https://doi.org/10.1021/acs.jcim.9b00470Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Yadav Vikas, Sharp Rebecca, and Bethard Steven. 2018. Deep affix features improve neural named entity recognizers. In Proceedings of the 7th Joint Conference on Lexical and Computational Semantics. Association for Computational Linguistics, 167172. DOI: https://doi.org/10.18653/v1/S18-2021Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Yang Jie, Liang Shuailong, and Zhang Yue. 2018. Design challenges and misconceptions in neural sequence labeling. In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, 38793889. Retrieved from https://www.aclweb.org/anthology/C18-1327.Google ScholarGoogle Scholar
  52. [52] Yang Jie and Zhang Yue. 2018. NCRF++: An open-source neural sequence labeling toolkit. In Proceedings of the ACL 2018, System Demonstrations. Association for Computational Linguistics, 7479. DOI: https://doi.org/10.18653/v1/P18-4013Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Yu Juntao, Bohnet Bernd, and Poesio Massimo. 2020. Named entity recognition as dependency parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 64706476. DOI: https://doi.org/10.18653/v1/2020.acl-main.577Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Zheng Changmeng, Cai Yi, Xu Jingyun, Leung Ho-fung, and Xu Guandong. 2019. A boundary-aware neural model for nested named entity recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, 357366. DOI: https://doi.org/10.18653/v1/D19-1034Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Joined Type Length Encoding for Nested Named Entity Recognition

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Asian and Low-Resource Language Information Processing
          ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 21, Issue 3
          May 2022
          413 pages
          ISSN:2375-4699
          EISSN:2375-4702
          DOI:10.1145/3505182
          Issue’s Table of Contents

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 13 December 2021
          • Accepted: 1 August 2021
          • Revised: 1 July 2021
          • Received: 1 May 2020
          Published in tallip Volume 21, Issue 3

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed
        • Article Metrics

          • Downloads (Last 12 months)80
          • Downloads (Last 6 weeks)3

          Other Metrics

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        View Full Text

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!