skip to main content
research-article
Public Access

Biomedical Named Entity Recognition via Knowledge Guidance and Question Answering

Authors Info & Claims
Published:18 July 2021Publication History
Skip Abstract Section

Abstract

In this work, we formulated the named entity recognition (NER) task as a multi-answer knowledge guided question-answer task (KGQA) and showed that the knowledge guidance helps to achieve state-of-the-art results for 11 of 18 biomedical NER datasets. We prepended five different knowledge contexts—entity types, questions, definitions, and examples—to the input text and trained and tested BERT-based neural models on such input sequences from a combined dataset of the 18 different datasets. This novel formulation of the task (a) improved named entity recognition and illustrated the impact of different knowledge contexts, (b) reduced system confusion by limiting prediction to a single entity-class for each input token (i.e., B, I, O only) compared to multiple entity-classes in traditional NER (i.e., Bentity1, Bentity2, Ientity1, I, O), (c) made detection of nested entities easier, and (d) enabled the models to jointly learn NER-specific features from a large number of datasets. We performed extensive experiments of this KGQA formulation on the biomedical datasets, and through the experiments, we showed when knowledge improved named entity recognition. We analyzed the effect of the task formulation, the impact of the different knowledge contexts, the multi-task aspect of the generic format, and the generalization ability of KGQA. We also probed the model to better understand the key contributors for these improvements.

References

  1. Asma Ben Abacha, Alba G. Seco de Herrera, Ke Wang, L. Rodney Long, Sameer Antani, and Dina Demner-Fushman. 2017. Named entity recognition in functional neuroimaging literature. In Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine (BIBM’17). IEEE, 2218–2220.Google ScholarGoogle Scholar
  2. Muhammad Amith, Licong Cui, Kirk Roberts, and Cui Tao. 2020. Towards an ontology-based medication conversational agent for PrEP and PEP. In Proceedings of the 1st Workshop on Natural Language Processing for Medical Conversations. Association for Computational Linguistics, 31–40. DOI:https://doi.org/10.18653/v1/2020.nlpmc-1.5Google ScholarGoogle Scholar
  3. Michael Bada, Miriam Eckert, Donald Evans, Kristin Garcia, Krista Shipley, Dmitry Sitnikov, William A. Baumgartner, K. Bretonnel Cohen, Karin Verspoor, Judith A. Blake, et al. 2012. Concept annotation in the CRAFT corpus. BMC Bioinform. 13, 1 (2012), 161.Google ScholarGoogle ScholarCross RefCross Ref
  4. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A pretrained language model for scientific text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). Association for Computational Linguistics, 3615–3620. DOI:https://doi.org/10.18653/v1/D19-1371Google ScholarGoogle Scholar
  5. Steven Bethard, Guergana Savova, Martha Palmer, and James Pustejovsky. 2017. SemEval-2017 Task 12: Clinical TempEval. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval’17). Association for Computational Linguistics, 565–572. DOI:https://doi.org/10.18653/v1/S17-2093Google ScholarGoogle ScholarCross RefCross Ref
  6. Olivier Bodenreider. 2004. The unified medical language system (UMLS): integrating biomedical terminology. Nucleic Acids Res. 32, suppl. 1 (2004), D267–D270.Google ScholarGoogle ScholarCross RefCross Ref
  7. Andrew Borthwick, John Sterling, Eugene Agichtein, and Ralph Grishman. 1998. Exploiting diverse knowledge sources via maximum entropy in named entity recognition. In Proceedings of the 6th Workshop on Very Large Corpora.Google ScholarGoogle Scholar
  8. Sanchari Chowdhuri, Sidnei McCrea, Dina Demner Fushman, and Casey Overby Taylor. 2019. Extracting biomedical terms from postpartum depression online health communities. In Proceedings of the AMIA Summits on Translational Science. 592.Google ScholarGoogle Scholar
  9. Massimiliano Ciaramita and Yasemin Altun. 2005. Named-entity recognition in novel domains with external lexical knowledge. In Proceedings of the NIPS Workshop on Advances in Structured Learning for Text and Speech Processing, Vol. 2005.Google ScholarGoogle Scholar
  10. K. Bretonnel Cohen and Lawrence Hunter. 2004. Natural language processing and systems biology. In Artificial Intelligence Methods and Tools for Systems Biology. Springer, 147–173.Google ScholarGoogle Scholar
  11. Gamal Crichton, Sampo Pyysalo, Billy Chiu, and Anna Korhonen. 2017. A neural network multi-task learning approach to biomedical named entity recognition. BMC Bioinform. 18, 1 (2017), 368.Google ScholarGoogle ScholarCross RefCross Ref
  12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 4171–4186. DOI:https://doi.org/10.18653/v1/N19-1423Google ScholarGoogle Scholar
  13. Rezarta Islamaj Doğan, Robert Leaman, and Zhiyong Lu. 2014. NCBI disease corpus: A resource for disease name recognition and concept normalization. J. Biomed. Inform. 47 (2014), 1–10.Google ScholarGoogle ScholarCross RefCross Ref
  14. Matt Gardner, Jonathan Berant, Hannaneh Hajishirzi, Alon Talmor, and Sewon Min. 2019. Question answering is a format; when is it useful?Retrieved from https://arXiv:1909.11291.Google ScholarGoogle Scholar
  15. Martin Gerner, Goran Nenadic, and Casey M. Bergman. 2010. LINNAEUS: A species name identification system for biomedical literature. BMC Bioinform. 11, 1 (2010), 85.Google ScholarGoogle ScholarCross RefCross Ref
  16. John M. Giorgi and Gary D. Bader. 2020. Towards reliable named entity recognition in the biomedical domain. Bioinformatics 36, 1 (2020), 280–286.Google ScholarGoogle ScholarCross RefCross Ref
  17. Luheng He, Mike Lewis, and Luke Zettlemoyer. 2015. Question-answer driven semantic role labeling: Using natural language to annotate natural language. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 643–653.Google ScholarGoogle Scholar
  18. Arzoo Katiyar and Claire Cardie. 2018. Nested named entity recognition revisited. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 861–871.Google ScholarGoogle ScholarCross RefCross Ref
  19. Jun’ichi Kazama and Kentaro Torisawa. 2007. Exploiting Wikipedia as external knowledge for named entity recognition. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL’07). 698–707.Google ScholarGoogle Scholar
  20. Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun’ichi Tsujii. 2009. Overview of BioNLP’09 shared task on event extraction. In Proceedings of the BioNLP Workshop Companion Volume for Shared Task. 1–9.Google ScholarGoogle Scholar
  21. J.-D. Kim, Tomoko Ohta, Yuka Tateisi, and Jun’ichi Tsujii. 2003. GENIA corpus—A semantically annotated corpus for bio-textmining. Bioinformatics 19, suppl_1 (2003), i180–i182.Google ScholarGoogle ScholarCross RefCross Ref
  22. Jin-Dong Kim, Tomoko Ohta, Yoshimasa Tsuruoka, Yuka Tateisi, and Nigel Collier. 2004. Introduction to the bio-entity recognition task at JNLPBA. In Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications. Citeseer, 70–75.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Martin Krallinger, Florian Leitner, Obdulia Rabal, Miguel Vazquez, Julen Oyarzabal, and Alfonso Valencia. 2015. CHEMDNER: The drugs and chemical names extraction challenge. J. Cheminform. 7, S1 (2015), S1.Google ScholarGoogle ScholarCross RefCross Ref
  24. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. RACE: Large-scale reading comprehension dataset from examinations. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 785–794. DOI:https://doi.org/10.18653/v1/D17-1082Google ScholarGoogle Scholar
  25. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36, 4 (2020), 1234–1240.Google ScholarGoogle Scholar
  26. Xiaoya Li, Jingrong Feng, Yuxian Meng, Qinghong Han, Fei Wu, and Jiwei Li. 2020. A unified MRC framework for named entity recognition. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 5849–5859. DOI:https://doi.org/10.18653/v1/2020.acl-main.519Google ScholarGoogle Scholar
  27. Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun. 2019. Sequence-to-nuggets: Nested entity mention detection via anchor-region networks. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 5182–5192.Google ScholarGoogle ScholarCross RefCross Ref
  28. Hongfang Liu, Zhang-Zhi Hu, Manabu Torii, Cathy Wu, and Carol Friedman. 2006. Quantitative assessment of dictionary-based protein named entity tagging. J. Amer. Med. Inform. Assoc. 13, 5 (2006), 497–507.Google ScholarGoogle ScholarCross RefCross Ref
  29. Tianyu Liu, Jin-Ge Yao, and Chin-Yew Lin. 2019. Towards improving neural named entity recognition with gazetteers. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 5301–5307. DOI:https://doi.org/10.18653/v1/P19-1524Google ScholarGoogle Scholar
  30. Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hannaneh Hajishirzi. 2019. A general framework for information extraction using dynamic span graphs. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 3036–3046.Google ScholarGoogle Scholar
  31. Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. 2018. The natural language decathlon: Multitask learning as question answering. Retrieved from https://arXiv:1806.08730.Google ScholarGoogle Scholar
  32. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? A new dataset for open book question answering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’18).Google ScholarGoogle Scholar
  33. Claire Nédellec, Robert Bossy, Jin-Dong Kim, Jung-Jae Kim, Tomoko Ohta, Sampo Pyysalo, and Pierre Zweigenbaum. 2013. Overview of BioNLP shared task 2013. In Proceedings of the BioNLP Shared Task Workshop. 1–7.Google ScholarGoogle Scholar
  34. Tomoko Ohta, Robert Bossy, Ngan Nguyen, Jun’ichi Tsujii, Jin-Dong Kim, and Sampo Pyysalo. 2011. Overview of bionlp shared task 2011. In Proceedings of the BioNLP Shared Task Workshop. 1–6.Google ScholarGoogle Scholar
  35. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, 8024–8035. Retrieved from http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf.Google ScholarGoogle Scholar
  36. Sampo Pyysalo and Sophia Ananiadou. 2014. Anatomical entity mention recognition at literature scale. Bioinformatics 30, 6 (2014), 868–875.Google ScholarGoogle ScholarCross RefCross Ref
  37. Sampo Pyysalo, Tomoko Ohta, Makoto Miwa, and Jun’ichi Tsujii. 2011. Towards exhaustive protein modification event extraction. In Proceedings of the BioNLP Workshop. 114–123.Google ScholarGoogle Scholar
  38. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2383–2392.Google ScholarGoogle ScholarCross RefCross Ref
  39. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. 2019. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’19).Google ScholarGoogle Scholar
  40. Max E. Savery, Willie J. Rogers, Malvika Pillai, James G. Mork, and Dina Demner-Fushman. 2020. Chemical entity recognition for MEDLINE indexing. In Proceedings of the AMIA Summits on Translational Science. 561.Google ScholarGoogle Scholar
  41. Takashi Shibuya and Eduard Hovy. 2020. Nested named entity recognition via second-best sequence learning and decoding. Trans. Assoc. Comput. Linguist. 8 (2020), 605–620.Google ScholarGoogle Scholar
  42. Yuqi Si, Jingqi Wang, Hua Xu, and Kirk Roberts. 2019. Enhancing clinical concept extraction with contextual embeddings. J. Amer. Med. Inform. Assoc. 26, 11 (2019), 1297–1304.Google ScholarGoogle ScholarCross RefCross Ref
  43. Larry Smith, Lorraine K. Tanabe, Rie Johnson nee Ando, Cheng-Ju Kuo, I.-Fang Chung, Chun-Nan Hsu, Yu-Shi Lin, Roman Klinger, Christoph M. Friedrich, Kuzman Ganchev, et al. 2008. Overview of biocreative II gene mention recognition. Genome Biol. 9, S2 (2008), S2.Google ScholarGoogle ScholarCross RefCross Ref
  44. Hye-Jeong Song, Byeong-Cheol Jo, Chan-Young Park, Jong-Dae Kim, and Yu-Seop Kim. 2018. Comparison of named entity recognition methodologies in biomedical documents. Biomed. Eng. Online 17, 2 (2018), 158.Google ScholarGoogle Scholar
  45. Jana Straková, Milan Straka, and Jan Hajic. 2019. Neural architectures for nested NER through linearization. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 5326–5331.Google ScholarGoogle ScholarCross RefCross Ref
  46. Weiyi Sun, Anna Rumshisky, and Ozlem Uzuner. 2013. Evaluating temporal relations in clinical text: 2012 i2b2 challenge. J. Amer. Med. Inform. Assoc. 20, 5 (2013), 806–813.Google ScholarGoogle ScholarCross RefCross Ref
  47. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 4149–4158. DOI:https://doi.org/10.18653/v1/N19-1421Google ScholarGoogle Scholar
  48. Ozlem Uzuner, Andreea Bodnari, Shuying Shen, Tyler Forbush, John Pestian, and Brett R. South. 2012. Evaluating the state of the art in coreference resolution for electronic medical records. J. Amer. Med. Inform. Assoc. 19, 5 (2012), 786–791.Google ScholarGoogle ScholarCross RefCross Ref
  49. Özlem Uzuner, Imre Solti, and Eithon Cadag. 2010. Extracting medication information from clinical text. J. Amer. Med. Inform. Assoc. 17, 5 (2010), 514–518.Google ScholarGoogle ScholarCross RefCross Ref
  50. Özlem Uzuner, Brett R. South, Shuying Shen, and Scott L. DuVall. 2011. 2010 i2b2/VA challenge on concepts, assertions, and relations in clinical text. J. Amer. Med. Inform. Assoc. 18, 5 (2011), 552–556.Google ScholarGoogle ScholarCross RefCross Ref
  51. Xi Wang, Jiagao Lyu, Li Dong, and Ke Xu. 2019. Multitask learning for biomedical named entity recognition with cross-sharing structure. BMC Bioinform. 20, 1 (2019), 427.Google ScholarGoogle ScholarCross RefCross Ref
  52. Xuan Wang, Yu Zhang, Qi Li, Xiang Ren, Jingbo Shang, and Jiawei Han. 2019. Distantly supervised biomedical named entity recognition with dictionary expansion. In Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine (BIBM’19). IEEE, 496–503.Google ScholarGoogle Scholar
  53. Xuan Wang, Yu Zhang, Xiang Ren, Yuhao Zhang, Marinka Zitnik, Jingbo Shang, Curtis Langlotz, and Jiawei Han. 2018. Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics 35, 10 (2018), 1745–1752.Google ScholarGoogle Scholar
  54. Chih-Hsuan Wei, Yifan Peng, Robert Leaman, Allan Peter Davis, Carolyn J. Mattingly, Jiao Li, Thomas C. Wiegers, and Zhiyong Lu. 2015. Overview of the biocreative v chemical disease relation (CDR) task. In Proceedings of the 5th Biocreative Challenge Evaluation Workshop, Vol. 14.Google ScholarGoogle Scholar
  55. Edwin B. Wilson. 1927. Probable inference, the law of succession, and statistical inference. J. Amer. Statist. Assoc. 22, 158 (1927), 209–212.Google ScholarGoogle Scholar
  56. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. 2019. HuggingFace’s transformers: State-of-the-art natural language processing. Retrieved from https://abs/1910.03771.Google ScholarGoogle Scholar
  57. Stephen Wu, Kirk Roberts, Surabhi Datta, Jingcheng Du, Zongcheng Ji, Yuqi Si, Sarvesh Soni, Qiong Wang, Qiang Wei, Yang Xiang, et al. 2020. Deep learning in clinical natural language processing: A methodical review. J. Amer. Med. Info. Assoc. 27, 3 (2020), 457–470.Google ScholarGoogle Scholar
  58. Vikas Yadav and Steven Bethard. 2018. A survey on recent advances in named entity recognition from deep learning models. In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, Santa Fe, NM, 2145–2158. Retrieved from https://www.aclweb.org/anthology/C18-1182.Google ScholarGoogle Scholar
  59. Vikas Yadav, Rebecca Sharp, and Steven Bethard. 2018. Deep affix features improve neural named entity recognizers. In Proceedings of the 7th Joint Conference on Lexical and Computational Semantics. 167–172.Google ScholarGoogle Scholar
  60. Wonjin Yoon, Chan Ho So, Jinhyuk Lee, and Jaewoo Kang. 2019. Collabonet: Collaboration of deep neural networks for biomedical named entity recognition. BMC Bioinform. 20, 10 (2019), 249.Google ScholarGoogle Scholar
  61. Juntao Yu, Bernd Bohnet, and Massimo Poesio. 2020. Named entity recognition as dependency parsing. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 6470–6476. DOI:https://doi.org/10.18653/v1/2020.acl-main.577Google ScholarGoogle Scholar

Index Terms

  1. Biomedical Named Entity Recognition via Knowledge Guidance and Question Answering

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!