skip to main content
research-article

Joint Model of Entity Recognition and Relation Extraction with Self-attention Mechanism

Authors Info & Claims
Published:23 May 2020Publication History
Skip Abstract Section

Abstract

In recent years, the joint model of entity recognition (ER) and relation extraction (RE) has attracted more and more attention in the healthcare and medical domains. However, there are some problems with the prior work. The joint model cannot extract all the relations for a specific entity, and the majority of joint models heavily rely on complex artificial features or professional natural language processing (NLP) tools. In this article, we construct a novel joint model that can simultaneously extract all medical entities and relations from medicine Chinese instructions. Moreover, the self-attention mechanism is introduced to the joint model to learn word intra-sentence dependencies. The proposed model is evaluated using a medicine Chinese instruction dataset that we collect and an open dataset provided in CoNLL-2004. Experimental results show that the model with self-attention achieves the state-of-the-art performance.

References

  1. Harsha Gurulingappa, Abdul M. Rajput, Augus Roberts, Juliane Fluck, Martin Hofmann-Apitius, and Luca Toldo. 2012. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. J. Biomed. Inf. 45, 5 (2012), 885--892.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. Joint entity recognition and relation extraction as a multi-head selection problem. Expert Syst. Appl. 114 (2018), 34--45. https://bmcbioinformatics.biomedcentral.com/track/pdf/10.1186/1471-2105-6-S1-S14.Google ScholarGoogle ScholarCross RefCross Ref
  3. Jianpeng Cheng, Li Dong, and Mirella Lapata. 2016. Long short-term memory-networks for machine reading. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP’16), 551--561.Google ScholarGoogle ScholarCross RefCross Ref
  4. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of 31st Conference on Neural Information Processing Systems (NIPS’17), 5999--6009.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In Proceedings of the 6th International Conference on Learning Representations (ICLR’17).Google ScholarGoogle Scholar
  6. Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. 2018. Self-attention with relative position representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), 464--468.Google ScholarGoogle ScholarCross RefCross Ref
  7. Zhouhan Lin, Minwei Feng, Cicero N. Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In Proceedings of the 6th International Conference on Learning Representations (ICLR’17).Google ScholarGoogle Scholar
  8. Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. 2018. DiSAN: Directional self-attention network for RNN/CNN-free language understanding. In Proceeding of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18), 5446--5455.Google ScholarGoogle Scholar
  9. Patrick Verga, Emma Strubell, and Andrew McCallum. 2018. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), 872--884.Google ScholarGoogle ScholarCross RefCross Ref
  10. Zhixing Tan, Mingxuan Wang, Jun Xie, Yidong Chen, and Xiaodong Shi. 2018. Deep semantic role labeling with self-attention. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18), 4929--4936.Google ScholarGoogle ScholarCross RefCross Ref
  11. Daniel Hanisch, Katrin Fundel, Heinz-Theodor Mevissen, Ralf Zimmer, and Juliane Fluck. 2005. ProMiner: Rule-based protein and gene entity recognition. BMC Bioinf. 6, Suppl 1 (2005), S14. https://bmcbioinformatics.biomedcentral.com/track/pdf/10.1186/1471-2105-6-S1-S14.Google ScholarGoogle ScholarCross RefCross Ref
  12. Christopher S. G. Khoo, Syin Chan, and Yun Niu. 2000. Extracting causal knowledge from a medical database using graphical patterns. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics (ACL’00), 336--343.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Oana Frunza and Diana Inkpen. 2010. Extraction of disease-treatment semantic relations from biomedical sentences. In Proceedings of the 2010 Workshop on Biomedical Natural Language Processing, 91--98.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Maofu Liu, Li Jiang, and Huijun Hu. 2017. Automatic extraction and visualization of semantic relations between medical entities from medicine instructions. Multimedia Tools Appl. 76, 8 (2017) 10555--10573.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Abhyuday N. Jagannatha and Hong Yu. 2016. Bidirectional RNN for medical event detection in electronic health records. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16), 473--482.Google ScholarGoogle Scholar
  16. Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16), 2124--2133.Google ScholarGoogle ScholarCross RefCross Ref
  17. Chanqin Quan, Lei Hua, Xiao Sun, and Wenjun Bai. 2016. Multichannel convolutional neural network for biological relation extraction, BioMed Research International 2016 (2016), 1–10. http://downloads.hindawi.com/journals/bmri/2016/1850404.pdf.Google ScholarGoogle Scholar
  18. Bishan Yang and Claire Cardie. 2013. Joint inference for fine-grained opinion extraction. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL’13), 1640--1649.Google ScholarGoogle Scholar
  19. Sameer Singh, Sebastian Riedel, Brian Martin, Jiaping Zheng, and Andrew McCallum. 2013. Joint inference of entities, relations, and coreference. In Proceedings of the 2013 Workshop on Automated Knowledge Base Construction, 1--6.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Suncong Zheng, Yuexing Hao, Dongyuan Lu, Hongyun Bao, Jiaming Xu, Hongwei Hao and Bo Xu. 2017. Joint entity and relation extraction based on a hybrid neural network. Neurocomputing 257 (2017), 59--66. https://www.sciencedirect.com/science/article/abs/pii/S0925231217301613.Google ScholarGoogle ScholarCross RefCross Ref
  21. Pankaj Gupta, Hinrich Schütze, and Bernt Andrassy. 2016. Table filling multi-task recurrent neural network for joint entity and relation extraction. In Proceedings of the 26th International Conference on Computational Linguistics (COLING’16), 2537--2547.Google ScholarGoogle Scholar
  22. Heike Adel and Hinrich Schütze. 2017. Global normalization of convolutional neural networks for joint entity and relation classification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), 1723--1729.Google ScholarGoogle ScholarCross RefCross Ref
  23. Maria Pershina, Bonan Min, Wei Xu, and Ralph Grishman. 2014. Infusion of labeled data into distant supervision for relation extraction. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL’14), 732--738.Google ScholarGoogle ScholarCross RefCross Ref
  24. Lisheng Fu, Bonan Min, Thien H. Nguyen, and Ralph Grishman. 2018. A case study on learning a unified encoder of relations. In Proceedings of the 4th Workshop on Noisy User-generated Text (W-NUT) at EMNLP, 202--207.Google ScholarGoogle ScholarCross RefCross Ref
  25. Lisheng Fu, Thien H. Nguyen, Bonan Min, and Ralph Grishman. 2017. Domain adaptation for relation extraction with domain adversarial neural network. In Proceedings of the 8th International Joint Conference on Natural Language Processing (JCNLP’17), 425--429.Google ScholarGoogle Scholar
  26. Yee S. Chan, Joshua Fasching, Haoling Qiu, and Bonan Min. 2019. Rapid customization for event extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL’19), 31--36.Google ScholarGoogle ScholarCross RefCross Ref
  27. Makoto Miwa, and Mohit Bansal. 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16), 1105--1116.Google ScholarGoogle ScholarCross RefCross Ref
  28. Fei Li, Meishan Zhang, Guohong Fu, and Donghong Ji. 2017. A neural joint model for entity and relation extraction from biomedical text, BMC Bioinf. 18, 1 (2017) 198.Google ScholarGoogle ScholarCross RefCross Ref
  29. Arzoo Katiyar and Claire Cardie. 2017. Going out on a limb: Joint extraction of entity mentions and relations without dependency trees. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), 917--928.Google ScholarGoogle ScholarCross RefCross Ref
  30. Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. An attentive neural architecture for joint segmentation and parsing and its application to real estate ads. Expert Syst. Appl. 102 (2018), 100--112. https://www.sciencedirect.com/science/article/abs/pii/S0957417418301192.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Meishan Zhang, Yue Zhang, and Guohong Fu. 2017. End-to-End Neural Relation Extraction with Global Optimization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), 1730--1740.Google ScholarGoogle ScholarCross RefCross Ref
  32. Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. 2017. Joint extraction of entities and relations based on a novel tagging scheme. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), 1227--1236.Google ScholarGoogle ScholarCross RefCross Ref
  33. Supot Nitsuwat and Wansa Paoin. 2004. Development of ICD-10-TM ontology for a semi-automated morbidity coding system in Thailand. Methods Inf. Med. 51, 6 (2004) 519--528.Google ScholarGoogle ScholarCross RefCross Ref
  34. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyerl. 2016. Neural Architectures for Named Entity Recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16), 260--270.Google ScholarGoogle ScholarCross RefCross Ref
  35. Dan Roth, and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the 8th Conference on Computational Natural Language Learning (CoNLL’04) at NAACL-HLT, 1--8.Google ScholarGoogle Scholar
  36. Makoto Miwa and Yutaka Sasaki. 2014. Modeling joint entity and relation extraction with table representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14), 1858--1869.Google ScholarGoogle ScholarCross RefCross Ref
  37. Rohit J. Kate and Raymond Mooney. 2010. Joint entity and relation extraction using card-pyramid parsing. In Proceedings of the 14th Conference on Computational Natural Language Learning (CoNLL’10), 203--212.Google ScholarGoogle Scholar

Index Terms

  1. Joint Model of Entity Recognition and Relation Extraction with Self-attention Mechanism

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!