Abstract
In recent years, the joint model of entity recognition (ER) and relation extraction (RE) has attracted more and more attention in the healthcare and medical domains. However, there are some problems with the prior work. The joint model cannot extract all the relations for a specific entity, and the majority of joint models heavily rely on complex artificial features or professional natural language processing (NLP) tools. In this article, we construct a novel joint model that can simultaneously extract all medical entities and relations from medicine Chinese instructions. Moreover, the self-attention mechanism is introduced to the joint model to learn word intra-sentence dependencies. The proposed model is evaluated using a medicine Chinese instruction dataset that we collect and an open dataset provided in CoNLL-2004. Experimental results show that the model with self-attention achieves the state-of-the-art performance.
- Harsha Gurulingappa, Abdul M. Rajput, Augus Roberts, Juliane Fluck, Martin Hofmann-Apitius, and Luca Toldo. 2012. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. J. Biomed. Inf. 45, 5 (2012), 885--892.Google Scholar
Digital Library
- Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. Joint entity recognition and relation extraction as a multi-head selection problem. Expert Syst. Appl. 114 (2018), 34--45. https://bmcbioinformatics.biomedcentral.com/track/pdf/10.1186/1471-2105-6-S1-S14.Google Scholar
Cross Ref
- Jianpeng Cheng, Li Dong, and Mirella Lapata. 2016. Long short-term memory-networks for machine reading. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP’16), 551--561.Google Scholar
Cross Ref
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of 31st Conference on Neural Information Processing Systems (NIPS’17), 5999--6009.Google Scholar
Digital Library
- Romain Paulus, Caiming Xiong, and Richard Socher. 2018. A deep reinforced model for abstractive summarization. In Proceedings of the 6th International Conference on Learning Representations (ICLR’17).Google Scholar
- Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. 2018. Self-attention with relative position representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), 464--468.Google Scholar
Cross Ref
- Zhouhan Lin, Minwei Feng, Cicero N. Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A structured self-attentive sentence embedding. In Proceedings of the 6th International Conference on Learning Representations (ICLR’17).Google Scholar
- Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. 2018. DiSAN: Directional self-attention network for RNN/CNN-free language understanding. In Proceeding of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18), 5446--5455.Google Scholar
- Patrick Verga, Emma Strubell, and Andrew McCallum. 2018. Simultaneously self-attending to all mentions for full-abstract biological relation extraction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’18), 872--884.Google Scholar
Cross Ref
- Zhixing Tan, Mingxuan Wang, Jun Xie, Yidong Chen, and Xiaodong Shi. 2018. Deep semantic role labeling with self-attention. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI’18), 4929--4936.Google Scholar
Cross Ref
- Daniel Hanisch, Katrin Fundel, Heinz-Theodor Mevissen, Ralf Zimmer, and Juliane Fluck. 2005. ProMiner: Rule-based protein and gene entity recognition. BMC Bioinf. 6, Suppl 1 (2005), S14. https://bmcbioinformatics.biomedcentral.com/track/pdf/10.1186/1471-2105-6-S1-S14.Google Scholar
Cross Ref
- Christopher S. G. Khoo, Syin Chan, and Yun Niu. 2000. Extracting causal knowledge from a medical database using graphical patterns. In Proceedings of the 38th Annual Meeting on Association for Computational Linguistics (ACL’00), 336--343.Google Scholar
Digital Library
- Oana Frunza and Diana Inkpen. 2010. Extraction of disease-treatment semantic relations from biomedical sentences. In Proceedings of the 2010 Workshop on Biomedical Natural Language Processing, 91--98.Google Scholar
Digital Library
- Maofu Liu, Li Jiang, and Huijun Hu. 2017. Automatic extraction and visualization of semantic relations between medical entities from medicine instructions. Multimedia Tools Appl. 76, 8 (2017) 10555--10573.Google Scholar
Digital Library
- Abhyuday N. Jagannatha and Hong Yu. 2016. Bidirectional RNN for medical event detection in electronic health records. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16), 473--482.Google Scholar
- Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16), 2124--2133.Google Scholar
Cross Ref
- Chanqin Quan, Lei Hua, Xiao Sun, and Wenjun Bai. 2016. Multichannel convolutional neural network for biological relation extraction, BioMed Research International 2016 (2016), 1–10. http://downloads.hindawi.com/journals/bmri/2016/1850404.pdf.Google Scholar
- Bishan Yang and Claire Cardie. 2013. Joint inference for fine-grained opinion extraction. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL’13), 1640--1649.Google Scholar
- Sameer Singh, Sebastian Riedel, Brian Martin, Jiaping Zheng, and Andrew McCallum. 2013. Joint inference of entities, relations, and coreference. In Proceedings of the 2013 Workshop on Automated Knowledge Base Construction, 1--6.Google Scholar
Digital Library
- Suncong Zheng, Yuexing Hao, Dongyuan Lu, Hongyun Bao, Jiaming Xu, Hongwei Hao and Bo Xu. 2017. Joint entity and relation extraction based on a hybrid neural network. Neurocomputing 257 (2017), 59--66. https://www.sciencedirect.com/science/article/abs/pii/S0925231217301613.Google Scholar
Cross Ref
- Pankaj Gupta, Hinrich Schütze, and Bernt Andrassy. 2016. Table filling multi-task recurrent neural network for joint entity and relation extraction. In Proceedings of the 26th International Conference on Computational Linguistics (COLING’16), 2537--2547.Google Scholar
- Heike Adel and Hinrich Schütze. 2017. Global normalization of convolutional neural networks for joint entity and relation classification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), 1723--1729.Google Scholar
Cross Ref
- Maria Pershina, Bonan Min, Wei Xu, and Ralph Grishman. 2014. Infusion of labeled data into distant supervision for relation extraction. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL’14), 732--738.Google Scholar
Cross Ref
- Lisheng Fu, Bonan Min, Thien H. Nguyen, and Ralph Grishman. 2018. A case study on learning a unified encoder of relations. In Proceedings of the 4th Workshop on Noisy User-generated Text (W-NUT) at EMNLP, 202--207.Google Scholar
Cross Ref
- Lisheng Fu, Thien H. Nguyen, Bonan Min, and Ralph Grishman. 2017. Domain adaptation for relation extraction with domain adversarial neural network. In Proceedings of the 8th International Joint Conference on Natural Language Processing (JCNLP’17), 425--429.Google Scholar
- Yee S. Chan, Joshua Fasching, Haoling Qiu, and Bonan Min. 2019. Rapid customization for event extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL’19), 31--36.Google Scholar
Cross Ref
- Makoto Miwa, and Mohit Bansal. 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16), 1105--1116.Google Scholar
Cross Ref
- Fei Li, Meishan Zhang, Guohong Fu, and Donghong Ji. 2017. A neural joint model for entity and relation extraction from biomedical text, BMC Bioinf. 18, 1 (2017) 198.Google Scholar
Cross Ref
- Arzoo Katiyar and Claire Cardie. 2017. Going out on a limb: Joint extraction of entity mentions and relations without dependency trees. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), 917--928.Google Scholar
Cross Ref
- Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. An attentive neural architecture for joint segmentation and parsing and its application to real estate ads. Expert Syst. Appl. 102 (2018), 100--112. https://www.sciencedirect.com/science/article/abs/pii/S0957417418301192.Google Scholar
Digital Library
- Meishan Zhang, Yue Zhang, and Guohong Fu. 2017. End-to-End Neural Relation Extraction with Global Optimization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP’17), 1730--1740.Google Scholar
Cross Ref
- Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. 2017. Joint extraction of entities and relations based on a novel tagging scheme. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), 1227--1236.Google Scholar
Cross Ref
- Supot Nitsuwat and Wansa Paoin. 2004. Development of ICD-10-TM ontology for a semi-automated morbidity coding system in Thailand. Methods Inf. Med. 51, 6 (2004) 519--528.Google Scholar
Cross Ref
- Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyerl. 2016. Neural Architectures for Named Entity Recognition. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16), 260--270.Google Scholar
Cross Ref
- Dan Roth, and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. In Proceedings of the 8th Conference on Computational Natural Language Learning (CoNLL’04) at NAACL-HLT, 1--8.Google Scholar
- Makoto Miwa and Yutaka Sasaki. 2014. Modeling joint entity and relation extraction with table representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14), 1858--1869.Google Scholar
Cross Ref
- Rohit J. Kate and Raymond Mooney. 2010. Joint entity and relation extraction using card-pyramid parsing. In Proceedings of the 14th Conference on Computational Natural Language Learning (CoNLL’10), 203--212.Google Scholar
Index Terms
Joint Model of Entity Recognition and Relation Extraction with Self-attention Mechanism
Recommendations
An Attention-based Model for Joint Extraction of Entities and Relations with Implicit Entity Features
WWW '19: Companion Proceedings of The 2019 World Wide Web ConferenceExtracting entities and relations is critical to the understanding of massive text corpora. Recently, neural joint models have shown promising results for this task. However, the entity features are not effectively used in these joint models. In this ...
Enhancing Joint Entity and Relation Extraction with Language Modeling and Hierarchical Attention
Web and Big DataAbstractBoth entity recognition and relation extraction can benefit from being performed jointly, allowing them to enhance each other. However, existing methods suffer from the sparsity of relevant labels and strongly rely on external natural language ...
Joint extraction of entities and relations by a novel end-to-end model with a double-pointer module
Highlights- An end-to-end model with the double-pointer module is proposed that can extract entities and relations jointly.
AbstractJoint extraction of entities and relations is to detect entities and recognize semantic relations simultaneously. However, some existing joint models predict relations on words, instead of entities. These models cannot make full use of ...






Comments