research-article

A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction

Authors:

Yongliang Shen,

Yechun Tang, and

Weiming LuAuthors Info & Claims

WWW '21: Proceedings of the Web Conference 2021

April 2021

Pages 1704 - 1715

https://doi.org/10.1145/3442381.3449895

Published: 03 June 2021 Publication History

Abstract

Joint entity and relation extraction framework constructs a unified model to perform entity recognition and relation extraction simultaneously, which can exploit the dependency between the two tasks to mitigate the error propagation problem suffered by the pipeline model. Current efforts on joint entity and relation extraction focus on enhancing the interaction between entity recognition and relation extraction through parameter sharing, joint decoding, or other ad-hoc tricks (e.g., modeled as a semi-Markov decision process, cast as a multi-round reading comprehension task). However, there are still two issues on the table. First, the interaction utilized by most methods is still weak and uni-directional, which is unable to model the mutual dependency between the two tasks. Second, relation triggers are ignored by most methods, which can help explain why humans would extract a relation in the sentence. They’re essential for relation extraction but overlooked. To this end, we present a Trigger-Sense Memory Flow Framework (TriMF) for joint entity and relation extraction. We build a memory module to remember category representations learned in entity recognition and relation extraction tasks. And based on it, we design a multi-level memory flow attention mechanism to enhance the bi-directional interaction between entity recognition and relation extraction. Moreover, without any human annotations, our model can enhance relation trigger information in a sentence through a trigger sensor module, which improves the model performance and makes model predictions with better interpretation. Experiment results show that our proposed framework achieves state-of-the-art results by improves the relation F1 to 52.44% (+3.2%) on SciERC, 66.49% (+4.9%) on ACE05, 72.35% (+0.6%) on CoNLL04 and 80.66% (+2.3%) on ADE.

References

[1]

Eugene Agichtein and Luis Gravano. 2000. Snowball: Extracting relations from large plain-text collections. In Proceedings of the fifth ACM conference on Digital libraries. 85–94.

Digital Library

[2]

Chinatsu Aone, Lauren Halverson, Tom Hampton, and Mila Ramos-Santacruz. 1998. SRA: Description of the IE2 system used for MUC-7. In Seventh Message Understanding Conference (MUC-7): Proceedings of a Conference Held in Fairfax, Virginia, April 29-May 1, 1998.

[3]

David S Batista, Bruno Martins, and Mário J Silva. 2015. Semi-supervised bootstrapping of relationship extractors with distributional semantics. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 499–504.

[4]

Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. Adversarial training for multi-context joint entity and relation extraction. arXiv preprint arXiv:1808.06876(2018).

[5]

Giannis Bekoulis, Johannes Deleu, Thomas Demeester, and Chris Develder. 2018. Joint entity recognition and relation extraction as a multi-head selection problem. Expert Systems with Applications 114 (2018), 34–45.

[6]

Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A pretrained language model for scientific text. arXiv preprint arXiv:1903.10676(2019).

[7]

Quoc-Chinh Bui, Sophia Katrenko, and Peter MA Sloot. 2011. A hybrid approach to extract protein–protein interactions. Bioinformatics 27, 2 (2011), 259–265.

Digital Library

[8]

Razvan Bunescu and Raymond Mooney. 2005. A shortest path dependency kernel for relation extraction. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing. 724–731.

Digital Library

[9]

Yee Seng Chan and Dan Roth. 2011. Exploiting syntactico-semantic structures for relation extraction. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 551–560.

Digital Library

[10]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).

[11]

Markus Eberts and Adrian Ulges. 2019. Span-based Joint Entity and Relation Extraction with Transformer Pre-training. arXiv preprint arXiv:1909.07755(2019).

[12]

Tsu-Jui Fu, Peng-Hsuan Li, and Wei-Yun Ma. 2019. GraphRel: Modeling text as relational graphs for joint entity and relation extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1409–1418.

[13]

Katrin Fundel, Robert Küffner, and Ralf Zimmer. 2007. RelEx-Relation extraction using dependency parse trees. Bioinformatics 23, 3 (2007), 365–371.

Digital Library

[14]

Pankaj Gupta, Hinrich Schütze, and Bernt Andrassy. 2016. Table filling multi-task recurrent neural network for joint entity and relation extraction. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. 2537–2547.

[15]

Harsha Gurulingappa, Abdul Mateen Rajput, Angus Roberts, Juliane Fluck, Martin Hofmann-Apitius, and Luca Toldo. 2012. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. Journal of biomedical informatics 45, 5 (2012), 885–892.

Digital Library

[16]

Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2020. More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction. arXiv preprint arXiv:2004.03186(2020).

[17]

Marti A Hearst. 1992. Automatic acquisition of hyponyms from large text corpora. In Coling 1992 volume 2: The 15th international conference on computational linguistics.

[18]

Jing Jiang and ChengXiang Zhai. 2007. A systematic exploration of the feature space for relation extraction. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference. 113–120.

[19]

Rosie Jones, Andrew McCallum, Kamal Nigam, and Ellen Riloff. 1999. Bootstrapping for text learning tasks. In IJCAI-99 Workshop on Text Mining: Foundations, Techniques and Applications, Vol. 1.

[20]

Nanda Kambhatla. 2004. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proceedings of the ACL 2004 on Interactive poster and demonstration sessions. 22–es.

Digital Library

[21]

Arzoo Katiyar and Claire Cardie. 2016. Investigating lstms for joint extraction of opinion entities and relations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 919–929.

[22]

Qi Li and Heng Ji. 2014. Incremental joint extraction of entity mentions and relations. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 402–412.

[23]

Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity-relation extraction as multi-turn question answering. arXiv preprint arXiv:1905.05529(2019).

[24]

Bill Yuchen Lin, Dong-Ho Lee, Ming Shen, Ryan Moreno, Xiao Huang, Prashant Shiralkar, and Xiang Ren. 2020. Triggerner: Learning with entity triggers as explanations for named entity recognition. In ACL.

[25]

Yankai Lin, Shiqi Shen, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. 2016. Neural relation extraction with selective attention over instances. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2124–2133.

[26]

Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. arXiv preprint arXiv:1808.09602(2018).

[27]

Yi Luan, Dave Wadden, Luheng He, Amy Shah, Mari Ostendorf, and Hannaneh Hajishirzi. 2019. A general framework for information extraction using dynamic span graphs. arXiv preprint arXiv:1904.03296(2019).

[28]

Scott Miller, Heidi Fox, Lance Ramshaw, and Ralph Weischedel. 2000. A novel use of statistical parsing to extract information from text. In 1st Meeting of the North American Chapter of the Association for Computational Linguistics.

Digital Library

[29]

Makoto Miwa and Mohit Bansal. 2016. End-to-end relation extraction using lstms on sequences and tree structures. arXiv preprint arXiv:1601.00770(2016).

[30]

Dan Roth and Wen-tau Yih. 2004. A linear programming formulation for global inference in natural language tasks. Technical Report. ILLINOIS UNIV AT URBANA-CHAMPAIGN DEPT OF COMPUTER SCIENCE.

[31]

Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In European Semantic Web Conference. Springer, 593–607.

Digital Library

[32]

Ryuichi Takanobu, Tianyang Zhang, Jiexi Liu, and Minlie Huang. 2019. A hierarchical framework for relation extraction with reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7072–7079.

Digital Library

[33]

David Wadden, Ulme Wennberg, Yi Luan, and Hannaneh Hajishirzi. 2019. Entity, relation, and event extraction with contextualized span representations. arXiv preprint arXiv:1909.03546(2019).

[34]

Shaolei Wang, Yue Zhang, Wanxiang Che, and Ting Liu. 2018. Joint extraction of entities and relations based on a novel graph scheme. In IJCAI. 4461–4467.

[35]

Ziqi Wang, Yujia Qin, Wenxuan Zhou, Jun Yan, Qinyuan Ye, Leonardo Neves, Zhiyuan Liu, and Xiang Ren. 2019. Learning from explanations with neural execution tree. In International Conference on Learning Representations.

[36]

Zhepei Wei, Jianlin Su, Yue Wang, Yuan Tian, and Yi Chang. 2020. A Novel Cascade Binary Tagging Framework for Relational Triple Extraction. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 1476–1488.

[37]

Bishan Yang and Claire Cardie. 2013. Joint inference for fine-grained opinion extraction. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1640–1649.

[38]

Yue Yuan, Xiaofei Zhou, Shirui Pan, Qiannan Zhu, Zeliang Song, and Li Guo. 2020. A Relation-Specific Attention Network for Joint Entity and Relation Extraction. In International Joint Conference on Artificial Intelligence 2020. Association for the Advancement of Artificial Intelligence (AAAI), 4054–4060.

[39]

Xiangrong Zeng, Daojian Zeng, Shizhu He, Kang Liu, and Jun Zhao. 2018. Extracting relational facts by an end-to-end neural model with copy mechanism. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 506–514.

[40]

Min Zhang, Jie Zhang, and Jian Su. 2006. Exploring syntactic features for relation extraction using a convolution tree kernel. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference. 288–295.

Digital Library

[41]

Tianyang Zhao, Zhao Yan, Y. Cao, and Zhoujun Li. 2020. Asking Effective and Diverse Questions: A Machine Reading Comprehension based Framework for Joint Entity-Relation Extraction. In IJCAI.

[42]

Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, and Bo Xu. 2017. Joint extraction of entities and relations based on a novel tagging scheme. arXiv preprint arXiv:1706.05075(2017).

[43]

Wenxuan Zhou, Hongtao Lin, Bill Yuchen Lin, Ziqi Wang, Junyi Du, Leonardo Neves, and Xiang Ren. 2020. Nero: A neural rule grounding framework for label-efficient relation extraction. In Proceedings of The Web Conference 2020. 2166–2176.

Digital Library

Cited By

Dai QYang WWei FHe LLiao Y(2024)FSN: Joint Entity and Relation Extraction Based on Filter Separator NetworkEntropy10.3390/e2602016226:2(162)Online publication date: 12-Feb-2024
https://doi.org/10.3390/e26020162
Liu YFu LXia XZhang Y(2024)Exploring the Role of Self-Adaptive Feature Words in Relation Quintuple Extraction for Scientific LiteratureApplied Sciences10.3390/app1410402014:10(4020)Online publication date: 9-May-2024
https://doi.org/10.3390/app14104020
Zhao XYang DYang MWang LZhang RCheng HLam WSHEN YXu R(2024)A Comprehensive Survey on Relation Extraction: Recent Advances and New FrontiersACM Computing Surveys10.1145/3674501Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3674501
Show More Cited By

Recommendations

REKER: Relation Extraction with Knowledge of Entity and Relation
Natural Language Processing and Chinese Computing
Abstract
Relation Extraction (RE) is an important task to mine knowledge from massive text corpus. Existing relation extraction methods usually purely rely on the textual information of sentences to predict the relations between entities. The useful ...
Read More
A Relation-Oriented Method for Joint Entity and Relation Extraction Based on Neural Network
EITCE '21: Proceedings of the 2021 5th International Conference on Electronic Information Technology and Computer Engineering

Entity and relation extraction is a basic task of information extraction in natural language processing. At present, Entity and relation extraction based on artificial intelligence has been widely studied, but most methods adopt the idea of identifying ...
Read More
A Flexible Text Mining System for Entity and Relation Extraction in PubMed
DTMBIO '15: Proceedings of the ACM Ninth International Workshop on Data and Text Mining in Biomedical Informatics

Due to an enormous number of scientific publications that cannot be handled manually, there is a rising interest in text-mining techniques for automated information extraction, especially in the biomedical field. Such techniques provide effective means ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '21: Proceedings of the Web Conference 2021

April 2021

4054 pages

ISBN:9781450383127

DOI:10.1145/3442381

Editors:
Jure Leskovec
Stanford
,
Marko Grobelnik
Jožef Stefan Institute
,
Marc Najork
Google
,
Jie Tang
Tsinghua University
,
Leila Zia
Wikimedia Foundation

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 June 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '21

Sponsor:

SIGWEB

WWW '21: The Web Conference 2021

April 19 - 23, 2021

Ljubljana, Slovenia

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
370
Total Downloads

Downloads (Last 12 months)71
Downloads (Last 6 weeks)3

Other Metrics

View Author Metrics

Citations

Cited By

Dai QYang WWei FHe LLiao Y(2024)FSN: Joint Entity and Relation Extraction Based on Filter Separator NetworkEntropy10.3390/e2602016226:2(162)Online publication date: 12-Feb-2024
https://doi.org/10.3390/e26020162
Liu YFu LXia XZhang Y(2024)Exploring the Role of Self-Adaptive Feature Words in Relation Quintuple Extraction for Scientific LiteratureApplied Sciences10.3390/app1410402014:10(4020)Online publication date: 9-May-2024
https://doi.org/10.3390/app14104020
Zhao XYang DYang MWang LZhang RCheng HLam WSHEN YXu R(2024)A Comprehensive Survey on Relation Extraction: Recent Advances and New FrontiersACM Computing Surveys10.1145/3674501Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1145/3674501
Mengjun DJin QAng LXue FYi Y(2024)A Dual Information Flow Model For Entity Relation Extraction2023 8th International Conference on Information Systems Engineering10.1145/3641032.3641044(95-100)Online publication date: 11-Jun-2024
https://doi.org/10.1145/3641032.3641044
Zhang YMeng FChen YXu JZhou J(2024)Complex Question Enhanced Transfer Learning for Zero-Shot Joint Information ExtractionIEEE/ACM Transactions on Audio, Speech, and Language Processing10.1109/TASLP.2023.330448132(261-275)Online publication date: 2024
https://doi.org/10.1109/TASLP.2023.3304481
Wu YChen YQin YTang RZheng Q(2024)A recollect-tuning method for entity and relation extractionExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.123000245:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.123000
Chen JHu JLi TTeng FDu S(2024)An effective relation-first detection model for relational triple extractionExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122007238:PBOnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122007
Ji BLi SXu HYu JMa JLiu HYang J(2024)Span-based joint entity and relation extraction augmented with sequence tagging mechanismScience China Information Sciences10.1007/s11432-022-3608-y67:5Online publication date: 3-Apr-2024
https://doi.org/10.1007/s11432-022-3608-y
Liao GTang XLi TZhong LZeng P(2024)A Review of Relationship Extraction Based on Deep LearningArtificial Intelligence and Machine Learning10.1007/978-981-97-1277-9_6(73-84)Online publication date: 3-Apr-2024
https://doi.org/10.1007/978-981-97-1277-9_6
Sun WLiu SLiu YKong LJian Z(2023)Information Extraction Network Based on Multi-Granularity Attention and Multi-Scale Self-LearningSensors10.3390/s2309425023:9(4250)Online publication date: 25-Apr-2023
https://doi.org/10.3390/s23094250
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents