skip to main content
research-article

An Extensible Framework of Leveraging Syntactic Skeleton for Semantic Relation Classification

Published:27 September 2020Publication History
Skip Abstract Section

Abstract

Relation classification is one of the most fundamental upstream tasks in natural language processing and information extraction. State-of-the-art approaches make use of various deep neural networks (DNNs) to extract higher-level features directly. They can easily access to accurate classification results by taking advantage of both local entity features and global sentential features. Recent works on relation classification devote efforts to modify these neural networks, but less attention has been paid to the feature design concerning syntax. However, from a linguistic perspective, syntactic features are essential for relation classification. In this article, we present a novel linguistically motivated approach that enhances relation classification by imposing additional syntactic constraints. We investigate to leverage syntactic skeletons along with the sentential contexts to identify hidden relation types. The syntactic skeletons are extracted under the guidance of prior syntax knowledge. During extraction, the input sentences are recursively decomposed into syntactically shorter and simpler chunks. Experimental results on the SemEval-2010 Task 8 benchmark show that incorporating syntactic skeletons into current DNN models enhances the task of relation classification. Our systems significantly surpass two strong baseline systems. One of the substantial advantages of our proposal is that this framework is extensible for most current DNN models.

References

  1. Nguyen Bach and Sameer Badaskar. 2007. A Review of Relation Extraction. Carnegie Mellon University.Google ScholarGoogle Scholar
  2. Jari Björne, Juho Heimonen, Filip Ginter, Antti Airola, Tapio Pahikkala, and Tapio Salakoski. 2011. Extracting contextualized complex biological events with rich graph-based feature sets. Computational Intelligence 27, 4 (2011), 541--557.Google ScholarGoogle ScholarCross RefCross Ref
  3. Razvan Bunescu and Raymond J. Mooney. 2005. A shortest path dependency kernel for relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 724--731.Google ScholarGoogle Scholar
  4. Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel P. Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12 (2011), 2493--2537.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Andrew M. Dai and Quoc V. Le. 2015. Semi-supervised sequence learning. In Advances in Neural Information Processing Systems. 3079--3087.Google ScholarGoogle Scholar
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171--4186.Google ScholarGoogle Scholar
  7. Cicero dos Santos, Bing Xiang, and Bowen Zhou. 2015. Classifying relations by ranking with convolutional neural networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 626--634.Google ScholarGoogle ScholarCross RefCross Ref
  8. Anthony Fader, Stephen Soderland, and Oren Etzioni. 2011. Identifying relations for open information extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1535--1545.Google ScholarGoogle Scholar
  9. Anthony Fader, Luke Zettlemoyer, and Oren Etzioni. 2014. Open question answering over curated and extracted knowledge bases. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 1156--1165.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Renxu Sun, Jing Jiang, Yee Fan, Tan, Hang Cui, Tat-Seng Chua, and Min-Yen Kan. 2005. Using syntactic and semantic relation analysis in question answering. In Proceedings of the 14th Text REtrieval Conference (TREC’05). Special Publication 500-266. National Institute of Standards and Technology.Google ScholarGoogle Scholar
  11. Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid Ó Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2009. SemEval-2010 Task 8: Multi-way classification of semantic relations between pairs of nominals. In Proceedings of the Workshop on Semantic Evaluations: Recent Achievements and Future Directions. 94--99.Google ScholarGoogle ScholarCross RefCross Ref
  12. Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 541--550.Google ScholarGoogle Scholar
  13. Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 328--339.Google ScholarGoogle ScholarCross RefCross Ref
  14. Yatian Shen and Xuanjing Huang. 2016. Attention-based convolutional neural network for semantic relation extraction. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING’16). 2526--2536.Google ScholarGoogle Scholar
  15. Nanda Kambhatla. 2004. Combining lexical, syntactic, and semantic features with maximum entropy models for extracting relations. In Proceedings of ACL 2004 on Interactive Poster and Demonstration Sessions. 22.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1746--1751.Google ScholarGoogle ScholarCross RefCross Ref
  17. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP’04). 388--395.Google ScholarGoogle Scholar
  18. Joohong Lee, Sangwoo Seo, and Yong Suk Choi. 2019. Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing. Symmetry 11, 6 (2019), 785.Google ScholarGoogle ScholarCross RefCross Ref
  19. Bart Mellebeek, Karolina Owczarzak, Declan Groves, Josef Van Genabith, and Andy Way. 2006. A syntactic skeleton for statistical machine translation. In Proceedings of the 11th Conference of the European Association for Machine Translation. 195--202.Google ScholarGoogle Scholar
  20. George A. Miller and Christiane Fellbaum. 1991. Semantic networks of English. Cognition 41, 1–3 (1991), 197--229.Google ScholarGoogle ScholarCross RefCross Ref
  21. Bonan Min, Ralph Grishman, Li Wan, Chang Wang, and David Gondek. 2013. Distant supervision for relation extraction with an incomplete knowledge base. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 777--782.Google ScholarGoogle Scholar
  22. Mike Mintz, Steven Bills, Rion Snow, and Dan Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. 1003--1011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Raymond J. Mooney and Razvan C. Bunescu. 2006. Subsequence kernels for relation extraction. In Advances in Neural Information Processing Systems. 171--178.Google ScholarGoogle Scholar
  24. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP’14)., 1532--1543.Google ScholarGoogle Scholar
  25. Matthew E. Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1756--1765.Google ScholarGoogle ScholarCross RefCross Ref
  26. Longhua Qian, Guodong Zhou, Fang Kong, Qiaoming Zhu, and Peide Qian. 2008. Exploiting constituent dependencies for tree kernel-based semantic relation extraction. In Proceedings of the 22nd International Conference on Computational Linguistics, Vol. 1. 697--704.Google ScholarGoogle ScholarCross RefCross Ref
  27. Sebastian Riedel, Limin Yao, and Andrew McCallum. 2010. Modeling relations and their mentions without labeled text. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part III. 148--163.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1715--1725.Google ScholarGoogle ScholarCross RefCross Ref
  29. Livio Baldini Soares, Nicholas Fitzgerald, Jeffrey Ling, and Tom Kwiatkowski. 2019. Matching the blanks: Distributional similarity for relation learning. In Proceedings of Annual Meeting of the Association for Computational Linguistics. 2895--2905.Google ScholarGoogle Scholar
  30. Richard Socher, Brody Huval, Christopher D. Manning, and Andrew Y. Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. 1201--1211.Google ScholarGoogle Scholar
  31. Fabian M. Suchanek, Georgiana Ifrim, and Gerhard Weikum. 2006. Combining linguistic and statistical analysis to extract relations from web documents. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, 712--717.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. 2012. Multi-instance multi-label learning for relation extraction. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 455--465.Google ScholarGoogle Scholar
  33. Ryuichi Takanobu, Tianyang Zhang, Jiexi Liu, and Minlie Huang. 2019. A hierarchical framework for relation extraction with reinforcement learning. In Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI’19), the 31st Innovative Applications of Artificial Intelligence Conference (IAAI’19), and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI’19). 7072--7079.Google ScholarGoogle Scholar
  34. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems. 5998--6008.Google ScholarGoogle Scholar
  35. Piek Vossen. 1998. A multilingual database with lexical semantic networks. Natural Language Engineering 10 (1998), 978--994.Google ScholarGoogle Scholar
  36. Linlin Wang, Zhu Cao, Gerard De Melo, and Zhiyuan Liu. 2016. Relation classification via multi-level attention CNNs. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Vol. 1. 1298--1307.Google ScholarGoogle ScholarCross RefCross Ref
  37. Fei Wu and Daniel S. Weld. 2010. Open information extraction using Wikipedia. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. 118--127.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Shanchan Wu and Yifan He. 2019. Enriching pre-trained language model with entity information for relation classification. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM’19). ACM, New York, NY, 2361--2364.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Minguang Xiao and Cong Liu. 2016. Semantic relation classification via hierarchical recurrent neural network with attention. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING’16). 1254--1263.Google ScholarGoogle Scholar
  40. Tong Xiao, Jingbo Zhu, and Chunliang Zhang. 2014. A hybrid approach to skeleton-based translation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 563--568.Google ScholarGoogle ScholarCross RefCross Ref
  41. Tong Xiao, Jingbo Zhu, Chunliang Zhang, and Tongran Liu. 2016. Syntactic skeleton-based translation. In Proceedings of the National Conference on Artificial Intelligence. 2856--2862.Google ScholarGoogle Scholar
  42. Wen-Tau Yih, Xiaodong He, and Christopher Meek. 2014. Semantic parsing for single-relation question answering. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 643--648.Google ScholarGoogle ScholarCross RefCross Ref
  43. Mo Yu, Matthew R. Gormley, and Mark Dredze. 2014. Factor-based compositional embedding models. In Proceedings of the NIPS Workshop on Learning Semantics.Google ScholarGoogle Scholar
  44. Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1753--1762.Google ScholarGoogle ScholarCross RefCross Ref
  45. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of the 25th International Conference on Computational Linguistics: Technical Papers (COLING’14). 2335--2344.Google ScholarGoogle Scholar
  46. Dongxu Zhang and Dong Wang. 2015. Relation classification via recurrent neural network. arXiv:1508.01006Google ScholarGoogle Scholar
  47. Shu Zhang, Dequan Zheng, Xinchen Hu, and Ming Yang. 2015. Bidirectional long short-term memory networks for relation classification. In Proceedings of the 29th Pacific Asia Conference on Language, Information, and Computation. 73--78.Google ScholarGoogle Scholar
  48. Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 207--212.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. An Extensible Framework of Leveraging Syntactic Skeleton for Semantic Relation Classification

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 19, Issue 6
      November 2020
      277 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3426881
      Issue’s Table of Contents

      Copyright © 2020 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 27 September 2020
      • Accepted: 1 May 2020
      • Revised: 1 April 2020
      • Received: 1 December 2019
      Published in tallip Volume 19, Issue 6

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!