skip to main content
research-article

Neural Unsupervised Semantic Role Labeling

Authors Info & Claims
Published:12 August 2021Publication History
Skip Abstract Section

Abstract

The task of semantic role labeling (SRL) is dedicated to finding the predicate-argument structure. Previous works on SRL are mostly supervised and do not consider the difficulty in labeling each example which can be very expensive and time-consuming. In this article, we present the first neural unsupervised model for SRL. To decompose the task as two argument related subtasks, identification and clustering, we propose a pipeline that correspondingly consists of two neural modules. First, we train a neural model on two syntax-aware statistically developed rules. The neural model gets the relevance signal for each token in a sentence, to feed into a BiLSTM, and then an adversarial layer for noise-adding and classifying simultaneously, thus enabling the model to learn the semantic structure of a sentence. Then we propose another neural model for argument role clustering, which is done through clustering the learned argument embeddings biased toward their dependency relations. Experiments on the CoNLL-2009 English dataset demonstrate that our model outperforms the previous state-of-the-art baseline in terms of non-neural models for argument identification and classification.

References

  1. Omri Abend and Ari Rappoport. 2010. Fully unsupervised core-adjunct argument classification. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL). 226–236. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Omri Abend, Roi Reichart, and Ari Rappoport. 2009. Unsupervised argument identification for semantic role labeling. In Proceedings of the Joint Conference of the 47th Annual Meeting of the Association for Comutational Linguistics (ACL) and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing (AFNLP): Volume 1-Volume 1. Association for Computational Linguistics, 28–36. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Ashraf Ali, Fayez Alfayez, and Hani Alquhayz. 2018. Semantic similarity measures between words: A brief survey. Scientific International Journal 30, 6 (2018), 907–914.Google ScholarGoogle Scholar
  4. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1533–1544.Google ScholarGoogle Scholar
  5. Chris Biemann. 2006. Chinese whispers: An efficient graph clustering algorithm and its application to natural language processing problems. In Proceedings of the 1st Workshop on Graph Based Methods for Natural Language Processing. 73–80. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Wanxiang Che, Min Zhang, AiTi Aw, ChewLim Tan, Ting Liu, and Sheng Li. 2008. Using a hybrid convolution tree kernel for semantic role labeling. ACM Transactions on Asian Language Information Processing (TALIP) 7, 4 (2008), 1–23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171–4186.Google ScholarGoogle Scholar
  8. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, (July 2011), 2121–2159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. William Foland and James Martin. 2015. Dependency-based semantic role labeling using convolutional neural networks. In Proceedings of the 4th Joint Conference on Lexical and Computational Semantics. 279–288.Google ScholarGoogle ScholarCross RefCross Ref
  10. Hagen Fürstenau and Mirella Lapata. 2009. Graph alignment for semi-supervised semantic role labeling. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP): Volume 1-Volume 1. 11–20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Nikhil Garg and James Henderson. 2012. Unsupervised semantic role induction with global role ordering. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL): Short Papers-Volume 2. 145–149. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics 28, 3 (2002), 245–288. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Andrew S. Gordon and Reid Swanson. 2007. Generalizing Semantic Role Annotations Across Syntactically Similar Verbs. Technical Report. Institute for Creative Technologies, University of Southern California, Marina Del Ray, CA.Google ScholarGoogle Scholar
  14. Trond Grenager and Christopher D. Manning. 2006. Unsupervised discovery of a statistical verb lexicon. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1–8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Shexia He, Zuchao Li, and Hai Zhao. 2019a. Syntax-aware multilingual semantic role labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP).Google ScholarGoogle ScholarCross RefCross Ref
  16. Shexia He, Zuchao Li, and Hai Zhao. 2019b. Syntax-aware multilingual semantic role labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP). 5353–5362.Google ScholarGoogle ScholarCross RefCross Ref
  17. Shexia He, Zuchao Li, Hai Zhao, and Hongxiao Bai. 2018. Syntax for semantic role labeling, to be, or not to be. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 1: Long Papers). 2061–2071.Google ScholarGoogle ScholarCross RefCross Ref
  18. Richard Johansson and Pierre Nugues. 2008. Dependency-based semantic role labeling of PropBank. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP). 69–78. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Karin Kipper, Hoa Trang Dang, Martha Palmer, et al. 2000. Class-based construction of a verb lexicon. In Proceedings of the 17th National Conference on Artificial Intelligence and 12th Conference on Innovative Applications of Artificial Intelligence (AAAI/IAAI). 691–696. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Joel Lang and Mirella Lapata. 2011a. Unsupervised semantic role induction via split-merge clustering. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL): Human Language Technologies-Volume 1. 1117–1126. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Joel Lang and Mirella Lapata. 2011b. Unsupervised semantic role induction with graph partitioning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). 1320–1331. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 2: Short Papers). 302–308.Google ScholarGoogle ScholarCross RefCross Ref
  23. Zuchao Li, Chaoyu Guan, Hai Zhao, Rui Wang, Kevin Parnow, and Zhuosheng Zhang. 2020a. Memory network for linguistic structure parsing. IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP) 28 (2020), 2743–2755.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Zuchao Li, Shexia He, Jiaxun Cai, Zhuosheng Zhang, Hai Zhao, Gongshen Liu, Linlin Li, and Luo Si. 2018. A unified syntax-aware framework for semantic role labeling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2401–2411.Google ScholarGoogle ScholarCross RefCross Ref
  25. Zuchao Li, Shexia He, Hai Zhao, Yiqing Zhang, Zhuosheng Zhang, Xi Zhou, and Xiang Zhou. 2019. Dependency or span, end-to-end uniform semantic role labeling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6730–6737.Google ScholarGoogle ScholarCross RefCross Ref
  26. Zuchao Li, Hai Zhao, Rui Wang, and Kevin Parnow. 2020b. High-order semantic role labeling. In Findings of the Association for Computational Linguistics: EMNLP 2020. 1134–1151.Google ScholarGoogle ScholarCross RefCross Ref
  27. Yi Luan, Yangfeng Ji, Hannaneh Hajishirzi, and Boyang Li. 2016. Multiplicative representations for unsupervised semantic role induction. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 2: Short Papers). 118–123.Google ScholarGoogle ScholarCross RefCross Ref
  28. Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1506–1515.Google ScholarGoogle ScholarCross RefCross Ref
  29. Lluís Màrquez, Xavier Carreras, Kenneth C Litkowski, and Suzanne Stevenson. 2008. Semantic role labeling: An introduction to the special issue. Computational Linguistics 34, 2 (2008), 145–159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111–3119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Kashif Munir, Hai Zhao, and Zuchao Li. 2021. Adaptive convolution for semantic role labeling. IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021), 782–791.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Joakim Nivre, Johan Hall, and Jens Nilsson. 2006. Maltparser: A data-driven parser-generator for dependency parsing. In LREC, Vol. 6. 2216–2219.Google ScholarGoogle Scholar
  33. Sebastian Padó and Mirella Lapata. 2009. Cross-lingual annotation projection for semantic roles. Journal of Artificial Intelligence Research 36 (2009), 307–340. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics 31, 1 (2005), 71–106. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Sameer S. Pradhan, Wayne Ward, and James H. Martin. 2008. Towards robust semantic role labeling. Computational Linguistics 34, 2 (2008), 289–310. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Michael Roth and Mirella Lapata. 2016. Neural semantic role labeling with dependency path embeddings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). 1192–1202.Google ScholarGoogle ScholarCross RefCross Ref
  37. Chen Shi, Shujie Liu, Shuo Ren, Shi Feng, Mu Li, Ming Zhou, Xu Sun, and Houfeng Wang. 2016. Knowledge-based semantic embedding for machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 1: Long Papers). 2245–2254.Google ScholarGoogle ScholarCross RefCross Ref
  38. Mihai Surdeanu, Sanda Harabagiu, John Williams, and Paul Aarseth. 2003. Using predicate-argument structures for information extraction. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL). 8–15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Robert Swier and Suzanne Stevenson. 2005. Exploiting a verb lexicon in automatic semantic role labelling. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (EMNLP). 883–890. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Robert S. Swier and Suzanne Stevenson. 2004. Unsupervised semantic role labelling. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP). 95–102.Google ScholarGoogle Scholar
  41. Ivan Titov and Ehsan Khoddam. 2014. Unsupervised induction of semantic roles within a reconstruction-error minimization framework. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics (ACL): Human Language Technologies. 1–10.Google ScholarGoogle Scholar
  42. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research 11, (Dec. 2010), 3371–3408. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Haitong Yang and Chengqing Zong. 2016. Learning generalized features for semantic role labeling. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 15, 4 (2016), 1–16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Wen-tau Yih, Matthew Richardson, Chris Meek, Ming-Wei Chang, and Jina Suh. 2016. The value of semantic parse labeling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 2: Short Papers). 201–206.Google ScholarGoogle Scholar
  45. Hai Zhao, Wenliang Chen, and Chunyu Kit. 2009. Semantic dependency parsing of NomBank and PropBank: An efficient integrated approach via a large-scale feature selection. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP): Volume 1-Volume 1. Association for Computational Linguistics, 30–39. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Junru Zhou, Zuchao Li, and Hai Zhao. 2020. Parsing all: Syntax and semantics, dependencies and spans. In Findings of the Association for Computational Linguistics: EMNLP 2020. 4438–4449.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Neural Unsupervised Semantic Role Labeling

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!