Abstract
The task of semantic role labeling (SRL) is dedicated to finding the predicate-argument structure. Previous works on SRL are mostly supervised and do not consider the difficulty in labeling each example which can be very expensive and time-consuming. In this article, we present the first neural unsupervised model for SRL. To decompose the task as two argument related subtasks, identification and clustering, we propose a pipeline that correspondingly consists of two neural modules. First, we train a neural model on two syntax-aware statistically developed rules. The neural model gets the relevance signal for each token in a sentence, to feed into a BiLSTM, and then an adversarial layer for noise-adding and classifying simultaneously, thus enabling the model to learn the semantic structure of a sentence. Then we propose another neural model for argument role clustering, which is done through clustering the learned argument embeddings biased toward their dependency relations. Experiments on the CoNLL-2009 English dataset demonstrate that our model outperforms the previous state-of-the-art baseline in terms of non-neural models for argument identification and classification.
- Omri Abend and Ari Rappoport. 2010. Fully unsupervised core-adjunct argument classification. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL). 226–236. Google Scholar
Digital Library
- Omri Abend, Roi Reichart, and Ari Rappoport. 2009. Unsupervised argument identification for semantic role labeling. In Proceedings of the Joint Conference of the 47th Annual Meeting of the Association for Comutational Linguistics (ACL) and the 4th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing (AFNLP): Volume 1-Volume 1. Association for Computational Linguistics, 28–36. Google Scholar
Digital Library
- Ashraf Ali, Fayez Alfayez, and Hani Alquhayz. 2018. Semantic similarity measures between words: A brief survey. Scientific International Journal 30, 6 (2018), 907–914.Google Scholar
- Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1533–1544.Google Scholar
- Chris Biemann. 2006. Chinese whispers: An efficient graph clustering algorithm and its application to natural language processing problems. In Proceedings of the 1st Workshop on Graph Based Methods for Natural Language Processing. 73–80. Google Scholar
Digital Library
- Wanxiang Che, Min Zhang, AiTi Aw, ChewLim Tan, Ting Liu, and Sheng Li. 2008. Using a hybrid convolution tree kernel for semantic role labeling. ACM Transactions on Asian Language Information Processing (TALIP) 7, 4 (2008), 1–23. Google Scholar
Digital Library
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171–4186.Google Scholar
- John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, (July 2011), 2121–2159. Google Scholar
Digital Library
- William Foland and James Martin. 2015. Dependency-based semantic role labeling using convolutional neural networks. In Proceedings of the 4th Joint Conference on Lexical and Computational Semantics. 279–288.Google Scholar
Cross Ref
- Hagen Fürstenau and Mirella Lapata. 2009. Graph alignment for semi-supervised semantic role labeling. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP): Volume 1-Volume 1. 11–20. Google Scholar
Digital Library
- Nikhil Garg and James Henderson. 2012. Unsupervised semantic role induction with global role ordering. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL): Short Papers-Volume 2. 145–149. Google Scholar
Digital Library
- Daniel Gildea and Daniel Jurafsky. 2002. Automatic labeling of semantic roles. Computational Linguistics 28, 3 (2002), 245–288. Google Scholar
Digital Library
- Andrew S. Gordon and Reid Swanson. 2007. Generalizing Semantic Role Annotations Across Syntactically Similar Verbs. Technical Report. Institute for Creative Technologies, University of Southern California, Marina Del Ray, CA.Google Scholar
- Trond Grenager and Christopher D. Manning. 2006. Unsupervised discovery of a statistical verb lexicon. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1–8. Google Scholar
Digital Library
- Shexia He, Zuchao Li, and Hai Zhao. 2019a. Syntax-aware multilingual semantic role labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP).Google Scholar
Cross Ref
- Shexia He, Zuchao Li, and Hai Zhao. 2019b. Syntax-aware multilingual semantic role labeling. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP). 5353–5362.Google Scholar
Cross Ref
- Shexia He, Zuchao Li, Hai Zhao, and Hongxiao Bai. 2018. Syntax for semantic role labeling, to be, or not to be. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 1: Long Papers). 2061–2071.Google Scholar
Cross Ref
- Richard Johansson and Pierre Nugues. 2008. Dependency-based semantic role labeling of PropBank. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP). 69–78. Google Scholar
Digital Library
- Karin Kipper, Hoa Trang Dang, Martha Palmer, et al. 2000. Class-based construction of a verb lexicon. In Proceedings of the 17th National Conference on Artificial Intelligence and 12th Conference on Innovative Applications of Artificial Intelligence (AAAI/IAAI). 691–696. Google Scholar
Digital Library
- Joel Lang and Mirella Lapata. 2011a. Unsupervised semantic role induction via split-merge clustering. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL): Human Language Technologies-Volume 1. 1117–1126. Google Scholar
Digital Library
- Joel Lang and Mirella Lapata. 2011b. Unsupervised semantic role induction with graph partitioning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP). 1320–1331. Google Scholar
Digital Library
- Omer Levy and Yoav Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 2: Short Papers). 302–308.Google Scholar
Cross Ref
- Zuchao Li, Chaoyu Guan, Hai Zhao, Rui Wang, Kevin Parnow, and Zhuosheng Zhang. 2020a. Memory network for linguistic structure parsing. IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP) 28 (2020), 2743–2755.Google Scholar
Digital Library
- Zuchao Li, Shexia He, Jiaxun Cai, Zhuosheng Zhang, Hai Zhao, Gongshen Liu, Linlin Li, and Luo Si. 2018. A unified syntax-aware framework for semantic role labeling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2401–2411.Google Scholar
Cross Ref
- Zuchao Li, Shexia He, Hai Zhao, Yiqing Zhang, Zhuosheng Zhang, Xi Zhou, and Xiang Zhou. 2019. Dependency or span, end-to-end uniform semantic role labeling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 6730–6737.Google Scholar
Cross Ref
- Zuchao Li, Hai Zhao, Rui Wang, and Kevin Parnow. 2020b. High-order semantic role labeling. In Findings of the Association for Computational Linguistics: EMNLP 2020. 1134–1151.Google Scholar
Cross Ref
- Yi Luan, Yangfeng Ji, Hannaneh Hajishirzi, and Boyang Li. 2016. Multiplicative representations for unsupervised semantic role induction. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 2: Short Papers). 118–123.Google Scholar
Cross Ref
- Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1506–1515.Google Scholar
Cross Ref
- Lluís Màrquez, Xavier Carreras, Kenneth C Litkowski, and Suzanne Stevenson. 2008. Semantic role labeling: An introduction to the special issue. Computational Linguistics 34, 2 (2008), 145–159. Google Scholar
Digital Library
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111–3119. Google Scholar
Digital Library
- Kashif Munir, Hai Zhao, and Zuchao Li. 2021. Adaptive convolution for semantic role labeling. IEEE/ACM Transactions on Audio, Speech, and Language Processing 29 (2021), 782–791.Google Scholar
Digital Library
- Joakim Nivre, Johan Hall, and Jens Nilsson. 2006. Maltparser: A data-driven parser-generator for dependency parsing. In LREC, Vol. 6. 2216–2219.Google Scholar
- Sebastian Padó and Mirella Lapata. 2009. Cross-lingual annotation projection for semantic roles. Journal of Artificial Intelligence Research 36 (2009), 307–340. Google Scholar
Digital Library
- Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: An annotated corpus of semantic roles. Computational Linguistics 31, 1 (2005), 71–106. Google Scholar
Digital Library
- Sameer S. Pradhan, Wayne Ward, and James H. Martin. 2008. Towards robust semantic role labeling. Computational Linguistics 34, 2 (2008), 289–310. Google Scholar
Digital Library
- Michael Roth and Mirella Lapata. 2016. Neural semantic role labeling with dependency path embeddings. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL). 1192–1202.Google Scholar
Cross Ref
- Chen Shi, Shujie Liu, Shuo Ren, Shi Feng, Mu Li, Ming Zhou, Xu Sun, and Houfeng Wang. 2016. Knowledge-based semantic embedding for machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 1: Long Papers). 2245–2254.Google Scholar
Cross Ref
- Mihai Surdeanu, Sanda Harabagiu, John Williams, and Paul Aarseth. 2003. Using predicate-argument structures for information extraction. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics (ACL). 8–15. Google Scholar
Digital Library
- Robert Swier and Suzanne Stevenson. 2005. Exploiting a verb lexicon in automatic semantic role labelling. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (EMNLP). 883–890. Google Scholar
Digital Library
- Robert S. Swier and Suzanne Stevenson. 2004. Unsupervised semantic role labelling. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP). 95–102.Google Scholar
- Ivan Titov and Ehsan Khoddam. 2014. Unsupervised induction of semantic roles within a reconstruction-error minimization framework. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics (ACL): Human Language Technologies. 1–10.Google Scholar
- Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of Machine Learning Research 11, (Dec. 2010), 3371–3408. Google Scholar
Digital Library
- Haitong Yang and Chengqing Zong. 2016. Learning generalized features for semantic role labeling. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 15, 4 (2016), 1–16. Google Scholar
Digital Library
- Wen-tau Yih, Matthew Richardson, Chris Meek, Ming-Wei Chang, and Jina Suh. 2016. The value of semantic parse labeling for knowledge base question answering. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL) (Volume 2: Short Papers). 201–206.Google Scholar
- Hai Zhao, Wenliang Chen, and Chunyu Kit. 2009. Semantic dependency parsing of NomBank and PropBank: An efficient integrated approach via a large-scale feature selection. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP): Volume 1-Volume 1. Association for Computational Linguistics, 30–39. Google Scholar
Digital Library
- Junru Zhou, Zuchao Li, and Hai Zhao. 2020. Parsing all: Syntax and semantics, dependencies and spans. In Findings of the Association for Computational Linguistics: EMNLP 2020. 4438–4449.Google Scholar
Cross Ref
Index Terms
Neural Unsupervised Semantic Role Labeling
Recommendations
Semi-Supervised Semantic Role Labeling with Bidirectional Language Models
The recent success of neural networks in NLP applications has provided a strong impetus to develop supervised models for semantic role labeling (SRL) that forego the requirement for extensive feature engineering. Recent state-of-the-art approaches require ...
Semantic Role Labeling System for Persian Language
In this article, we present an automatic semantic role labeling system in Persian consisting of two modules: argument identification for specifying argument spans and argument classification for categorizing their semantic roles. Our modules have been ...
Predicate-attention neural model for Chinese semantic role labeling
AbstractSemantic role labeling functions to convey the meaning of a sentence through forming a predicate-argument structure directed at the specific predicate. In recent years, end-to-end semantic role labeling methods associated with the deep ...






Comments