Abstract
Dependency parsing is a fundamental problem in natural language processing. We introduce a novel dependency-parsing framework called head-pointing--based dependency parsing. In this framework, we cast the Korean dependency parsing problem as a statistical head-pointing and arc-labeling problem. To address this problem, a novel neural network called the multitask pointer network is devised for a neural sequential head-pointing and type-labeling architecture. Our approach does not require any handcrafted features or language-specific rules to parse dependency. Furthermore, it achieves state-of-the-art performance for Korean dependency parsing.
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations.Google Scholar
- Xavier Carreras. 2007. Experiments with a higher-order projective dependency parser. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL’07). 957--961.Google Scholar
- Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 740--750.Google Scholar
- Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, et al. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14).Google Scholar
Cross Ref
- Jinho D. Choi and Martha Palmer. 2011. Statistical dependency parsing in Korean: From corpus generation to automatic parsing. In Proceedings of the 2nd Workshop on Statistical Parsing of Morphologically Rich Languages. 1--11. Google Scholar
Digital Library
- Jan Chorowski, Michał Zapotoczny, and Paweł Rychlikowski. 2016. Read, tag, and parse all at once, or fully-neural dependency parsing. arXiv:1609.03441.Google Scholar
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12, 2493--2537. Google Scholar
Digital Library
- Michael A. Covington. 2001. A fundamental algorithm for dependency parsing. In Proceedings of the 39th Annual ACM Southeast Conference. 95--102.Google Scholar
- Aron Culotta and Jeffrey Sorensen. 2004. Dependency tree kernels for relation extraction. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. 423. Google Scholar
Digital Library
- Yuan Ding and Martha Palmer. 2005. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 541--548. Google Scholar
Digital Library
- Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 334--343.Google Scholar
- Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of the 16th Conference on Computational Linguistics, Vol. 1. 340--345. Google Scholar
Digital Library
- Yoav Goldberg and Joakim Nivre. 2013. Training deterministic parsers with non-deterministic oracles. Transactions of the Association for Computational Linguistics 1, 403--414.Google Scholar
Cross Ref
- James Henderson. 2003. Inducing history representations for broad coverage statistical parsing. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, Vol. 1. 24--31. Google Scholar
Digital Library
- James Henderson, Paola Merlo, Ivan Titov, and Gabriele Musillo. 2013. Multilingual joint parsing of syntactic and semantic dependencies with a latent variable model. Computational Linguistics 39, 4 (2013), 949--998. Google Scholar
Digital Library
- James Henderson and Ivan Titov. 2010. Incremental sigmoid belief networks for grammar learning. Journal of Machine Learning Research 11, 3541--3570. Google Scholar
Digital Library
- Terry Koo and Michael Collins. 2010. Efficient third-order dependency parsers. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. 1--11. Google Scholar
Digital Library
- Changki Lee, Junseok Kim, and Jeonghee Kim. 2014. Korean dependency parsing using deep learning. In Proceedings of the Annual Conference on Human and Cognitive Language Technology.Google Scholar
- Changki Lee, Junseok Kim, Jeonghee Kim, and Hyunki Kim. 2013. Joint models for Korean word spacing and POS tagging using structural SVM. Journal of KIISE: Software and Applications 40, 12 (2013), 826--832.Google Scholar
- Jianri Li and Jong-Hyeok Lee. 2015. Korean transition-based dependency parsing with recurrent neural network. KIISE Transactions on Computing Practices 21, 8 (2015), 567--571.Google Scholar
Cross Ref
- Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1412--1421.Google Scholar
- Xuezhe Ma and Hai Zhao. 2012. Fourth-order dependency parsing. In Proceedingsof the 25th International Conference on Computational Linguistics (Posters). 785--796.Google Scholar
- Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 91--98. Google Scholar
Digital Library
- Ryan T. McDonald and Fernando C. N. Pereira. 2006. Online learning of approximate dependency parsing algorithms. In Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics (EACL’06). 81--88.Google Scholar
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781.Google Scholar
- Jin-Woo Min and Seung-Hoon Na. 2016. Korean dependency parsing using sequence to sequence model. In Proceedings of Korean Institute of Information Scientists and Engineers (KISSE’16). 491--493.Google Scholar
- Jin-Woo Min and Seung-Hoon Na. 2017. SyntaxNet models using transition based recurrent unit for Korean dependency parsing. In Proceedings of Korean Institute of Information Scientists and Engineers (KISSE’17). 602--604.Google Scholar
- Seung-Hoon Na, Kangil Kim, and Young-Kil Kim. 2016. Stack LSTMs for transition-based Korean dependency parsing. In Proceedings of the Korea Computer Congress (KCC’16). 732--734.Google Scholar
- Seung-Hoon Na, Jong-Hoon Shin, and Kangil Kim. 2016. Improving stack LSTMs by combining syllables and morphemes for Korean dependency parsing. In Proceedings of the Annual Conference on Human and Cognitive Language Technology.Google Scholar
- Joakim Nivre. 2003. An efficient algorithm for projective dependency parsing. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT’03).Google Scholar
- Joakim Nivre and Mario Scholz. 2004. Deterministic dependency parsing of English text. In Proceedings of the 20th International Conference on Computational Linguistics. 64. Google Scholar
Digital Library
- Kenji Sagae. 2009. Analysis of discourse structure with syntactic dependencies and data-driven shift-reduce parsing. In Proceedings of the 11th International Conference on Parsing Technologies. 81--84. Google Scholar
Digital Library
- Yusuke Shinyama, Satoshi Sekine, and Kiyoshi Sudo. 2002. Automatic paraphrase acquisition from news articles. In Proceedings of the 2nd International Conference on Human Language Technology Research. 313--318. Google Scholar
Digital Library
- Richard Socher, John Bauer, Christopher D. Manning, and Andrew Y. Ng. 2013. Parsing with compositional vector grammars. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL’13), Vol. 1. 455--465.Google Scholar
- Richard Socher, Cliff C. Lin, Chris Manning, and Andrew Y. Ng. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML’11). 129--136. Google Scholar
Digital Library
- Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS’14). 3104--3112. Google Scholar
Digital Library
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, et al. 2017. Attention is all you need. arXiv:1706.03762.Google Scholar
- Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’15). 2692--2700. Google Scholar
Digital Library
- Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’15). 2773--2781. Google Scholar
Digital Library
- Wenhui Wang and Baobao Chang. 2016. Graph-based dependency parsing with bidirectional LSTM. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16).Google Scholar
Cross Ref
- Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT’03), Vol. 3. 195--206.Google Scholar
- J. T. Yoon and K.-S. Choi. 1999. Study on Kaist Corpus. Technical Report CS-TR-99-139. Department of Computer Science, KAIST.Google Scholar
- Xingxing Zhang, Jianpeng Cheng, and Mirella Lapata. 2016. Dependency parsing as head selection. arXiv:1606.01280.Google Scholar
- Yue Zhang and Stephen Clark. 2008. A tale of two parsers: Investigating and combining graph-based and transition-based dependency parsing using beam-search. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 562--571. Google Scholar
Digital Library
Index Terms
Multitask Pointer Network for Korean Dependency Parsing
Recommendations
Multitask Pointer Network for multi-representational parsing
AbstractDependency and constituent trees are widely used by many artificial intelligence applications for representing the syntactic structure of human languages. Typically, these structures are separately produced by either dependency or ...
Highlights- Dependency and constituent trees are used by many artificial intelligence applications.
Korean Dependency Parsing with Proper Noun Encodin
ICIIT '19: Proceedings of the 2019 4th International Conference on Intelligent Information TechnologyDependency parsing is the process of analyzing the linguistic relationship of words that make up a sentence. In natural language processing using deep learning, previously registered words are represented in a continuous vector space. However, if proper ...
Dependency parsing with bottom-up Hierarchical Pointer Networks
AbstractDependency parsing is a crucial step towards deep language understanding and, therefore, widely demanded by numerous Natural Language Processing applications. In particular, left-to-right and top-down transition-based algorithms that ...
Highlights- The left-to-right transition-based algorithm is a state-of-the-art dependency parser.






Comments