Abstract
Recently, neural approaches for transition-based dependency parsing have become one of the state-of-the art methods for performing dependency parsing tasks in many languages. In neural transition-based parsing, a parser state representation is first computed from the configuration of a stack and a buffer, which is then fed into a feed-forward neural network model that predicts the next transition action. Given that words are basic elements of a stack and buffer, a parser state representation is considerably affected by how a word representation is defined. In particular, word representation issues become more critical in morphologically rich languages such as Korean, as the set of potential words is not bound but introduce the second-order vocabulary complexity, called the phrase vocabulary complexity due to the agglutinative characteristics of the language. In this article, we propose a hybrid word representation that combines two compositional word representations, each of which is derived from representations of syllables and morphemes, respectively. Our underlying assumption for this hybrid word representation is that, because both syllables and morphemes are two common ways of decomposing Korean words, it is expected that their effects in inducing word representation are complementary to one another. Experimental results carried on Sejong and SPMRL 2014 datasets show that our proposed hybrid word representation leads to the state-of-the-art performance.
- Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. 2016. Globally normalized transition-based neural networks. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 2442--2452.Google Scholar
Cross Ref
- Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2015. Improved transition-based parsing by modeling characters instead of words with LSTMs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 349--359.Google Scholar
- Miguel Ballesteros, Yoav Goldberg, Chris Dyer, and Noah A. Smith. 2016. Training with exploration improves a greedy stack LSTM parser. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP’16). 2005--2010.Google Scholar
- Yoshua Bengio. 2009. Learning deep architectures for AI. Found. Trends Mach. Learn. 2, 1 (2009), 1--127. Google Scholar
Digital Library
- Anders Björkelund, Özlem Çetinoğlu, Agnieszka Faleńska, Richárd Farkas, Thomas Mueller, Wolfgang Seeker, and Zsolt Szántó. 2014. Introducing the IMS-Wrocław-Szeged-CIS entry at the SPMRL 2014 shared task: Reranking and morpho-syntax meet unlabeled data. In Proceedings of the 1st Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages. 97--102.Google Scholar
- Anders Björkelund and Joakim Nivre. 2015. Non-deterministic oracles for unrestricted non-projective transition-based dependency parsing. In Proceedings of the 14th International Conference on Parsing Technologies. 76--86.Google Scholar
Cross Ref
- Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In Empirical Methods in Natural Language Processing (EMNLP’14). 740--750.Google Scholar
- Maengsik Choi, Seokwon Jeong, and Harksoo Kim. 2014. Dependency structure analysis and dependency label annotation using CRFs. KIISE Trans. Comput. Pract. (in Korean) 41, 4 (2014), 302--308.Google Scholar
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12 (2011), 2493--2537. Google Scholar
Digital Library
- Cícero Nogueira dos Santos and Bianca Zadrozny. 2014. Learning character-level representations for part-of-speech tagging. In Proceedings of the 31th International Conference on Machine Learning (ICML’14). 1818--1826. Google Scholar
Digital Library
- Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL’15). 334--343.Google Scholar
- Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A. Smith. 2016. Recurrent neural network grammars. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 199--209.Google Scholar
- Yoav Goldberg. 2015. A primer on neural network models for natural language processing. arXiv Preprint arXiv:1510.00726 (2015).Google Scholar
- Sébastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On using very large target vocabulary for neural machine translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP’15). 1--10.Google Scholar
Cross Ref
- Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional LSTM feature representations. Trans. Assoc. Comput. Linguist. 4 (2016), 313--327.Google Scholar
Cross Ref
- Sandra Kubler, Ryan McDonald, Joakim Nivre, and Graeme Hirst. 2009. Dependency Parsing. Morgan and Claypool Publishers. Google Scholar
Digital Library
- Taku Kudo and Yuji Matsumoto. 2002. Japanese dependency analysis using cascaded chunking. In Proceedings of the 6th Conference on Natural Language Learning (COLING’02). 1--7. Google Scholar
Digital Library
- Adhiguna Kuncoro, Miguel Ballesteros, Lingpeng Kong, Chris Dyer, Graham Neubig, and Noah A. Smith. 2017. What do recurrent neural network grammars learn about syntax? In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’17). 1249--1258.Google Scholar
- Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 260--270.Google Scholar
Cross Ref
- Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 260--270.Google Scholar
Cross Ref
- Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521 (2015), 436--444.Google Scholar
Cross Ref
- Changki Lee, Junseok Kim, and Jeonghee Kim. 2014. Korean dependency parsing using deep learning. In Proceedings of the 26th Annual Conference on Human and Cognitive Language Technology (in Korean).Google Scholar
- Jianri Li and Jong-Hyeok Lee. 2015. Korean transition-based dependency parsing with recurrent neural network. KIISE Trans. Comput. Pract. (in Korean) 21, 8 (2015), 567--571.Google Scholar
Cross Ref
- Joon-Ho Lim, Yeo-Chan Yoon, Yongjin Bae, Hyunki Kim, and Kyu-Chul Lee. 2014. Korean dependency parsing model based on transition system using head final constraint. In Proceedings of Korea Computer Congress 2014 (in Korean).Google Scholar
- Soojong Lim, Youngtae Kim, and Dongyul Ra. 2011. Korean dependency parsing based on machine learning of feature weights. KIISE Trans. Comput. Pract. (in Korean) 38, 4 (2011), 214--223.Google Scholar
- Wang Ling, Chris Dyer, Alan W. Black, Isabel Trancoso, Ramon Fermandez, Silvio Amir, Luis Marujo, and Tiago Luis. 2015. Finding function in form: Compositional character models for open vocabulary word representation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 1520--1530.Google Scholar
Cross Ref
- Minh-Thang Luong and Christopher D. Manning. 2016. Achieving open vocabulary neural machine translation with hybrid word-character models. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 1054--1063.Google Scholar
- Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, and Wojciech Zaremba. 2015. Addressing the rare word problem in neural machine translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP’15). 11--19.Google Scholar
Cross Ref
- Seung-Hoon Na. 2014. Third-order dependency parsing of korean. In Proceedings of the 26th Annual Conference on Human and Cognitive Language Technology (in Korean).Google Scholar
- Seung-Hoon Na. 2015. Conditional random fields for korean morpheme segmentation and POS tagging. ACM Trans. Asian Low-Res. Lang. Info. Process. 14, 3 (2015), 10:1--10:16. Google Scholar
Digital Library
- Seung-Hoon Na and Sangkeun Jung. 2014. Integrated model of graph-based and transition-based methods for korean dependency parsing. In Proceedings of the 26th Annual Conference on Human and Cognitive Language Technology (in Korean).Google Scholar
- Seung-Hoon Na, Kangil Kim, and Young-Kil Kim. 2016. Stack LSTMs for transition-based korean dependency parsing. In Proceedings of Korea Computer Congress 2016 (in Korean).Google Scholar
- Seung-Hoon Na, Jong-Hoon Shin, and Kangil Kim. 2016. Improving stack LSTMs by combining syllables and morphemes for korean dependency parsing. In Proceedings of the 28th Annual Conference on Human and Cognitive Language Technology (in Korean).Google Scholar
- Hwee Tou Ng and Jin Kiat Low. 2004. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP’04). 277--284.Google Scholar
- Joakim Nivre. 2003. An efficient algorithm for projective dependency parsing. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT’03). 149--160.Google Scholar
- Joakim Nivre. 2008. Algorithms for deterministic incremental dependency parsing. Comput. Linguist. 34, 4 (2008), 513--553. Google Scholar
Digital Library
- Joakim Nivre. 2009. Non-projective dependency parsing in expected linear time. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing (ACL-IJCNLP’09). 351--359. Google Scholar
Digital Library
- Jin-Young Oh and Jeong-Won Cha. 2013. Korean dependency parsing using key eojoel. KIISE Trans. Comput. Pract. (in Korean) 40, 10 (2013), 600--608.Google Scholar
- Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1532--1543. Retrieved from http://www.aclweb.org/anthology/D14-1162.Google Scholar
- Djamé Seddah, Sandra Kübler, and Reut Tsarfaty. 2014. Introducing the SPMRL 2014 shared task on parsing morphologically-rich languages. In Proceedings of the 1st Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages. 103--109.Google Scholar
- Richard Socher, Cliff C. Lin, Chris Manning, and Andrew Y. Ng. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML’11). 129--136. Google Scholar
Digital Library
- David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured training for neural network transition-based parsing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP’09). 323--333.Google Scholar
Cross Ref
- Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In The 8th International Workshop of Parsing Technologies (IWPT’03). 195--206.Google Scholar
- Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT’11). 188--193. Google Scholar
Digital Library
Index Terms
Transition-Based Korean Dependency Parsing Using Hybrid Word Representations of Syllables and Morphemes with LSTMs
Recommendations
Improving Vietnamese Dependency Parsing Using Distributed Word Representations
SoICT '15: Proceedings of the 6th International Symposium on Information and Communication TechnologyDependency parsing has become an important line of research in natural language processing in recent years. This is due to its usefulness in a wide variety of real world applications. This paper presents the improvement of Vietnamese dependency parsing ...
Korean Dependency Parsing with Proper Noun Encodin
ICIIT '19: Proceedings of the 2019 4th International Conference on Intelligent Information TechnologyDependency parsing is the process of analyzing the linguistic relationship of words that make up a sentence. In natural language processing using deep learning, previously registered words are represented in a continuous vector space. However, if proper ...
Comparison of performance of enhanced morpheme-based language model with different word-based language models for improving the performance of Tamil speech recognition system
This paper describes a new technique of language modeling for a highly inflectional Dravidian language, Tamil. It aims to alleviate the main problems encountered in processing of Tamil language, like enormous vocabulary growth caused by the large number ...






Comments