skip to main content
research-article

Transition-Based Korean Dependency Parsing Using Hybrid Word Representations of Syllables and Morphemes with LSTMs

Authors Info & Claims
Published:14 December 2018Publication History
Skip Abstract Section

Abstract

Recently, neural approaches for transition-based dependency parsing have become one of the state-of-the art methods for performing dependency parsing tasks in many languages. In neural transition-based parsing, a parser state representation is first computed from the configuration of a stack and a buffer, which is then fed into a feed-forward neural network model that predicts the next transition action. Given that words are basic elements of a stack and buffer, a parser state representation is considerably affected by how a word representation is defined. In particular, word representation issues become more critical in morphologically rich languages such as Korean, as the set of potential words is not bound but introduce the second-order vocabulary complexity, called the phrase vocabulary complexity due to the agglutinative characteristics of the language. In this article, we propose a hybrid word representation that combines two compositional word representations, each of which is derived from representations of syllables and morphemes, respectively. Our underlying assumption for this hybrid word representation is that, because both syllables and morphemes are two common ways of decomposing Korean words, it is expected that their effects in inducing word representation are complementary to one another. Experimental results carried on Sejong and SPMRL 2014 datasets show that our proposed hybrid word representation leads to the state-of-the-art performance.

References

  1. Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. 2016. Globally normalized transition-based neural networks. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 2442--2452.Google ScholarGoogle ScholarCross RefCross Ref
  2. Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2015. Improved transition-based parsing by modeling characters instead of words with LSTMs. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 349--359.Google ScholarGoogle Scholar
  3. Miguel Ballesteros, Yoav Goldberg, Chris Dyer, and Noah A. Smith. 2016. Training with exploration improves a greedy stack LSTM parser. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP’16). 2005--2010.Google ScholarGoogle Scholar
  4. Yoshua Bengio. 2009. Learning deep architectures for AI. Found. Trends Mach. Learn. 2, 1 (2009), 1--127. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Anders Björkelund, Özlem Çetinoğlu, Agnieszka Faleńska, Richárd Farkas, Thomas Mueller, Wolfgang Seeker, and Zsolt Szántó. 2014. Introducing the IMS-Wrocław-Szeged-CIS entry at the SPMRL 2014 shared task: Reranking and morpho-syntax meet unlabeled data. In Proceedings of the 1st Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages. 97--102.Google ScholarGoogle Scholar
  6. Anders Björkelund and Joakim Nivre. 2015. Non-deterministic oracles for unrestricted non-projective transition-based dependency parsing. In Proceedings of the 14th International Conference on Parsing Technologies. 76--86.Google ScholarGoogle ScholarCross RefCross Ref
  7. Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In Empirical Methods in Natural Language Processing (EMNLP’14). 740--750.Google ScholarGoogle Scholar
  8. Maengsik Choi, Seokwon Jeong, and Harksoo Kim. 2014. Dependency structure analysis and dependency label annotation using CRFs. KIISE Trans. Comput. Pract. (in Korean) 41, 4 (2014), 302--308.Google ScholarGoogle Scholar
  9. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12 (2011), 2493--2537. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Cícero Nogueira dos Santos and Bianca Zadrozny. 2014. Learning character-level representations for part-of-speech tagging. In Proceedings of the 31th International Conference on Machine Learning (ICML’14). 1818--1826. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL’15). 334--343.Google ScholarGoogle Scholar
  12. Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A. Smith. 2016. Recurrent neural network grammars. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 199--209.Google ScholarGoogle Scholar
  13. Yoav Goldberg. 2015. A primer on neural network models for natural language processing. arXiv Preprint arXiv:1510.00726 (2015).Google ScholarGoogle Scholar
  14. Sébastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On using very large target vocabulary for neural machine translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP’15). 1--10.Google ScholarGoogle ScholarCross RefCross Ref
  15. Eliyahu Kiperwasser and Yoav Goldberg. 2016. Simple and accurate dependency parsing using bidirectional LSTM feature representations. Trans. Assoc. Comput. Linguist. 4 (2016), 313--327.Google ScholarGoogle ScholarCross RefCross Ref
  16. Sandra Kubler, Ryan McDonald, Joakim Nivre, and Graeme Hirst. 2009. Dependency Parsing. Morgan and Claypool Publishers. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Taku Kudo and Yuji Matsumoto. 2002. Japanese dependency analysis using cascaded chunking. In Proceedings of the 6th Conference on Natural Language Learning (COLING’02). 1--7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Adhiguna Kuncoro, Miguel Ballesteros, Lingpeng Kong, Chris Dyer, Graham Neubig, and Noah A. Smith. 2017. What do recurrent neural network grammars learn about syntax? In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics (EACL’17). 1249--1258.Google ScholarGoogle Scholar
  19. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 260--270.Google ScholarGoogle ScholarCross RefCross Ref
  20. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proceedings of the 15th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT’16). 260--270.Google ScholarGoogle ScholarCross RefCross Ref
  21. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521 (2015), 436--444.Google ScholarGoogle ScholarCross RefCross Ref
  22. Changki Lee, Junseok Kim, and Jeonghee Kim. 2014. Korean dependency parsing using deep learning. In Proceedings of the 26th Annual Conference on Human and Cognitive Language Technology (in Korean).Google ScholarGoogle Scholar
  23. Jianri Li and Jong-Hyeok Lee. 2015. Korean transition-based dependency parsing with recurrent neural network. KIISE Trans. Comput. Pract. (in Korean) 21, 8 (2015), 567--571.Google ScholarGoogle ScholarCross RefCross Ref
  24. Joon-Ho Lim, Yeo-Chan Yoon, Yongjin Bae, Hyunki Kim, and Kyu-Chul Lee. 2014. Korean dependency parsing model based on transition system using head final constraint. In Proceedings of Korea Computer Congress 2014 (in Korean).Google ScholarGoogle Scholar
  25. Soojong Lim, Youngtae Kim, and Dongyul Ra. 2011. Korean dependency parsing based on machine learning of feature weights. KIISE Trans. Comput. Pract. (in Korean) 38, 4 (2011), 214--223.Google ScholarGoogle Scholar
  26. Wang Ling, Chris Dyer, Alan W. Black, Isabel Trancoso, Ramon Fermandez, Silvio Amir, Luis Marujo, and Tiago Luis. 2015. Finding function in form: Compositional character models for open vocabulary word representation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP’15). 1520--1530.Google ScholarGoogle ScholarCross RefCross Ref
  27. Minh-Thang Luong and Christopher D. Manning. 2016. Achieving open vocabulary neural machine translation with hybrid word-character models. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). 1054--1063.Google ScholarGoogle Scholar
  28. Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, and Wojciech Zaremba. 2015. Addressing the rare word problem in neural machine translation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP’15). 11--19.Google ScholarGoogle ScholarCross RefCross Ref
  29. Seung-Hoon Na. 2014. Third-order dependency parsing of korean. In Proceedings of the 26th Annual Conference on Human and Cognitive Language Technology (in Korean).Google ScholarGoogle Scholar
  30. Seung-Hoon Na. 2015. Conditional random fields for korean morpheme segmentation and POS tagging. ACM Trans. Asian Low-Res. Lang. Info. Process. 14, 3 (2015), 10:1--10:16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Seung-Hoon Na and Sangkeun Jung. 2014. Integrated model of graph-based and transition-based methods for korean dependency parsing. In Proceedings of the 26th Annual Conference on Human and Cognitive Language Technology (in Korean).Google ScholarGoogle Scholar
  32. Seung-Hoon Na, Kangil Kim, and Young-Kil Kim. 2016. Stack LSTMs for transition-based korean dependency parsing. In Proceedings of Korea Computer Congress 2016 (in Korean).Google ScholarGoogle Scholar
  33. Seung-Hoon Na, Jong-Hoon Shin, and Kangil Kim. 2016. Improving stack LSTMs by combining syllables and morphemes for korean dependency parsing. In Proceedings of the 28th Annual Conference on Human and Cognitive Language Technology (in Korean).Google ScholarGoogle Scholar
  34. Hwee Tou Ng and Jin Kiat Low. 2004. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (EMNLP’04). 277--284.Google ScholarGoogle Scholar
  35. Joakim Nivre. 2003. An efficient algorithm for projective dependency parsing. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT’03). 149--160.Google ScholarGoogle Scholar
  36. Joakim Nivre. 2008. Algorithms for deterministic incremental dependency parsing. Comput. Linguist. 34, 4 (2008), 513--553. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Joakim Nivre. 2009. Non-projective dependency parsing in expected linear time. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing (ACL-IJCNLP’09). 351--359. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Jin-Young Oh and Jeong-Won Cha. 2013. Korean dependency parsing using key eojoel. KIISE Trans. Comput. Pract. (in Korean) 40, 10 (2013), 600--608.Google ScholarGoogle Scholar
  39. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 1532--1543. Retrieved from http://www.aclweb.org/anthology/D14-1162.Google ScholarGoogle Scholar
  40. Djamé Seddah, Sandra Kübler, and Reut Tsarfaty. 2014. Introducing the SPMRL 2014 shared task on parsing morphologically-rich languages. In Proceedings of the 1st Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages. 103--109.Google ScholarGoogle Scholar
  41. Richard Socher, Cliff C. Lin, Chris Manning, and Andrew Y. Ng. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML’11). 129--136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. David Weiss, Chris Alberti, Michael Collins, and Slav Petrov. 2015. Structured training for neural network transition-based parsing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP’09). 323--333.Google ScholarGoogle ScholarCross RefCross Ref
  43. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In The 8th International Workshop of Parsing Technologies (IWPT’03). 195--206.Google ScholarGoogle Scholar
  44. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT’11). 188--193. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Transition-Based Korean Dependency Parsing Using Hybrid Word Representations of Syllables and Morphemes with LSTMs

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Article Metrics

        • Downloads (Last 12 months)11
        • Downloads (Last 6 weeks)2

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!