skip to main content
short-paper

Multitask Pointer Network for Korean Dependency Parsing

Published:08 February 2019Publication History
Skip Abstract Section

Abstract

Dependency parsing is a fundamental problem in natural language processing. We introduce a novel dependency-parsing framework called head-pointing--based dependency parsing. In this framework, we cast the Korean dependency parsing problem as a statistical head-pointing and arc-labeling problem. To address this problem, a novel neural network called the multitask pointer network is devised for a neural sequential head-pointing and type-labeling architecture. Our approach does not require any handcrafted features or language-specific rules to parse dependency. Furthermore, it achieves state-of-the-art performance for Korean dependency parsing.

References

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations.Google ScholarGoogle Scholar
  2. Xavier Carreras. 2007. Experiments with a higher-order projective dependency parser. In Proceedings of the Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL’07). 957--961.Google ScholarGoogle Scholar
  3. Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). 740--750.Google ScholarGoogle Scholar
  4. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, et al. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14).Google ScholarGoogle ScholarCross RefCross Ref
  5. Jinho D. Choi and Martha Palmer. 2011. Statistical dependency parsing in Korean: From corpus generation to automatic parsing. In Proceedings of the 2nd Workshop on Statistical Parsing of Morphologically Rich Languages. 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Jan Chorowski, Michał Zapotoczny, and Paweł Rychlikowski. 2016. Read, tag, and parse all at once, or fully-neural dependency parsing. arXiv:1609.03441.Google ScholarGoogle Scholar
  7. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12, 2493--2537. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Michael A. Covington. 2001. A fundamental algorithm for dependency parsing. In Proceedings of the 39th Annual ACM Southeast Conference. 95--102.Google ScholarGoogle Scholar
  9. Aron Culotta and Jeffrey Sorensen. 2004. Dependency tree kernels for relation extraction. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics. 423. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Yuan Ding and Martha Palmer. 2005. Machine translation using probabilistic synchronous dependency insertion grammars. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 541--548. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Chris Dyer, Miguel Ballesteros, Wang Ling, Austin Matthews, and Noah A. Smith. 2015. Transition-based dependency parsing with stack long short-term memory. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. 334--343.Google ScholarGoogle Scholar
  12. Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of the 16th Conference on Computational Linguistics, Vol. 1. 340--345. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Yoav Goldberg and Joakim Nivre. 2013. Training deterministic parsers with non-deterministic oracles. Transactions of the Association for Computational Linguistics 1, 403--414.Google ScholarGoogle ScholarCross RefCross Ref
  14. James Henderson. 2003. Inducing history representations for broad coverage statistical parsing. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, Vol. 1. 24--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. James Henderson, Paola Merlo, Ivan Titov, and Gabriele Musillo. 2013. Multilingual joint parsing of syntactic and semantic dependencies with a latent variable model. Computational Linguistics 39, 4 (2013), 949--998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. James Henderson and Ivan Titov. 2010. Incremental sigmoid belief networks for grammar learning. Journal of Machine Learning Research 11, 3541--3570. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Terry Koo and Michael Collins. 2010. Efficient third-order dependency parsers. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Changki Lee, Junseok Kim, and Jeonghee Kim. 2014. Korean dependency parsing using deep learning. In Proceedings of the Annual Conference on Human and Cognitive Language Technology.Google ScholarGoogle Scholar
  19. Changki Lee, Junseok Kim, Jeonghee Kim, and Hyunki Kim. 2013. Joint models for Korean word spacing and POS tagging using structural SVM. Journal of KIISE: Software and Applications 40, 12 (2013), 826--832.Google ScholarGoogle Scholar
  20. Jianri Li and Jong-Hyeok Lee. 2015. Korean transition-based dependency parsing with recurrent neural network. KIISE Transactions on Computing Practices 21, 8 (2015), 567--571.Google ScholarGoogle ScholarCross RefCross Ref
  21. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. 1412--1421.Google ScholarGoogle Scholar
  22. Xuezhe Ma and Hai Zhao. 2012. Fourth-order dependency parsing. In Proceedingsof the 25th International Conference on Computational Linguistics (Posters). 785--796.Google ScholarGoogle Scholar
  23. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. 91--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Ryan T. McDonald and Fernando C. N. Pereira. 2006. Online learning of approximate dependency parsing algorithms. In Proceedings of the 10th Conference of the European Chapter of the Association for Computational Linguistics (EACL’06). 81--88.Google ScholarGoogle Scholar
  25. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv:1301.3781.Google ScholarGoogle Scholar
  26. Jin-Woo Min and Seung-Hoon Na. 2016. Korean dependency parsing using sequence to sequence model. In Proceedings of Korean Institute of Information Scientists and Engineers (KISSE’16). 491--493.Google ScholarGoogle Scholar
  27. Jin-Woo Min and Seung-Hoon Na. 2017. SyntaxNet models using transition based recurrent unit for Korean dependency parsing. In Proceedings of Korean Institute of Information Scientists and Engineers (KISSE’17). 602--604.Google ScholarGoogle Scholar
  28. Seung-Hoon Na, Kangil Kim, and Young-Kil Kim. 2016. Stack LSTMs for transition-based Korean dependency parsing. In Proceedings of the Korea Computer Congress (KCC’16). 732--734.Google ScholarGoogle Scholar
  29. Seung-Hoon Na, Jong-Hoon Shin, and Kangil Kim. 2016. Improving stack LSTMs by combining syllables and morphemes for Korean dependency parsing. In Proceedings of the Annual Conference on Human and Cognitive Language Technology.Google ScholarGoogle Scholar
  30. Joakim Nivre. 2003. An efficient algorithm for projective dependency parsing. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT’03).Google ScholarGoogle Scholar
  31. Joakim Nivre and Mario Scholz. 2004. Deterministic dependency parsing of English text. In Proceedings of the 20th International Conference on Computational Linguistics. 64. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kenji Sagae. 2009. Analysis of discourse structure with syntactic dependencies and data-driven shift-reduce parsing. In Proceedings of the 11th International Conference on Parsing Technologies. 81--84. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Yusuke Shinyama, Satoshi Sekine, and Kiyoshi Sudo. 2002. Automatic paraphrase acquisition from news articles. In Proceedings of the 2nd International Conference on Human Language Technology Research. 313--318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Richard Socher, John Bauer, Christopher D. Manning, and Andrew Y. Ng. 2013. Parsing with compositional vector grammars. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (ACL’13), Vol. 1. 455--465.Google ScholarGoogle Scholar
  35. Richard Socher, Cliff C. Lin, Chris Manning, and Andrew Y. Ng. 2011. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML’11). 129--136. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS’14). 3104--3112. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, et al. 2017. Attention is all you need. arXiv:1706.03762.Google ScholarGoogle Scholar
  38. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’15). 2692--2700. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Oriol Vinyals, Łukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’15). 2773--2781. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Wenhui Wang and Baobao Chang. 2016. Graph-based dependency parsing with bidirectional LSTM. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16).Google ScholarGoogle ScholarCross RefCross Ref
  41. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT’03), Vol. 3. 195--206.Google ScholarGoogle Scholar
  42. J. T. Yoon and K.-S. Choi. 1999. Study on Kaist Corpus. Technical Report CS-TR-99-139. Department of Computer Science, KAIST.Google ScholarGoogle Scholar
  43. Xingxing Zhang, Jianpeng Cheng, and Mirella Lapata. 2016. Dependency parsing as head selection. arXiv:1606.01280.Google ScholarGoogle Scholar
  44. Yue Zhang and Stephen Clark. 2008. A tale of two parsers: Investigating and combining graph-based and transition-based dependency parsing using beam-search. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 562--571. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Multitask Pointer Network for Korean Dependency Parsing

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 18, Issue 3
      September 2019
      386 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3305347
      Issue’s Table of Contents

      Copyright © 2019 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 8 February 2019
      • Accepted: 1 September 2018
      • Revised: 1 March 2018
      • Received: 1 June 2017
      Published in tallip Volume 18, Issue 3

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • short-paper
      • Research
      • Refereed
    • Article Metrics

      • Downloads (Last 12 months)12
      • Downloads (Last 6 weeks)0

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!