skip to main content
research-article

Enhancing Shift-Reduce Constituent Parsing with Action N-Gram Model

Published:17 February 2016Publication History
Skip Abstract Section

Abstract

Current shift-reduce parsers “understand” the context by embodying a large number of binary indicator features with a discriminative model. In this article, we propose the action n-gram model, which utilizes the action sequence to help parsing disambiguation. The action n-gram model is trained on action sequences produced by parsers with the n-gram estimation method, which gives a smoothed maximum likelihood estimation of the action probability given a specific action history. We show that incorporating action n-gram models into a state-of-the-art parsing framework could achieve parsing accuracy improvements on three datasets across two languages.

References

  1. Daniel M. Bikel. 2004. On the Parameter Space of Generative Lexicalized Statistical Parsing Models. Ph.D. Dissertation. University of Pennsylvania. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Ted Briscoe and John Carroll. 1993. Generalized probabilistic LR parsing of natural language (corpora) with unification-based grammars. Computational Linguistics 19, 1 (1993), 25--59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Peter F. Brown, Vincent J. Della Pietra, Robert L. Mercer, Stephen A. Della Pietra, and Jennifer C. Lai. 1992. An estimate of an upper bound for the entropy of English. Computational Linguistics 18, 1 (1992), 31--40. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Xavier Carreras, Michael Collins, and Terry Koo. 2008. TAG, dynamic programming, and the perceptron for efficient, feature-rich parsing. In Proceedings of the 12th Conference on Computational Natural Language Learning. Association for Computational Linguistics, 9--16. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Eugene Charniak. 2000. A maximum-entropy-inspired parser. In Proceedings of the 1st North American Chapter of the Association for Computational Linguistics Conference. Association for Computational Linguistics, 132--139. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Eugene Charniak and Mark Johnson. 2005. Coarse-to-fine n-best parsing and MaxEnt discriminative reranking. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 173--180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Stanley F. Chen and Joshua Goodman. 1996. An empirical study of smoothing techniques for language modeling. In Proceedings of the 34th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 310--318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Wenliang Chen, Min Zhang, and Haizhou Li. 2012. Utilizing dependency language models for graph-based dependency parsing models. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, Vol. 1. Association for Computational Linguistics, 213--222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Michael Collins. 1997. Three generative, lexicalised models for statistical parsing. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 16--23. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Michael Collins. 1999. Head-Driven Statistical Models for Natural Language Parsing. Ph.D. Dissertation. University of Pennsylvania. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Michael Collins. 2002. Discriminative training methods for hidden Markov models: Theory and experiments with perceptron algorithms. In Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, Vol. 10. Association for Computational Linguistics, 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Michael Collins and Brian Roark. 2004. Incremental parsing with the perceptron algorithm. In Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 111. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jason M. Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In Proceedings of the 16th Conference on Computational Linguistics, Vol. 1. Association for Computational Linguistics, 340--345. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Yoav Goldberg and Michael Elhadad. 2010. An efficient algorithm for easy-first non-directional dependency parsing. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 742--750. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Yoav Goldberg and Joakim Nivre. 2013. Training deterministic parsers with non-deterministic oracles. Transactions of the Association for Computational Linguistics 1 (2013), 403--414.Google ScholarGoogle ScholarCross RefCross Ref
  16. Jun Hatori, Takuya Matsuzaki, Yusuke Miyao, and Jun’ichi Tsujii. 2012. Incremental joint approach to word segmentation, pos tagging, and dependency parsing in chinese. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers,, Vol. 1. Association for Computational Linguistics, 1045--1053. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Liang Huang. 2008. Forest reranking: Discriminative parsing with non-local features. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics. 586--594.Google ScholarGoogle Scholar
  18. Liang Huang, Wenbin Jiang, and Qun Liu. 2009. Bilingually-constrained (monolingual) shift-reduce parsing. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Vol. 3. Association for Computational Linguistics, 1222--1231. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Liang Huang and Kenji Sagae. 2010. Dynamic programming for linear-time incremental parsing. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1077--1086. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Zhongqiang Huang and Mary Harper. 2009. Self-training PCFG grammars with latent annotations across languages. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, Vol. 2. Association for Computational Linguistics, 832--841. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Zhongqiang Huang, Mary Harper, and Slav Petrov. 2010. Self-training with products of latent variable grammars. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 12--22. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Wenbin Jiang, Liang Huang, Qun Liu, and Yajuan Lü. 2008. A cascaded linear model for joint chinese word segmentation and part-of-speech tagging. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics. Citeseer.Google ScholarGoogle Scholar
  23. Inui Kentaro, Virach Sornlertlamvanich, Tanaka Hozumi, and Tokunaga Takenobu. 1998. Probabilistic GLR parsing: A new formalization and its impact on parsing performance. Journal of Natural Language Processing 5, 3, 33--52. Citeseer.Google ScholarGoogle ScholarCross RefCross Ref
  24. Alon Lavie. 1996. GLR*: A robust parser for spontaneously spoken language. In Proceedings of the ESSLLI-1996 Workshop on Robust Parsing, Prague, Czech Republic.Google ScholarGoogle Scholar
  25. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The penn treebank. Computational Linguistics 19, 2 (1993), 313--330. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. David McClosky, Eugene Charniak, and Mark Johnson. 2006. Effective self-training for parsing. In Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics. Association for Computational Linguistics, 152--159. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 91--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Joakim Nivre, Johan Hall, Jens Nilsson, Atanas Chanev, Gülsen Eryigit, Sandra Kübler, Svetoslav Marinov, and Erwin Marsi. 2007. MaltParser: A language-independent system for data-driven dependency parsing. Natural Language Engineering 13, 2 (2007), 95--135.Google ScholarGoogle ScholarCross RefCross Ref
  29. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, Vol. 1. Association for Computational Linguistics, 160--167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Slav Petrov and Dan Klein. 2007. Improved inference for unlexicalized parsing. In Proceedings of the Human Language Technology Conference of the HLT-NAACL. Association for Computational Linguistics, 404--411.Google ScholarGoogle Scholar
  31. Adwait Ratnaparkhi. 1997. A linear observed time statistical parser based on maximum entropy models. arXiv preprint cmp-lg/9706014 (1997).Google ScholarGoogle Scholar
  32. Kenji Sagae and Alon Lavie. 2005. A classifier-based parser with linear run-time complexity. In Proceedings of the 9th International Workshop on Parsing Technology. Association for Computational Linguistics, 125--132. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Kenji Sagae and Alon Lavie. 2006. Parser combination by reparsing. In Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers. Association for Computational Linguistics, 129--132. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Libin Shen, Jinxi Xu, and Ralph M. Weischedel. 2008. A new string-to-dependency machine translation algorithm with a target dependency language model. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics. 577--585.Google ScholarGoogle Scholar
  35. Stephen M. Stigler. 1989. Francis Galton’s account of the invention of correlation. Statistical Sciences 4, 2 (1989), 73--79.Google ScholarGoogle ScholarCross RefCross Ref
  36. Masaru Tomita. 1987. An efficient augmented-context-free parsing algorithm. Computational Linguistics 13, 1--2 (1987), 31--46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Mengqiu Wang, Kenji Sagae, and Teruko Mitamura. 2006. A fast, accurate deterministic parser for Chinese. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 425--432. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Zhiguo Wang and Nianwen Xue. 2014. Joint POS tagging and transition-based constituent parsing in chinese with non-local features. In Proceedings of the 52th Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics.Google ScholarGoogle ScholarCross RefCross Ref
  39. Naiwen Xue, Fei Xia, Fu-Dong Chiou, and Martha Palmer. 2005. The penn chinese treebank: Phrase structure annotation of a large corpus. Natural Language Engineering 11, 2 (2005), 207--238. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of IWPT, Vol. 3.Google ScholarGoogle Scholar
  41. Meishan Zhang, Yue Zhang, Wanxiang Che, and Ting Liu. 2013. Chinese parsing exploiting characters. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics.Google ScholarGoogle Scholar
  42. Yue Zhang and Stephen Clark. 2009. Transition-based parsing of the Chinese treebank using a global discriminative model. In Proceedings of the 11th International Conference on Parsing Technologies. Association for Computational Linguistics, 162--171. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers, Vol. 2. Association for Computational Linguistics, 188--193. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Muhua Zhu, Yue Zhang, Wenliang Chen, Min Zhang, and Jingbo Zhu. 2013. Fast and accurate shift-reduce constituent parsing. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics.Google ScholarGoogle Scholar
  45. Muhua Zhu, Jingbo Zhu, and Wang Huizhen. 2012. Exploiting lexical dependencies from large scale data for better shift-reduce constituent parsing. In Proceedings of the International Conference on Computational Linguistics. Association for Computational Linguistics.Google ScholarGoogle Scholar

Index Terms

(auto-classified)
  1. Enhancing Shift-Reduce Constituent Parsing with Action N-Gram Model

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Asian and Low-Resource Language Information Processing
        ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 15, Issue 3
        March 2016
        220 pages
        ISSN:2375-4699
        EISSN:2375-4702
        DOI:10.1145/2876004
        Issue’s Table of Contents

        Copyright © 2016 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 17 February 2016
        • Accepted: 1 September 2015
        • Revised: 1 January 2015
        • Received: 1 September 2014
        Published in tallip Volume 15, Issue 3

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Refereed

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!