Abstract
Tree-LSTM algorithm accommodates tree structure processing to extract information outside the linear sequence pattern. The use of Tree-LSTM in text generation problems requires the help of an external parser at each generation iteration. Developing a good parser demands the representation of complex features and relies heavily on the grammar of the corpus. The limited corpus results in an insufficient number of vocabs for a grammar-based parser, making it less natural to link the text generation process. This research aims to solve the problem of limited corpus by proposing the use of a Reinforcement Learning algorithm in the formation of constituency trees, which link the sentence generation process given a seed phrase as the input in the Tree-LSTM model. The tree production process is modeled as a Markov’s decision process, where a set of states consists of word embedding vectors, and a set of actions of {Shift, Reduce}. The Deep Q-Network model as an approximator of the Q-Learning algorithm is trained to obtain optimal weights in representing the Q-value function.
The test results on perplexity-based evaluation show that the proposed Tree-LSTM and Q-Learning combination model achieves values 9.60 and 4.60 for two kinds of corpus with 205 and 1,000 sentences, respectively, better than the Shift-All model. Human evaluation of Friedman test and posthoc analysis showed that all five respondents tended to give the same assessment for the combination model of Tree-LSTM and Q-Learning, which on average outperforms two other nongrammar models, i.e., Shift-All and Reduce-All.
- [1] . 2018. Affective neural response generation. In Advances in Information Retrieval. Springer International Publishing, Cham, 154–166.Google Scholar
- [2] . 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics 5 (2017), 135–146.Google Scholar
Cross Ref
- [3] . 2016. A fast unified model for parsing and sentence understanding. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, 1466–1477.
DOI: Google ScholarCross Ref
- [4] . 2017. Generating fake but realistic headlines using deep neural networks. In International Conference on Database and Expert Systems Applications. Springer International Publishing, Cham, 427–440.Google Scholar
Cross Ref
- [5] . 2016. Developing (and utilizing) an Indonesian treebank. In The Second Wordnet Bahasa Workshop. Nanyang Technological University, 15–16.Google Scholar
- [6] . 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems, , , , , and (Eds.), Vol. 27. Curran Associates, Inc.Google Scholar
Digital Library
- [7] . 2018. Learning word vectors for 157 languages. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018). European Language Resources Association (ELRA), Miyazaki, Japan. https://aclanthology.org/L18-1550.Google Scholar
- [8] . 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735–1780.Google Scholar
Digital Library
- [9] . 1993. Using statistical testing in the evaluation of retrieval experiments. In Proceedings of the 16th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, New York, NY, USA, 329–338.
DOI: Google ScholarDigital Library
- [10] . 2019. An introduction to variational autoencoders. Found. Trends Mach. Learn. 12, 4 (2019), 307–392.
DOI: Google ScholarDigital Library
- [11] . 2012. IDENTIC corpus: Morphologically enriched Indonesian-English parallel corpus. In LREC. European Language Resources Association (ELRA), 902–906.Google Scholar
- [12] . 2015. When are tree structures necessary for deep learning of representations?. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, 2304–2314.
DOI: Google ScholarCross Ref
- [13] . 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, 2122–2132.
DOI: Google ScholarCross Ref
- [14] . 2017. Shift-reduce constituent parsing with neural lookahead features. Transactions of the Association for Computational Linguistics 5 (2017), 45–58.Google Scholar
Cross Ref
- [15] . 2018. Latent tree learning with differentiable parsers: Shift-reduce parsing and chart parsing. In Proceedings of the Workshop on the Relevance of Linguistic Structure in Neural Architectures for NLP. Association for Computational Linguistics, Melbourne, Australia, 13–18.
DOI: Google ScholarCross Ref
- [16] . 2016. End-to-end relation extraction using LSTMs on sequences and tree structures. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, 1105–1116.
DOI: Google ScholarCross Ref
- [17] . 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529–533.Google Scholar
- [18] . 2016. Neural paraphrase generation with stacked residual LSTM networks. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. The COLING 2016 Organizing Committee, Osaka, Japan, 2923–2934. https://aclanthology.org/C16-1275Google Scholar
- [19] . 2019. A tree-to-sequence model for neural NLG in task-oriented dialog. In Proceedings of the 12th International Conference on Natural Language Generation. Association for Computational Linguistics, Tokyo, Japan, 95–100.
DOI: Google ScholarCross Ref
- [20] . 1999. Using NLP or NLP resources for information retrieval tasks. In Natural Language Information Retrieval. Springer Netherlands, Dordrecht, 99–111.Google Scholar
Cross Ref
- [21] . 2017. A minimal span-based neural constituency parser. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, 818–827.Google Scholar
Cross Ref
- [22] . 2018. Reinforcement Learning: An Introduction. MIT press.Google Scholar
Digital Library
- [23] . 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Beijing, China, 1556–1566.Google Scholar
Cross Ref
- [24] . 2019. A shift-reduce dependency parser based on reinforcement learning. DEStech Transactions on Computer Science and Engineering (2019).Google Scholar
Cross Ref
- [25] . 2017. Learning to compose words into sentences with reinforcement learning. In 5th International Conference on Learning Representations (ICLR 2017). International Conference on Learning Representations, OpenReview.net, Toulon, France.Google Scholar
- [26] . 1989. Simple fast algorithms for the editing distance between trees and related problems. SIAM Journal on Computing 18, 6 (1989), 1245–1262.Google Scholar
Digital Library
- [27] . 2009. Dependency parsing with energy-based reinforcement learning. In Proceedings of the 11th International Conference on Parsing Technologies (IWPT’09). Association for Computational Linguistics, Paris, France, 234–237.Google Scholar
Digital Library
- [28] . 2018. Learning structured representation for text classification via reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.Google Scholar
Cross Ref
- [29] . 2019. Neural conversation generation with auxiliary emotional supervised models. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 19, 2 (2019), 1–17.Google Scholar
Index Terms
Q-Learning for Shift-Reduce Parsing in Indonesian Tree-LSTM-Based Text Generation
Recommendations
Shift-reduce CCG parsing
HLT '11: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1CCGs are directly compatible with binary-branching bottom-up parsing algorithms, in particular CKY and shift-reduce algorithms. While the chart-based approach has been the dominant approach for CCG, the shift-reduce method has been little explored. In ...
Deterministic shift-reduce parsing for unification-based grammars
Many parsing techniques assume the use of a packed parse forest to enable efficient and accurate parsing. However, they suffer from an inherent problem that derives from the restriction of locality in the packed parse forest. Deterministic parsing is ...
GLL parse-tree generation
Backtracking techniques which are often used to extend recursive descent (RD) parsers can have explosive run-times and cannot deal with grammars with left recursion. GLL parsers are fully general, worst-case cubic parsers which have the recursive ...






Comments