Abstract
Quality estimation is an important task in machine translation that has attracted increased interest in recent years. A key problem in translation-quality estimation is the lack of a sufficient amount of the quality annotated training data. To address this shortcoming, the Predictor-Estimator was proposed recently by introducing “word prediction” as an additional pre-subtask that predicts a current target word with consideration of surrounding source and target contexts, resulting in a two-stage neural model composed of a predictor and an estimator. However, the original Predictor-Estimator is not trained on a continuous stacking model but instead in a cascaded manner that separately trains the predictor from the estimator. In addition, the Predictor-Estimator is trained based on single-task learning only, which uses target-specific quality-estimation data without using other training data that are available from other-level quality-estimation tasks. In this article, we thus propose a multi-task stack propagation, which extensively applies stack propagation to fully train the Predictor-Estimator on a continuous stacking architecture and multi-task learning to enhance the training data from related other-level quality-estimation tasks. Experimental results on WMT17 quality-estimation datasets show that the Predictor-Estimator trained with multi-task stack propagation provides statistically significant improvements over the baseline models. In particular, under an ensemble setting, the proposed multi-task stack propagation leads to state-of-the-art performance at all the sentence/word/phrase levels for WMT17 quality estimation tasks.
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the ICLR 2015.Google Scholar
- Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017. Findings of the 2017 conference on machine translation (WMT17). In Proceedings of the 2nd Conference on Machine Translation, Volume 2: Shared Task Papers. Association for Computational Linguistics, 169--214. Retrieved from http://www.aclweb.org/anthology/W17-4717.Google Scholar
- Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurelie Neveol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turchi, Karin Verspoor, and Marcos Zampieri. 2016. Findings of the 2016 conference on machine translation. In Proceedings of the 1st Conference on Machine Translation. Association for Computational Linguistics, 131--198. Retrieved from http://www.aclweb.org/anthology/W/W16/W16-2301.Google Scholar
- Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’14). Association for Computational Linguistics, 1724--1734. Retrieved from http://www.aclweb.org/anthology/D14-1179.Google Scholar
Cross Ref
- Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12 (Nov. 2011), 2493--2537. http://dl.acm.org/citation.cfm?id=1953048.2078186. Google Scholar
Digital Library
- Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. Multi-task learning for multiple language translation. In Proceedings of the 53rd Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, 1723--1732.Google Scholar
- Mariano Felice and Lucia Specia. 2012. Linguistic features for quality estimation. In Proceedings of the 7th Workshop on Statistical Machine Translation. Association for Computational Linguistics, 96--103. Retrieved from http://www.aclweb.org/anthology/W12-3110. Google Scholar
Digital Library
- Jesús González-Rubio, J. Ramón Navarro-Cerdán, and Francisco Casacuberta. 2013. Dimensionality reduction methods for machine translation quality estimation. Mach. Trans. 27, 3 (2013), 281--301. Google Scholar
Digital Library
- Jesús González-Rubio, Alberto Sanchís, and Francisco Casacuberta. 2012. PRHLT submission to the WMT12 quality estimation task. In Proceedings of the 7th Workshop on Statistical Machine Translation. Association for Computational Linguistics, 104--108. Retrieved from http://www.aclweb.org/anthology/W12-3111.Google Scholar
Digital Library
- Kazuma Hashimoto, Caiming Xiong, Yoshimasa Tsuruoka, and Richard Socher. 2017. A joint many-task model: Growing a neural network for multiple NLP tasks. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 1923--1933. Retrieved from http://aclweb.org/anthology/D17-1206.Google Scholar
Cross Ref
- Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2016. Log-linear combinations of monolingual and bilingual neural machine translation models for automatic post-editing. In Proceedings of the 1st Conference on Machine Translation (WMT’16). 751--758.Google Scholar
Cross Ref
- Lukasz Kaiser, Aidan N. Gomez, and François Chollet. 2017. Depthwise separable convolutions for neural machine translation. CoRR abs. Retrieved from http://arxiv.org/abs/1706.03059.Google Scholar
- Hyun Kim, Hun-Young Jung, Hongseok Kwon, Jong-Hyeok Lee, and Seung-Hoon Na. 2017. Predictor-estimator: Neural quality estimation based on target word prediction for machine translation. ACM Trans. Asian Low-Resour. Lang. Inf. Process. 17, 1, Article 3 (Sept. 2017), 22 pages. Google Scholar
Digital Library
- Hyun Kim and Jong-Hyeok Lee. 2016. Recurrent neural network based translation quality estimation. In Proceedings of the 1st Conference on Machine Translation. Association for Computational Linguistics, 787--792. Retrieved from http://www.aclweb.org/anthology/W/W16/W16-2384.Google Scholar
Cross Ref
- Hyun Kim and Jong-Hyeok Lee. 2016. A recurrent neural networks approach for estimating the quality of machine translation output. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 494--498. Retrieved from http://www.aclweb.org/anthology/N16-1059.Google Scholar
Cross Ref
- Hyun Kim, Jong-Hyeok Lee, and Seung-Hoon Na. 2017. Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation. In Proceedings of the 2nd Conference on Machine Translation, Volume 2: Shared Task Papers. Association for Computational Linguistics, 562--568. Retrieved from http://www.aclweb.org/anthology/W17-4763.Google Scholar
Cross Ref
- Anna Kozlova, Mariya Shmatova, and Anton Frolov. 2016. YSDA participation in the WMT’16 quality estimation shared task. In Proceedings of the 1st Conference on Machine Translation. Association for Computational Linguistics, 793--799. Retrieved from http://www.aclweb.org/anthology/W/W16/W16-2385.Google Scholar
Cross Ref
- Julia Kreutzer, Shigehiko Schamoni, and Stefan Riezler. 2015. QUality Estimation from scraTCH (QUETCH): Deep learning for word-level translation quality estimation. In Proceedings of the 10th Workshop on Statistical Machine Translation. Association for Computational Linguistics, 316--322. Retrieved from http://aclweb.org/anthology/W15-3037.Google Scholar
Cross Ref
- Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2016. Multi-task sequence to sequence learning. In Proceedings of the International Conference on Learning Representations.Google Scholar
- André F. T. Martins, Ramón Astudillo, Chris Hokamp, and Fabio Kepler. 2016. Unbabel’s participation in the WMT16 word-level translation quality estimation shared task. In Proceedings of the 1st Conference on Machine Translation. Association for Computational Linguistics, 806--811. Retrieved from http://www.aclweb.org/anthology/W/W16/W16-2387.Google Scholar
Cross Ref
- André F. T. Martins, Junczys-Dowmunt Marcin, Fabio Kepler, and Ramón Astudillo. 2017. Pushing the limits of translation quality estimation. Trans. Assoc. Comput. Ling. 5 (2017) 205--218.Google Scholar
Cross Ref
- Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Meeting on Association for Computational Linguistics (ACL’03). Volume 1. 160--167. Google Scholar
Digital Library
- Raj Nath Patel and Sasikumar M. 2016. Translation quality estimation using recurrent neural network. In Proceedings of the First Conference on Machine Translation. Association for Computational Linguistics, 819--824. Retrieved from http://www.aclweb.org/anthology/W/W16/W16-2389.Google Scholar
- Hao Peng, Sam Thomson, and Noah A. Smith. 2017. Deep multitask learning for semantic dependency parsing. In Proceedings of the 55th Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2037--2048.Google Scholar
- Raphael Rubino, José de Souza, Jennifer Foster, and Lucia Specia. 2013. Topic models for translation quality estimation for gisting purposes. In Proceedings of the 14th Machine Translation Summit. 295--302.Google Scholar
- Kashif Shah, Trevor Cohn, and Lucia Specia. 2015. A Bayesian non-linear method for feature selection in machine translation quality estimation. Mach. Trans. 29, 2 (2015), 101--125. Google Scholar
Digital Library
- Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the Association for Machine Translation in the Americas. 223--231.Google Scholar
- Anders Søgaard and Yoav Goldberg. 2016. Deep multi-task learning with low level tasks supervised at lower layers. In Proceedings of the 54th Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, 231--235.Google Scholar
Cross Ref
- Radu Soricut and Abdessamad Echihabi. 2010. TrustRank: Inducing trust in automatic translations via ranking. In Proceedings of the 48th Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 612--621. Retrieved from http://www.aclweb.org/anthology/P10-1063. Google Scholar
Digital Library
- Lucia Specia and Varvara Logacheva. 2017. WMT17 Quality estimation shared task training and development data. LINDAT/CLARIN digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. Retrieved from http://hdl.handle.net/11372/LRT-1974.Google Scholar
- Lucia Specia, Kashif Shah, José G. C. de Souza, and Trevor Cohn. 2013. QuEst—A translation quality estimation framework. In Proceedings of the 51st Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 79--84. Retrieved from http://www.aclweb.org/anthology/P13-4014.Google Scholar
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 5998--6008. Retrieved from http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf. Google Scholar
Digital Library
- Yuan Zhang and David Weiss. 2016. Stack-propagation: Improved representation learning for syntax. In Proceedings of the 54th Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 1557--1566. Retrieved from http://www.aclweb.org/anthology/P16-1147.Google Scholar
Cross Ref
Index Terms
Multi-task Stack Propagation for Neural Quality Estimation
Recommendations
Uniformly Interpolated Balancing for Robust Prediction in Translation Quality Estimation: A Case Study of English-Korean Translation
There has been growing interest among researchers in quality estimation (QE), which attempts to automatically predict the quality of machine translation (MT) outputs. Most existing works on QE are based on supervised approaches using quality-annotated ...
A Neural Network Model for Online Multi-Task Multi-Label Pattern Recognition
Proceedings of the 23rd International Conference on Artificial Neural Networks and Machine Learning ICANN 2013 - Volume 8131This paper presents a new sequential multi-task learning model with the following functions: one-pass incremental learning, task allocation, knowledge transfer, task consolidation, learning of multi-label data, and active learning. This model learns ...
Multi-Task Self-Supervised Learning for Script Event Prediction
CIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge ManagementMost existing approaches to script event prediction rely on manually labeled data heavily, which is often expensive to obtain. To cope with the training data bottleneck, we investigate methods of combining multiple self-supervised tasks, i.e. tasks ...






Comments