Abstract
Neural Machine Translation (NMT) systems are undesirably slow as the decoder often has to compute probability distributions over large target vocabularies. In this work, we propose a coarse-to-fine approach to reduce the complexity of the decoding process, using only the information of the weight matrix in the Softmax layer. The large target vocabulary is first trimmed to a small candidate set in the coarse-grained phase, and from this candidate set the final top-k results are generated in the fine-grained phase. Tested on an RNN-based NMT system and a Transformer-based NMT system separately, our GPU-friendly method achieved a significant speed-up without harming the translation quality.
- [1] . 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations (ICLR’15), and (Eds.). San Diego, CA. http://arxiv.org/abs/1409.0473.Google Scholar
- [2] . 2016. Findings of the 2016 conference on machine translation. In Proceedings of the 1st Conference on Machine Translation (WMT’16). The Association for Computer Linguistics, Berlin, 131–198. Google Scholar
Cross Ref
- [3] . 2017. Sharp models on dull hardware: Fast and accurate neural machine translation decoding on the CPU. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’17), , , and (Eds.). Association for Computational Linguistics, Copenhagen, 2820–2825. Google Scholar
Cross Ref
- [4] . 2015. On using very large target vocabulary for neural machine translation. CoRR abs/1412.2007 (2014). arXiv:1412.2007 http://arxiv.org/abs/1412.2007.Google Scholar
- [5] . 2016. Sequence-level knowledge distillation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP’16), , , and (Eds.). The Association for Computational Linguistics, Austin, TX, 1317–1327. Google Scholar
Cross Ref
- [6] . 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics (ACL’07), , , and (Eds.). The Association for Computational Linguistics, Prague. https://aclanthology.org/P07-2045/.Google Scholar
Digital Library
- [7] . 2003. Statistical phrase-based translation. In Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL’03), and (Eds.). The Association for Computational Linguistics, Edmonton. https://aclanthology.org/N03-1017/.Google Scholar
Cross Ref
- [8] . 2016. Vocabulary selection strategies for neural machine translation. arXiv:1610.00072. Retrieved from http://arxiv.org/abs/1610.00072.Google Scholar
- [9] . 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’15), , , , , and (Eds.). The Association for Computational Linguistics, Lisbon, 1412–1421.
DOI: Google ScholarCross Ref
- [10] . 2016. Vocabulary manipulation for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL’16). The Association for Computer Linguistics, Berlin. Google Scholar
Cross Ref
- [11] . 2008. Coarse-to-fine syntactic machine translation using language projections. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP’08). ACL, Honolulu, Hawaii, 108–116. Retrieved from https://aclanthology.org/D08-1012/.Google Scholar
Digital Library
- [12] . 2017. Attention-based vocabulary selection for NMT decoding. arXiv:1706.03824. Retrieved from http://arxiv.org/abs/1706.03824.Google Scholar
- [13] . 2017. Attention-based vocabulary selection for NMT decoding. arXiv:1706.03824. Retrieved from http://arxiv.org/abs/1706.03824.Google Scholar
- [14] . 2016. Compression of neural machine translation models via pruning. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL’16), and (Eds.). ACL, Berlin, 291–301. Google Scholar
Cross Ref
- [15] . 2016. Edinburgh neural machine translation systems for WMT 16. In Proceedings of the 1st Conference on Machine Translation (WMT’16). The Association for Computer Linguistics, Berlin, 371–376. Google Scholar
Cross Ref
- [16] . 2017. Speeding up neural machine translation decoding by shrinking run-time vocabulary. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL’17), and (Eds.). Association for Computational Linguistics, Vancouver, 574–579. Google Scholar
Cross Ref
- [17] . 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, , , , , , , and (Eds.). Long Beach, 5998–6008. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html.Google Scholar
- [18] . 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv:1609.08144. Retrieved from http://arxiv.org/abs/1609.08144.Google Scholar
- [19] . 2019. Sharing attention weights for fast transformer. In Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI’19), (Ed.). ijcai.org, Macao, 5292–5298. Google Scholar
Cross Ref
Index Terms
Coarse-to-Fine Output Predictions for Efficient Decoding in Neural Machine Translation
Recommendations
Neural Machine Translation of Indian Languages
Compute '17: Proceedings of the 10th Annual ACM India Compute ConferenceNeural Machine Translation (NMT) is a new technique for machine translation that has led to remarkable improvements compared to rule-based and statistical machine translation (SMT) techniques, by overcoming many of the weaknesses in the conventional ...
Approach of English to Sanskrit machine translation based on case-based reasoning, artificial neural networks and translation rules
In Machine Translation (MT), we reuse past translation that is encoded into a set of cases, where case is the input sentence and its corresponding translation. A case which is similar to the input sentence will be retrieved and a solution is produced by ...
A Neural Network Classifier Based on Dependency Tree for English-Vietnamese Statistical Machine Translation
Computational Linguistics and Intelligent Text ProcessingAbstractReordering in MT is a major challenge when translating between languages with different of sentence structures. In Phrase-based statistical machine translation (PBSMT) systems, syntactic pre-ordering is a commonly used pre-processing technique. ...






Comments