Abstract
Subword segmentation plays an important role in Tibetan neural machine translation (NMT). The structure of Tibetan words consists of two levels. First, words consist of a sequence of syllables, and then a syllable consists of a sequence of characters. According to this special word structure, we propose two methods for Tibetan subword segmentation, namely syllable-based and character-based methods. The former generates subwords based on the Tibetan syllables, and the latter is based on Tibetan characters. In addition, we carry out experiments with these two subword segmentation methods on low-resource Tibetan-to-Chinese NMT, respectively. The experimental results show that both of them can improve translation performance, in which the subword segmentation based on character sequences can achieve better results. Overall, our proposed character-based subword segmentation is more simple and effective. Moreover, it can achieve better experimental results without paying much attention to the linguistic features of Tibetan.
- Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. arXiv:1607.06450. Retrieved from https://arxiv.org/abs/1607.06450.Google Scholar
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR.Google Scholar
- Ankur Bapna, Mia Chen, Orhan Firat, Yuan Cao, and Yonghui Wu. 2018. Training deeper neural machine translation models with transparent attention. In Proceedings of EMNLP. 3028–3033. DOI:https://doi.org/10.18653/v1/D18-1338Google Scholar
Cross Ref
- Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5, 2 (1994), 157–166. Google Scholar
Digital Library
- Mia Xu Chen, Orhan Firat, and Ankur et al. Bapna. 2018. The best of both worlds: Combining recent advances in neural machine translation. In Proceedings of ACL. 76–86. DOI:https://doi.org/10.18653/v1/P18-1008Google Scholar
Cross Ref
- Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machinetranslation: Encoder-decoder approaches. In Proceedings of SSST. 103–111.Google Scholar
- Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of EMNLP. Association for Computational Linguistics, 1724–1734. DOI:https://doi.org/10.3115/v1/D14-1179Google Scholar
Cross Ref
- Jonas Gehring, Michael Auli, David Grangier, and Yann Dauphin. 2017. A convolutional encoder model for neural machine translation. In Proceedings of ACL. Association for Computational Linguistics, 123–135. DOI:https://doi.org/10.18653/v1/P17-1012Google Scholar
Cross Ref
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of CVPR. 770–778.Google Scholar
Cross Ref
- Nathan Hill and Marieke Meelen. 2017. Segmenting and POS tagging classical tibetan using a memory-based tagger. Himalay. Ling. 16, 2 (2017), 64--86. DOI:https://doi.org/10.5070/H916234501Google Scholar
- Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580 (2012). Retrieved from https://arxiv.org/abs/1207.0580.Google Scholar
- Caijun Kang, Di Jiang, and Congjun Long. 2013. Tibetan word segmentation based on word-position tagging. In Proceedings of IALP. 239–242. Google Scholar
Digital Library
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR.Google Scholar
- Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of ACL. 67–72.Google Scholar
Cross Ref
- Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of ACL 2018. Association for Computational Linguistics, 66–75.Google Scholar
Cross Ref
- Wen Lai, Xiaobing Zhao, and Wei Bao. 2018. Tibetan-chinese neural machine translation based on syllable segmentation. In Proceedings of LoResMT. Association for Machine Translation in the Americas, 21–29.Google Scholar
- Junhui Li, Deyi Xiong, Zhaopeng Tu, Muhua Zhu, Min Zhang, and Guodong Zhou. 2017. Modeling source syntax for neural machine translation. In Proceedings of ACL. 688–697. DOI:https://doi.org/10.18653/v1/P17-1064Google Scholar
Cross Ref
- Yachao Li, Junhui Li, and Min Zhang. 2018. Adaptive weighting for neural machine translation. In Proceedings of COLING. Association for Computational Linguistics, 3038–3048.Google Scholar
- Yachao Li, Junhui Li, Min Zhang, Yixin Li, and Peng Zou. 2020. Improving neural machine translation with linear interpolation of a short-path unit. ACM Trans. Asian Low-Resource Lang. Inf. Process. 19, 3 (2020), 1--16. DOI:https://doi.org/10.1145/3377851 Google Scholar
Digital Library
- Yachao Li, Deyi Xiong, Min Zhang, Jing Jiang, Ning Ma, and Jianmin Yin. 2017. Research on tibetan-chinese neural machine translation. J. Chin. Inf. Process. 31, 6 (2017), 104–109.Google Scholar
- Yachao Li, Jam Yangkyi, Chengqing Zong, and Hongzhi Yu. 2013. Research and implementation of tibetan automatic word segmentation based on conditional random field. J. Chin. Inf. Process. 27, 4 (2013), 52–58.Google Scholar
- Huidan Liu, Minghua Nuo, Longlong Ma, Jian Wu, and Yeping He. 2011. Tibetan word segmentation as syllable tagging using conditional random field. In Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation. Institute of Digital Enhancement of Cognitive Processing, 168–177.Google Scholar
- Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the EMNLP. Association for Computational Linguistics, 1412–1421. DOI:https://doi.org/10.18653/v1/D15-1166Google Scholar
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of ACL. 311–318. DOI:https://doi.org/10.3115/1073083.1073135 Google Scholar
Digital Library
- Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of NAACL. 2227–2237. DOI:https://doi.org/10.18653/v1/N18-1202Google Scholar
Cross Ref
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of ACL 2016. 1715–1725. DOI:https://doi.org/10.18653/v1/P16-1162Google Scholar
Cross Ref
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NIPS. 5998–6008. Google Scholar
Digital Library
- Mingxuan Wang, Zhengdong Lu, Jie Zhou, and Qun Liu. 2017. Deep neural machine translation with linear associative unit. In Proceedings of ACL 2017. Association for Computational Linguistics, 136–145. DOI:https://doi.org/10.18653/v1/P17-1013Google Scholar
Cross Ref
- Xing Wang, Zhaopeng Tu, Longyue Wang, and Shuming Shi. 2019. Exploiting sentential context for neural machine translation. In Proceedings of ACL. 6197–6203. DOI:https://doi.org/10.18653/v1/P19-1624Google Scholar
Cross Ref
- Yonghui Wu, Mike Schuster, Zhifeng Chen, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. In arXiv:1609.08144. Retrieved from https://arxiv.org/abs/1609.08144.Google Scholar
- Yingting Wu and Hai Zhao. 2018. Finding better subword segmentation for neural machine translation. In Proceedings of CCL.Google Scholar
Cross Ref
- Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, and Wei Xu. 2016. Deep recurrent models with fast-forward connections for neural machine translation. Trans. Assoc. Comput. Ling. 4 (2016), 371–383. DOI:https://doi.org/10.1162/tacl_a_00105Google Scholar
Cross Ref
Index Terms
Finding Better Subwords for Tibetan Neural Machine Translation
Recommendations
Low Resource Neural Machine Translation: Assamese to/from Other Indo-Aryan (Indic) Languages
Machine translation (MT) systems have been built using numerous different techniques for bridging the language barriers. These techniques are broadly categorized into approaches like Statistical Machine Translation (SMT) and Neural Machine Translation (...
Chinese---Spanish neural machine translation enhanced with character and word bitmap fonts
Recently, machine translation systems based on neural networks have reached state-of-the-art results for some pairs of languages (e.g., German---English). In this paper, we are investigating the performance of neural machine translation in Chinese---...
The Study on the Disambiguation Method of Tibetan Same Shape Different Pronunciation Words
ICCPR '21: Proceedings of the 2021 10th International Conference on Computing and Pattern RecognitionWith the in-depth study of Tibetan phonetic synthesis, the problem of disambiguation of Tibetan same writing different pronunciation words has gradually been exposed as a major obstacle to the naturalness and intelligibility of the synthesis system. ...






Comments