skip to main content
research-article

Finding Better Subwords for Tibetan Neural Machine Translation

Authors Info & Claims
Published:15 March 2021Publication History
Skip Abstract Section

Abstract

Subword segmentation plays an important role in Tibetan neural machine translation (NMT). The structure of Tibetan words consists of two levels. First, words consist of a sequence of syllables, and then a syllable consists of a sequence of characters. According to this special word structure, we propose two methods for Tibetan subword segmentation, namely syllable-based and character-based methods. The former generates subwords based on the Tibetan syllables, and the latter is based on Tibetan characters. In addition, we carry out experiments with these two subword segmentation methods on low-resource Tibetan-to-Chinese NMT, respectively. The experimental results show that both of them can improve translation performance, in which the subword segmentation based on character sequences can achieve better results. Overall, our proposed character-based subword segmentation is more simple and effective. Moreover, it can achieve better experimental results without paying much attention to the linguistic features of Tibetan.

References

  1. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. arXiv:1607.06450. Retrieved from https://arxiv.org/abs/1607.06450.Google ScholarGoogle Scholar
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR.Google ScholarGoogle Scholar
  3. Ankur Bapna, Mia Chen, Orhan Firat, Yuan Cao, and Yonghui Wu. 2018. Training deeper neural machine translation models with transparent attention. In Proceedings of EMNLP. 3028–3033. DOI:https://doi.org/10.18653/v1/D18-1338Google ScholarGoogle ScholarCross RefCross Ref
  4. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5, 2 (1994), 157–166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Mia Xu Chen, Orhan Firat, and Ankur et al. Bapna. 2018. The best of both worlds: Combining recent advances in neural machine translation. In Proceedings of ACL. 76–86. DOI:https://doi.org/10.18653/v1/P18-1008Google ScholarGoogle ScholarCross RefCross Ref
  6. Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machinetranslation: Encoder-decoder approaches. In Proceedings of SSST. 103–111.Google ScholarGoogle Scholar
  7. Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of EMNLP. Association for Computational Linguistics, 1724–1734. DOI:https://doi.org/10.3115/v1/D14-1179Google ScholarGoogle ScholarCross RefCross Ref
  8. Jonas Gehring, Michael Auli, David Grangier, and Yann Dauphin. 2017. A convolutional encoder model for neural machine translation. In Proceedings of ACL. Association for Computational Linguistics, 123–135. DOI:https://doi.org/10.18653/v1/P17-1012Google ScholarGoogle ScholarCross RefCross Ref
  9. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of CVPR. 770–778.Google ScholarGoogle ScholarCross RefCross Ref
  10. Nathan Hill and Marieke Meelen. 2017. Segmenting and POS tagging classical tibetan using a memory-based tagger. Himalay. Ling. 16, 2 (2017), 64--86. DOI:https://doi.org/10.5070/H916234501Google ScholarGoogle Scholar
  11. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580 (2012). Retrieved from https://arxiv.org/abs/1207.0580.Google ScholarGoogle Scholar
  12. Caijun Kang, Di Jiang, and Congjun Long. 2013. Tibetan word segmentation based on word-position tagging. In Proceedings of IALP. 239–242. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR.Google ScholarGoogle Scholar
  14. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. OpenNMT: Open-source toolkit for neural machine translation. In Proceedings of ACL. 67–72.Google ScholarGoogle ScholarCross RefCross Ref
  15. Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of ACL 2018. Association for Computational Linguistics, 66–75.Google ScholarGoogle ScholarCross RefCross Ref
  16. Wen Lai, Xiaobing Zhao, and Wei Bao. 2018. Tibetan-chinese neural machine translation based on syllable segmentation. In Proceedings of LoResMT. Association for Machine Translation in the Americas, 21–29.Google ScholarGoogle Scholar
  17. Junhui Li, Deyi Xiong, Zhaopeng Tu, Muhua Zhu, Min Zhang, and Guodong Zhou. 2017. Modeling source syntax for neural machine translation. In Proceedings of ACL. 688–697. DOI:https://doi.org/10.18653/v1/P17-1064Google ScholarGoogle ScholarCross RefCross Ref
  18. Yachao Li, Junhui Li, and Min Zhang. 2018. Adaptive weighting for neural machine translation. In Proceedings of COLING. Association for Computational Linguistics, 3038–3048.Google ScholarGoogle Scholar
  19. Yachao Li, Junhui Li, Min Zhang, Yixin Li, and Peng Zou. 2020. Improving neural machine translation with linear interpolation of a short-path unit. ACM Trans. Asian Low-Resource Lang. Inf. Process. 19, 3 (2020), 1--16. DOI:https://doi.org/10.1145/3377851 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Yachao Li, Deyi Xiong, Min Zhang, Jing Jiang, Ning Ma, and Jianmin Yin. 2017. Research on tibetan-chinese neural machine translation. J. Chin. Inf. Process. 31, 6 (2017), 104–109.Google ScholarGoogle Scholar
  21. Yachao Li, Jam Yangkyi, Chengqing Zong, and Hongzhi Yu. 2013. Research and implementation of tibetan automatic word segmentation based on conditional random field. J. Chin. Inf. Process. 27, 4 (2013), 52–58.Google ScholarGoogle Scholar
  22. Huidan Liu, Minghua Nuo, Longlong Ma, Jian Wu, and Yeping He. 2011. Tibetan word segmentation as syllable tagging using conditional random field. In Proceedings of the 25th Pacific Asia Conference on Language, Information and Computation. Institute of Digital Enhancement of Cognitive Processing, 168–177.Google ScholarGoogle Scholar
  23. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the EMNLP. Association for Computational Linguistics, 1412–1421. DOI:https://doi.org/10.18653/v1/D15-1166Google ScholarGoogle Scholar
  24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of ACL. 311–318. DOI:https://doi.org/10.3115/1073083.1073135 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of NAACL. 2227–2237. DOI:https://doi.org/10.18653/v1/N18-1202Google ScholarGoogle ScholarCross RefCross Ref
  26. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of ACL 2016. 1715–1725. DOI:https://doi.org/10.18653/v1/P16-1162Google ScholarGoogle ScholarCross RefCross Ref
  27. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NIPS. 5998–6008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Mingxuan Wang, Zhengdong Lu, Jie Zhou, and Qun Liu. 2017. Deep neural machine translation with linear associative unit. In Proceedings of ACL 2017. Association for Computational Linguistics, 136–145. DOI:https://doi.org/10.18653/v1/P17-1013Google ScholarGoogle ScholarCross RefCross Ref
  29. Xing Wang, Zhaopeng Tu, Longyue Wang, and Shuming Shi. 2019. Exploiting sentential context for neural machine translation. In Proceedings of ACL. 6197–6203. DOI:https://doi.org/10.18653/v1/P19-1624Google ScholarGoogle ScholarCross RefCross Ref
  30. Yonghui Wu, Mike Schuster, Zhifeng Chen, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. In arXiv:1609.08144. Retrieved from https://arxiv.org/abs/1609.08144.Google ScholarGoogle Scholar
  31. Yingting Wu and Hai Zhao. 2018. Finding better subword segmentation for neural machine translation. In Proceedings of CCL.Google ScholarGoogle ScholarCross RefCross Ref
  32. Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, and Wei Xu. 2016. Deep recurrent models with fast-forward connections for neural machine translation. Trans. Assoc. Comput. Ling. 4 (2016), 371–383. DOI:https://doi.org/10.1162/tacl_a_00105Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Finding Better Subwords for Tibetan Neural Machine Translation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Asian and Low-Resource Language Information Processing
      ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 20, Issue 2
      March 2021
      313 pages
      ISSN:2375-4699
      EISSN:2375-4702
      DOI:10.1145/3454116
      Issue’s Table of Contents

      Copyright © 2021 Copyright held by the owner/author(s).

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 15 March 2021
      • Accepted: 1 August 2020
      • Revised: 1 July 2020
      • Received: 1 January 2020
      Published in tallip Volume 20, Issue 2

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!