skip to main content
research-article

Reinforced NMT for Sentiment and Content Preservation in Low-resource Scenario

Authors Info & Claims
Published:30 June 2021Publication History
Skip Abstract Section

Abstract

The preservation of domain knowledge from source to the target is crucial in any translation workflows. Hence, translation service providers that use machine translation (MT) in production could reasonably expect that the translation process should transfer both the underlying pragmatics and the semantics of the source-side sentences into the target language. However, recent studies suggest that the MT systems often fail to preserve such crucial information (e.g., sentiment, emotion, gender traits) embedded in the source text in the target. In this context, the raw automatic translations are often directly fed to other natural language processing (NLP) applications (e.g., sentiment classifier) in a cross-lingual platform. Hence, the loss of such crucial information during the translation could negatively affect the performance of such downstream NLP tasks that heavily rely on the output of the MT systems.

In our current research, we carefully balance both the sides (i.e., sentiment and semantics) during translation, by controlling a global-attention-based neural MT (NMT), to generate translations that encode the underlying sentiment of a source sentence while preserving its non-opinionated semantic content. Toward this, we use a state-of-the-art reinforcement learning method, namely, actor-critic, that includes a novel reward combination module, to fine-tune the NMT system so that it learns to generate translations that are best suited for a downstream task, viz. sentiment classification while ensuring the source-side semantics is intact in the process. Experimental results for Hindi–English language pair show that our proposed method significantly improves the performance of the sentiment classifier and alongside results in an improved NMT system.

References

  1. Haithem Afli, Sorcha Maguire, and Andy Way. 2017. Sentiment translation for low resourced languages: Experiments on Irish general election tweets. In Proceedings of the 18th International Conference on Computational Linguistics and Intelligent Text Processing.Google ScholarGoogle Scholar
  2. Sweta Agrawal and Marine Carpuat. 2019. Controlling text complexity in neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 1549–1564.DOI:https://doi.org/10.18653/v1/D19-1166Google ScholarGoogle ScholarCross RefCross Ref
  3. Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. An actor-critic algorithm for sequence prediction. In Proceedings of the 5th International Conference on Learning Representations.Google ScholarGoogle Scholar
  4. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations.Google ScholarGoogle Scholar
  5. Alexandra Balahur and Marco Turchi. 2012. Multilingual sentiment analysis using machine translation? In Proceedings of the 3rd Workshop in Computational Approaches to Subjectivity and Sentiment Analysis. Association for Computational Linguistics, 52–60. Retrieved from https://www.aclweb.org/anthology/W12-3709. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Alexandra Balahur, Marco Turchi, Ralf Steinberger, José Manuel Perea Ortega, Guillaume Jacquet, Dilek Küçük, Vanni Zavarella, and Adil El Ghali. 2014. Resource creation and evaluation for multilingual sentiment analysis in social media texts. In Proceedings of the International Conference on Language Resources and Evaluation (LREC’14). Citeseer, 4265–4269.Google ScholarGoogle Scholar
  7. A. R. Balamurali, Mitesh M. Khapra, and Pushpak Bhattacharyya. 2013. Lost in translation: Viability of machine translation for cross language sentiment analysis. In Proceedings of the 14th International Conference on Computational Linguistics and Intelligent Text Processing (CICLing’13). 38–49. DOI:https://doi.org/10.1007/978-3-642-37256-8_4 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Carmen Banea, Rada Mihalcea, Janyce Wiebe, and Samer Hassan. 2008. Multilingual subjectivity analysis using machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 127–135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. P. Basile, F. Cutugno, M. Nissim, V. Patti, and R. Sprugnoli. 2016. EVALITA 2016: Overview of the 5th evaluation campaign of natural language processing and speech tools for italian. In Proceedings of the CEUR Workshop. 1749.Google ScholarGoogle Scholar
  10. Boxing Chen and Colin Cherry. 2014. A systematic comparison of smoothing techniques for sentence-level bleu. In Proceedings of the Association for Computational Linguistic. 362–367.Google ScholarGoogle ScholarCross RefCross Ref
  11. Boxing Chen and Xiaodan Zhu. 2014. Bilingual sentiment consistency for statistical machine translation. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. 607–615. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Michael Denkowski and Alon Lavie. 2011. Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. In Proceedings of the 6th Workshop on Statistical Machine Translation. 85–91. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 4171–4186.DOI:https://doi.org/10.18653/v1/N19-1423Google ScholarGoogle Scholar
  14. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4171–4186.DOI:https://doi.org/10.18653/v1/N19-1423Google ScholarGoogle Scholar
  15. Marie Escribe. 2019. Human evaluation of neural machine translation: The case of deep learning. In Proceedings of the 2nd Workshop on Human-Informed Translation and Interpreting Technology (HiT-IT’19). 36.Google ScholarGoogle ScholarCross RefCross Ref
  16. Joseph L. Fleiss. 1971. Measuring nominal scale agreement among many raters.Psychol. Bull. 76, 5 (1971), 378.Google ScholarGoogle ScholarCross RefCross Ref
  17. Markus Freitag and Yaser Al-Onaizan. 2016. Fast domain adaptation for neural machine translation. Retrieved from https://arXiv:1612.06897.Google ScholarGoogle Scholar
  18. Markus Freitag, Isaac Caswell, and Scott Roy. 2019. APE at scale and its implications on MT evaluation biases. In Proceedings of the 4th Conference on Machine Translation. 34–44.Google ScholarGoogle ScholarCross RefCross Ref
  19. Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter sentiment classification using distant supervision. CS224N Project Report, Stanford 1, 12 (2009).Google ScholarGoogle Scholar
  20. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput. 9, 8 (1997), 1735–1780. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Hiroshi Kanayama, Tetsuya Nasukawa, and Hideo Watanabe. 2004. Deeper sentiment analysis using machine translation technology. In Proceedings of the 20th International Conference on Computational Linguistics (COLING’04). 494–500. Retrieved from https://www.aclweb.org/anthology/C04-1071. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR’15).Google ScholarGoogle Scholar
  23. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 388–395.Google ScholarGoogle Scholar
  24. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics. 177–180. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Anoop Kunchukuttan, Pratik Mehta, and Pushpak Bhattacharyya. 2018. The IIT Bombay English-Hindi parallel corpus. In Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC’18). European Language Resources Association, 3473–3476.Google ScholarGoogle Scholar
  26. Tsz Kin Lam, Julia Kreutzer, and Stefan Riezler. 2018. A reinforcement learning approach to interactive-predictive neural machine translation. In Proceedings of the European Association for Machine Translation conference. 169–178.Google ScholarGoogle Scholar
  27. Tsz Kin Lam, Shigehiko Schamoni, and Stefan Riezler. 2019. Interactive-predictive neural machine translation through reinforcement and imitation. In Proceedings of Machine Translation Summit. 96–106.Google ScholarGoogle Scholar
  28. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. Retrieved from https://arXiv:1910.13461.Google ScholarGoogle Scholar
  29. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. Retrieved from https://arXiv:1907.11692.Google ScholarGoogle Scholar
  30. Pintu Lohar, Haithem Afli, and Andy Way. 2017. Maintaining sentiment polarity in translation of user-generated content. Prague Bull. Math. Linguist. 108, 1 (2017), 73–84.Google ScholarGoogle ScholarCross RefCross Ref
  31. Pintu Lohar, Haithem Afli, and Andy Way. 2018. Balancing translation quality and sentiment preservation. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas. 81–88.Google ScholarGoogle Scholar
  32. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1412–1421. DOI:https://doi.org/10.18653/v1/D15-1166Google ScholarGoogle Scholar
  33. Evgeny Matusov. 2019. The challenges of using neural machine translation for literature. In Proceedings of the Qualities of Literary Machine Translation. 10–19.Google ScholarGoogle Scholar
  34. Paul Michel and Graham Neubig. 2018. Extreme adaptation for personalized neural machine translation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 312–318. Retrieved from https://www.aclweb.org/anthology/P18-2050.Google ScholarGoogle ScholarCross RefCross Ref
  35. Hideki Mima, Osamu Furuse, and Hitoshi Iida. 1997. Improving performance of transfer-driven machine translation with extra-linguistic informatioon from context, situation, and environment. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’97). 983–989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Shachar Mirkin, Scott Nowson, Caroline Brun, and Julien Perez. 2015. Motivating personality-aware machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1102–1108. DOI:https://doi.org/10.18653/v1/D15-1130Google ScholarGoogle ScholarCross RefCross Ref
  37. Saif M. Mohammad, Mohammad Salameh, and Svetlana Kiritchenko. 2016. How translation alters sentiment. J. Artific. Intell. Res. 55 (2016), 95–130. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Khanh Nguyen, Hal Daumé III, and Jordan Boyd-Graber. 2017. Reinforcement learning for bandit neural machine translation with simulated human feedback. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 1464–1474.DOI:https://doi.org/10.18653/v1/D17-1153Google ScholarGoogle ScholarCross RefCross Ref
  39. Xing Niu, Marianna Martindale, and Marine Carpuat. 2017. A study of style in machine translation: Controlling the formality of machine translation output. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 2814–2819.Google ScholarGoogle ScholarCross RefCross Ref
  40. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on Association for Computational Linguistics. 311–318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Alberto Poncelas, Pintu Lohar, James Hadley, and Andy Way. 2020. The impact of indirect machine translation on sentiment classification. In Proceedings of the 14th Conference of the Association for Machine Translation in the Americas. Association for Machine Translation in the Americas, 78–88.Google ScholarGoogle Scholar
  42. Ella Rabinovich, Raj Nath Patel, Shachar Mirkin, Lucia Specia, and Shuly Wintner. 2017. Personalized machine translation: Preserving original author traits. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics. 1074–1084.Google ScholarGoogle ScholarCross RefCross Ref
  43. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Controlling politeness in neural machine translation via side constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 35–40.DOI:https://doi.org/10.18653/v1/N16-1005Google ScholarGoogle ScholarCross RefCross Ref
  44. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Controlling politeness in neural machine translation via side constraints. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 35–40.Google ScholarGoogle ScholarCross RefCross Ref
  45. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 1715–1725. DOI:https://doi.org/10.18653/v1/P16-1162Google ScholarGoogle ScholarCross RefCross Ref
  46. Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 1679–1684. https://doi.org/ 10.18653/v1/P19-1164Google ScholarGoogle ScholarCross RefCross Ref
  47. Amirhossein Tebbifakhr, Luisa Bentivogli, Matteo Negri, and Marco Turchi. 2019. Machine translation for machines: The sentiment classification use case. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 1368–1374.DOI:https://doi.org/10.18653/v1/D19-1140Google ScholarGoogle ScholarCross RefCross Ref
  48. Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting gender right in neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 3003–3008. https://doi.org/ 10.18653/v1/D18-1334Google ScholarGoogle ScholarCross RefCross Ref
  49. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Conference on Advances in Neural Information Processing Systems. 5998–6008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8, 3–4 (May 1992), 229–256.DOI:https://doi.org/10.1007/BF00992696 Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Shuly Wintner, Shachar Mirkin, Lucia Specia, Ella Rabinovich, and Raj Nath Patel. 2017. Personalized machine translation: Preserving original author traits. In Proceedings of the European Chapter of the Association for Computational Linguistics (EACL’17). 1074–1084.Google ScholarGoogle Scholar
  52. Lijun Wu, Fei Tian, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2018. A study of reinforcement learning for neural machine translation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. 3612–3621.DOI:https://doi.org/10.18653/v1/D18-1397Google ScholarGoogle ScholarCross RefCross Ref
  53. Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xuancheng Ren, Houfeng Wang, and Wenjie Li. 2018. Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 979–988.DOI:https://doi.org/10.18653/v1/P18-1090Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Reinforced NMT for Sentiment and Content Preservation in Low-resource Scenario

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Article Metrics

      • Downloads (Last 12 months)40
      • Downloads (Last 6 weeks)9

      Other Metrics

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!