skip to main content
research-article

Do Multilingual Users Prefer Chat-bots that Code-mix? Let's Nudge and Find Out!

Published:29 May 2020Publication History
Skip Abstract Section

Abstract

Despite their pervasiveness, current text-based conversational agents (chatbots) are predominantly monolingual, while users are often multilingual. It is well-known that multilingual users mix languages while interacting with others, as well as in their interactions with computer systems (such as query formulation in text-/voice-based search interfaces and digital assistants). Linguists refer to this phenomenon as code-mixing or code-switching. Do multilingual users also prefer chatbots that can respond in a code-mixed language over those which cannot? In order to inform the design of chatbots for multilingual users, we conduct a mixed-method user-study (N=91) where we examine how conversational agents, that code-mix and reciprocate the users' mixing choices over multiple conversation turns, are evaluated and perceived by bilingual users. We design a human-in-the-loop chatbot with two different code-mixing policies -- (a) always code-mix irrespective of user behavior, and (b) nudge with subtle code-mixed cues and reciprocate only if the user, in turn, code-mixes. These two are contrasted with a monolingual chatbot that never code-mixed. Users are asked to interact with the bots, and provide ratings on perceived naturalness and personal preference. They are also asked open-ended questions around what they (dis)liked about the bots. Analysis of the chat logs, users' ratings, and qualitative responses reveal that multilingual users strongly prefer chatbots that can code-mix. We find that self-reported language proficiency is the strongest predictor of user preferences. Compared to the Always code-mix policy, Nudging emerges as a low-risk low-gain policy which is equally acceptable to all users. Nudging as a policy is further supported by the observation that users who rate the code-mixing bot higher typically tend to reciprocate the language mixing pattern of the bot. These findings present a first step towards developing conversational systems that are more human-like and engaging by virtue of adapting to the users' linguistic style.

References

  1. Ana Inés Ansaldo, Karine Marcotte, Lilian Scherer, and Gaelle Raboyeau. 2008. Language therapy and bilingual aphasia: Clinical implications of psycholinguistic and neuroimaging research. Journal of Neurolinguistics, Vol. 21, 6 (2008), 539--557.Google ScholarGoogle ScholarCross RefCross Ref
  2. Peter Auer. 1995. The pragmatics of code-switching: A sequential approach. One speaker, two languages: Cross-disciplinary perspectives on code-switching (1995), 115--135.Google ScholarGoogle Scholar
  3. Peter Auer. 1999. From codeswitching via language mixing to fused lects: Toward a dynamic typology of bilingual speech. International journal of bilingualism, Vol. 3, 4 (1999), 309--332.Google ScholarGoogle ScholarCross RefCross Ref
  4. Peter Auer. 2005. A postscript: Code-switching and social identity. Journal of pragmatics, Vol. 37, 3 (2005), 403--410.Google ScholarGoogle ScholarCross RefCross Ref
  5. Peter Auer. 2013. Code-switching in conversation: Language, interaction and identity. Routledge.Google ScholarGoogle Scholar
  6. Ashutosh Baheti, Alan Ritter, Jiwei Li, and Bill Dolan. 2018. Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 3970--3980. https://doi.org/10.18653/v1/D18--1431Google ScholarGoogle ScholarCross RefCross Ref
  7. Gene Ball, Dan Ling, David Kurlander, John Miller, David Pugh, Tim Skelly, Andy Stankosky, David Thiel, Maarten Van Dantzich, and Trace Wax. 1997. Lifelike computer characters: The persona project at Microsoft research. Software agents (1997), 191--222.Google ScholarGoogle Scholar
  8. Anshul Bawa, Monojit Choudhury, and Kalika Bali. 2018a. Accommodation of Conversational Code Choice. Third Workshop on Computational Approaches to Linguistic Code-switching, ACL 2018 (2018).Google ScholarGoogle Scholar
  9. Anshul Bawa, Monojit Choudhury, and Kalika Bali. 2018b. User Perception of Code-Switching Dialog Systems. In 15th International Conference on Natural Language Processing. 171.Google ScholarGoogle Scholar
  10. Rafiya Begum, Kalika Bali, Monojit Choudhury, Koustav Rudra, and Niloy Ganguly. 2016. Functions of Code-Switching in Tweets: An Annotation Framework and Some Initial Experiments. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association (ELRA).Google ScholarGoogle Scholar
  11. Linda Bell and Joakim Gustafson. 1999. Interaction with an animated agent in a spoken dialogue system. In Sixth European Conference on Speech Communication and Technology .Google ScholarGoogle ScholarCross RefCross Ref
  12. JULIA CAMBRE and CHINMAY KULKARNI. 2019. One Voice Fits All? Social Implications and Research Challenges of Designing Voices for Smart Devices. (2019).Google ScholarGoogle Scholar
  13. Antoinette Camilleri. 1996. Language values and identities: Code switching in secondary classrooms in Malta. Linguistics and education, Vol. 8, 1 (1996), 85--103.Google ScholarGoogle Scholar
  14. Justine Cassell, Timothy Bickmore, Mark Billinghurst, Lee Campbell, Kenny Chang, Hannes Vilhjálmsson, and Hao Yan. 1999. Embodiment in conversational interfaces: Rea. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 520--527.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Justine Cassell and Kristinn R Thorisson. 1999. The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence, Vol. 13, 4--5 (1999), 519--538.Google ScholarGoogle ScholarCross RefCross Ref
  16. Leon Ciechanowski, Aleksandra Przegalinska, Mikolaj Magnuski, and Peter Gloor. 2018. In the shades of the uncanny valley: An experimental study of human--chatbot interaction. Future Generation Computer Systems (2018).Google ScholarGoogle Scholar
  17. Kevin Corti and Alex Gillespie. 2016. Co-constructing intersubjectivity with artificial conversational agents: people are more likely to initiate repairs of misunderstandings with agents represented as human. Computers in Human Behavior, Vol. 58 (2016), 431--442.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Sonam Damani, Nitya Raviprakash, Umang Gupta, Ankush Chatterjee, Meghana Joshi, Khyatti Gupta, Kedhar Nath Narahari, Puneet Agrawal, Manoj Kumar Chinnakotla, Sneha Magapu, et al. 2018. Ruuh: A Deep Learning Based Conversational Social Agent. arXiv preprint arXiv:1810.12097 (2018).Google ScholarGoogle Scholar
  19. Cristian Danescu-Niculescu-Mizil, Michael Gamon, and Susan Dumais. 2011. Mark my words!: linguistic style accommodation in social media. In Proceedings of the 20th international conference on World wide web. ACM, 745--754.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Sofia, Bulgaria, 250--259.Google ScholarGoogle Scholar
  21. Anna De Fina. 2007. Code-switching and the construction of ethnic identity in a community of practice. Language in society, Vol. 36, 3 (2007), 371--392.Google ScholarGoogle Scholar
  22. Jean-Marc Dewaele and Li Wei. 2013. Is multilingualism linked to a higher tolerance of ambiguity? Bilingualism: Language and Cognition, Vol. 16, 1 (2013), 231--240.Google ScholarGoogle ScholarCross RefCross Ref
  23. Jean-Marc Dewaele and Li Wei. 2014. Attitudes towards code-switching among adult mono-and multilingual language users. Journal of Multilingual and Multicultural Development, Vol. 35, 3 (2014), 235--251.Google ScholarGoogle ScholarCross RefCross Ref
  24. Jessica Ficler and Yoav Goldberg. 2017. Controlling Linguistic Style Aspects in Neural Language Generation. CoRR, Vol. abs/1707.02633 (2017). arxiv: 1707.02633 http://arxiv.org/abs/1707.02633Google ScholarGoogle Scholar
  25. Saurabh Garg, Tanmay Parekh, and Preethi Jyothi. 2018. Code-switched Language Models Using Dual RNNs and Same-Source Pretraining. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 3078--3083. https://doi.org/10.18653/v1/D18--1346Google ScholarGoogle ScholarCross RefCross Ref
  26. Registrar General and Census Commissioner India. 2011. Census of India 2011. (2011).Google ScholarGoogle Scholar
  27. Howard Giles. 2007. Communication accommodation theory .Wiley Online Library.Google ScholarGoogle Scholar
  28. Howard Giles, Donald M Taylor, and Richard Bourhis. 1973. Towards a theory of interpersonal accommodation through language: Some Canadian data. Language in society, Vol. 2, 2 (1973), 177--192.Google ScholarGoogle Scholar
  29. John J Gumperz. 1982. Discourse strategies. Vol. 1. Cambridge University Press.Google ScholarGoogle Scholar
  30. Khyatti Gupta, Meghana Joshi, Ankush Chatterjee, Sonam Damani, Kedhar Nath Narahari, and Puneet Agrawal. 2019. Insights from Building an Open-Ended Conversational Agent. In Proceedings of the First Workshop on NLP for Conversational AI. 106--112.Google ScholarGoogle ScholarCross RefCross Ref
  31. Annabell Ho, Jeff Hancock, and Adam S Miner. 2018. Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. Journal of Communication, Vol. 68, 4 (2018), 712--733.Google ScholarGoogle ScholarCross RefCross Ref
  32. Netiks International. 2019. Genetiks. (2019). https://genetiks.net/Google ScholarGoogle Scholar
  33. Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS '18). ACM, New York, NY, USA, 895--906. https://doi.org/10.1145/3196709.3196735Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Chaitanya K. Joshi, Fei Mi, and Boi Faltings. 2017. Personalization in Goal-Oriented Dialog. CoRR, Vol. abs/1706.07503 (2017). arxiv: 1706.07503 http://arxiv.org/abs/1706.07503Google ScholarGoogle Scholar
  35. Naveena Karusala, Aditya Vishwanath, Aditya Vashistha, Sunita Kumar, and Neha Kumar. 2018. " Only if you use English you will get to more things" Using Smartphones to Navigate Multilingualism. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Nicole C Kr"amer, Astrid von der Pütten, and Sabrina Eimler. 2012. Human-agent and human-robot interaction theory: Similarities to and differences from human-human interaction. In Human-computer interaction: The agency perspective. Springer, 215--240.Google ScholarGoogle Scholar
  37. Ying Li and Pascale Fung. 2012. Code-Switch Language Model with Inversion Constraints for Mixed Language Speech Recognition. In Proceedings of COLING 2012. The COLING 2012 Organizing Committee, Mumbai, India, 1671--1680. https://www.aclweb.org/anthology/C12--1102Google ScholarGoogle Scholar
  38. Max M Louwerse, Arthur C Graesser, Shulan Lu, and Heather H Mitchell. 2005. Social cues in animated conversational agents. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, Vol. 19, 6 (2005), 693--704.Google ScholarGoogle ScholarCross RefCross Ref
  39. JM Lucas, F Fernández, J Salazar, J Ferreiros, and R San Segundo. 2009. Managing speaker identity and user profiles in a spoken dialogue system. Procesamiento del lenguaje natural 43 (2009), 77--84.Google ScholarGoogle Scholar
  40. Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). ACM, New York, NY, USA, 5286--5297. https://doi.org/10.1145/2858036.2858288Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. 2019. Personalizing Dialogue Agents via Meta-Learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 5454--5459. https://doi.org/10.18653/v1/P19--1542Google ScholarGoogle ScholarCross RefCross Ref
  42. Rada Mihalcea and Stephen Pulman. 2007. Characterizing humour: An exploration of features in humorous texts. In International Conference on Intelligent Text Processing and Computational Linguistics. Springer, 337--347.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Carol Myers-Scotton. 1993. Dueling Languages: Grammatical Structure in Code-Switching .Claredon, Oxford.Google ScholarGoogle Scholar
  44. Carol Myers-Scotton. 1995. Social motivations for codeswitching: Evidence from Africa .Oxford University Press.Google ScholarGoogle Scholar
  45. Carol Myers-Scotton. 2005. Multiple voices: An introduction to bilingualism .Wiley-Blackwell.Google ScholarGoogle Scholar
  46. Mark Mysl'in and Roger Levy. 2015. Code-switching and predictability of meaning in discourse. Language, Vol. 91, 4 (2015), 871--905.Google ScholarGoogle ScholarCross RefCross Ref
  47. Clifford Nass, BJ Fogg, and Youngme Moon. 1996. Can computers be teammates? International Journal of Human-Computer Studies, Vol. 45, 6 (1996), 669--678.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Clifford Ivar Nass and Scott Brave. 2005. Wired for speech: How voice activates and advances the human-computer relationship .MIT press Cambridge, MA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Ocelot. 2019. Ocelot Bilingual Chatbot. (2019). https://www.ocelotbot.com/news/bilingual-chatbot/Google ScholarGoogle Scholar
  50. Adithya Pratapa, Gayatri Bhat, Monojit Choudhury, Sunayana Sitaram, Sandipan Dandapat, and Kalika Bali. 2018. Language modeling for code-mixing: The role of linguistic theory based synthetic data. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1543--1553.Google ScholarGoogle ScholarCross RefCross Ref
  51. Byron Reeves and Clifford Ivar Nass. 1996. The media equation: How people treat computers, television, and new media like real people and places. Cambridge university press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Shruti Rijhwani, Royal Sequiera, Monojit Choudhury, Kalika Bali, and Chandra Shekhar Maddila. 2017. Estimating code-switching on twitter with a novel generalized word-level language detection technique. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1971--1982.Google ScholarGoogle ScholarCross RefCross Ref
  53. Dorsa Sadigh, S Shankar Sastry, Sanjit A Seshia, and Anca Dragan. 2016. Information gathering actions over human internal state. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 66--73.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Bidisha Samanta, Sharmila Reddy, Hussain Jagirdar, Niloy Ganguly, and Soumen Chakrabarti. 2019. A Deep Generative Model for Code-Switched Text. arXiv preprint arXiv:1906.08972 (2019).Google ScholarGoogle Scholar
  55. Carol Myers Scotton and William Ury. 1977. Bilingual strategies: The social functions of code-switching. International Journal of the sociology of language, Vol. 1977, 13 (1977), 5--20.Google ScholarGoogle ScholarCross RefCross Ref
  56. Abigail See, Stephen Roller, Douwe Kiela, and Jason Weston. 2019. What makes a good conversation? How controllable attributes affect human judgments. CoRR, Vol. abs/1902.08654 (2019). arxiv: 1902.08654 http://arxiv.org/abs/1902.08654Google ScholarGoogle Scholar
  57. Heung-Yeung Shum, Xiaodong He, and Di Li. 2018. From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots. arXiv preprint arXiv:1801.01957 (2018).Google ScholarGoogle Scholar
  58. Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology, Vol. 29, 1 (2010), 24--54.Google ScholarGoogle Scholar
  59. Genta Indra Winata, Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2019. Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences. arXiv preprint arXiv:1909.08582 (2019).Google ScholarGoogle Scholar
  60. Donald Winford. 2003. An introduction to contact linguistics. Wiley-Blackwell.Google ScholarGoogle Scholar
  61. Yossi Matias Yaniv Leviathan. 2018. Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone. (2018). https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.htmlGoogle ScholarGoogle Scholar
  62. Jennifer Zamora. 2017. I'm Sorry, Dave, I'm Afraid I Can'T Do That: Chatbot Perception and Expectations. In Proceedings of the 5th International Conference on Human Agent Interaction (Bielefeld, Germany) (HAI '17). ACM, New York, NY, USA, 253--260. https://doi.org/10.1145/3125739.3125766Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing Dialogue Agents: I have a dog, do you have pets too?. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, 2204--2213. https://doi.org/10.18653/v1/P18--1205Google ScholarGoogle ScholarCross RefCross Ref
  64. Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2018. The design and implementation of XiaoIce, an empathetic social chatbot. arXiv preprint arXiv:1812.08989 (2018).Google ScholarGoogle Scholar

Index Terms

  1. Do Multilingual Users Prefer Chat-bots that Code-mix? Let's Nudge and Find Out!

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!