Abstract
Despite their pervasiveness, current text-based conversational agents (chatbots) are predominantly monolingual, while users are often multilingual. It is well-known that multilingual users mix languages while interacting with others, as well as in their interactions with computer systems (such as query formulation in text-/voice-based search interfaces and digital assistants). Linguists refer to this phenomenon as code-mixing or code-switching. Do multilingual users also prefer chatbots that can respond in a code-mixed language over those which cannot? In order to inform the design of chatbots for multilingual users, we conduct a mixed-method user-study (N=91) where we examine how conversational agents, that code-mix and reciprocate the users' mixing choices over multiple conversation turns, are evaluated and perceived by bilingual users. We design a human-in-the-loop chatbot with two different code-mixing policies -- (a) always code-mix irrespective of user behavior, and (b) nudge with subtle code-mixed cues and reciprocate only if the user, in turn, code-mixes. These two are contrasted with a monolingual chatbot that never code-mixed. Users are asked to interact with the bots, and provide ratings on perceived naturalness and personal preference. They are also asked open-ended questions around what they (dis)liked about the bots. Analysis of the chat logs, users' ratings, and qualitative responses reveal that multilingual users strongly prefer chatbots that can code-mix. We find that self-reported language proficiency is the strongest predictor of user preferences. Compared to the Always code-mix policy, Nudging emerges as a low-risk low-gain policy which is equally acceptable to all users. Nudging as a policy is further supported by the observation that users who rate the code-mixing bot higher typically tend to reciprocate the language mixing pattern of the bot. These findings present a first step towards developing conversational systems that are more human-like and engaging by virtue of adapting to the users' linguistic style.
- Ana Inés Ansaldo, Karine Marcotte, Lilian Scherer, and Gaelle Raboyeau. 2008. Language therapy and bilingual aphasia: Clinical implications of psycholinguistic and neuroimaging research. Journal of Neurolinguistics, Vol. 21, 6 (2008), 539--557.Google Scholar
Cross Ref
- Peter Auer. 1995. The pragmatics of code-switching: A sequential approach. One speaker, two languages: Cross-disciplinary perspectives on code-switching (1995), 115--135.Google Scholar
- Peter Auer. 1999. From codeswitching via language mixing to fused lects: Toward a dynamic typology of bilingual speech. International journal of bilingualism, Vol. 3, 4 (1999), 309--332.Google Scholar
Cross Ref
- Peter Auer. 2005. A postscript: Code-switching and social identity. Journal of pragmatics, Vol. 37, 3 (2005), 403--410.Google Scholar
Cross Ref
- Peter Auer. 2013. Code-switching in conversation: Language, interaction and identity. Routledge.Google Scholar
- Ashutosh Baheti, Alan Ritter, Jiwei Li, and Bill Dolan. 2018. Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 3970--3980. https://doi.org/10.18653/v1/D18--1431Google Scholar
Cross Ref
- Gene Ball, Dan Ling, David Kurlander, John Miller, David Pugh, Tim Skelly, Andy Stankosky, David Thiel, Maarten Van Dantzich, and Trace Wax. 1997. Lifelike computer characters: The persona project at Microsoft research. Software agents (1997), 191--222.Google Scholar
- Anshul Bawa, Monojit Choudhury, and Kalika Bali. 2018a. Accommodation of Conversational Code Choice. Third Workshop on Computational Approaches to Linguistic Code-switching, ACL 2018 (2018).Google Scholar
- Anshul Bawa, Monojit Choudhury, and Kalika Bali. 2018b. User Perception of Code-Switching Dialog Systems. In 15th International Conference on Natural Language Processing. 171.Google Scholar
- Rafiya Begum, Kalika Bali, Monojit Choudhury, Koustav Rudra, and Niloy Ganguly. 2016. Functions of Code-Switching in Tweets: An Annotation Framework and Some Initial Experiments. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (Eds.). European Language Resources Association (ELRA).Google Scholar
- Linda Bell and Joakim Gustafson. 1999. Interaction with an animated agent in a spoken dialogue system. In Sixth European Conference on Speech Communication and Technology .Google Scholar
Cross Ref
- JULIA CAMBRE and CHINMAY KULKARNI. 2019. One Voice Fits All? Social Implications and Research Challenges of Designing Voices for Smart Devices. (2019).Google Scholar
- Antoinette Camilleri. 1996. Language values and identities: Code switching in secondary classrooms in Malta. Linguistics and education, Vol. 8, 1 (1996), 85--103.Google Scholar
- Justine Cassell, Timothy Bickmore, Mark Billinghurst, Lee Campbell, Kenny Chang, Hannes Vilhjálmsson, and Hao Yan. 1999. Embodiment in conversational interfaces: Rea. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 520--527.Google Scholar
Digital Library
- Justine Cassell and Kristinn R Thorisson. 1999. The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence, Vol. 13, 4--5 (1999), 519--538.Google Scholar
Cross Ref
- Leon Ciechanowski, Aleksandra Przegalinska, Mikolaj Magnuski, and Peter Gloor. 2018. In the shades of the uncanny valley: An experimental study of human--chatbot interaction. Future Generation Computer Systems (2018).Google Scholar
- Kevin Corti and Alex Gillespie. 2016. Co-constructing intersubjectivity with artificial conversational agents: people are more likely to initiate repairs of misunderstandings with agents represented as human. Computers in Human Behavior, Vol. 58 (2016), 431--442.Google Scholar
Digital Library
- Sonam Damani, Nitya Raviprakash, Umang Gupta, Ankush Chatterjee, Meghana Joshi, Khyatti Gupta, Kedhar Nath Narahari, Puneet Agrawal, Manoj Kumar Chinnakotla, Sneha Magapu, et al. 2018. Ruuh: A Deep Learning Based Conversational Social Agent. arXiv preprint arXiv:1810.12097 (2018).Google Scholar
- Cristian Danescu-Niculescu-Mizil, Michael Gamon, and Susan Dumais. 2011. Mark my words!: linguistic style accommodation in social media. In Proceedings of the 20th international conference on World wide web. ACM, 745--754.Google Scholar
Digital Library
- Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Sofia, Bulgaria, 250--259.Google Scholar
- Anna De Fina. 2007. Code-switching and the construction of ethnic identity in a community of practice. Language in society, Vol. 36, 3 (2007), 371--392.Google Scholar
- Jean-Marc Dewaele and Li Wei. 2013. Is multilingualism linked to a higher tolerance of ambiguity? Bilingualism: Language and Cognition, Vol. 16, 1 (2013), 231--240.Google Scholar
Cross Ref
- Jean-Marc Dewaele and Li Wei. 2014. Attitudes towards code-switching among adult mono-and multilingual language users. Journal of Multilingual and Multicultural Development, Vol. 35, 3 (2014), 235--251.Google Scholar
Cross Ref
- Jessica Ficler and Yoav Goldberg. 2017. Controlling Linguistic Style Aspects in Neural Language Generation. CoRR, Vol. abs/1707.02633 (2017). arxiv: 1707.02633 http://arxiv.org/abs/1707.02633Google Scholar
- Saurabh Garg, Tanmay Parekh, and Preethi Jyothi. 2018. Code-switched Language Models Using Dual RNNs and Same-Source Pretraining. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 3078--3083. https://doi.org/10.18653/v1/D18--1346Google Scholar
Cross Ref
- Registrar General and Census Commissioner India. 2011. Census of India 2011. (2011).Google Scholar
- Howard Giles. 2007. Communication accommodation theory .Wiley Online Library.Google Scholar
- Howard Giles, Donald M Taylor, and Richard Bourhis. 1973. Towards a theory of interpersonal accommodation through language: Some Canadian data. Language in society, Vol. 2, 2 (1973), 177--192.Google Scholar
- John J Gumperz. 1982. Discourse strategies. Vol. 1. Cambridge University Press.Google Scholar
- Khyatti Gupta, Meghana Joshi, Ankush Chatterjee, Sonam Damani, Kedhar Nath Narahari, and Puneet Agrawal. 2019. Insights from Building an Open-Ended Conversational Agent. In Proceedings of the First Workshop on NLP for Conversational AI. 106--112.Google Scholar
Cross Ref
- Annabell Ho, Jeff Hancock, and Adam S Miner. 2018. Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. Journal of Communication, Vol. 68, 4 (2018), 712--733.Google Scholar
Cross Ref
- Netiks International. 2019. Genetiks. (2019). https://genetiks.net/Google Scholar
- Mohit Jain, Pratyush Kumar, Ramachandra Kota, and Shwetak N. Patel. 2018. Evaluating and Informing the Design of Chatbots. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS '18). ACM, New York, NY, USA, 895--906. https://doi.org/10.1145/3196709.3196735Google Scholar
Digital Library
- Chaitanya K. Joshi, Fei Mi, and Boi Faltings. 2017. Personalization in Goal-Oriented Dialog. CoRR, Vol. abs/1706.07503 (2017). arxiv: 1706.07503 http://arxiv.org/abs/1706.07503Google Scholar
- Naveena Karusala, Aditya Vishwanath, Aditya Vashistha, Sunita Kumar, and Neha Kumar. 2018. " Only if you use English you will get to more things" Using Smartphones to Navigate Multilingualism. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--14.Google Scholar
Digital Library
- Nicole C Kr"amer, Astrid von der Pütten, and Sabrina Eimler. 2012. Human-agent and human-robot interaction theory: Similarities to and differences from human-human interaction. In Human-computer interaction: The agency perspective. Springer, 215--240.Google Scholar
- Ying Li and Pascale Fung. 2012. Code-Switch Language Model with Inversion Constraints for Mixed Language Speech Recognition. In Proceedings of COLING 2012. The COLING 2012 Organizing Committee, Mumbai, India, 1671--1680. https://www.aclweb.org/anthology/C12--1102Google Scholar
- Max M Louwerse, Arthur C Graesser, Shulan Lu, and Heather H Mitchell. 2005. Social cues in animated conversational agents. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, Vol. 19, 6 (2005), 693--704.Google Scholar
Cross Ref
- JM Lucas, F Fernández, J Salazar, J Ferreiros, and R San Segundo. 2009. Managing speaker identity and user profiles in a spoken dialogue system. Procesamiento del lenguaje natural 43 (2009), 77--84.Google Scholar
- Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI '16). ACM, New York, NY, USA, 5286--5297. https://doi.org/10.1145/2858036.2858288Google Scholar
Digital Library
- Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. 2019. Personalizing Dialogue Agents via Meta-Learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Florence, Italy, 5454--5459. https://doi.org/10.18653/v1/P19--1542Google Scholar
Cross Ref
- Rada Mihalcea and Stephen Pulman. 2007. Characterizing humour: An exploration of features in humorous texts. In International Conference on Intelligent Text Processing and Computational Linguistics. Springer, 337--347.Google Scholar
Digital Library
- Carol Myers-Scotton. 1993. Dueling Languages: Grammatical Structure in Code-Switching .Claredon, Oxford.Google Scholar
- Carol Myers-Scotton. 1995. Social motivations for codeswitching: Evidence from Africa .Oxford University Press.Google Scholar
- Carol Myers-Scotton. 2005. Multiple voices: An introduction to bilingualism .Wiley-Blackwell.Google Scholar
- Mark Mysl'in and Roger Levy. 2015. Code-switching and predictability of meaning in discourse. Language, Vol. 91, 4 (2015), 871--905.Google Scholar
Cross Ref
- Clifford Nass, BJ Fogg, and Youngme Moon. 1996. Can computers be teammates? International Journal of Human-Computer Studies, Vol. 45, 6 (1996), 669--678.Google Scholar
Digital Library
- Clifford Ivar Nass and Scott Brave. 2005. Wired for speech: How voice activates and advances the human-computer relationship .MIT press Cambridge, MA.Google Scholar
Digital Library
- Ocelot. 2019. Ocelot Bilingual Chatbot. (2019). https://www.ocelotbot.com/news/bilingual-chatbot/Google Scholar
- Adithya Pratapa, Gayatri Bhat, Monojit Choudhury, Sunayana Sitaram, Sandipan Dandapat, and Kalika Bali. 2018. Language modeling for code-mixing: The role of linguistic theory based synthetic data. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1543--1553.Google Scholar
Cross Ref
- Byron Reeves and Clifford Ivar Nass. 1996. The media equation: How people treat computers, television, and new media like real people and places. Cambridge university press.Google Scholar
Digital Library
- Shruti Rijhwani, Royal Sequiera, Monojit Choudhury, Kalika Bali, and Chandra Shekhar Maddila. 2017. Estimating code-switching on twitter with a novel generalized word-level language detection technique. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 1971--1982.Google Scholar
Cross Ref
- Dorsa Sadigh, S Shankar Sastry, Sanjit A Seshia, and Anca Dragan. 2016. Information gathering actions over human internal state. In Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE, 66--73.Google Scholar
Digital Library
- Bidisha Samanta, Sharmila Reddy, Hussain Jagirdar, Niloy Ganguly, and Soumen Chakrabarti. 2019. A Deep Generative Model for Code-Switched Text. arXiv preprint arXiv:1906.08972 (2019).Google Scholar
- Carol Myers Scotton and William Ury. 1977. Bilingual strategies: The social functions of code-switching. International Journal of the sociology of language, Vol. 1977, 13 (1977), 5--20.Google Scholar
Cross Ref
- Abigail See, Stephen Roller, Douwe Kiela, and Jason Weston. 2019. What makes a good conversation? How controllable attributes affect human judgments. CoRR, Vol. abs/1902.08654 (2019). arxiv: 1902.08654 http://arxiv.org/abs/1902.08654Google Scholar
- Heung-Yeung Shum, Xiaodong He, and Di Li. 2018. From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots. arXiv preprint arXiv:1801.01957 (2018).Google Scholar
- Yla R Tausczik and James W Pennebaker. 2010. The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology, Vol. 29, 1 (2010), 24--54.Google Scholar
- Genta Indra Winata, Andrea Madotto, Chien-Sheng Wu, and Pascale Fung. 2019. Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences. arXiv preprint arXiv:1909.08582 (2019).Google Scholar
- Donald Winford. 2003. An introduction to contact linguistics. Wiley-Blackwell.Google Scholar
- Yossi Matias Yaniv Leviathan. 2018. Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone. (2018). https://ai.googleblog.com/2018/05/duplex-ai-system-for-natural-conversation.htmlGoogle Scholar
- Jennifer Zamora. 2017. I'm Sorry, Dave, I'm Afraid I Can'T Do That: Chatbot Perception and Expectations. In Proceedings of the 5th International Conference on Human Agent Interaction (Bielefeld, Germany) (HAI '17). ACM, New York, NY, USA, 253--260. https://doi.org/10.1145/3125739.3125766Google Scholar
Digital Library
- Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing Dialogue Agents: I have a dog, do you have pets too?. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, 2204--2213. https://doi.org/10.18653/v1/P18--1205Google Scholar
Cross Ref
- Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2018. The design and implementation of XiaoIce, an empathetic social chatbot. arXiv preprint arXiv:1812.08989 (2018).Google Scholar
Index Terms
Do Multilingual Users Prefer Chat-bots that Code-mix? Let's Nudge and Find Out!
Recommendations
Toward a Multilingual Conversational Agent: Challenges and Expectations of Code-mixing Multilingual Users
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing SystemsMultilingual speakers tend to interleave two or more languages when communicating. This communication strategy is called code-mixing, and it has surged with today’s ever-increasing linguistic and cultural diversity. Because of their communication style,...
A Comparative User Study of Interactive Multilingual Search Interfaces
CHIIR '18: Proceedings of the 2018 Conference on Human Information Interaction & RetrievalWhile the number of polyglot Web users across the globe has increased dramatically, little human-centered research has been conducted to better understand and support multilingual user abilities and preferences. In particular, in the fields of cross-...
Bilingual by default: Voice Assistants and the role of code-switching in creating a bilingual user experience
CUI '22: Proceedings of the 4th Conference on Conversational User InterfacesConversational User Interfaces such as Voice Assistants are hugely popular. Yet they are designed to be monolingual by default, lacking support for, or sensitivity to, the bilingual dialogue experience. In this provocation paper, we highlight the ...






Comments