Goes to the Heart: Speaking the User's Native Language

We are developing a social robot to work alongside human support workers who help new arrivals in a country to navigate the necessary bureaucratic processes in that country. The ultimate goal is to develop a robot that can support refugees and asylum seekers in the UK. As a first step, we are targeting a less vulnerable population with similar support needs: international students in the University of Glasgow. As the target users are in a new country and may be in a state of stress when they seek support, forcing them to communicate in a foreign language will only fuel their anxiety, so a crucial aspect of the robot design is that it should speak the users' native language if at all possible. We provide a technical description of the robot hardware and software, and describe the user study that will shortly be carried out. At the end, we explain how we are engaging with refugee support organisations to extend the robot into one that can also support refugees and asylum seekers.


INTRODUCTION
Millions of people worldwide fee their countries due to reasons such as wars, climate change and hostile environments.Host countries such as the UK provide services and support for these refugees and asylum seekers, which includes helping them to navigate the necessary bureaucratic processes in the new country.The ultimate goal of our current project is to develop a social robot that can work alongside human support workers to help with these processes.As the target users are likely in a state of stress and in a new country, forcing them to communicate in a foreign language will only fuel their anxiety: a crucial aspect of the robot design is therefore that it should speak the users' native language if at all possible.
As a step towards developing the fnal robot system, we have begun by considering a less vulnerable population with similar needs: namely, international students in the University of Glasgow.Although these students are highly skilled and will all have passed an English test, in practice, they still often have limited conversational English skills which can lead to challenges navigating bureaucracy in a foreign language.The robot will be deployed in the student services enquiry hall (Figure 1), which is the main place where students go for assistance concerning academic and fnancial issues.It will work alongside student support workers, helping users engage with local services such as council tax and fnancial aid.
Non-UK students comprise almost a third of all the students in the UK, with one in fve students being both non-EU and non-UK.Looking into home countries, close to a third of all non-EU students are from China, and an increase of students coming from the United Arab Emirates [10].This distribution is mirrored by the international student demographics at the target university, so the robot will speak Mandarin and Standard Arabic in addition to English.
This work forms part of an increasing number of social robot studies which are being conducted in public spaces -"anywhere that groups of people who may or may not know each other can freely assemble, move about, and interact" [18].Our specifc aim is to use a social robot in a context where it is truly needed, where the users are members of the public rather than specifcally recruited participants, and where there is the clear potential for a concrete positive impact of the robot's presence on the users it interacts with.Evaluating the initial system with international students will provide a proof-of-concept of the technical details of the system and will also provide design feedback to help develop the details of the fnal system which aims to support the needs of refugees and asylum seekers, always alongside human experts.

RELATED WORK
Social robots are leaving the controlled environment of the lab and being increasingly deployed in public spaces, where the technical challenges are signifcant but the application areas and evaluations are much more realistic [20,25].For example, the Furhat robot [1] can be found acting as a receptionist [19], a museum guide [22], and a barista [15], while Pepper has been deployed in a wide range of contexts including shopping malls [7], restaurants [27], libraries [21], and train stations [29].User responses to these robots have generally been positive, although technical limitations on audiovisual sensing in public spaces do sometimes have an impact.
While the current prototype system is targeted at international students, the ultimate goal of the current project is to support the needs of refugees and asylum seekers, which falls into the general area of social care; this is an area where there is an increasing shortage of workers to help an ever-growing population, both in the UK and worldwide.In 2023, for example, it was reported that UK social workers are leaving their posts in record number [30], which posses a problem as there is a rising demand and decreasing resources [6].Moreover, there is also a drop in the number of qualifed nurses and midwives [26].In social care, there has been a serious and chronic under-funding of services for the last decade in the UK [24].One cannot ignore that there is a growing need for a technological solution, not to replace the human workers but to support them to allow them to complete their jobs efectively.
In Japan, robots are being developed for the beneft of the elderly population.In a study conducted in a Japanese care home, the staf had more complaints than praises, e.g., "staf stopped using Hug after only a few days, saying it was cumbersome and time consuming to wheel from room to room-cutting into the time they had to interact with the residents" [34].The staf was frustrated, being busy maintaining robots, instead of looking after the residents.
In a survey [14] of robotics publications between 2015 and 2020, the authors found that older adults did not want their caregivers to be replaced by robots [5], but as a geriatric nurse said: "...they help when no one else is there to help." [32].In addition, nurses in a US study envisioned that robots could empower them when confronting unsafe physicians behavior, contributing to patients' safety [28].An important aspect of well-being is a sense of belonging, the lack of which can lead to stress and uncertainty [33].This is especially crucial when dealing with refugees and asylum seekers.Concerning the sense of belonging to the host country, one of the challenges they face is learning a new language.Foreign Language Anxiety (FLA) is described as "the worry and negative emotional reaction aroused when learning or using a second language" [17], and was found to be closely related to test-anxiety, "a type of performance anxiety stemming from a fear of failure" [12].Chatbots and robots can be instrumental in providing help, as evident from a study where the participants reported a reduction of anxiety levels after several weeks of learning English with a chatbot [3].However, care must be taken when developing any artifcial speech-based system that it takes into account the linguistic needs of its target population and does not perpetuate existing biases from the natural-language processing and speech technology communities [e.g., 4, 8, 13].

RESEARCH HYPOTHESES
Using one's native language in human-robot interaction will contribute to: (1) A positive efect on communication efcacy.
(2) An increased sense of belonging.
(3) A reduction of anxiety levels.

CASE STUDY: STUDENT SUPPORT
While the eventual goal of this project is to develop a robot that is able to support refugees and asylum seekers, for the current study we have implemented a robot system to support international students, with the goal of deploying it in the student services enquiry hall at the University of Glasgow.The technical components of this system will also form the basis for the fnal planned system, although the details of user interactions with the fnal system will be developed together with the fnal target stakeholders.

Stakeholders
The process of developing the university support robot included a lengthy consultation with the student support team in order to ensure that the robot meets the needs of all the parties involved.We observed the support ofcers while they were interacting with international students on several occasions and also held meetings with the managers of the student support centre, in regards to usefulness and data safety.Ultimately, the system is built around four frequently asked topics: fnancial aid, council tax exemption, ofcial documents, and navigating the campus.We kept the interaction short, as the information is very technical, ending each session with a QR code that the student can scan, which will lead to information translated into their preferred language by a native speaker.

The System
The full system incorporates the Furhat robot and a Kindle Fire tablet.The user initiates an interaction with the system by using the tablet, where they can choose their preferred language: English, Mandarin, or Standard Arabic (Figure 2).After choosing a language on the screen, Furhat comes to life and starts the interaction in their language.Each session ends with a QR code that they can scan, which will lead to further information, again, in their preferred language.Figure 3 shows a sample interaction between the student and the robot, in English; the Mandarin and Standard Arabic interactions follow an identical model.
On a technical level, speech recognition is done through Google Cloud Speech-to-Text [9], using the microphone provided with the Furhat robot.The dialogue manager was coded in Python [31] as a state machine [16], advancing the user through the conversation in a task-based manner.Some states which require Natural Language Understanding to interpret the user's intent use OpenAI [23] to generate a classifcation and move to the next state.The use of OpenAI provides fexibility and robustness to potential speech-recognition errors in the public context.If the user asks for something outside the task-based domain, OpenAI is used to generate a short answer, followed by a question relevant to the support dialogue, allowing for a degree of social interaction in the context of the overall task-based dialogue.
The robot will speak in 3 diferent synthetic female voices, one for each language, provided by Amazon Polly [2].The choice of a female robot was made because the female voices sounded better than the male voices across the three target languages.The appearance of the robot is ethnically ambiguous, so it would seem plausible to the student that it could be fuent in all three languages.The online informative English texts have been translated into Mandarin and Standard Arabic by native speaking volunteers, as automatic translations are not always correct or natural to native speakers.

Support Information
All the information provided by the robot was verifed by the student support team.The authors created HTML pages for each of the sub-topics and had native speakers translate them into Mandarin and Modern Standard Arabic.We kept the interaction short, as the information is very technical, ending each session with a QR code that the student can scan and will lead to one of the aforementioned HTML pages.OpenAI is only used for a chitchat feature and for student: [The user clicks on "English" on the tablet] system: How can I help you?student: Will robots take over the world?system: Unlikely, it's a sci-f concept.How can I help you?student: I'm looking for a building in the campus.system: Where would you like to go? student: The admissions ofce, please.system: For a map to the admissions ofce, please scan the QR code below.[After 20 seconds] Have you scanned the QR code?student: Yes, thank you. system: Can I help you with anything else? student: No, that will be all.system: I would appreciate it if you'd rate our conversation by clicking on the tablet.Thank you and have a lovely day.student: [The user clicks on a happy emoji on the tablet]

Evaluation
At the end of each interaction, the student will give short feedback by choosing one of three emoji options (happy, neutral, or sad) on the screen, as shown in Figure 3.There will be a longer optional feedback questionnaire with general demographic details and 18 statements drawn from the SASSI questionnaire [11] concerning cognitive demands (e.g., I felt calm using the system), habitability (e.g., I always knew what to say to the system) and likeability (e.g., The system is friendly); these sections were chosen as they address the research hypotheses identifed above.In addition to the subjective feedback, all interactions with the system will be logged for further analysis.This will only be done on the intent level, e.g., the states which were used in the interaction, for privacy reasons.This data will be used to help guide the development of further versions of the robot system, both for this student population and for the ultimate target population of refugees and asylum seekers.

Ethics and Deployment
Ethical approval has been obtained for the deployment and the user study, and the robot will be deployed in the University's student services enquiry hall in early 2024.This is a period where a large number of new international students arrive and potentially need help with any of the tasks supported by the robot, so should provide an excellent test of the robot system.

SUMMARY AND FURTHER WORK
Social workers and people working in care cannot be replaced by robots, but can be empowered by them.Our goal is to develop a social robot system to support front-line workers who help new arrivals to the UK -initially we target international students, but our eventual goal is to develop a version of the system that can help refugees and asylum seekers.In both cases, we take into account the linguistic needs of the target users and aim, by speaking their own language, to strengthen their sense of belonging as well as easing the workload of the human support ofcers by automating some of their straightforward tasks.The robot will soon be deployed in the university student services enquiry hall in early 2024, and the insights from this initial study will be benefcial for our next study with disadvantaged and vulnerable individuals.

Next Case Study: Refugee Support
As part of developing the student support robot, we engaged with a range of university stakeholders to ensure that the robot met the needs of the target population.For the next version, our eventual goal is to support Syrian refugees and Asylum seekers, and the system will be implemented to support English, Modern Standard Arabic, and the Damascus Arabic Dialect to assess the impact of the diferent speaking styles.
As part of developing the next version of the robot, which should help refugees and asylum seekers, it is crucial to engage with them to fully understand their needs and preferences.Our frst step will be to carry out consultations including focus groups and informal interviews where we will present the Furhat robot and see how it can meet their needs and be of help.We are already engaging with local organisations that support refugees and asylum seekers, with the intent to develop the interaction style that will make them feel more at home and more welcome in their new host countries.

Figure 1 :
Figure 1: The student services enquiry hall in the university.

Figure 2 :
Figure 2: A system comprised of the Furhat robot and a tablet.

Figure 3 :
Figure 3: Sample interaction between a student and Furhat