Designing Social Educational Robots: How User Participation Shapes Acceptance

Recent trends in human-robot interaction emphasize the importance of user participation to foster acceptance. However, current research overlooks the influence of varying participation levels and lacks insights for non-participating users. We conducted two studies to explore user involvement in designing educational robots. The first study examined how different levels affected users' acceptance. Highly involved participants contributed to context, requirements, and prototyping, while those with low involvement only prototyped within predetermined parameters. Results showed that increased participation led to significantly higher utilitarian attitudes, hedonic attitudes, and intention to use. To validate this for non-involved users, a second study was conducted to assess whether prototypes designed with higher participation positively impacted non-participatory groups. Surprisingly, no significant differences were revealed. This finding raises questions about how participatory design processes should be structured to yield outcomes applicable to the broader user group not directly involved in the design process.


INTRODUCTION
In today's rapidly evolving technological landscape, robots are increasingly playing diverse roles, spanning education, healthcare, and home entertainment industries.However, the development and integration of robots with social attributes and interactive qualities have only yielded a limited number of success stories, while the majority of robots remain in the research phase others struggle in the market [2,23,27].Social robots have distinctive behavioral traits, blurring the line between machines and humans, engaging in interactive conversations, and responding to social cues [1,25,44,47].Because they share features with both machines and humans, social robots are seen as a complex new genre of technology [1,25,47].To develop social robots successfully, it is essential to assess user acceptance, defned as the user's willingness to use technology consistently over an extended period [17].
Large-scale European research reveals that participants' acceptance of robots varies depending on the application context, with greater acceptance of machine-like robots in perceived dangerous jobs but discomfort regarding robots caring for children, the elderly, and people with disabilities [10,12,40].In the educational context, concerns among teachers and parents include fears of dehumanizing pupils, fostering dependence, and limitations in recognizing children's emotional needs [33,41,42], leading to neutral attitudes and skepticism about actual deployment [13,33,41,42].Aiming to explain acceptance issues of social robots within society, some researchers advocate that there is a tendency to prioritize technological advancements in the development of these systems over understanding user needs, leading to a gap between technology and user requirements [20,23,26,29,43].
To address these challenges, participatory design (PD) methodologies in the feld of human-robot interaction (HRI) are gaining momentum.These methodologies aim to create robot designs that are socially acceptable and meaningful, aligning with user perspectives by involving potential users throughout the design process.Primarily rooted in the healthcare sector, PD has been employed to design robots that assist older adults dealing with depression [26], children with autism [24], individuals with visual impairments [4], and stakeholders involved in dementia care [30].In the educational sector, [37] explored the use of educational robots with teenagers to help them manage stress.The methods and degree of user involvement in these processes remain topics of ongoing debate, with variations in approaches and extensions.
In general, three primary study approaches can be identifed: evaluating existing robotic systems with participants [4,8,14,24,37], building robots from the ground up [11,30,38], and shaping design principles for PD methods in HRI [3,26,36].While research on user involvement in HRI is somewhat sparse, recent studies have started to demonstrate the positive efects of incorporating participant input throughout the design process [7,22,34,46].For example, [34] found that heightened participation in the prototyping phase of an educational robot-where participants selected features from a predefned list-positively infuenced their attitudes and diminished anxiety towards the robot.Additionally, they noted a non-signifcant trend indicating an increased willingness to own and learn with the educational robot.
In the discourse of conceptualizing the most efective approach to integrating PD methodologies in the design and research process, it is argued that using an extended, hands-on, and mixed-method approach incorporating techniques like scenarios, storyboarding, and card sorting, efectively encourages participant engagement and creative contribution [6,30,36].It stands to reason that extensive PD practices, by incorporating a diverse range of user perspectives and needs during the development process, would not only enhance the overall quality and relevance of the designed systems but also ensure that they are more universally accessible and benefcial, even for potential users who did not directly participate in the design process.However, the latter has never been validated in research.
This work contributes to the explorations of PD methodologies in HRI, specifcally focusing on educational robots.It hypothesizes that extensive, hands-on user participation in the PD process will lead to higher user acceptance of participants' developed robots.The efect of user participation on acceptance is assessed through participants' utilitarian and hedonic attitudes, along with their intention to use the robots.These correlates are derived from an acceptance model specifcally formulated by [16] to address the complexities associated with social robots functioning as autonomous and interactive social agents.To assess the broader applicability of participatory-designed robots, a second study evaluates the acceptance of the designed robots among participants not involved in the design process.The aim is to determine if design outcomes created with higher user participation in the design phase correlate with higher acceptance scores from a wider audience.Ultimately, the studies seek to uncover how PD processes can be optimized to yield design outcomes that are not only acceptable to participants involved in the design process but also resonate with a broader community represented by the participants.

STUDY 1
The frst study explored whether the degree of PD that participants experienced infuences the acceptance of educational robots.As suggested in [34] and [46], it was hypothesized that increasing participation in the design process leads to higher acceptance of the developed educational robot designs.Thus, a high PD group developed a robot's use case, design requirements, and prototype, while those in the low PD group only prototyped within predetermined parameters.

Methods
The study was evaluated by a checklist of the local ethics committee and via the Helsinki guidelines [21].The checklist, prior data collection published study plan and collected data are accessible via the Open Science Framework (https://osf.io/38xah/).

Participants.
To indicate an appropriate sample size, the authors were guided by similar studies using PD methodologies.Based on this a sample size of = 40 was chosen to efectively engage participants in design tasks and cover multiple steps of a typical design journey, while also considering the time-intensive nature of facilitating PD journeys.Participants, primarily students, were recruited from the university's participant pool.Participants were evenly distributed to the high PD involvement group ( = 20, with 7 males, 12 females, and one diverse identifying person; = 26.35,= 2.18), and the low PD involvement group ( = 20, with 8 males, 12 females, and no diverse identifying person; = 26.55,= 2.87).
2.1.2Apparatus and task.The study's experimental setup included PD activities, and a survey to evaluate participants' acceptance of the developed educational robots, and was conducted remotely using Zoom [49] and Miro [32].The two participatory conditions varied in the provided material.In the high PD condition, participants were tasked with creating their own educational robot.This involved identifying a learning-related problem scenario, defning design requirements, and prototyping the robot.In contrast, participants in the low PD condition were given a pre-defned use case and tasked with prototyping an educational robot, using a problem scenario and design requirements randomly retrieved from a participant in the high PD condition.Each design task was displayed on a separate template.The methodological setup was largely inspired by the social robot design canvas from [3], which provides design task templates for guiding stakeholders through a PD design journey tailored to social robots.For this study, the templates were adapted for individual use, made self-explanatory, and translated into German.To keep the time constant between the two experimental variations, participants in the low PD group flled out an alternate task, in which they defned a problem scenario and design requirements for an ergonomic workspace.The author devised a comparable mental challenge unrelated to social robots as an alternative task to maintain consistent timing between the two levels of PD involvement.

Design & Dependent variables.
The present study employed a between-subjects experimental design to investigate the impact of PD involvement on participants' acceptance of educational robots.Participants were assigned to one of two conditions: high PD or low PD.Participant levels of acceptance compromised utilitarian attitudes, hedonic attitudes, and intention to use their educational robot.For each of the dependent variables, an overall score was calculated based on the sub-measures defned by [16].For utilitarian attitudes, the subscales usefulness, ease of use, and adaptability were assessed.Hedonic attitudes were surveyed with the subscales enjoyment, attractiveness, animacy, social presence, sociability, and companionship with the associated scales.Prior robot experience was obtained as a control variable to detect and account for any potential biases.
2.1.4Procedure.Before the experiment, registered participants selected a date and time for their sessions and were randomly assigned to either the low or the high PD group, until a target of 20 participants was reached.Each experiment was conducted with a maximum of three participants.Upon entering the experiment via a Zoom Link, participants were informed about the study's setup, and the confdentiality of their data, as well as their rights regarding data deletion.Participants were then guided through the individual design tasks on Miro.After completing the tasks, participants were given a survey to evaluate their designs.The survey, which took 20-30 minutes to complete, collected demographic data, evaluations based on the robots participants developed, and information about their prior robot experience.The entire procedure took approximately 120 minutes.

Discussion
The frst study investigated how user participation in the design process impacts the acceptance of educational robots.We hypothesized that higher user participation in PD would lead to more favorable evaluations of the robot in terms of utilitarian and hedonic attitudes, and the intention to use it.The results supported all expectations, adding to the expanding corpus of research, indicating that PD yields positive outcomes for people involved in the design and conceptualization of the robot [34,46].This study builds on the fndings of [34] by demonstrating that the degree of user involvement can signifcantly afect behavioral intentions like the intention to use the robot.Moreover, we observed that in the group with high PD involvement, customizing robot functionalities to specifc tasks resulted in superior assessments of both utilitarian and hedonic attitudes.This aligns with fndings by [46], which showed that therapists' involvement in the design of Socially Assistive Robots (SARs) increased their willingness to use them, their perceived utility, and their interaction with robots.Higher evaluations in the high PD group were likely infuenced by the adoption of a mixed-method approach, allowing participants to dive into diferent usage scenarios, ideate features, and prototype the robot without any limitations.Through shaping the robots' purpose and tasks the robot is considered more useful, and participants are more willing to use and employ the technology in their day-to-day lives [15,18].
Participants designed educational robots on an individual basis rather than in interdisciplinary teams, leading to a diverse spectrum of robot designs tailored to personal preferences, lacking the collaborative character typically associated with PD [39].The study difered from prior research by having participants assess their own robot in the survey, instead of obtaining attitudes towards assessing existing robots.This self-assessment was likely infuenced by cognitive biases such as the "Ikea Efect", and the "I designed it myself efect" [19,31], where the creative efort and successful task completion foster feelings of competence and ownership towards the object [5,28]).However, the current fndings highlight the potential of extensive and hands-on user participation on participants' levels of acceptance towards individually customized educational robots.As we seek to determine whether in-depth user participation in PD journeys should be a fundamental component in developing social robots that meaningfully refect users' worlds, it is vital to explore how extensive PD benefts those who are not directly involved in the design process.The second study will expand upon the established foundation, providing insights into the generalizability of PD benefts across a wider audience.

STUDY 2
The rationale behind employing in-depth PD practices is to not only improve the system's overall quality and relevance to the participant involved but also to ensure its universal accessibility and advantages for even those potential users, uninvolved in the design phase.It is assumed that following an extensive hands-on approach in the design journey reveals designs that are sensitive to the users' requirements, cultural contexts, emotional experiences, and design preferences, and realistic in the sense that they are feasible for realworld usage [26,30,36].However, does involving potential users in the creative process indeed result in robots that are more likely to resonate with the broader population?This central question has rarely been the focus of contemporary research.Therefore, a second study was conducted, to explore the promoted potential of PD, gauging the level of acceptance of these robots among individuals who did not participate in the design stage.The objective is to establish whether a design outcome created with higher PD involvement, compared to lower PD involvement during the design phase leads to increased acceptance ratings from a broader demographic of students.

Participants.
The sample size of the second study was matched to the number of educational robot prototypes created in the frst study.Therefore, a fresh sample of = 40 participants were recruited to evaluate one randomly assigned robot per participant in an online survey.The participants, mostly students, were evenly distributed to the two groups: = 20 participants evaluated a design outcome of high PD (8 males, 12 females; = 29.4,= 2.23), and = 20 participants evaluated design outcomes based on low PD (5 male, 15 female; = 28.5, = 2.35).

Apparatus and task.
Participants in this study were tasked with evaluating an educational robot developed through a PD approach of study one.The assessment was carried out via an online survey, where each participant was randomly assigned an educational robot, presented using the prototyping template completed by a participant from the frst study.

Design & Dependent variables.
In line with the frst study, the second study obtained a between-subjects experimental design to investigate whether a design outcome based on higher PD resulted in higher acceptance of a participant who was not involved in the design process.Participants were assigned to evaluate a design outcome based on high PD or low PD.Participant levels of acceptance were measured in utilitarian attitudes, hedonic attitudes, and intention to use the educational robot.
3.1.4Procedure.Participants could enroll in the study through the university's participant pool.Upon registration, they received a link to the online survey, where they were randomly assigned an educational robot for evaluation based on their personal preferences.The participants were provided with a description of the robot, encouraged to familiarize themselves with it, and then evaluated their connection to the robot's application context.Finally, the acceptance of the robot was gauged using the same set of questions as in the frst study, with the robot being displayed once again before each set of questions.

Discussion
The second study aimed to explore the benefts of extensive user participation in the design of educational robots on acceptance levels among participants who did not participate in the design process.We hypothesized that robots developed with extensive user participation revealed higher scores in utilitarian and hedonic attitudes, as well as the intention to use, compared to those created with lower user participation.Contrary to our expectations, we found no evidence that in-depth user participation in the design process benefts a wider audience.
Although the results suggest that using comprehensive PD methods does not produce positive outcomes for individuals not involved in the design process, a few limitations may have afected these fndings.Participants formed their judgments based on the solution space template, which primarily used text descriptions and a user storyboard to detail the robot's role, features, interaction modes, and appearance, supplemented by a simple visual prototype.Previous work by [35,45] has shown that a two-dimensional representation of a robot falls short of conveying the full experience of its physical presence and interactive capabilities.The participants' limited prior exposure to social robots likely contributed to their conservative evaluations of the robots' utility and user experience.[9,16,26].To assess the advantages of participatory design (PD) robots for non-design participants, future research should evaluate both of-the-shelf robots and PD robots.This enables a comprehensive comparison between a user-centric, bottom-up designed robot and one developed by researchers in a controlled environment.

GENERAL DISCUSSION
The studies revealed that the benefts of increased user participation in the design process were manifested as higher acceptance scores exclusively from participants who were involved in conceptualizing the educational robot.For those who merely evaluated the robotic designs without being part of the design journey, the degree of user participation did not seem to infuence their perception of the design's quality.These results prompt a reevaluation of PD processes to ensure that the outcomes resonate with and apply to a broader audience that includes individuals not involved in the design process.
A limitation was the diverse designs of robots, each shaped by a single participant's perspective, leading to varied application contexts and behaviors.This diversity may have hindered clear acceptance patterns among non-participatory evaluators.Future research should adopt a participatory approach centered on a specifc topic, accommodating user needs in collaborative teams, as is customary in PD.This might balance between individual creativity and group consensus, potentially leading to more universally acceptable and applicable robot designs.
The study's reliance on visual prototypes, instead of physical ones, may have restricted a comprehensive evaluation.Physical prototypes are crucial in representing a social robot's embodiment, interaction design, and contextual use, which signifcantly afects human-robot interaction (HRI) [14,48].The social robot's physical form can more accurately refect its hedonic qualities, making interactive, real-world prototypes essential for future PD methodologies.
Our results underscore the complexity of PD and its nuanced impact on user acceptance of educational robots.While increased user participation is benefcial for those involved in the design process, its advantages are not automatically extended to uninvolved users.While our fndings are most directly applicable to educational robots and university students, further research could explore their relevance to other social robot contexts and age groups, ideally with larger sample sizes.The broader implications of our work point to the potential of user customization in enhancing the acceptance of social robots [19,31].As we further integrate robots into educational environments, it is essential to focus not just on technological advancements but also on ensuring these technologies are inclusive and adaptable to a diverse range of user needs.