Human Impression of Humanoid Robots Mirroring Social Cues

Mirroring non-verbal social cues such as affect or movement can enhance human-human and human-robot interactions in the real world. The robotic platforms and control methods also impact people's perception of human-robot interaction. However, limited studies have compared robot imitation across different platforms and control methods. Our research addresses this gap by conducting two experiments comparing people's perception of affective mirroring between the iCub and Pepper robots and movement mirroring between vision-based iCub control and Inertial Measurement Unit (IMU)-based iCub control. We discovered that the iCub robot was perceived as more humanlike than the Pepper robot when mirroring affect. A vision-based controlled iCub outperformed the IMU-based controlled one in the movement mirroring task. Our findings suggest that different robotic platforms impact people's perception of robots' mirroring during HRI. The control method also contributes to the robot's mirroring performance. Our work sheds light on the design and application of different humanoid robots in the real world.


ABSTRACT
Mirroring non-verbal social cues such as affect or movement can enhance human-human and human-robot interactions in the real world.The robotic platforms and control methods also impact people's perception of human-robot interaction.However, limited studies have compared robot imitation across different platforms and control methods.Our research addresses this gap by conducting two experiments comparing people's perception of affective mirroring between the iCub and Pepper robots and movement mirroring between vision-based iCub control and Inertial Measurement Unit (IMU)-based iCub control.We discovered that the iCub robot was perceived as more humanlike than the Pepper robot when mirroring affect.A vision-based controlled iCub outperformed the IMU-based

INTRODUCTION
The mirror neuron system (MNS) in humans facilitates the understanding of others by simulating their behaviors via sensorimotor processes [5].Mirroring, a fundamental element of social interaction, involves subconsciously imitating another individual's nonverbal cues, such as gestures, expressions, and postures [10].It can reflect an adaptive integration and utilization of social cues within the social context [22].This mechanism often leads individuals to collaborate with those who exhibit similar and familiar behaviors [7].Mirror system dysfunction contributes to difficulties in social communication for individuals with Autism Spectrum Disorders (ASD) [18].Mirroring also plays a significant role in human-robot social interaction.By mimicking non-verbal social cues, humans feel socially closer to the robot and perceive it as more aware of the intentions behind their social behaviors [14].
For robots, affective mirroring causes people to perceive the robot as an agent capable of conveying internal states, displaying social intelligence, and expressing humanlike characteristics [3,6].Gonsior et al. [11] investigated the impact of mirroring facial expressions on empathy and perceived subjective performance in interactions with the robot head EDDIE [21], revealing that adaptive modes of robot behavior, where the robot mirrored human expressions, led to increased levels of human empathy and improved perceived task performance compared to a non-adaptive mode-without facial expression mimicry.Although most previous research shows consistent findings, few studies compare people's perceptions of affective mirroring on different humanoid robots.Robots convey emotional signals in various ways.For instance, the iCub robot can display simplified facial expressions with LED light pattern changes, and the Pepper robot can change the color of the shoulder and eyelids to represent emotions.It may cause people to interpret them differently for the same expression.
Movement mirroring enhances robots' sociability during humanrobot interactions, making them more humanlike, empathetic, and socially intelligent [4].Two primary methods of enabling robots to mirror human movements include IMU-based controlled and vision-based controlled imitations.IMU-based controlled mirroring uses readings from an IMU attached to a head-mounted eye tracker worn by an actor to directly translate their head movements into robotic actions [9].In contrast, vision-based controlled mirroring uses external cameras and pose estimation algorithms to interpret an actor's head movements and mirror them through a robot [8].Liu et al. [17] show that the lightweight model surpasses the other state-of-the-art models on the same robot doing the head movement mirroring.Geminiani et al. [9] find that the Microsoft Kinect-based controlled NAO robot outperforms the IMU-based controlled NAO robot regarding limb movement mirroring in the autism treatment.However, comparing different control methods of robots on doing head and gaze mirroring remains to be studied.
Social robots are designed to aid people, but individuals have been adapting to the robots instead.This is due to the fact that robots are not always designed with human preferences and interactive needs [15,19].Researchers in robotic mirroring are constantly improving humanoid robots' accuracy and timeliness in simulating social cues.However, research about subjective evaluation and preference of the robotic platform and control method is limited.
In this study, we conducted two experiments with two humanoids, the iCub and Pepper robots, as shown in Figure 1.The first experiment compared people's perceptions of affective mirroring on different humanoid robots.The second experiment assessed the impact of various control methods on the same robot platform doing movement mirroring.We evaluated the robots' performance by their mirroring speed and accuracy.People's perception of the robots was measured from four dimensions-Socially Intelligent, Mechanical, Responsive, and Humanlike.Through these investigations, our goal is to enhance the alignment of robotic design with human interaction preferences.We aim to solve these issues by investigating the following research questions (RQ): RQ1 How do different robotics platforms, specifically the iCub and Pepper robots, compare in affective mirroring?RQ2 How do various robotic control methods, especially visionbased controlled and IMU-based controlled methods, impact the iCub robot's performance in movement mirroring tasks?

STUDY DESIGN 2.1 Affective Mirroring Task
In this experiment, participants were asked to make eight facial expressions-Anger, Fear, Happiness, Disgust, Sadness, Neutral, Surprise, and Contempt-in front of the Pepper or iCub robots.The expressions were to be performed within one minute in any order.The robot mirrored participants' expressions either through affective signaling-by changing the Pepper robot's eye and shoulder LED colors [13,16]-or robotic facial expressions-by changing the iCub robot's eyebrow and mouth LED patterns [2].Next, participants were asked to match the colors displayed on the Pepper robot (depicted in the top row of Figure 2) and facial expressions on the iCub robot (depicted in the bottom row of Figure 2) to emotion categories.Technical details for running the experiment are provided as part of the Wrapyfi [1] tutorial series 1 .Upon completion of the task, participants were asked to scan a QR code appearing on the Pepper's tablet using their cell phones to complete a three-item questionnaire, evaluating their experiences with either robot.In both questionnaires, participants were asked to rate their interaction with the robots using a 5-point Likert scale: Q1 How precise was the robot in mirroring your facial expressions?(1 = very imprecise, 5 = very precise) Q2 Did the robot mirror your expressions with major delay?(1 = no significant delay, 5 = significant delay) Participants rated their impression of the robots on four dimensions -Socially Intelligent, Mechanical, Responsive, and Humanlike-using a 5-point Likert scale (1 = not at all, 5 = yes, a lot).

Gaze and Head Movement Mirroring Task
In this experiment, participants interacted with the iCub robot given two conditions.Under the vision-based controlled condition, the iCub robot's movements were actuated by a vision-based head pose estimation model.Under the inertial measurement unit (IMU) controlled condition, the orientation readings arrived instead from an IMU attached to a wearable eye tracker.Participants wore the eye tracker and were asked to look at the iCub robot, freely moving their eyes and head.Participants observed the movements of the iCub robot to evaluate the interaction.Technical details for running the experiment are provided as part of the Wrapyfi [1] tutorial series 2 .
Participants were asked to rate their interaction with the iCub robot using a 5-point Likert scale: Q1 How precise was the robot in mirroring your head movements?
(1 = very imprecise, 5 = very precise) Q2 Did the robot mirror your head movements with major delay?
(1 = no significant delay, 5 = significant delay) Q3 Did the robot move its eyes?(Yes/No) Q4 How precise was the robot in mirroring your eye movements?
(1 = very imprecise, 5 = very precise) Q5 Did the robot mirror your eye movements with major delay?
(1 = no significant delay, 5 = significant delay) Participants rated their impression of the iCub robot on four dimensions-Socially Intelligent, Mechanical, Responsive, and Humanlikeusing a 5-point Likert scale (1 = not at all, 5 = yes, a lot).

Experimental Setup
The participants were seated 80 cm away from the iCub robot's head, adjusting its height to match their eye level.A circular marker was placed beside the iCub robot to calibrate the Pupil Core eye tracker.Situated in front of the iCub robot was a Logitech C920 webcam facing the participants to perform tasks requiring a fixed view of their faces while the iCub robot moved its head and eyes.The Pepper robot stood facing the participants at an angle of 45 degrees with a distance of 1.2 m.The Pepper robot displayed an illustration of the ongoing task on its tablet and communicated the instructions verbally.The interaction was one minute long per task condition and the condition order was randomized.We used the Wrapyfi [1] framework for managing the task order, transmitting data between models and robots using various middleware, and orchestrating the experimental pipeline.

Participants
30 participants (female = 7, male = 22, preferred not to say = 1) took part in both studies.Participants were between 24 and 41 years of age, with a mean age of 28.7.All participants reported no history of neurological conditions-seizures, epilepsy, stroke, etc.-and had normal or corrected-to-normal vision and hearing.One participant's data was excluded from the Pepper robot's affective mirroring experiment because of self-reported color blindness.Another participant's data was excluded from the iCub robot's movement mirroring experiment due to technical issues.This study adhered to the principles expressed in the Declaration of Helsinki.
Participants signed consent forms approved by the Ethics Committee at the Department of Informatics, University of Hamburg.

RESULTS
We evaluated the results of both mirroring tasks, studying the perceived impression of the robot in each separate condition, as well as comparing the paired conditions within each respective task.Normality tests were conducted on the participants' answers 2 https://wrapyfi.readthedocs.io/en/latest/tutorials/Multiple%20Sensors.html to each dimension of the questionnaires.Results showed that their responses were normally distributed.In addition, all Post hoc tests in this study used Bonferroni correction.

Affective Mirroring
For the affective mirroring task on either robot, the recognition accuracy is listed in Figure 2.For the Pepper robot, participants were most accurate in recognizing anger (86.2%) and least accurate in recognizing fear (3.4%).For the iCub robot, participants were most accurate in recognizing happiness (100%) and least accurate in recognizing disgust (16.7%).

DISCUSSION
Participants associated the iCub robot's facial expressions with emotions more than the Pepper robot's affective signaling and found the iCub robot more humanlike.Another observation relates to     [20,23].Participants more accurately identified expressions of Happiness, Neutral, and Surprise on the iCub robot compared to the Pepper robot.This can be attributed to humans primarily relying on observing the mouth and eyebrows to recognize these facial expressions [12], features that the Pepper robot lacks.We compared two movement mirroring methods.The visionbased controlled method produced smoother, more precise, and more responsive movements than the IMU-based controlled method.The IMU-based controlled method transfers the IMU readings at a faster rate, but this causes jittery movements due to hardware limitations.These findings are also consistent with Geminiai et al. [9] that the IMU-based NAO robot is more intrusive and requires longer setup time than the Kinect-based NAO robot during the limb movement mirroring.However, in our study, both methods were perceived as equally humanlike, implying that less responsiveness does not contradict humanlikeness.
Several limitations could be addressed and investigated in future research.We could not compare movement mirroring on the two humanoid robots.This is because the Pepper robot is not able to roll its head or move its eyes, unlike the iCub robot.Our iCub robot doesn't have a full body, hence, we cannot study the limb mirroring between the two robots.Future studies could address the interaction effect between affective and movement mirroring.Moreover, researchers could investigate how different humanoid robots and control methods impact children with ASD, and whether it affects their social functions [24].

CONCLUSIONS
We investigated human perceptions of two humanoid robots in the affective and movement mirroring tasks.Our findings revealed that a robot displaying facial expressions like an iCub robot was perceived as more humanlike than a robot conveying affective signals like a Pepper robot.For gaze and head mirroring, a vision-based controlled robot performed better than an IMU-based controlled robot.This could be attributed to latency in processing and transmitting the filtered IMU readings.In summary, we showed that robotic platforms and robot control methods played an essential role in mirroring tasks during HRI.It may guide future humanoid robot design decisions to align with humans' needs.

Figure 1 :
Figure 1: A participant performing the four mirroring tasks in random order: A) The iCub robot mirroring facial expressions; B) The Pepper robot affectively signaling through LED color changes; C) The iCub robot mirroring head movement based on an inertial measurement unit (IMU) readings.The red circle shows the IMU; D) The iCub robot mirroring head movement according to a vision-based model.The red circle shows the camera.

Figure 2 :
Figure 2: Eight emotion categories mimicked on the Pepper (Top) and iCub (Bottom) robots in the form of affective signaling and robotic facial expressions, respectively.Results of the human study are reported below each image in terms of the average accuracy in matching each affective signal or facial expression to an emotion category.

Table 1 :
Impression of the robots under different task conditions (Mean ± SE)