ShareYourReality: Investigating Haptic Feedback and Agency in Virtual Avatar Co-embodiment

Virtual co-embodiment enables two users to share a single avatar in Virtual Reality (VR). During such experiences, the illusion of shared motion control can break during joint-action activities, highlighting the need for position-aware feedback mechanisms. Drawing on the perceptual crossing paradigm, we explore how haptics can enable non-verbal coordination between co-embodied participants. In a within-subjects study (20 participant pairs), we examined the effects of vibrotactile haptic feedback (None, Present) and avatar control distribution (25-75%, 50-50%, 75-25%) across two VR reaching tasks (Targeted, Free-choice) on participants’ Sense of Agency (SoA), co-presence, body ownership, and motion synchrony. We found (a) lower SoA in the free-choice with haptics than without, (b) higher SoA during the shared targeted task, (c) co-presence and body ownership were significantly higher in the free-choice task, (d) players’ hand motions synchronized more in the targeted task. We provide cautionary considerations when including haptic feedback mechanisms for avatar co-embodiment experiences.

In a within-subjects study (20 participant pairs), we examined the effects of vibrotactile haptic feedback (None, Present) and avatar

INTRODUCTION
Virtual Reality (VR) technologies enable people to not only immerse themselves in artificial digital worlds, but also enable previously impossible interactions that challenge our assumptions about our virtual bodies and social coordination processes in such virtual spaces.The proliferation of consumer head-mounted displays (HMDs) have made VR systems an increasingly common platform through which virtual social interactions can take place [16,45].VR enables meeting in a shared, immersive virtual environment [20,34], and interacting with virtual representations of human avatars and virtual agents.Social interactions are a key factor in VR -not only to prevent isolation of individuals in the virtual environment [49], but also to enable joint social activities and interactions [48,58] in and beyond social VR platforms such as VRChat and Rec Room [42].
Despite the myriad ways that social VR platforms can implement multi-user functionality, the most commonly used method today is giving each user their own individual avatar to navigate through the virtual environment.
Typically, such interactions adopt a first-person perspective, which is an important contributing factor to creating a sense of body ownership (i.e., the experience of having a virtual body [27]) and agency (i.e., involving a sense of control over a virtual body [4]) towards the user's virtual avatar.Extending beyond common social VR interactions, researchers have recently explored the concept of "virtual co-embodiment" [11,15], where two users embody a single, shared avatar.This inherently differs from a shared visual experience where multiple users would only share the same viewing perspective [66].Virtual co-embodiment offers a multi-user experience characterized by shared control over the avatar's movement.Since two or more people share control over the avatar, there is an increase in social coordination [11].Such 'fusionary' interactions are a component of what is dubbed the "JIZAI Body" [21]: the concept by which a computer-mediated human body can seamlessly adapt to social structure changes, such that any additions or alterations (virtual or physical) would feel as much their own as their original body.This relates to recent efforts toward sensible human-computer integration [7,40], which extends the notion of cybernetics [62].Indeed, experiments have shown that participants who co-embody a virtual avatar report high levels of perceived control, with lower levels of actual control [17,28].This enhanced perception of control can be useful for rehabilitation specialists and support personnel that use immersive technology for the treatment of physical and cognitive function of individuals that have suffered from stroke [37] and dementia [69], respectively.The shared control of virtual co-embodiment provides an assistive methodology to improve accessibility for these vulnerable individuals during the setup and navigation of virtual environments.In cases of stroke rehabilitation, co-embodiment can also be incorporated alongside techniques Manuscript submitted to ACM such as Mirror Therapy [36], to enrich motor support in the VR system by incorporating shared (human) assistance to adapt to requirements for each individual need.
Research has shown that individuals can adapt to different media to achieve their communication goals [43], where the degree of embodiment and time required for such embodiment may vary depending on the visual and visuo-motor consistency of the artificial (virtual) and biological body [1,10,25].Furthermore, when another entity that is sentient or appears to be sentient is present in the same environment, another dimension called 'social presence' comes into play [42,54].The degree of experienced social presence depends on the person's perceived ability to access another individual's intentions, intelligence, and sensory impressions.Immersive qualities, contextual properties, and individual differences, can predict the extent to which social presence is experienced by users in VR [42].Without a sufficient level of social presence, the other entity is perceived as artificial and not as an intentional social being [42].Without such perceived intentionality, shared and collaborative tasks become difficult, making a high sense of social presence vital for a smooth collaborative experience to occur during virtual co-embodiment.
However, a specific challenge in co-embodiment is that the visual feedback alone of the combined motion of the shared avatar does not fully provide the intentional information of one's partner because the shared avatar's motion is partially determined by one's own motion.Virtual co-embodiment leads to a situation where there is an intermingling of self-presence with social presence, where the identity of the self is intrinsically linked to the avatar and the presence of another intentional subject.This intentionality is translated through the amount of control available for each person over the shared avatar.While researchers have explored virtual co-embodiment in relation to users' perception of their embodiment of the shared avatar [11,15] and its use for motor skill learning [28,29], the role of social presence and its influence on social coordination within this context has not yet been fully characterized.Extensions of this concept by Hapuarachchi and Kitazaki [18] and Hapuarachchi et al. [17] have highlighted the need for non-verbal communication mechanisms to be implemented in the paradigm of co-embodiment to enhance sense of embodiment towards the co-embodied avatar.Indeed, during such co-embodiment experiences, the illusion of shared motion control can be coupled with user communication techniques that support coordination, highlighting the need for position-aware feedback mechanisms [59].Haptic feedback may be especially appropriate here because it can positively influence the experience of social presence [42].For this, we draw on Auvray et al. [2]'s haptic 'perceptual crossing paradigm', which was conceived to study social interaction dynamics in real time through tactile sensorimotor interactions [30].
Perceptual crossing refers to situations where two perceptual activities of the same kind meet each other, such as when two people catch each other's eye (joint gaze and attention [56]), or mutual social touch [50].This paradigm offers us a foundation to build systems that enable studying of factors involved in mutual recognition between people in remote interactions.Given the focus of co-embodiment scenarios on a sense of shared control with a remotely located other person, we believe the perceptual crossing paradigm lends itself well to studying sensorimotor interactions in co-embodied VR.
In this paper, we draw on the perceptual crossing paradigm [2], to explore how haptic feedback can be integrated into a virtual co-embodiment scenario, where pairs of participants share control over a virtual hand.Building on the idea of haptic perceptual crossing, we implemented haptic feedback (on/off) when users' hand positions overlap in virtual space.
We examine how haptic cues can enable awareness of each other and coordination between co-embodied participants.
We ask: (RQ) How do haptic feedback mechanisms and varied avatar control distribution influence users' sense of agency, co-presence, body ownership, and motion synchrony in targeted and free-choice virtual avatar co-embodiment tasks?In a controlled, within-subjects study with 20 participant pairs, we examined the effects of positional haptic feedback (None, Present) and avatar control distribution (25-75%, 50-50%, 75-25%) across two cube selection tasks  (Targeted, Free-choice) on participants' sense of agency (SoA), co-presence, body ownership, and motion synchrony.
Our findings showed (a) a lower sense of agency in the free-choice with haptics compared to no haptic feedback, (b) higher agency during the shared target task, (c) co-presence and embodiment were significantly higher in tasks where there were multiple targets, and (d) players' hand motions synchronized more in the targeted task.
Our exploratory work offers two primary contributions: (1) We integrate the concept of perceptual crossing into the paradigm of virtual co-embodiment to enable position-aware haptic non-verbal communication cues between two users; (2) We provide empirically backed insights showing the influence of haptics on the perceived sense of agency during targeted and free-choice selection tasks under variable control of shared virtual hands.

BACKGROUND AND RELATED WORK
In this section, we describe prior work on virtual co-embodiment, control sharing techniques, haptics integrated into co-embodiment experiences, and the perceptual crossing paradigm.
Manuscript submitted to ACM

Virtual avatar co-embodiment and the Sense of Agency
Virtual co-embodiment refers to occurrences where multiple users can simultaneously interact with the virtual environment using a shared avatar.Given that two or more individuals share control over the avatar, there is an increase in social coordination [11].To this end, prior work has shown that participants who co-embody a virtual avatar reported high levels of perceived control, with lower levels of weighted percentage of actual control [17,28].This makes it a promising tool for VR-based rehabilitation [23] and training [14,19] applications, since a learner with low control can feel a stronger sense of agency (SoA) while performing the activity with a teacher with high control.One domain in which researchers are trying to leverage the immersive capabilities of VR is in the support and treatment of dementia patients [69] -these individuals are considered vulnerable and they typically find it challenging to operate basic VR controls [31].In such cases, co-embodiment can enable assistive accessibility for these individuals, guided by support personnel.This would enable them to not only train, but also maintain a high level of agency during such immersive experiences.Furthermore, VR has shown potential for its ability to stimulate Mirror Neurons (MNs) of the internal sensorimotor system of stroke patients [5].In such settings, patients are immersed in training scenarios in virtual environments that involve executing motor actions, such as observing and visualizing mirror limb movements with the intent to imitate these actions.These have shown enhanced MN activation, leading to faster post-stroke recovery [37].
To that end, co-embodiment can be further leveraged within these techniques in order to improve the effectiveness of such treatments through enhanced agency and nuanced control over movements executed by the patients actively guided by their trainers or caregivers.
In the context of avatar co-embodiment, the 'Sense Of Embodiment' (SoE) can be manifested through three main components: Sense of Self-Location, Sense of Body Ownership, and the Sense of Agency [27].Sense of Self-Location refers to the feeling of 'being inside' a virtual body, while sense of body ownership and agency refers to the feeling of 'having' and 'controlling' the virtual body, respectively.Studies have explored various factors and their influence on these components, and have shown that manipulations of the overall SoE are possible through changes in avatar representations, degree of control, and perspective of the users [10,41].Similarly, the influence of sharing the virtual body with another user and its effect on SoE was studied in experiments of virtual co-embodiment.Here, the sense of agency and body ownership play a pivotal role that determines the engagement level during the shared perceptual activity [11,15,17,18,29].

Avatar co-embodiment control sharing techniques
To realize such co-embodiment, the motion of a shared avatar has previously been generated using two techniques: the weighted average co-embodiment method [11,15,28] and the body-part-segmented co-embodiment method [17,18].
The weighted-average method involves assigning a weight between 0 and 100 percent to each user and generating the movement of the shared avatar by interpolating the weighted average of the real-time position and orientation of the controllers of both the users (Figure 2) [11,15,28].The body-part-segmented co-embodiment method is a technique where the motion of independent limbs of a shared avatar is controlled by each user separately [17,18].In this paper we focus on the first method, where shared interactions can be manipulated by both the users and their influence is determined by the percentage of control they possess.Since we focus on position-aware feedback mechanisms, we draw on the weighted-average method to enable this.Results from [11,15,28] all showed that sense of agency increased with the increase in the control weight for the participant during avatar co-embodiment.In all the studies, participants could coordinate their movements in joint action, leading to the sharing of motor intention and synchronization.In a Manuscript submitted to ACM follow-up study by Kodama et al. [29], they evaluated participants' task performance and motor skill learning ability.
They concluded that learning using virtual co-embodiment was more efficient than the perspective-sharing method, in which a translucent teacher avatar was superimposed on the learner's first-person perspective view.However, contrary to the previous studies, no significant differences were observed between the different control weight conditions.We draw on the design considerations from [11,15,28] to design our study protocol.

Haptics for virtual co-embodied experiences
Since the early days of VR, haptic feedback has been a central component in many VR systems [55] and has been used to enable a diversity of touch-based interactions in VR [60], with the most common type of haptic feedback in VR applications being vibrotactile and force feedback [65].Studies that have explored haptics as a communication medium in the context of shared virtual spaces report enhanced user experiences [24,68].Important to our present purposes, the addition of haptic feedback to social VR has been found to consistently enhance perceived social presence [42].In work on co-embodiment, the application of haptic feedback is essentially understudied.Hapuarachchi and Kitazaki [18] explored the manipulation of the sense of agency by providing visual feedback of the partner's target during co-embodiment, and Hapuarachchi et al. [17] implemented passive haptics by attaching a back brace to both the users, allowing them to maintain consistent shoulder posture while controlling the shared avatar using the bodypart-segmented co-embodiment method.These explorations highlight the value of identifying what type of feedback modalities can be integrated into the virtual co-embodiment paradigm to provide users with advanced perceptual capabilities.While visual feedback offers more information to the user, it leads to cluttered, chaotic experiences when scaled up.Thus, haptic feedback provides an alternative to overcome this limitation.The challenge in the context of co-embodiment is to design the feedback mechanism in a way that does not increase the cognitive load required to differentiate between the interaction with the environment and the presence of the other user.Given the foregoing, we implement haptic feedback as a communication medium to indicate the other users' position during co-embodiment.

Perceptual crossing paradigm for enhancing social coordination
The perceptual crossing paradigm [2] was conceived to study social interaction dynamics in real time through tactile sensorimotor interactions [30].The classical paradigm features a minimalist 1D environment-a line-that loops around creating a continuous interaction space not visible to the users.Two users are each represented in the virtual space by an avatar (a dot) that they can control using a standard computer mouse.When their avatar encounters a virtual object in the 1D space they receive haptic feedback.There are three types of objects in the environment: a static object, the user's avatars, and the user's 'shadow,' an object that moves with the users' avatars at a set distance.All objects have exactly the same size and produce the same haptic feedback when encountered.When one user encounters the shadow of the other user, only the interacting user receives haptic feedback while the other user (to whom the shadow belongs), does not.The only condition when both users receive haptic feedback simultaneously is when both users' avatars encounter each other.Users are tasked with clicking the mouse when they think they are interacting with the avatar of the other user.In the original studies [2,3], users were successfully able to locate each other in the 1D virtual space.However, the probability of clicking when encountering another user's avatar was not significantly higher than clicking when encountering another user's shadow.Successful identification of the other could only be explained by the stability of mutual recognition; users would encounter one another, move back, encounter each other again, and repeat, creating an oscillating movement pattern of repeated encounters.In other words, users would only successfully recognize each other during perceptual crossing (i.e., perceptual activities of the same kind meet each other).
Manuscript submitted to ACM instructed to collaborate, participants successfully identified the other's avatar and, for those encounters, reported the clearest awareness of the others' presence [12].A version of the paradigm that used a following task in a skewed 1D environment highlighted that users were successful in following each other's movements through haptic perceptual crossing [32].Though part of the strength of the perceptual crossing paradigm lies in the minimalist approach, 2D extensions have already been successful [33].To the best of our knowledge, no 3D implementations of the paradigm have ever been attempted.We see an interesting opportunity in the implementation of the paradigm's basic premise (i.e., haptic feedback upon contact in a virtual space to signify the presence of the other) as an interactive cue that could aid movement coordination as well as enhance perceived social presence in virtual co-embodiment scenarios.

METHODS
In this section, we describe our research methodology, including the study design, experimental protocol, objective and subjective measures, our hardware and software setup, study procedure, and participant sample.
The control distribution consisted of three sets of weighing player one and player two's control over the shared avatar: 25-75%, 50-50%, 75-25% (referred to as W25, W50, W75).There was either no haptic feedback (NH) or haptic feedback when participants' hands overlapped (H) in the virtual space.This interaction was designed using virtual spheres (radius 8 cm), which is approximately the size of the virtual hand mesh that was attached to the controllers in the virtual environment.When the spheres of each user intersect with each other, the haptic feedback is triggered for both the users.The study was divided into three phases: Training, Task 1, and Task 2 (Figure 3).There were six distinct conditions (3 control distribution x 2 haptic feedback) for each task that was performed by a pair of participants.The experiment consisted of two tasks where each of these conditions were repeated twice, bringing the total to 24 (2 tasks x 2 repetitions) trials for the entire study.The subjective responses of the participants on the sense of agency were collected with a questionnaire after each trial, while the sense of co-presence and embodiment questionnaires were collected after each haptic feedback condition / block (after six trials).Task 1 was always performed before Task 2, and the two haptic feedback conditions were counterbalanced according to a Latin square design such that starting trial Manuscript submitted to ACM of each session consisted of all possible combinations of haptic and control conditions, with the remainder of trials subsequently randomized to mitigate order effects.The study was designed such that a sample consists of pairs of participants.For example, in a session, participant 1 would perform the task with 25% control four times (4x), twice with and twice without haptic feedback, while their counterpart had 75% control (performed also 4x).Similarly for 50% (4x) and 75% (4x).Therefore for "25%", "75%", and "50%", there were 4 samples for each task (twice with and twice without haptic feedback).

Protocol
3.2.1 Joint action reaching tasks.The most common method to evaluate virtual co-embodiment is using a reaching task.In this task, participants touch an object such as a cube [15] or sphere [11,18] using a shared avatar.This task typically focuses only on participants' motion, as adding additional interactions (e.g., button presses) can increase task complexity, which may render the task unsuitable for studying shared control.Fribourg et al. [11] introduced a reaching task, for three scenarios: free, target, and trajectory.During the free task, each participant was free to choose any sphere to touch, while the sphere to be touched was highlighted in the target task.The trajectory task involved following a particular path before touching a highlighted sphere, and it focused more on precision.To help answer our RQ, we need to better understand the influence of movement freedom and intention on the level of embodiment (sense of agency and body ownership) over the shared avatar using haptics.Therefore, we implemented two reaching tasks: targeted (Task 1) and free-choice (Task 2), which we describe below.

Training.
In the training phase, basic controls of our VR system were explained to each participant, including how to use the controller buttons to interact with widgets in the scene.Afterward, each participant performed an individual training trial, which showed the participants how to complete Task 1.Since the training session was performed individually, no haptic feedback was provided beforehand, since this can only occur in the later part of the study involving co-embodiment.

Task 1:
Targeted.In Task 1, participants used the shared right hand of the avatar to touch a cube that spawned in their field of view (Figure 4(a)).Once the shared hand collided with the cube, the cube would be removed.After a second delay, another cube was spawned at a pseudo-randomized location 1 (Figure 4(b)).The location was pseudo-randomized, instead of pre-generated, to minimize learning effects.The delay provided a small reset time for the participants to avoid physical and cognitive fatigue.A spatial chime sound also originated from the spawned location of the cube, to indicate to participants the location of the new cube as it is difficult for users to realize if the cube has appeared at a location that is not within their field of view.Participants had to touch the cube a total of 17 times in each trial during Task 1.

Task 2:
Free-choice.In Task 2, five cubes are spawned in front of the participants, who had to move the shared hand to touch any of them to progress (Figure 5).In this case, when the shared hand collided with any of the cubes, all the cubes would get removed, and after a second delay, all the cubes would re-spawn back in the same positions.During Task 2, in each trial, participants had to touch one of the five cubes a total of five times to proceed.This task was designed to simulate a scenario where participants would have to collaboratively choose the co-embodied movement without verbal communication.This provided a suitable scenario to investigate if the position-aware feedback mechanism modeled on the perceptual crossing paradigm will enable the co-embodied users to work together.Manuscript submitted to ACM Additionally, the start and end time of each trial and the duration of overlap of the participants' right hands were recorded.
3.3.2Subjective measures.Participants filled in the Simulator Sickness Questionnaire(SSQ) [26] before and after the study.Additionally, participants filled in the Igroup Presence Questionnaire (IPQ) [51] at the end of the study.
During Task 1 and Task 2, participants used the Oculus motion controllers to provide a Likert-scale rating ranging from "not at all" (1) to "fully in control" (7) for the question "How much do you feel in control?" after each trial to measure their subjective "Sense of Agency" over the shared avatar.These questions were embedded as panels in VR, allowing participants to stay immersed in the VR experience [47].After each haptic feedback condition, participants would answer three questions about their "sense of co-presence" and three questions about their "sense of body ownership", taken from standard questionnaires of co-presence [46] and avatar embodiment [44].Given these questions belong to two different questionnaires, we calculate the reliability scores separately.These six questions were selected based on their relevance to the study design, while reducing participants' workload and the total session time compared to using the full questionnaires.
Co-presence (CP) questionnaire (Cronbach's =0.87) (1) I felt that I was in the presence of the other person (2) I felt that the other person and I were together in the same space (3) I felt that the other person responded to shifts in my movement (e.g., posture, position) Body Ownership (BO) questionnaire (Cronbach's =0.58) (1) I felt as if my (real) hands were drifting toward the virtual hands or as if the virtual hands were drifting toward my (real) hands (2) I felt as if the movements of the virtual hands were influencing my own movements (3) At some point, it felt as if my real hands were starting to take on the posture or shape of the virtual hands that I saw

Hardware and software setup
Participants performed the study using Oculus Quest 2 Head-Mounted Displays (HMDs) and Oculus Touch VR motion controllers connected to desktop computers.These computers ran the virtual environment we created using Unreal Engine 5.12 , and were connected with Ethernet to ensure minimum latency between the computers.One computer hosts a local server while the second computer joined this server as a client.Each computer recorded the rotation and position of their respective users.To create a co-embodied avatar, the level spawns a "shared hands" avatar in the virtual world.This virtual representation was chosen to model a gender neutral representation of hands (cf., [52]).Since the controller was not represented in virtual space, we did not have the position of the hand to be wrapped around the controller, and instead showed a default open palm position pose.To determine the position of each of the shared hands, the avatar linearly interpolates between the position of User 1 and User 2, expressed by the following equation: I n t e r mi t t e n t S i n u s o i d a l He a r t b e a t Co n s t a n t where  controls the interpolation such that the resulting position is 100% of Player 1 when  is 1 and 100% of Player 2's position when  is 0. This value can be set to vary the control over the shared hands in each level to W25/W50/W75 to create the conditions outlined in the study design.

Haptic feedback design.
Previous experiments by Wentzel et al. [61] tested techniques to modulate amplification levels of vibrations and found that it impacted the user's comfort.Since our study is modelled on the perceptual crossing paradigm [2], which uses simplistic ON/OFF feedback for interactions between the participants, we also implement the haptic feedback to ON when participants hands are in the same virtual position and OFF otherwise.No additional hardware is implemented for sophisticated vibrotactile feedback cues as the scope of this study is limited to using the standard oculus motion controllers.Therefore, we conducted a pre-study to make an informed decision on the intensity and pattern of the vibrations.Fifteen Participants (Mean age = 25.66,SD = 2.09) tested four common type vibration patterns [53]: Intermittent, Sinusoidal, Heartbeat, and Constant (Figure 6) in combination with four intensity levels: 10, 20, 30, and 40.The intermittent pattern consisted of two vibration occurrences every second, the sinusoidal had one occurrence per second, the heartbeat pattern consisted of two short occurrences followed by a pause every second, and the constant pattern had a continuous vibration throughout.These were tested in VR using the same motion controllers that would be used for the main study.While receiving haptic feedback, participants rated on a Likert scale their perceived comfort ranging from very uncomfortable (1) to very comfortable (7), and their perceived intensity level ranging from very calm (1) to very intense (7).  1. Results of our pre-study on perceived mean intensity and comfort for 16 haptic patterns.The mean perceived intensity ratings range between 1.8 to 6.7 (where 1 is very calm and 7 very intense) and the mean perceived comfort ratings range between 2.2 to 5.2 (where 1 is very uncomfortable and 7 very comfortable).The chosen variant "sinusoidal pattern with intensity 20" received a mean intensity rating of 4.2 and comfort rating of 4.4.
From Table 1, results showed that high-intensity vibrations (30,40) had lower comfort ratings overall.The highest comfort rating was given to the heartbeat and sinusoidal patterns at intensity 10.However, this intensity level was often hardly noticeable by some participants.The variant with the sinusoidal pattern at intensity 20, provided a balanced level of intensity (mean score = 4.266) while still being comfortable (mean score = 4.4).Therefore, we chose to implement this variant of the haptic feedback for our main study.

Study procedure
At the study location, a table was placed where participants would fill out all the forms: demographics, informed consent, pre-and post-study SSQ, and IPQ.Two computers were placed side by side on a separate table and were connected to HMDs (Figure 7).One participant was randomly assigned to the computer acting as User 1, and the other to the computer acting as User 2. A video camera was also in place to record both participants' motions while they performed the tasks.A video showing this interaction is provided in Supplementary Material A. The position in which both participants would stand was marked on the floor.To reduce any possibility of injury and to simplify the interactions, participants conducted the trials while standing, and only used their right hand for the motion task.The spatial chime sound was channeled through the HMD speakers, set at a comfortable 60% volume.
Upon their arrival, participants were asked to read and sign the informed consent form and fill in a pre-study SSQ.Similar to Fribourg et al. [11], participants were briefed that they would be sharing the avatar during all trials and instructed to avoid communicating with each other verbally.No instructions were provided regarding the haptic feedback, allowing participants to interpret the meaning of the vibrotactile cue when it occurs during the tasks.This was done to evaluate the effectiveness of the chosen vibration pattern in establishing synchronization patterns between participants autonomously, as was done in the original perceptual crossing experiments [2,3].Participants provided subjective ratings on their sense of agency, co-presence, and body ownership after each set of trials using questionnaires Manuscript submitted to ACM impression of the study, their perceptions of the shared motion and the haptic feedback, their impressions regarding the two tasks, and provocations regarding further use cases of virtual co-embodiment.The complete interview guide is provided in Supplementary Material B. Sessions lasted an average of 60 minutes, whereas the within-VR portion took approximately 25 minutes.Each participant was compensated with a €/$10 gift voucher for participating.Our study received approval from our institute's ethics and data protection committee, where we also followed any guidelines pertaining to any prevailing cleanliness (cf., COVID-19) regulations.Participants were recruited primarily from the first author's university.All were right-handed.Fifteen participants reported no prior VR experience; 17 reported being novice users (having used VR at least once), and eight reported occasional VR use.Of the 20 pairs, three were couples, 12 were friends, and five did not know each other (i.e., strangers).There were three male-male pairs (two friend pairs, one stranger pair).There were six female-female pairs (five friend pairs, one stranger pair).There were eleven male-female mixed pairs (three were couples, six were friend pairs, two were stranger pairs). 3For effect size f=0.25 under =0.05 and power (1-)=0.95,with 24 repeated measurements within factors, a minimum of 12 participants is needed.

ANALYSIS AND RESULTS
A mixed-methods approach was adopted for analysis, which means the results of the quantitative analysis are interpreted along with the qualitative analysis to explain the phenomena observed.
4.1 Pre-processing and analysis approach 4.1.1Synchrony measure.Users' hand motion data was re-sampled to 100Hz.We cleaned the data by removing missing values, NAs (Not Available) due to logging errors, and duplicates.This resulted in the removal of 11,679 records, with a final dataset size = 880,549 records.Several measures of inter-personal synchrony exist, from dyadic synchrony in VR as was done by Sun et al. (2019) [57], to breathing synchrony as was done by El Ali et al. ( 2023) [8].Given our dataset, we analyze joint motion synchrony by adapting Sun et al. 's [57] approach -we perform the following steps to obtain our synchrony measures: The extracted <X,Y,Z> positional movement data was used to calculate the distance moved between each timestamp for each participant.We calculate the Euclidean distances for the movement of both participants, by taking the square of the difference between the consecutive positions in each direction (X, Y and Z).We then compute the square root of the sum of these squared differences to calculate the overall Euclidean distance for the movement between each timestamp.The intervals of the timestamps that are considered for this calculation are short (in the order of milliseconds), therefore any repeated movements (left and right) that occur over an interval will be captured, and will be different from a continuous motion in a single direction.
We then compute the rolling Spearman correlation between each participants' summed (right hand) Euclidean movement.Since our Euclidean measures were not normally distributed, we used rolling Spearman's Rank Correlation Coefficient with a window size of 450 samples to compare the two movement series.The mean of the rolling Spearman's Rank Correlation Coefficient was then calculated for all 24 trials across the 20 sessions4 .

Statistical analysis approach.
The combined effects of task, control, and haptics on participants' subjective ratings of perceived Sense of Agency (SoA), co-presence, body ownership, and mean Spearman's Rank Correlation Coefficient were analyzed by fitting a full mixed-effects model for each dataset.First, the normality of the data was tested using the Shapiro-Wilk test.Results for all dependent variables showed that the data distribution significantly deviated from normality (p < 0.05).Therefore, aligned rank transforms were applied to the data before fitting it to the model [63].
Holm-Bonferroni corrections were applied to the datasets, and contrast tests were conducted using ART-C [9].The results of the analysis of variance for all response variables are provided in Table 2.

Sense of Agency.
The analysis of the Sense of Agency (SoA) ratings are shown as boxplots in Figure 8(a), where lines with asterisks indicate pairwise, Holm-Bonferroni corrected, significance.A full mixed-effects model showed significance for Task and Haptics (p < 0.000).Significant interaction effects were also found between Task and Haptics (p < 0.000).Contrast tests for the main effect of Task revealed that responses were significantly higher in Task 1 compared to Task 2.Moreover, the contrast test for Haptics revealed that participants' feelings of control were significantly greater in conditions without haptic feedback compared to conditions with haptic feedback.The contrast test on the interaction effects between Task and Haptics showed significant differences across all levels except between Task 1 -No haptics condition x Task 1 -Haptics condition (p = 0.219).

Response Variable
Factor  2. Analysis of Deviance on the full mixed-effects model for Sense of Angency (SoA), co-presence, body ownership and mean rolling Spearman Rank Correlation using Aligned Rank Transformed data.For Sense of Agency, the model shows significance for the factors Task, Haptics and the interaction of Task and Haptics (***p<0.001).For Co-presence 1, Co-presence 2 and Co-presence 3, the model shows significance only for the Task factor (***p<0.001).For Body ownership 2, the model shows significance only for the Task factor (**p<0.01) and for Mean rolling Spearman's rank correlation coefficient, the model shows significance only for the Task factor (***p<0.001).

Manuscript submitted to ACM
T a s k 1 t r i a l s T a s k 2 t r i a l s 4.2.6 SSQ Motion sickness.Participants' reported motion sickness was measured before and after the study using the SSQ questionnaire [26].A Wilcoxon signed-rank test was conducted since the data did not have a normal distribution.

Qualitative results
We used an inductive thematic analysis [6] approach.First, the lead author extensively reviewed the interview transcripts and recorded videos, generating initial codes and themes.Then, the other authors reviewed the codes and themes for consistency and offered additional themes as needed.Quotes are attributed to participants by indicating which pair (P1-P20) they belonged to, followed by the specific participant (PN-1 or PN-2) where appropriate.Overall, not only did participants feel more at ease with sharing the motion when a common target was presented, but the addition of free choices in Task 2 added confusion and frustration.
4.3.2Perception of shared motion.Only about half of the participants (21/40) were consciously aware that the motion of the avatar was shared between them and their partner during Task 1, while the rest expressed that it only became evident to them during Task 2, when differences in choices emerged between them and their partners.Participants attributed the differences between their motion and that of the avatar as "glitches" or "delays", rather than the input of the other person in the pair.For instance, [P8-1] mentioned, "In the beginning, it felt like the hand was not working well", and [P16-2] remarked, "I saw this (movement), and I thought it was an algorithm".Importantly, during Task 2, when participants felt a diminished sense of agency or their partners' movements were not well coordinated, they exaggerated their movements.This compensated for a perceived lack of responsiveness in shared motion: "When I moved my hand, I noticed the hand didn't move that much, so to compensate for it, I had to reach out more" [P8-1].This was particularly evident when there was a substantial difference in height between the pairs of participants, where there was an imbalance of control due to the taller participants' extended reach, which resulted in a frustrating experience with shared motion for the partner with less reach.mentioned that they would be more attentive to the other person's movement had it been with an unfamiliar person.

Motion synchrony.
A high level of motion synchronization was observed during Task 1; participants started the study with distinct motions, which eventually joined when one participant began mimicking the hand motion of the other.Participants made similar observations, referring to these synchronizations as "rhythms" or "flows".For example, [P1-1] mentioned: "after a few rounds it felt like we were getting into this rhythm, " and [P2-2] stated: "I started with arc motion and [P2-1] was doing a different motion, then [P2-1] started moving with arc motion".Manuscript submitted to ACM

DISCUSSION
Below we discuss our study limitations and future work, and thereafter discuss our key findings by interpreting and synthesizing the results of our quantitative and qualitative analysis.

Study limitations and future work
First, we tested only a subset of questions in common co-presence and embodiment questionnaires -while such additional measures could shed further light on the experience of virtual co-embodiment, this was a deliberate design choice to ensure users do not experience fatigue and overload during the study.Second, while we followed closely Jeunet et al. [22]'s question regarding the sense of agency, we found that some participants may have misinterpreted what was meant by 'feeling of control'.They judged the question to be related to success in the task rather than actual control over their body movements.Indeed, agency within HCI can have multiple interpretations (see [4] for a review), and we see this as a promising avenue for future work to explore other methods for evaluating the sense of agency in such shared virtual co-embodiment experiences.Third, it may be worthwhile to further extend the basic perceptual crossing paradigm in future work, by systematically investigating how varying the presence and type of haptics-related instructions and training beforehand would influence participants' shared agency during co-embodiment tasks.Fourth, we restricted ourselves to studying hand ownership, we do not investigate realistic full-body avatar representations (cf., [11,28]).Furthermore, previous studies have shown that the realism of the avatar [10] and users' choice of avatar [35] impacts their sense of embodiment.Given our focus was on better understanding the role of haptic feedback and shared control distribution across targeted and free-choice tasks, we kept our study variables to a minimum to avoid blowing up the parameter space.However, this provides an interesting area for further research -does the type of avatar body, or mixed hand representation shared amongst users similarly influences the sense of agency and co-presence?
Fifth, it worth exploring how height differences between participants and their reach can impact experiences of shared avatar control.To this end, prior work has developed methods that can generate avatar body characteristics that can adapt to variable heights of participants that can be used [67] -this would help ensure that control is distributed precisely between the participants, even if this does not necessarily reflect real-world user characteristics.Finally, given our finding that the type of relationship with another person can influence following and leading behavior (cf., Sec 4.3.3),this opens up opportunities to further examine co-embodiment interactions in different dyad compositions.

Elucidating the role of haptic feedback and avatar control distribution for virtual avatar co-embodiment
Our study explored the impact of including haptic feedback and varying avatar control distribution on users' sense of agency, co-presence, body ownership and motion synchrony across reaching tasks in a virtual co-embodiment scenario (RQ).To this end, one key objective was to assess how the condition in which participants would receive haptic feedback when their hands overlapped would affect these three factors in scenarios involving shared goals and free choice.Our findings indicate that the presence of haptic feedback yielded a significant effect on the sense of agency during our study, though in unexpected ways.Participants felt a significantly greater sense of agency during conditions without haptic feedback compared to conditions with haptic feedback.Given that haptic feedback is well-suited for conveying non-verbal cues [39], we expected that haptics would facilitate, not hinder sensori-motor coordination and guidance [38].Furthermore, our qualitative findings (Sec 4.3.5)indicated that the vibrotactile patterns within our haptic feedback were perceived to be a hindrance and at times intrusive.Despite that we took care to ensure pleasant Manuscript submitted to ACM vibrotactile patterns through a pre-study, as participants reported, they were reminded of smartphone and smartwatch vibrations and notifications.To interpret this finding, we first note that given our focus on translating elements of perceptual crossing into the avatar co-embodiment paradigm, we restricted our study to the context of autonomous interaction processes during shared perceptual activities.This means that even without conscious awareness of the vibrotactile cues, we expected that such position-aware haptic feedback mechanisms would support shared perceptual experiences, in this case, shared motor activity during targeted and free-choice reaching tasks.However, given the salience of the haptic stimuli, we suspect that haptics may have lowered the sense of agency for participants as they may have felt overwhelmed by the other users' guidance.This, along with the interplay of control, coordination, and physical attributes, would have then played a role in shaping the strategies the participants used in synchronizing with the other user during the haptic feedback conditions.Together, the foregoing raise cautions about how haptics can be integrated, suggesting that including vibrotactile-based haptic feedback as a positional guidance mechanism in 3D virtual space during such shared control interactions may not be an effective means for improving shared avatar co-embodiment experiences.

Shared control, motor synchrony, and perceptual crossing across targeted and free-choice tasks
Our findings indicate that participants' reported feelings of control (SoA) do not align with the actual levels of control.
We found that participants' sense of agency increased between 25% and 50% control conditions, while a decrease was observed between 50% and 75% conditions.This result echoes the findings of Kodama et al. (2023) [29], who did not find a clear differentiation between the tested levels of control.Moreover, we found that participants felt a significantly greater sense of agency in Task 1 (targeted) compared with Task 2 (free-choice).Since the conditions were counterbalanced and trials randomized, participants may have had a difficult time to judge absolute control levels, as they had no relative comparison to indicate such experienced control levels.We use absolute judgements for measurement of subjective responses since it is the standard practice across prior work [11,15] that investigates the sense of agency and also provided a means to test if haptics would lead to higher (perceived) sense of agency in a given trial, without referencing back to earlier trials (which may not have had haptics activated).As such, overestimation of control was apparent during Task 1.This lends credence to the findings by Fribourg et al. [11], who also found that participants perceived a greater sense of agency when the goal was shared compared to situations where participants pursued different goals.Furthermore, participants felt a significantly greater sense of co-presence during Task 2 compared with Task 1, suggesting that strong motion synchronization effects may have diminished the awareness of the other.This was further echoed by participants, where some reported a lack of awareness that they were sharing an avatar with their partners during Task 1. Qualitative analyses of user responses in the perceptual crossing paradigm also highlight that movement synchronization might not always necessarily indicate recognition of each other [30].In our specific implementation, in Task 1 (targeted), it might have been the case that the straight-forward task environment afforded high synchronization, but no real space for active exploration to take place (i.e., similar to the oscillating movements found upon successful recognition of each other in perceptual crossing studies) limiting opportunities to become aware of the other participant.
As such, participants' attention to their partners' movement and need to explicitly communicate verbally with their partners during Task 2 also indicates that they were more inclined to consciously co-ordinate, compared to the more autonomous interaction that was observed during Task 1.
We also found that participants felt that the movements of the virtual hands were influencing their own movements (Body ownership 2; cf., Sec.4.2.3)significantly greater during Task 2 compared with Task 1.This was further reflected upon by participants who stated they actively strategized to either exert more control over the virtual hand or to Manuscript submitted to ACM follow its movements during Task 2. Interestingly, similar strategizing about movements when encountering other users are found in the perceptual crossing paradigm, where in some cases users would spontaneously adopt leader and follower roles [13].For example, users may choose to remain stationary and passively receive the other's touch [30].
These parallels are interesting given the stark differences in available sensory information in our tasks compared to the perceptual crossing paradigm.It raises interesting questions about ways in which the basic paradigm can be extended and integrated into more multi-sensory shared virtual environments.The foregoing raise fundamental questions about the nature of shared control and social coordination as we integrate with machines and one another [40]: to what extent should we be consciously aware of bodily feedback mechanisms during shared activities?Given the importance of motion synchrony in varying the levels of conscious awareness of the virtual other, to what extent should shared body control systems, whether with humans or machines, leverage this without impeding on users' sense of perceived and actual agency?

CONCLUSION
We investigated whether integrating haptics into shared avatar co-embodiment can enhance users' shared VR experiences.Drawing on the perceptual crossing paradigm, we examine whether implementing non-verbal feedback mechanisms (namely, haptic feedback) within embodied interaction between two users can improve such social coordination experiences.Insights from this work provide a deeper understanding of the dynamics between users during co-embodiment and its impact on the perceptions of their sense of agency, co-presence, and body ownership towards a virtual hand.We found that haptic feedback given to participants when their hands overlapped led to a diminished sense of agency during co-embodiment.Our findings showed (a) a lower sense of agency in the free-choice with haptics compared to no feedback, (b) higher agency during the shared target task, (c) co-presence and embodiment were significantly higher in tasks where there were multiple targets, (d) users' hand motions synchronized more in the targeted task.Our work contributes a deeper understanding and cautionary considerations for the role of vibrotactile haptic feedback and shared control distribution in the emerging area of virtual avatar co-embodiment.

Us e r 1 J o i n t a c t i o n t a s k Us e r 2 Fig. 1 .
Fig. 1.Two users co-embodying virtual hands performing two types of joint action tasks in a shared virtual reality environment Virtual co-embodiment enables two users to share a single avatar in Virtual Reality (VR).During such experiences, the illusion of shared motion control can break during joint-action activities, highlighting the need for position-aware feedback mechanisms.Drawing on the perceptual crossing paradigm, we explore how haptics can enable non-verbal coordination between co-embodied participants.

Fig. 2 .
Fig. 2. Illustration of the weighted average virtual co-embodiment method, where the motion of the shared avatar (center) is generated by taking the weighted average of the motion of User 1 (left) and User 2 (right)

Fig. 3 .
Fig. 3. Diagram illustrating the different phases of the Study procedure, along with textual labels explaining each component

Fig. 6 .
Fig. 6.Waveforms of four vibration patterns used in pre-study (created in Unreal Engine): Intermittent (top left), Sinusoidal (top right), Heartbeat (bottom left) and Constant (bottom right)

Fig. 7 .
Fig. 7.An image of the study setup that shows two users performing a task, and the two computer screens showing the perspective view of each user

Fig. 11 .
Fig. 11.Motion controller synchronization (using rolling Spearman's Rank Correlation with 450 samples) between users.Plot shows synchrony across trials for each session

4. 3 . 1
Frustration with shared decisions.Participants associated their experience while performing Task 1 with being more comfortable than during Task 2, stating that "I didn't feel much [shared control] in the beginning but in the second Manuscript submitted to ACM task with the choice it felt horrible" [P10-2].In fact, many participants expressed that, without a shared goal, they often "[...] thought that I'm not controlling and somebody's here to control the hands, and it made me a bit angry" [P15-2].

4. 3 . 3
Following and leading.Participants spontaneously took on a more follower or leader role during the trials.While some participants focused on actively following their partners' movements, aiming to coordinate their actions better, others took over the lead by misbehaving and "[...] trying to check control by doing the opposite movement"[P3-2], so that the share of control was more apparent.During Task 2, this difference was more evident, with followers expressing that since, "[...]I had no control, I thought I would follow whatever pattern in movement the other person was doing" [P11-1].On the other hand, leaders used this ambiguity by moving their hands dramatically to shift the shared hand, regardless of the amount of control they had over the avatar: "I can sort of limit other persons' actions and actually feel more in control" [P14-1].Additionally, five participant pairs noted that the relationship with their partners also influenced the degree of co-operation they were inclined to achieve.For example, pairs that knew each other[P7]

4. 3 . 5
Perception of vibration patterns and associations.Several participants did not fully grasp during the study that haptic feedback would occur when their hands overlapped with their partner, while others inferred negative associations with the haptic feedback during the study, based on their prior experience with vibration feedback patterns.For example, participants expressed that they interpreted the haptics as hostile: "I thought maybe I was wrong that's why the vibrations are coming to push me in another direction" [P6-2] or that the haptic feedback was "[...] very random, like it was malfunctioning" [P7-2].Participants who viewed the haptics as positive feedback tended to associate it with video games: "I play the Nintendo Switch, and if you win in the game, it will have vibration" [P18-2].