“Tele” Me More: Using Telepresence Charades to Connect Strangers and Exhibits in Different Museums

The museum is changing from a place of passive consumption to a place of interactive experiences, opening up new ways of engaging with exhibits and others. As a promising direction, this paper explores the potential of telepresence stations in the museum context to enhance social connectedness among visitors over distance. Emphasizing the significance of social exchange, our research focuses on studying telepresence to foster interactions between strangers, share knowledge, and promote social connectedness. To do so, we first observe exhibitions and then interview individual visitors of a technical museum about their experiences and needs. Based on the results, we design appropriate voiceless and touchless communication channels and test them in a study. The findings of our in-situ user study with 24 visitors unfamiliar with each other in the museum provide insights into behaviors and perceptions, contributing valuable knowledge on seamlessly integrating telepresence technology in exhibitions, with a focus on enhancing learning, social connections, and the museum experience in general.


INTRODUCTION
The museum as a traditional public institution for explorative knowledge transfer is no longer understood as a place of pure presentation of content, but also as a place for dialogue and self-cocreation [56].Consequently, an exchange occurs not only among visitors and with the museum but also between museums themselves, facilitating the sharing of knowledge and exhibits [17,62].
With the advancement of technology, the Digital Social Museum leverages digital tools to enhance the museum experience, making it more interactive and personalized [41].This enables visitors to bring objects to life and interact with them without the risk of damage.In addition, telepresence allows exhibits to be integrated in novel ways outside of the hosting institution, where they are no longer tied to a physical location [24].The increased use of interactive technologies can, however, lead to a reduction of interpersonal interaction between visitors [69].Yet, it is through this exchange between people and the emotions it evokes that the museum experience remains memorable [6] and therefore benefits learning.Accordingly, there is a fundamental need to design the use of new technologies in the museum context in such a way that they promote and foster social interaction between visitors.In particular, this also includes exchanges with previously unknown people who have similar interests.This not only has the potential to broaden horizons [4] but can also create a sense of belonging [39].
In this paper, we expand the museum with telepresence technology to not only extend the exhibition space but also to foster social exchange among visitors in a seamless way.We therefore design the communication between both sides to analyze how a telepresence station can be implemented in this particular environment while providing a way to share exhibits and museum space.We investigate how the place, interaction type, and user role in telepresence interactions, along with the presentation of information on each side, impact telepresence, efficiency, and social connectedness.By addressing these questions, we seek to improve telepresence and enhance the social and engaging aspects of the museum experience, particularly in collaborative learning scenarios.
As a result, we contribute with the findings of the observations and interviews conducted in the Deutsches Museum to gain insights into visitor behaviors and perceptions.Further, we state the results of our quantitative and qualitative study.

RELATED WORK
In the following, we discuss the related work that set the foundation for our research consisting of telepresence, social learning as well as museum developments.
Telepresence has evolved in areas like medicine [66], psychology [3,54], education [34] or the general working environment [20,26] since its first appearance in 1980 [45].Besides 2D video streams, research proposed Head-Mounted Displays (HMDs) [63,65], cylindrical surfaces [36], or the real environment [52] in symmetrical or asymmetrical combinations [14,21,46] to create the feeling of telepresence.Factors like the modification of the view [7,15,43,48,50] or manipulation of the environment [63,68] can affect the experience, whereby limiting the modalities can sometimes enhance human performance [55].During our exploration, we noticed a significant gap in research on these technologies within the museum field outside of robotic telepresence.
Learning can be influenced by social factors.According to Maslow [42], social needs rank high in human psychological needs.Therefore, the desire for "Social Connectedness" serves as a motivational force.This includes encounters with strangers, providing emotional fulfillment and a sense of community [49].According to Nabavi [47], social learning occurs when an observer's behavior is altered based on the actions of another person.[18].Another way to obtain social knowledge is through collaborative problemsolving, wherein multiple individuals come together, share their skills, expertise, and efforts to address specific issues [1] to learn mutually.It enhances individual performance and fosters identification among team members [53].In today's world, collaborative problem-solving skills are important to address the growing complexity of issues [59].In the next step, we dive into social learning and telepresence applications in the museum context.
Museum implementations already incorporated social elements: Clarke et al. [12] emphasize task division and simultaneous interaction with exhibits and familiar companions, noting that these factors can distract from each other.Guo et al. [27] bring strangers together through interactive chairs focusing on visualizing social cues.While prior work explored methods to foster social exchange through on-side objects [13,16,37], still challenges remain with engaging solitary visitors in these activities [11].Looking at telepresence in museums, we could not find work that connects two exhibitions on-side.Instead most research focuses on allowing remote visits [33,38] through VR [30,44], AR [57], web [22,25,60,61], or robotic [2,10] interfaces.

DESIGN PROCESS
To gain more insights into the visitors, we conducted informal observations in a technology museum with different exhibitions in size and attendance as a basis for visitor interviews before designing a concept.

Formative Pre-Study
The observations revealed diverse visitors of various ages and mobility go alone to the museum observing others.We noticed that highly interactive exhibitions led to increased exchange between people with the most interaction occurring around central exhibits.Conversations mainly centered around exhibits while the person talking pointed at them.We observed visitors from diverse linguistic backgrounds, including sign language.
The semi-structured interviews of six males and four females (age: 19-86 years) who spent time alone at the museum revealed that half of the interviewees were open to socially connecting with unfamiliar visitors.An additional 40% stated they would consider an exchange if thereby they could gain additional information.Several interviewees faced communication challenges due to limitations or language barriers, making social interaction difficult.The interviews also revealed potential for engaging users who may not initially seek exchange through a safe environment, the feeling of contributing to something, interesting topics, or technology.

Concept
Based on these findings, we found that the target audience includes visitors alone at the museum who seek safe interaction with other visitors.Our objective is to foster a sense of connectedness, encourage social learning, and enable collaborative problem-solving beyond museum borders.We prioritize user-friendly and hygienic interaction, ensuring simple maintenance and cost-effectiveness to facilitate adoption by smaller museums while avoiding loose controllers to prevent theft in an unsupervised exhibition environment.
The experience should be seamless and accessible to reach a diverse range of users.A display solution offers the opportunity to automatically become part of the experience without overwhelming the user technically.Placing a screen in the exhibition room's center appears to be the most beneficial to interaction [51].However, we explore whether the physical location influences the interaction further in our study.We define the modalities [58] of our station, excluding smell, taste, and voice due to hygienic concerns, language differences and to avoid noise pollution.The selected action modalities are body and hand movement which can be accessed through a camera.We observed visitors often using context-dependent pointing [40] on exhibits to communicate.Pantomime, a context and culturally independent as well as a versatile communication form, enables communication without the use of tools or speaking [28,70,71].Additionally, we include drawing as one of the three main expressive modalities [67] that captures hand and body movements through displayed lines, which can contribute to collaborative problem-solving [5,9].As drawing was already implemented in many ways [23] we could not find touchless implementations in the museum context.We explore the roles of communication partners in telepresence interactions based on our observation.Additionally, we aim to understand the impact of different information presented to users on each side.Therefore, we carry out an experiment with a focus on addressing the following research questions: RQ1 How does the place affect the telepresence, efficiency, and social connectedness of the experience?RQ2 How does the interaction type affect the telepresence, efficiency, and social connectedness of the experience?RQ3 How does the user role affect the telepresence, efficiency, and social connectedness of the experience?

EVALUATION
To assess the impact of the independent variables place, interaction type, and role on participants' telepresence, efficiency, and social connectedness during voiceless collaborative problem-solving tasks in museums, we implemented a telepresence station to conduct a participant study within one week.Due to the limited time frame and a constrained number of participants, we employed a withinsubject design, ensuring each participant experienced each test situation at least once to avoid bias.We used a balanced Latin Square to order user role and interaction type combinations, minimizing consequential effects [35].

Apparatus
To address our research questions, we implement a telepresence charades game where two visitors silently explain exhibits to each other using pantomime or drawing.One person describes the object while the other selects from three options, creating an exchange between users.The metaphor of a "clear board" is used to avoid the negative impacts of viewing oneself [29].We implement the lowcost setup with a vertical screen, a deep camera, and a computer.Due to problems with the tracking of hand gestures and a lack of similar user interfaces, we decided to start the touchless drawing by bringing the hand closer to the screen, pausing it when moving away again.Both the describer and the guesser have the option to paint with different colors simultaneously.We kept the interface simple by slowly vanishing the drawing lines after 25 seconds so that no additional delete button is needed.This number was determined through prior tests.To avoid the Midas touch problem as known from gaze-based interactions [32], a 4-second cursor loading animation plays before activating the button.We positioned the buttons on the bottom of the screen so that everybody was able to reach them.

Study design
For the place variable, we selected a highly visited loud location near the entrance of a museum.We chose the other location outside of opening hours for its quiet environment.The interaction type was varied through pantomime and drawing, both to be performed collaboratively by participants.The user role variable includes the describer, who explains the exhibit, and the guesser, who selects one of the displayed objects.
To measure the dependent variables telepresence, efficiency, and social connectedness we utilized eleven questions each before and after the study, as well as 16 after each mode, quantitatively and qualitatively.To prevent participant exhaustion, we limit the questions in the main part to one section of a standardized questionnaire.Questions are available in English and the local language.The station documented the time and the success of the runs.Additional data can be found in the supplementary material.

Procedure
After we welcomed the participants, signed the consent forms together and the participants completing an introductory questionnaire on a tablet, we selected the program version from a paper sheet and started the application.Afterwards, we briefed the participants on the study's general course, station control, and interaction options.When both participants were visible on-screen, they could use the hand-tracking system to click the start symbol, initiating the first run.We noted down any occurring mistakes.After each run, participants filled in the corresponding tablet questionnaire, marked as completed by us.We repeated this process four times and then signaled the end of the first phase, leading to a location change (quiet vs loud).We marked completion, closed the current version, and selected the next one for the following run.The supervisor from the quiet side transported the participant to the loud location, ensuring participants did not meet.The other supervisor stayed at the loud station.After the location switch, the next version was opened, and again the process repeated four times.After completing the tests, participants answered the final questionnaire section.The participants were then officially released from the study and received a free museum ticket.

Participants
For the study, we recruited 24 participants (12 male, 12 female), aged between 18 and 66 (mean = 40.92,sd = 15.7) who did not know each other.All participants voluntarily took part and got a free museum ticket as a reward, whether or not they finished the study.

Analysis
Due to three false triggers by the participants, we had to remove 6 of the 192 runs that were carried out.This could be compensated in all cases by using a Linear Mixed Effects model which can handle missing data.We evaluated all non-parametric data of the Likert questions using an Aligned Rank Transformation as proposed by Wobbrock et al. [64].For significant effects, we performed post hoc pairwise comparisons by Elkin et al. [19].

RESULTS
In this section, we examine the results of efficiency, telepresence, and social connectedness questions in the main section, and conclude with subjective rankings.Before the study, we asked about the prior experience of the 24 participants: 20 used telepresence before, 13 had experience with AR, and 12 already tried VR and social online gaming.18 of the participants are satisfied with their social contacts and 17 feel on the same wavelength as the people in their social network.11 participants said that they often visit the museum, while only 4 would like to socialize at the museum.Finally, 16 do not lack company in the museum.

Efficiency
Regarding efficiency, there was a significant effect for the time needed in relation to the interaction type (F 1,155.86= 75.24,p<.001).Thus, the time was shorter when conversation partners used pantomime instead of drawing.We found a significant effect concerning the interaction type (F 1,156.42= 15.15,p<.001) for the successfully guessed runs.Among the 93 runs in groups, only 9 were incorrectly guessed, all in drawing mode.
The interaction type had a statistically significant (F 1,155.31= 22.72, p<.001) influence on the feeling that the interaction technique helped to solve the task, where pantomime was considered more helpful.Additionally, the user role had a significant (F 1,155.35= 12.32, p<.001) influence on the answering of this question where the guesser was rated higher.We found significant effects between the factors interaction type and user role (F 1,155.33 = 4.01, p<.05) with (drawing, describer) performing significantly worse than (drawing, guesser) and (pantomime, guesser) (both p<.01).The difference between (drawing, describer) and (pantomime, guesser) was significant (p<.001).Further the factors place, interaction type, and user role showed a significant interaction effect (F 1,155.24= 5.26, p<.05) for technique.The level combination of drawing and describer had a significant effect compared to pantomime and guesser when both were performed in the quiet environment (p<.05).Also (quiet, drawing, guesser) had a significant difference (p.<0.5) to (loud, drawing, describer) the first being rated better.Additionally, there was a highly significant effect in the participants' responses between (quiet, pantomime, guesser) and (loud, drawing, describer) (p<.001).When looking at the combination of quiet and drawing in relation to technique, participants responded significantly better on guesser (p<.05) in the loud location.For the levels loud and describer, pantomime performs significantly better than drawing (p<.01).Looking at the loud place, users significantly thought that the technique pantomime in guesser mode helped them more than drawing in describer mode (p<.001).

Telepresence
We observed no significant effects regarding place, interaction type, and user role in the question "I felt that the other person and I were in the same place".The overall results show that participants felt telepresence in 45% of the runs with 31% not feeling it.Overall, the median of all responses is 3, which means Neutral.

Social Connectedness
We found significant effects for the two questions around satisfaction with contact (F 1,155.14= 4.86, p<.05) and quality of communication (F 1,155.21= 26.76,p<.001) related to interaction type.For both, pantomime was rated higher than drawing.
If we look at the overall results of the modified social connectedness questions on an individual level from Bel et al. [8], 63% feel that they can communicate well with the other person with 16% saying the opposite resulting in a median of 4 (Agree).During the use of the station, 58% of the users said they got satisfaction from the contact with the other person with 20% not feeling it.This results in a median of 4, meaning Agree.54% of the participants felt on the same wavelength as the other person and 14% did not, leading to a median of 4 (Agree).The question of whether they felt a lack of company with the other person results in a median of 3 (Neutral), with 49% disagreeing and 30% agreeing.All significant results can be seen in Figure 2.

Subjective Rankings
After the study, the users estimated in which previous questions the variation of interaction type and user role applied the most: Regarding the question of where the communication went best, 45.8% answered with (pantomime, guessing).33.3% also felt pantomime was the best but while describing.20.8% felt they could communicate best during (drawing, guessing).When asked, based on the GEQ [31], on which task the participants admired their opposite the most, 37.5% answered in the (drawing, guessing) mode, 29.2% while guessing during pantomime and each 16.7% during (drawing, describing) and (pantomime, describing).In terms of telepresence, 54.2% of participants answered that they felt most in the same room with the other person while describing through pantomime, 29.2% during (pantomime, guessing), and 16.7% in drawing mode as a guesser.50% of the study participants felt connected to the other person during (pantomime, guessing), 33.3% during describing by pantomime, and 16.7% during (drawing, guessing).45.8% of the participants enjoyed being with the other person the most during (pantomime, describing) with 37.5% mostly enjoying it during guessing in pantomime mode, 12.5% during (drawing, guessing), and 4.2% when describing through drawing.33.3% of participants had the most empathy for the other person during (pantomime, guessing), 25% each during (drawing, describing) or (pantomime, describing), and only 16.7% while guessing during drawing.
Finally, participants answered four questions about the station, the interaction, and the other person.When asked if users learned anything from the interaction, 67% agreed with this statement and 25% disagreed resulting in a median of 4, which corresponds to Agree.To the statement "I felt safe during the interaction with the other person."based on the findings of our interviews, 79% responded that this statement was true and only 4.2% that it was not correct leading to a median of 4 (Agree).71% of respondents wanted to get to know the other person better, 4.2% were not interested showing a median of 4 (Agree).The telepresence station would be recommended by 83% of the users and only 4.2% would not recommend it resulting in a median of 5 (Strongly Agree).

Qualitative notes
Participants provided feedback on the station, expressing a desire for improved cursor and drawing controls.Some participants favored drawing as an interaction type once they understood it.A few preferred drawing because it helps them explain things.Challenges included participants attempting detailed drawings, hindered by issues like inaccurate tracking.Many wished for practice time before the study.Observations showed participants using gestures after each run to signal success or failure to each other like thumps up or shoulder shrugging.

DISCUSSION
The telepresence station successfully achieved its goals of providing users with a sense of security during interaction and facilitating social learning through mutual explanation.Although a quantitative assessment of learning success was not possible, participants expressed positive feedback.The majority of participants would recommend the station to others, emphasizing the enjoyable experience and potential for fostering social exchange.The main findings of our study indicate that variations in place, interaction type, and user role significantly influence efficiency and, to some extent, social connectedness, but not the feeling of telepresence.The questions about technique, satisfaction, communication, time, and correctness highlight a clear preference for the pantomime interaction type while the guesser user role received a better rating in terms of technique.

Optimizing Collaborative Problem-Solving
Pantomime outperformed drawing in efficiency, demonstrating faster and more reliable guessing of exhibits during our study.Drawing's slower speed, attributed to waiting time for erasing, led to all incorrectly guessed runs, indicating its lesser effectiveness in collaborative problem-solving.Challenges with tracking, participants' skills, experience, and the learning curve of the new drawing interface may have influenced results.Pantomime is in general more accessible and therefore more frequently used to communicate compared to drawing, which requires tools.The guesser user role was found more helpful in technique, being a passive conversational role but active in decision-making.In some cases the technique led to imbalances among the user pairs, emphasizing the importance of careful placement consideration in future telepresence station design especially in collaborative problem-solving tasks involving different user roles.Hereby it would be interesting to analyze if a better blend between real and virtual space could improve the telepresence and the interaction experience in general.

Tele-Pantomime enhances Social Connectedness
Among the factors analyzed, only the interaction type impacted social connectedness, with participants reporting higher satisfaction and better communication during pantomime compared to drawing.This highlights the positive association between the more commonly used pantomime method and increased satisfaction in social interactions.Less demanding communication methods, like pantomime, provide more room for social connectedness in the museum environment.Additionally, drawing directs less attention to the person and more to the drawing itself, potentially causing a loss of social cues.While efficiency is not the main focus, it could play a role in visitors' social connectedness, suggesting a potential correlation between task completion and effective communication.Regardless, we are convinced that interaction with exhibits and others depends on engagement rather than speed.Throughout the study, communication was generally perceived as good, with poststudy responses reinforcing the effectiveness of pantomime and guessing as the most liked communication method.Some participants favored the combination of drawing and guessing, suggesting possible advantages of drawing in collaborative problem-solving tasks.This highlights the drawing's general potential to contribute to social connectedness, even though executing it this way may be overwhelming for most participants.

Joy and Motivation
In terms of enjoyment and empathy, both describing and guessing roles showed balanced responses, indicating no recognizable influence of the user role on these factors during the telepresence exchange with strangers in the museum context.Visible joy and excitement during the interaction, reflected in concluding questions, may have a motivating impact on visitors' museum experience and learning behavior.A comparison of pre-questionnaire and subjective rankings responses suggests that telepresence tools can foster a feeling of social connectedness among visitors.Initially, the majority did not want to socialize and claimed no lack of companionship in the museum.However, in subjective rankings, most participants expressed a desire to deepen connections, indicating the telepresence station's potential to inspire users to seek social interaction.Nevertheless, these advantages require visitors to interact with the station, implying efforts will be required to encourage engagement when deploying it in the wild.

LIMITATIONS AND FUTURE WORK
This research contributes to the emerging field of social, touchless, and voiceless telepresence stations in museums.While confident that our results offer valuable insights, we acknowledge limitations in both the study design and outcomes.The study, conducted in a real museum under realistic conditions, required local monitoring by supervising personnel, potentially influencing results.Future work is essential to address how to motivate users in museum contexts.Our experiment focused on non-language-based communication channels like drawing and pantomime, excluding voice communication to bridge language barriers and preserve the museum experience for other visitors.Future exploration should include other non-language-based approaches, e.g., pointing, and diverse use cases for comprehensive insights.Additionally, a comparative study between speech-based and non-speech-based approaches is crucial for future system use.While our experiment set the groundwork for simple and playful communication with telepresence in museums, it is only a first step towards a digital museum that uses such systems to increase visitor engagement and encourage them to interact with exhibits in a playful way.Future work should investigate the relationship between telepresence, efficiency, and social connectedness in museums, addressing questions about its influence on learning success and the overall museum experience.Our work serves as a strong foundation for further research in this area.

CONCLUSION
In this paper, we looked at telepresence stations in museums to foster a sense of social connectedness between strangers through collaborative problem-solving tasks and therefore conducted a formative pre-study.In addition in a user study, we varied the place, interaction type, and user role and developed an application for this purpose.In this system, participants could engage with visitors and exhibits from other museums through charades.Our findings suggest that telepresence applications beyond robotic implementations in the museum environment can create a sense of social connectedness among users.The positive user experiences and feedback for the station underline its potential to foster interaction in this setting.However, we need to look more closely at how visitors can be motivated to interact with such stations in the future.We also investigated factors such as telepresence and efficiency using both qualitative and quantitative methods.We recognize the potential of implementing such systems in public spaces and advocate research in this area to refine voiceless and touchless telepresence technologies and uncover additional influencing factors.Our work represents only a first step towards a socially enriched museum experience and, potentially, an enhanced learning encounter in general.

Figure 2 :
Figure 2: Participants answers with significant results and the task completion time.The error bars depict the standard error.