LaSofa: Integrating Fantasy Storytelling in Human-Robot Interaction through an Interactive Sofa Robot

This paper presents LaSofa, an avant-garde interactive sofa that revolutionizes the concept of human-furniture interaction by integrating fantasy storytelling with conventional furniture design. LaSofa is equipped with pressure sensors that recognize user interactions, which in turn trigger character dialogues and adventure narratives set in an embedded fantasy world. These narratives are dynamically generated by advanced large language models (LLMs) and are delivered through state-of-the-art audio technology to create an enveloping auditory experience. This integration not only embodies the convergence of technology and storytelling within the realm of furniture but also marks a pioneering venture into augmenting human experiences through interactive design. The paper elaborates on the genesis, architecture, and prospective influence of LaSofa in the domains of interactive storytelling and innovative furniture design.


INTRODUCTION
People inherently possess a fondness for fantasy worlds, driven by their innate curiosity and desire to explore [4].Fantasy worlds can provide resistance to reality and spiritual regeneration, proving to be particularly valuable in today's high-stress context [14].Sofas, a primary furniture piece for relaxation in everyday life, are often seen in current home design as functional items meant to enhance physical comfort and visual aesthetics [6,9].However, people's daily use of sofas for sleeping, dreaming, watching TV, reading, and chatting transforms them into a space for fantasy and social interaction.Our goal is to integrate the concept of the fantasy world with traditional sofas, thereby altering the experience humans interact with sofas.This approach aims to meet not only physical but also psychological needs, ofering a novel form of human-robot interaction, where sofas are not merely passive objects but active participants in creating immersive, interactive experiences.Addressing this need, our paper introduces LaSofa, a pioneering robotic sofa that seamlessly integrates fantasy storytelling into its functionality.LaSofa is equipped with pressure sensors and is powered by LLMs, enabling it to initiate interactive narratives and dialogues.This integration transforms the user experience, ofering an immersive auditory environment and redefning the sofa from a mere piece of furniture to an interactive storytelling platform.This research contributes to the feld of human-robot interaction by demonstrating how the incorporation of fantasy and storytelling can enhance the functionality and user experience of robotic furniture.

RELATED WORKS
The integration of fantasy elements into interactive technology spans various domains, including narratives, the latest LLMs technology, and robotics.This section provides a concise overview of the relevant literature and technological advancements that underpin the development of LaSofa.

Fantasy Narratives and Interactive Games
A substantial body of research has explored the role of fantasy in novels, comics, movies, and games [5].These studies emphasize how fantasy worlds immerse individuals, sparking curiosity and a desire for exploration.The allure of these worlds lies in their capacity to ofer an escape from reality and to inspire imaginative thought.
Role-playing games (RPGs) are a prime example of how interactive storytelling can engage users.Research [10] notes that RPGs cater to individuals seeking escapism and adventure, ofering an alternative to mundane experiences.These games create intricate worlds where players can explore diferent facets of their personalities and decision-making skills.

Large Language Models in Storytelling and Gaming
Signifcant past research has revealed that Large Language Models (LLMs), a form of deep learning algorithm, possess the potential to create immersive fantasy experiences, like in storytelling[13].
In the text-based fantasy adventure game LIGHT, LLMs are employed to train goal-driven agents.These agents, through a combination of pre-training and reinforcement learning, have demonstrated enhanced behavior and communication within the game environment [1].In the context of Generative Agents, LLMs utilize natural language to comprehensively record an agent's experiences, synthesize these memories into higher-level refections over time, and dynamically retrieve them to plan future behaviors [8].LLMs have also shown impressive performance in communication games like Werewolf[12].

Robotics and Interactive Furniture
While the integration of fantasy in games is well-documented, its application in the feld of robotics, particularly in terms of physical engagement, remains less explored.There are discussions on how robot interactions, involving physical and tactile elements, difer signifcantly from virtual interactions [3].The development of robotic furniture, such as the Lift-Bit sofa, has shown functional advancements but lacks in providing comprehensive interactive experiences [2].Other interactive furniture pieces in the market are exploring various ways to engage users.The "Sparkle Bench" by NunoErin, for example, features LEDs and sensors that change color upon touch, creating an interactive visual experience [7].These designs showcase a trend in furniture towards interactivity and emotional engagement, beyond just functionality.

LASOFA SYSTEM BREAKDOWN
Bridging these gaps, this paper introduces LaSofa, a novel approach to integrating fantasy storytelling with functional furniture.LaSofa is equipped with pressure sensors and is powered by LLMs, enabling it to initiate interactive narratives and dialogues.

Interaction Design
Our interaction consists of two parts: When the "Tail " is picked up and brought close to other agent modules on the sofa, the robot begins to vibrate slightly.This vibration, with a "buzzing" sound, is as if the robot is being awakened.When touching robots of diferent personalities on the backrest, LaSofa provides varied auditory feedback according to the diferent personality identities of the pillow robots.LaSofa integrates RFID 1 technology within its robot modules, strategically concealed in the sofa's "Tail."This sophisticated use of RFID, which enables triggering from a distance, allows users to seamlessly initiate dialogues between the sofa and other agent modules [11].This feature enhances user interaction, making the experience more intuitive and engaging.
When users lie on LaSofa and embrace the "Tail", the sofa back's most pressured module triggers a conversation with the Tail's agent, recounting their adventure stories.If a user's posture presses on multiple modules, LaSofa selects for activation those at the edges of the pressure range, typically under hands and feet in a stretched posture, with up to two modules engaging in separate dialogues with the "Tail".As a result, the agents activated on LaSofa vary with each user's unique lying position.With distinct personalities and professions, these agents are activated at diferent frequencies based on posture-driven interactions, leading to a unique narrative journey for every LaSofa.This design ofers users a range of interactive experiences, forging an immersive adventure.3.2 System Architecture Design 3.2.1 Worldview Revealed.We aim to immerse users in a fantastical, adventure-flled world through our Sofa Robots.In this narrative, the sofa transforms into a continent, with its small robot modules embodying diverse characters, each with unique stories.The Sofa Tail, acting as a protagonist, is central to most narratives, engaging with other characters that have rich social dynamics, including friendships, family ties, and feuds.This interactive storytelling is shaped by user interactions, leading to varied storylines.For instance, when a user interacts with diferent characters like Jack the Soldier 2 and Amy the Farmer 3 , the Large Language Model crafts unique stories, such as their joint quest at an Enchanted Forest.Users can explore diferent plot lines by maintaining positions or shifting interactions, like engaging with Joe the Merchant, ensuring a personalized and immersive experience with each session.

System Design.
We propose an interactive storytelling and dialogue generation system, consisting of Planning, Dialogue, and Plot Progression modules. 2 The main character, whose name, occupation, and personality are generated by GPT-4 3 Agent C, whose name, occupation, and personality are also generated by  The Planning module includes Settings, Characters, and Outline.Settings detail the fantasy world's layout, created using GPT-4 to generate weather, terrain, beliefs, and history.Key locations such as squares, bars, and homes are established as default dialogue orientations.The Characters section uses GPT-4 to develop 8 virtual characters, specifying their names, professions, and personalities, which shape their dialogue topics and tones.The Outline section employs GPT-4 to formulate concise, coded story outlines for the whole fantasy world and each character, laying the framework for dialogue trends.Upon user interaction, LaSofa activates when a user sits and picks up LaSofa's tail, signaled by an RFID sensor module, with the system picks up the value on each pressure sensor and decide which character should be activated.The Planning module is continuously reused to generate dialogue prompts in the Dialogue module.
The Dialogue module consists of Input, Dialogue, and Outline.When a user embraces LaSofa's tail and leans against a specifc module on the sofa's backrest, it activates dialogues between the represented agent and the protagonist, Jack.For instance, Amy could be triggered when the user lies in the middle.The Dialogue module then utilizes the Planning module for prompt input, generating dialogues that align with the story context, as well as the characters' personalities and tones, which are then played for the user.After the dialogue, GPT-4 formulates a brief outline of the key points, integrating them into the Planning module's Outline section for future reference and re-prompting.If the user shifts their position, activating a diferent agent, the module dynamically produces their dialogue.The Outline Section ensures coherence in each interaction, enhancing the narrative arc of each character within the broader story.
The fnal component, the Plot Progression module, activates when users trigger the same agent more than three times.This module prompts GPT-4 to advance plots and write the ending of the current events in the dialogue.The purpose of this module is to avoid repetitive content and user fatigue when the same agent is activated in short succession.
Our robot presents stories in a conversational format.Therefore, compared to similar storytelling algorithms[13], our research places a stronger emphasis on the quality of the generated dialogue.Our system framework provides a more optimized solution.In terms of efciency, it ensures both the continuity of content in each interaction and minimizes the need for token calls.Regarding the entertainment value of the stories, we delve deeper into character personalities and backgrounds to better portray each character's unique traits.Additionally, our Plot Progression module is a crucial design element aimed at enhancing the user experience.It helps prevent content repetition and ensures a fresh experience for users while listening to the story.
Through these three modules, along with the sofa's sensors and auditory system, the system crafts an immersive experience.This experience parallels a Lilliput-like world within the confnes of the sofa, thereby introducing an innovative and captivating interaction form of human-sofa interaction.

Characters
Portrait of Jack and others

Outline
Outline the main plot points.

Lasofa turns off
Progress the storyline discussed in the current conversation and write the results of the current storyline.

Input
Plan module of Jack and agent C, relevant context

Dialogue
Speaker plays the generated dialogue.

Outline
Outline the main plot points of the dialogues.

Input
Plan module of Jack and agent E, relevant context

Dialogue
Speaker plays the generated dialogue.

Outline
Outline the main plot points of the dialogues.

FIELD STUDY
After developing the LaSofa prototype, we executed a pilot study and recruited fve participants to interact with LaSofa.Their reclining postures were recorded, and their experiences were subsequently interviewed.Observations on reclining postures indicated a natural preference among participants to embrace the "Tail", signifying this as a comfortable and accepted posture.Regarding module activation, the central agents-Agent E and Agent F-were most frequently activated in single-module scenarios, while the peripheral agents-Agent A, Agent C, Agent F, and Agent G-had higher activation in multimodule scenarios.These activation patterns provide insights for future enhancement of agent interrelationships.
Regarding user experience, the majority expressed fascination with the sofa's human-like conversation abilities.A minority desired ambient sounds alongside human voices.In response, we plan to integrate specifc sound efects into dialogues to enrich the immersive experience.Furthermore, we would like to introduce directional sound feld technology to create a spatial dimension in the audio experience, where the volume of voices changes with perceived distance, thus enhancing the sofa's ability to create a more spatially dynamic conversational environment.Looking ahead, we aim to expand LaSofa's human-machine interaction capabilities.Our focus will be on integrating multimodal interaction features, such as visual elements and gesture recognition, to enhance the immersive experience.Inspired by Interactive Drama Games, we also plan to introduce choice-based narrative elements, allowing users to infuence the story through an interactive tablet interface, further personalizing their experience.
We recognize the importance of understanding user experiences with LaSofa.Future studies will delve deeper into what aspects users fnd most appealing, whether it's the conversational abilities or other unique features of LaSofa.These insights will guide our ongoing development and refnement of the technology.
Additionally, we plan to integrate LaSofa into smart homes, enhancing its role as both an interactive storytelling device and an immersive audio platform, utilizing innovative sound feld localization technology for a richer auditory experience.

Figure 2 :
Figure 2: Trigger mode When you lie down holding the pillow in the position shown in the left image, it simultaneously triggers "Tail" and agents C and F. The interaction among these three creates a story.If you change your posture to the one shown in the right image, it simultaneously triggers "Tail" and agents A and F, resulting in a diferent story through their mutual interaction.