Pseudo-haptics Interfaces for Robotic Teleoperation

When remotely operating robotic systems, the situation awareness and the absence of the possibility of direct human touch in such a remote environment constitute major challenges in telerobotics. Haptic feedback has been playing an important role when people interact with remote environments (e.g., in robotic teleoperation) or provide more immersive experiences in virtual environments. Like haptic devices, pseudo-haptic techniques aim to simulate haptic sensations in human-computer interaction between real and remote or virtual worlds, by exploring multimodal feedback, mainly the visual, and the brain's capabilities and limitations, without needing a haptic device to be attached or applied to the body. The authors discuss the possibility of exploring pseudo-haptic techniques, notably combined multimodal techniques, to improve robotic teleoperation, in remote vehicle driving, object maneuvering, situation awareness, and collaborative tasks, which as per the best authors' knowledge has not been explored in the literature.

haptic devices, pseudo-haptic techniques aim to simulate haptic sensations in human-computer interaction between real and remote or virtual worlds, by exploring multimodal feedback, mainly the visual, and the brain's capabilities and limitations, without needing a haptic device to be attached or applied to the body.The authors discuss the possibility of exploring pseudo-haptic techniques, notably combined multimodal techniques, to improve robotic teleoperation, in remote vehicle driving, object maneuvering, situation awareness, and collaborative tasks, which as per the best authors' knowledge has not been explored in the literature.

INTRODUCTION
Robotic Teleoperation involves remotely driving and manipulating robotic systems, which is particularly applicable in hostile environments.However, Situation Awareness (SA) constitutes a major challenge in telerobotics [9].Humans perceive haptic sensations when manipulating a physical object.Haptic feedback has been playing an important role when people interact with remote environments (e.g., in robotic teleoperation) or provide more immersive experiences in virtual environments, where humans do not have the possibility to locally touch.Pseudo-haptics can be defned as a haptic sensation illusion perceived by playing with multimodal feedback, mainly visual, and exploring the brain's capabilities and limitations [11].By introducing subtle nuances of pseudo-haptic techniques (PHT), mapped to a user's action, allows the simulation of virtual tactile and kinesthetic sensations, induced without needing a haptic device to be attached or applied to the body.Those are perceived through multimodal simulated sensory efects, for example by visual and auditory efects, or embodied metaphors.In recent years, there was a signifcant increase in the number of pseudo-haptics literature published research work, simulating a wider variety of techniques and new application areas, mainly focused on extended reality and mid-air interactions [16].The authors consider further exploring those PHT, notably combined multimodal techniques, to improve robotic teleoperation, in remote vehicle driving, object manoeuvering, SA, and collaborative tasks.To the best authors' knowledge, the use of PHT for robotic teleoperation has been largely unexplored, with the exception [13] to present the compliance property and assist in surgery remote operation tasks.
This paper presents a defnition of the problem that PHT intends to address in robotic teleoperation.Alongside with a historical perspective, outlining how it has evolved over time and what previous attempts have been made to address those challenges.The next section describes the authors' proposed PHT solution approach, to address the identifed issues and the literature evidence sustaining those.A discussion section addresses the considerations to be taken and the reasons for the proposed approach.Followed by the conclusions, reinforcing the proposed approach position taken and identifying further future action.

ROBOTIC TELEOPERATION CHALLENGES
Situation awareness and the absence of touch are major challenges in robotic teleoperation [9].In this respect, haptic interfaces extend the traditional display-based human-computer interfaces by exploiting the sense of touch.Research into haptic devices aims to enable such haptic perception, mainly when people interact with virtual or remote environments, where the absence of haptic feedback and naturalness are signifcant challenges for a better immersive experience.In 2000 the term "pseudo-haptic feedback" was introduced, referred to as a haptic illusion meant to simulate a haptic sensation, combining the forces applied by the user to engage the virtual environment, with the incongruent visual information feedback from the simulation [7].Like haptic devices, PHT aim to simulate haptic sensations in Human-computer Interaction (HCI) between real and virtual worlds.For example, the manipulation of visual gain, when varying the mapping ratio of virtual displacement to the real-world movement [7].This Control/Display ratio distortion will afect the user's perception of an object's speed and body movements, e.g., walking [5], hand movements [6]).Or when on a personal computer, one drags a window between two monitors and has the resistance perception (haptic sensation) at the edge between the two monitors, and sticky widgets at the edge of a monitor.One key advantage of pseudo-haptics compared to haptics devices is their device-free nature, tending to make interfaces more fexible and portable by allowing the virtual properties of the objects, surfaces, or avatars to be dynamically modifed.PHTs ofer cost advantages with better mobility, portability, reduced size and weight, lower power consumption, and fewer hardware and maintenance constraints [11].This way, PHT allows addressing some of the haptic device constraints (limited degrees of freedom-DoF, not limited to the device and environment physics limitation) and brings more fexibility.

PHT FOR ROBOTIC TELEOPERATION
The PHT body-ownership and kinesthetic illusion is a research area foreseen to be further pursued in Extended Reality (XR) applications [16].Despite the associated challenges, the focus would be on better understanding multisensory integration in virtual and remote environments, by investigating vision and touch combinations, and exploring haptic and visual cues synchronization.Those techniques seem to signifcantly increase the experiences participant's belief, and enhance the sense of ownership in XR [6].In XR afordance tasks design, the PHT would be applied in multimodal methodologies, and to the input variables extracted from the natural tracking of motor activities.Furthermore, the HCI design could evolve to XR afordance tasks in the context of the Ecological Interface Design (EID) [14], allowing the user subconsciously to break down the information patterns of the new situations into familiar afordance perceptions.The goal of the PHT applied for Robotic Teleoperation would be to have an afordance perception of the work domain through the interface analogous to human capabilities, perceptual mechanisms, and actions, facilitating human problem-solving situations and reducing the cognitive load.Similarly to haptics devices, that arose as Virtual Reality (VR) applications and afterward were leveraged into robotic teleoperation, the authors' main contribution of this paper is the conceptual proposal, which consists of leveraging state-of-the-art PHT applied in XR applications, into robotic teleoperation of Remotely Operated Vehicles (ROV) and SA use cases.

Robotic teleoperation domains
To the best of authors' knowledge, the PHT applied to Space, underwater, ground robots, and aerial drone teleoperation, and SA has not yet been explored in literature.The PHT from the literature could be considered to improve the perception of remote vehicle driving, or remote object features and properties when operating a remote robot arm.Those could also be used in addition to traditional force-feedback haptic technologies used in telerobotics.As illustrated in Figure 1, pseudo-haptics can be introduced either as PHT stand-alone, or combined in conjunction with haptic devices, to enhance the user experience further and extend to other simultaneous sense perception simulations.For example, the authors consider the usage of pseudo-haptic multimodal interfaces to real-world underwater ROV teleoperation and to improve underwater SA.In this particular case, as described in PHT literature, where slow movements can contribute to an even better perception of virtual force created by PHT [10], [8].Since by physics nature, in underwater environments the movement tends to be slower than inland, one could consider that this could contribute to a better PHT efects perception in such environments.For example, by having PHT applied to the robotic teleoperation of ROV, which typically moves smoothly in aquatic conditions, this way would require only subtle discrepancy illusions to simulate motions properly.

Levaring Gaming PHT for Telerobotics
Besides the signifcant XR PHT techniques applications present in literature, one could also consider it interesting to extrapolate gaming industry techniques to PHT applied to teleoperation and SA.The video games commercially available have been taking advantage of visual illusions and tricks that make one think to see something that is not really like that.The Sony Echochrome (2008), started the idea of integrating visual illusions in the gameplay where a character is traversing a world where physics and reality depend on the observed visual perspective.Similarly, the Monument Valley (2014) and the Superliminal (2019) frst-person puzzle games are based on forced perspective and visual illusions.In another example, one of the game design concepts the "reveal" technique [15], which explicitly makes players realize that the game's optical visual illusion perception difers from reality.This can be done by providing a diferent kind of indication text or voice instructions options, by adding a visual indicator, changing perspective, or even removing the illusion interference.It could be interesting to investigate how to apply such gaming techniques in teleoperation to improve SA, helping to make teleoperators aware that remote reality conditions may have changed and become diferent from their perception, by adding such gaming techniques and widgets.Apart from visual illusions, user interface control/display input ratio manipulation has been also used in the game industry.For example, a vehicle in the virtual environment can become more challenging to control when driving over slippery areas (e.g., oil on the road), when the vehicle slows down if the vehicle becomes heavier during the game action, or when driving of-road sandy surface.The control/display ratio distortion is one of the most PHT techniques described in the literature and could be leveraged to assist operators in robotic teleoperation.As described, those XR and Gaming human-natural HCI design approaches could be further explored for robotic teleoperation, where such PHT more natural afordance can potentially be used to improve teleoperation tasks and SA, and also expected to reduce the cognitive load factors, such as fatigue, overload, attention, engagement, and expectancy, critical in long time duration robotic teleoperation navigation activities.

CONSIDERATIONS
Similar to the usage of haptics devices feedback in teleoperation, also the usage of PHT has challenges associated with such techniques that need to be taken into consideration and evaluated, in particular in real scenarios of safety-critical mission tasks.Nevertheless as previously described, in literature one can fnd PHT applied already to safety-critical tasks such as remote operation surgery [13].The usage of PHT has challenges for example associated with the need for the user's attention to be focused on the visual media display.For obvious reason, if that is not the case the perception of the efect will not be perceived by the operator.Nevertheless, if one is performing a mission safety-critical task, one could also expect that by default the operator is paying full attention, and/ or would be alerted.The possibility of visual displacement mismatch, or the simultaneous simulation, represents other associated challenges.In literature are described diferent workarounds and methods to address such constraints and limitations associated with pseudo-haptic methods [16], like by introducing a virtual visual cue add-on [2], or assuming the displacement as part of the design process [1], [12], among others.Also when considering continuous long-duration critical activities such as teleoperation, drone fight piloting, underwater ROVs, or space-critical maneuvering tasks.Is important to also consider that besides the efectiveness of the PHT for a certain teleoperation task, it seems to make worthwhile a consideration of biometric methods to evaluate the techniques' impact on cognitive load.The techniques described in [4], to measure the cognitive load by electroencephalogram (EEG), or by other devices such as functional near-infrared spectroscopy (fNIRS) in [3], have the advantage of data being collected at the exact moment that teleoperation experiments are being performed, not post-facto, as occurs in the case of the participants flling in post-experiment questionnaires.The use of such evaluation PHT methods and the comparison to other haptic device-based approaches would be important to perform and have an assessment of such techniques in real-world case situations in terms of efectiveness and cognitive load.awareness challenges.As far as the authors could identify from the literature, the PHT described in the literature in VR/ XR immersive applications, or the efects of the technique used in the gaming industry, have not yet been applied and explored in robotic teleoperation and SA until this moment [16].It describes concrete applications examples for robot teleoperation and furthermore how the PHT applied in HCI design could evolve in the context of the EID, allowing the user subconsciously to break down the information patterns of the new situations into familiar afordance tasks perceptions.The goal is to have PHT leveraging the task afordance perception of the work domain through the interface analogous to human capabilities, perceptual mechanisms, and actions, facilitating human problem-solving situations and reducing the cognitive load, particularly relevant when considering continuous long-duration critical activities such as robotic teleoperation.