vARitouch: Back of the Finger Device for Adding Variable Compliance to Rigid Objects

We present vARitouch, a back-of-the-finger wearable that can modify the perceived tactile material properties of the uninstrumented world around us: vARitouch can modulate the perceived softness of a rigid object through a vibrotactile compliance illusion. As vARitouch does not cover the fingertip, all-natural tactile properties are preserved. We provide three contributions: (1) We demonstrate the feasibility of the concept through a psychophysics study, showing that virtual compliance can be continuously modulated, and perceived softness can be increased by approximately 30 Shore A levels. (2) A qualitative study indicates the desirability of such a device, showing that a back-of-the-finger haptic device has many attractive qualities. (3) To implement vARitouch, we identify a novel way to measure pressure from the back of the finger by repurposing a pulse oximetry sensor. Based on these contributions, we present the finalized vARitouch system, accompanied by a series of application scenarios.


INTRODUCTION
Computers can influence, alter, or enhance our sensory perception.While visual augmented reality (AR) allows us to freely modify the appearance of the world around us, headphones and in-ear speakers allow us to modulate the sounds of the surrounding environment, our sense of touch remains largely unaltered; it appears that the mastery of perceptual augmentation ends there.
However, there is no principled reason why this should be the case.After all, there is an abundance of methods for creating various tactile material experiences [31,36,58].Most of these require instrumenting the object one interacts with by adding sensing and actuation mechanisms, thus limiting the number of objects that can be augmented.This limitation can be overcome by applying an actuation and sensing method to the fingertip, which enables presenting material experiences directly on the fingertip [38,61].However, whereas such devices can provide users with a virtual world of touch, they impede the fingertip, preventing it from interacting with the surrounding physical world.
Here, we present steps towards a tactile augmented reality.We focus on providing an experience of compliance to users as they interact with rigid everyday objects.Compliance is the ability of a material or mechanism to change its shape as pressure is applied to it.The ability to shape compliance provides rich opportunities to design virtual materials, as one might reduce rigidity of a solid table or add softness to a cushion.We build upon prior work in vibrotactile compliance rendering [30,36,37,56], systems which change experienced compliance by means of providing tactile impulses as pressure is exerted by the user.While these systems offer a robust compliance experience, even when interacting with rigid objects, they have limited suitability for universal, mobile AR applications, as, by their nature, they require a sensing device and an actuation device integrated into the object the user is interacting with.This paper builds on this work, providing the first steps towards a tactile Augmented Reality.We explore whether it is feasible to transform such an illusion into a tactile augmented reality device and whether there is utility in doing so, and then we demonstrate how to technically achieve this.
We address the aims of this work sequentially.(1) We explore whether compliance illusions can be provided by stimulating the back of the finger, how the experienced compliance varies with the number of tactile grains provided, and what the absolute reduction in hardness is.These questions are explored with a magnitude estimation and a staircase task, respectively.(2) We explore whether the device has real-world utility and how naive users react to the haptic experience.To this end, we asked users in a qualitative study to test a vARitouch prototype in a real-world application scenario.It became clear that to create a fully integrated wearable device, a back-of-the-finger pressure sensing solution is required.(3) We identify that a pulse oximetry sensor can be used for this purpose.We show that the perceptual principle is sound, that the device will have real-world utility, and that it can be technically implemented.(4) We present vARitouch, a back-of-the-finger wearable device that can alter the perceived compliance of objects and surfaces using pulses/vibrations applied to the fingernail.

RELATED WORK
From early exploratory actions performed by toddlers to the skilled and practiced actions of a grown-up, for example, playing an instrument, handling fragile glass, or assessing the ripeness of a piece of fruit, we experience the world through our fingers.To preserve this, real-world haptic attributes should not be occluded but augmented.In other words, we aim to enhance our tactile perception while still allowing us to receive information that real-world objects and surfaces offer.
Building upon this notion, we initially discuss relevant works on tactile perception and provide examples of devices designed to modify and enhance users' perception through vibrotactile feedback.Subsequently, we discuss finger augmentation devices for Mixed Reality (MR) scenarios.Lastly, we examine fingertip force sensing techniques with the potential for creating perceptually transparent systems, enhancing users' tactile perception while retaining access to haptic information from the physical world.

Vibrotactile Rendering of Material Experiences
There is a large body of work that demonstrates the use of vibrotactile rendering to augment the experience of interacting with objects.Generally speaking, coupling vibrotactile feedback with user movement has been shown to elicit a broad range of material experiences [31,58].
It is well understood that our experience of texture is mediated through vibration [10].This can be used to create virtual material experiences; for example, Romano and Kuchenbecker captured the vibration of a probe as it was moved over a texture, and these signals were used to generate an experience of texture for users moving a pen over a non-textured surface [57].Strohmeier et al. demonstrated that texture experiences can also be created by simple models, for example, by providing pulses at fixed intervals relative to user movement [65].
If such vibrotactile textures are provided relative to user pressure instead of movement, the vibration is experienced as recedes or complying of a material as pressure is applied -a sensation roughly related to softness.Kildal reported that providing vibrotactile pulses to users as they press into a rigid object made them feel the object to be more compliant [36].Later work by Kidal embedded this principle into Kooboh, a tangible user interface (TUI) able to provide varying experiences of compliance or deformation as users press the device [37].This work was followed by a broad range of explorations of virtual compliance, including the development of more effective haptic feedback of a virtual button using vibrotactile feedback along the entire length of a force-displacement curve [40] by Kim et al.Additionally, Heo's work proposed a method for compliance feedback that extends from normal force input to the case of a two-dimensional tangential force input [30].
This principle can be extended to other input modalities; for example, Strohmeier et al. presented a flexible device that couples the frequency at which pulses occurred to the extent a device is bent, resulting in an experience of changing material composition [63].Heo et al. showed that this approach was also able to create a broad range of experiences such as stretching, bending, and twisting for a rigid object [31].
Such augmentations have also been explored for wearables.Wittchen et al. and Strohmeier et al. both demonstrated the use of vibrotactile feedback coupled with user movement for modifying the experience of walking over soft surfaces [64,80].In the work presented in this paper, we go a step further by directly augmenting the body.Like previous work, vARitouch is a device that provides a virtual sense of compliance.Extending the work by Wittchen et al. and Strohmeier et al., vARitouch does not augment a proxy object such as a shoe or pen; instead, it directly augments the body.

Finger Augmentation Devices
Researchers have historically focused on grounded haptic interfaces, such as the Sigma or Phantom [45] devices, and glove-type haptic displays, such as the CyberGrasp [48] or the Rutgers Master [12], to create a more realistic feeling of touching virtual and remote environments [52,82].These devices provide compelling force sensations but are complex and expensive and usually not mobile.Using wearable devices instead enables haptic augmentation to better integrate with users' day-to-day activities.Such devices range from gloves for sensing to exoskeleton glove-based actuating systems [66].These systems rendered haptic feedback on the hand and vibrotactile feedback on the fingertips [52,73].Finger augmentation devices have become the norm to create and render vibrotactile devices as they are light, efficient, and leave the palm free for exploration.The finger-based devices are used for displaying normal indentation, skin stretch and rendering vibrations.For example, Prattichizzo et al. presented a wearable 3-DoF fingertip device for interaction with virtual and remote environments to provide indentation cues [55].Using an asymmetrical vibration, Choi et al. developed 'Grabity, ' and Culbertson et al. developed 'WAVES, ' both finger-worn devices which simulate weight and grasping in VR and present three-dimensional translation and rotation cues, respectively [17,19].However, many of the finger augmentation systems occlude the finger pad, thus obscuring or restricting the natural interactions the user has [39,61].
Such devices have a certain kinship with Virtual Reality goggles.Similarly to VR goggles, a new stimulus is presented to the user, while the real world is hidden from them.In the work discussed in this section, the finger pad is occluded, thus occluding tactile experiences of the surrounding natural world.However, vARitouch keeps the fingertip free for exploration by integrating sensing and actuating the back of the finger.

Perceptually Transparent Haptic Systems
While Virtual Reality (VR) goggles completely isolate users from their environment, Augmented Reality (AR) glasses take a different approach-they are designed to be perceptually transparent.That is, they can provide access to virtual information without impeding the perception of the physical world around us [47].
Similarly, finger augmentation devices can also be designed with this perceptually transparent concept in mind.Creating such systems is challenging, resulting in various hybrid prototypes that display aspects of transparency or partial transparency.One example of a partially transparent device is Touch&Fold, a fingernailmounted device, developed by Teng et al., that can be easily folded away when not in use, ensuring it does not interfere with natural interactions [70].Another system was published by Tran et al. [71], which augments on-body touch input with an actuator on the back of the fingernail [71].However, their system requires a sensor on the location that the user intends to touch, limiting the interaction to that location.
Other approaches include, for instance, Maeda's work, which utilizes side-of-finger rollers, actuated to generate skin vibrations that simulate a button press or interaction with a bumpy object [43].
McIntosh developed a wearable system for around-the-device interactions that tracks the finger using an array of magnetometers and provides haptic feedback through a magnet attached to the fingernail.This technology finds applications in 3D midair interactions [46].Ando proposed a nail-mounted device that delivers bump mapping information through haptic stimuli based on barcode patterns attached to the objects or surfaces to be augmented [5][6][7].Preechayasomboon contributed to this area with Haplets, wireless finger-worn devices that offer vibrotactile feedback using Linear Resonant Actuators (LRAs), particularly useful in hand-tracking virtual reality applications [56].Withana et al. demonstrated a tactile actuation system that provides tactile stimuli without interfering with natural perception.This system employs a thin-film electronic tattoo to deliver electrical stimulation to tactile receptors in the skin [78].Also, Tanaka et al. explored electro-tactile feedback, which allows tactile feedback without obstructing the palm of the hand [68].
An approach we find particularly elegant was proposed by Tao et al.By applying a mechanical frame around the fingerpad, Tao et al. were able to create the illusion of softness when touching objects [69].Their work acts as an inspiration for the current project.By using the well-understood grain-based compliance illusion, vARitouch not only allows users to experience compliance but enables programmatically manipulating perceived compliance so that, when touching the same surface, users can be made to experience different degrees of softness.It should also be highlighted that all of the above systems either do not require sensing, as they do not react to the environment, or they assume an external sensing device.vARitouch, on the other hand, is self-contained, integrating all elements required for the illusion to work in a single device (Figure 2).While there is a broad range of perceptually transparent haptic systems, these typically do not have integrated sensing, which might allow the device to react dynamically to its material surroundings.vARitouch senses the force users apply to objects without requiring external sensing technology and can react to the dynamics of the user's touch behavior in real time.This sets it apart from the other work highlighted in this section.

Fingertip Force Sensing
As our objective is to establish a perceptually transparent system, we require a sensing method that does not obstruct the user's genuine perception of the real world.A promising approach for achieving this is optical sensors.For example, Mascaro demonstrated that color changes of the fingernail correlate with applied pressure and that a paired light emitter and receiver located on the fingernail could provide accurate pressure information based on this color change [1,44].This principle also works with remote cameras.For example, Hwang tracks the color changes in the fingernail area with a mobile camera to use fingertip force as an input modality [33].
Another approach to measuring contact force is by means of skin deformation.Saito attached a device around the finger pad that uses photo-reflexive sensors to measure the contact force of the finger pad by measuring the finger deformation when it touches a surface [60].Furthermore, fingernail deformation is a viable metric for evaluating finger-force touch interactions.Hsiu developed Nail+, which uses a strain sensor array mounted onto the fingernail to facilitate force sensing on rigid surfaces [32].
Finger position can also be used to infer pressure.Dobinson developed a device to detect thumb-on-skin pressure using data from IMUs placed on each finger phalanx and an LSTM RNN network [22].This supports, within the constraints of the machine learning model, completely unoccluded sensing of fingertip contact and pressure.
vARitouch uses an optical approach, as it is best suited for integration on a back-of-the-finger device that is non-permanently attached and needs to work in a broad range of contexts.We show how an off-the-shelf pulse oximetry sensor is well suited for this purpose.
In summary, vARitouch is, to our knowledge, the first device that combines grain-based vibrotactile material illusions with a perceptually transparent finger augmentation device.As vARitouch integrates sensing, it can respond to the dynamics of the user's touch.vARitouch is also capable of modulating the strength of the compliance illusion it provides to suit the application at hand.

DESIGN RATIONALE
The overarching goal of this work is a step towards augmented tactile reality, that is, the ability to augment our tactile perception without inhibiting our physical access to the world around us.This can be thought of as analogous to our ability to provide visual information in AR glasses while still enabling the user to see their surroundings.To achieve this first step, we follow several design principles which directly and indirectly affect our design choices: Be transparent and self-contained.An AR technology must have the ability to vanish when not in use.In our context, this means that our sensory organ, the fingertip, must not be occluded.To be able to work anywhere, the technology must be self-contained.We want to create a design that requires no instrumentation of the external world.
Start small.Our focus is on a specific augmentation: reducing the perceived rigidity of touched objects.While this is somewhat arbitrary, using similar methods, we could have instead aimed to increase friction [62] or weight [17].However, we see practical value in this choice: For example, within interaction design, compliance of objects plays a significant role in shaping the user experience.This, for example, is evident in the detailed design of switches and buttons [2,77].Adding such compliance experiences to rigid objects would add a strong sense of interactivity.In rapid prototyping, it is often easiest to produce rigid prototypes with cheap 3D printers that can produce rigid objects in all forms and sizes using plastic filament, while printing soft materials is often challenging or expensive.Augmenting the compliance of rigid objects allows cheaply printing a shape and then fine-tuning compliance post-hoc.
Enable Embodied Experiences.When a technology presents information to a user, this information can be mediated in an embodied or in a hermeneutic, symbolic manner [72].For example, a visual display can be used to indicate temperature symbolically, with a number indicating the temperature in Fahrenheit or Celsius.A thermal display [54] might do so in an embodied manner by literally allowing the user to touch the target temperature.Similarly, current augmented reality technology might indicate softness symbolically by adding a visual overlay representing softness.However, this would require the user to continuously and actively interpret the overlay, which we do not believe would be a satisfying experience.Instead, we aim to create an embodied experience of softness 1 .
Preserve Sensorimotor Contingencies.We commonly assume that what makes our senses distinct is the physical phenomena they provide access to.This is true for vision and hearing, which enable us to perceive a slice of the electromagnetic spectrum (light) and pressure fluctuations (sound), respectively.As regards touch interactions, our tactile sense no longer maps as neatly to a single physical phenomenon.Our sense of touch responds to phenomena such as thermal conductivity, friction, shape, rigidity, texture, and even the behavior of our own body.O'Regan and Noë suggest that it is not the physical phenomena that distinguish sensory modalities but how information received by them is linked to motor actions [51].For example, when we move our eyes to the right, this always corresponds to a change in the neural signaling of our visual field moving to the left.O'Regan and Noë suggest that such learned structures, sensorimotor contingencies, form the basis from which we make sense of the world around us.An important sensorimotor contingency of tactile perception is that tactile cues are always coupled with user actions.We speculate that preserving this contingency is a basic requirement of tactile augmented reality.

Operating Principles of Compliance Illusion
This work builds upon the grain-based compliance illusion as introduced by Kildal [36].The grain-based compliance illusion can be used to make rigid objects feel compliant.This effect is achieved by measuring the applied force while users apply pressure and playing short vibration pulses (grains) when crossing predefined force values (Figure 3).This results in pulse frequencies that always correlate with the change in pressure.If the user keeps the pressure steady, they will feel no pulses.If the user changes the pressure, they feel rapid pulses.We refer to signals with this type of synchronization as coupled to human action.An important aspect of this illusion is that when the coupling is tight enough, the vibrotactile signal is no longer experienced as vibration but solely as an effect of the complying material. 1Interestingly, hermeneutic symbols might still be constructed from these embodied experiences [

Steps in Addressing our Design Goal
For this initial step towards tactile augmented reality, there are a number of open questions.In this paper, we address these questions one at a time and then, based on the insights collected, build our final prototype, vARitouch.Specifically, we address the following three questions: Q1: Does the compliance illusion work if the vibration is applied to the back of the finger?All previous work has vibrated the object that is being touched.It is unclear if vibrating the finger has the same effect.Additionally, when addressing this question, we want to establish the magnitude of the experience, that is, how much compliance is experienced.Here, we are interested in two questions.First, how does the number of grains influence the perceived magnitude of compliance, and how does this compare to the perceived changes in Shore A levels?This will help us design vibrotactile experiences.Second, to understand the strength of the experience, we also explore how much we can reduce the perceived hardness of a material in absolute terms.
Q2: Assuming the back-of-the-finger compliance illusion works, what benefits would users have from such a device?If we find that we can create such compliance experiences, it is relevant to observe if such experiences can be understood by nonexpert users.Should we find significant effects of our system, these effects might not have realworld relevance to users.To ensure that the system we envision is not only of theoretical interest, we conduct a qualitative study to also explore how nonexpert users would experience the system when engaged in a real-world task.Here, our focus is on both the efficacy of the device and the qualities that users experience when interacting in an environment that mixes physical and virtual tactile stimuli.
Q3: If such a device is, in fact desirable, how can we create a back-ofthe-finger sensor that can be fully integrated into a back-of-the-finger device?In practical terms, the biggest hurdle in implementing vARitouch is creating a sensing system that does not occlude the fingertip.None of the approaches found in the literature proved suitable for our needs.After much experimentation and many failed prototypes, the outcome of this final research question is a successful sensing method for vARitouch.
Having addressed these open questions, we are ready to present the design of vARitouch, together with hypothetical applications in which we believe vARitouch or similar systems will be useful in the future.

Q1: PSYCHOPHYSICAL EXPLORATION
We conducted two studies to identify whether the compliance illusion works when the stimulation is provided from the back of the finger and to assess the strength of the compliance experience.The first is a magnitude estimation task for finding how changes in parameters influence the relative change in the perceived magnitude of compliance.The second is a Staircase Procedure to identify the absolute change in compliance users' experience and to support the comparison of vARitouch with the work by Teo et al. [69].The data of these two experiments is available in an OSF repository2 .

Study #1: Magnitude Estimation
In this study, we examine how the perception of compliance changes with a change in the stimulus.Based on previous work [64], we expect that increasing the granularity of the vibrotactile rendering, delivered directly to the back-of-the-finger, will increase the perceived compliance.Assuming we find such an effect, we can also safely assume that the illusion is robust in principle, even if delivered to the back of the finger.
Participants were asked to evaluate the compliance of various objects with different hardness by physically interacting with them -squeezing and touching and then assigning a numerical rating based on their perception.While the participants explored an object, we provided the grain-based vibrotactile rendering to elicit the compliance illusion.Each participant went through three sets of 18 objects, resulting in 54 estimates per participant.Physical Samples: The selected samples had Shore A levels of 40, 50, 60, and 90.We 3D printed cuboids (6x6x7 mm) using a polyjet 3D printer (Stratasys J826).The Polyjet was chosen for its ability to print rubberlike materials (100% infill) with varying hardness.We initially printed nine Shore A levels on a linear scale for objects under testing, i.e., 10,20,30,40,50,60,70,80, and 90, and confirmed their compliance with an HBA 100-0 Shoremeter by Sauter.To keep the number of conditions of the factorial design manageable, we selected four Shore A levels for the experiment that we felt had roughly equidistant compliance.
Vibrotactile Stimuli: The number of grains selected for this experiment were 0 (i.e., no augmentation), 10, 20, and 40 grains.The vibrotactile stimuli were grain-based compliance illusions as introduced by Kildal [36] and implemented using Haptic Servos [58]: The pressure range was divided into discrete sections (bins), where a grain was placed on each crossing of a bin.The number of bins over the entire range defined the granularity of the stimulus.A single grain was a short vibrotactile pulse characterized by vibration properties such as frequency, amplitude (intensity), and duration.Each grain was a 5 ms long 200 Hz sinewave.
4.1.3Setup and apparatus.Participants were seated in a chair in front of a desk with a display and keyboard.Their dominant hand was positioned behind a black curtain, which ensured that no visual cues were available while exploring the objects.Participants were asked to rate each combination of physical sample and tactile feedback by entering a number in a GUI.
The LRA was attached with medical-grade double-sided tape between the nail and skin of the index finger on the participant's dominant hand.To provide motion-couple vibrotactile feedback, the FSR was placed at the bottom of each test object to sense the user-applied pressure (Figure 4).Conditions were automatically updated.
Each combination of parameters was repeated three times.Between repetitions, we presented the participants with two additional objects, as references for a very soft and a hard object.The soft object was a sponge, and the hard object was a solid resin cube (also 3D printed).
4.1.4Procedure.Before starting the experiment, participants were familiarized with compliance through the following script, which ensured all participants had a shared mental model of compliance: "Compliance is the property of an object to deform if you press against it.For example, a very soft mattress is more compliant than a wooden bench.A patch of moss is more compliant than a stone.A rubber cube is more compliant than a glass cube.In most contexts, you can think of compliance as corresponding to the softness of a material."To ensure their understanding, all participants completed a simple compliance discrimination test.This involved estimating the compliance of three objects with varying hardness.
For both the initial discrimination task and the actual experiment, participants were instructed not to rotate the objects during exploration and to grasp them only with their thumb and index finger.They were informed that a higher estimation should correspond to a more compliant object.They were also encouraged to use decimals in their estimates.Additionally, participants were Actuator FSR Test Object Figure 4: Participants held 3D-printed test objects of different softness (Shore A levels) between thumb and index finger to estimate their compliance.Additionally, virtual compliance was rendered with a vibrotactile actuator on the fingernail based on applied pressure (measured using an FSR).
advised to start with larger numbers, which would allow them to adjust their estimates downward if necessary.
After the discrimination test, the actuator was attached to the participant's fingernail to provide the vibrotactile augmentation.Subsequently, each participant performed three blocks of 18 trials.In each block, the participant explored four objects (Shore A of 40, 50, 60, 90) with four granularities (0, 10, 20, and 40 grains) and two non-augmented reference objects (sponge and plastic cube).The order and combination of hardness and granularity were counterbalanced.Throughout the experiment, the participant was guided by a PC monitor, and they entered their compliance estimates using a numerical pad on a GUI.4.1.5Data collection and processing.For the analysis, we incorporated data collected from all twelve participants.In total, 648 trials were recorded (12 participants × 3 repetitions × 18 trials = 648 trials).Data was standardized per participant by first removing the participant's average response from each estimate and then dividing each estimate by the standard deviation.The resulting data is in standardized units.
4.1.6Analysis.We present our results as interval estimates [41], as these provide a better intuition of the underlying phenomena of interest than p-values [24].However, though we prefer to avoid null hypothesis significance testing [8,11,20], we show that the interval estimates can be used to find equivalent information: To establish whether differences between levels are significant, we calculate the 95% confidence interval of the difference of adjacent estimates.The confidence intervals for the differences of means provide a range of likely values for  1 −  2 .If there is no difference between the population means, then the difference will be zero (i.e.,  1 −  2 = 0).If a 95% confidence interval includes the null value, then there is no statistically meaningful or statistically significant difference between the groups.If the confidence interval does not include the null value, then we conclude that there is a statistically significant difference between the groups [67].For all results, we will therefore report confidence intervals and, where relevant, confidence intervals of the differences of means.
4.1.7Results.In the standardized scale, an estimate of one indicates that this estimate is one standard deviation above the average estimate for that user.This shows differences in conditions while removing individual differences between participants.The response curves depicting virtual and physical compliance can be observed in Fig. 5.We found that, as expected, with increasing Shore A level, participants rated the samples less compliant (Red, left Figure 5).Notably, the response curve for virtual compliance shows a clear trend: as the number of grains in the vibrotactile rendering increases, the perceived softness of the explored object also increases (Blue, right Figure 5).This outcome effectively corroborates the initial hypothesis.The magnitude of this effect appears to be even greater than how the change in compliance of the physical samples was experienced.
We evaluated the calculated confidence intervals by checking whether the differences of adjacent ones do not contain zero, which means that the difference is significant at p = 0.05.We found the difference between Shore A levels of 50 and 60 to be significant (95% CI [0.71, 0.42]), as was the difference between 60 and 90 (95% CI [0.56, 0.26]).For virtual compliance, we found a significant difference between all levels -for 0 and 10 (95% CI [-0.35, -0.62]), 10 and 20 (95% CI [-0.25, -0.53]), and 20 and 40 (95% CI [-0.32, -0.61]).This shows that the perceived difference between conditions is greater than what we might expect to find by chance alone, given the variability of the data.Consequently, we can confidently say that the compliance illusion works and that within the parameter range we tested, increasing grains leads to greater compliance.Finally, the relative consistent size of the confidence intervals relative to the overall effects between tactile feedback parameters and evaluations of samples suggests that our ability to discriminate between virtual and physical compliance is similar.

Study #2: Staircase Procedure
The Magnitude Estimation experiment showed that the illusion is effective and revealed how parameter changes affect it.However, this only allows us to compare the relative compliance of the stimuli to one another without providing absolute values or real-world indicators of the illusion's strength.To judge the illusion's absolute effect, we conducted a Staircase experiment.

4.2.2
Procedure.The experiment aimed to determine the level of softness induced by the compliance illusion on rigid objects.We employed a 1-up, 1-down adaptive staircase design for the experiment.In each trial of the staircase, participants interacted simultaneously with two samples: (1) a test object touched by one of their index fingers, with an actuator attached to the fingernail, and (2) a reference object touched by the index finger of the opposite hand, without an actuator.
After exploring both objects, participants were asked to indicate whether the test object felt softer than the reference object.In the staircase design, if participants responded with "yes" (indicating that the test object felt softer), the hardness of the subsequent reference stimulus was decreased by 1 (resulting in a softer reference

So�er
Figure 5: Magnitude estimation for perceived compliance, in standardized units.We show response curves for virtual compliance (grain numbers -blue) and physical compliance (Shore A levels -red).We show the mean (circular mark) as well as the 95% confidence interval for each measure.A value of zero indicates an average rating that matches each participant's overall average.A value of 1 indicates that this rating is one Standard Deviation away from the participants' average.The data indicates that participants were not able to distinguish between Shore A levels of 40 and 50, but otherwise, all compliance levels significantly affected the ratings.object being presented).Conversely, if participants responded with "no" (indicating that the test object did not feel softer), the hardness of the next reference stimulus was increased by 1 (resulting in a harder reference object being presented), following the conventions of traditional staircase study design (see also Appendix A).Each participant completed four staircases (two test objects tested with both the dominant and non-dominant hand of each user), which took around 45 minutes to complete.
Starting conditions: Participants started each staircase procedure by comparing the test object against the softest stimuli (sponge) as a reference object.The test objects were of 70, and 90 Shore A level, and the reference objects were all the set of objects mentioned in subsubsection 4.1.3.
Stopping conditions: Each staircase was followed until five reversals were reached.The final discrimination value was obtained by calculating the average of the last three reversals.4.2.3Setup, apparatus, and stimuli.As before, participants were seated in a chair in front of a desk.Both hands were positioned behind a black curtain, which ensured that no visual cues were available while exploring the objects.The reference objects were the same 3D-printed cuboids (6x6x7 mm) of Shore A levels 10, 20, 30, 40, 50, 60, 70, 80, and 90.These were used to compare the test object to.We selected two relatively hard objects (70 and 90 Shore A) as test objects.We chose relatively hard objects, as these would allow for the largest reduction in hardness.Also, we imagine that hard objects would be used for most immediate application scenarios (for example, as demonstrated in our qualitative study in the next section).Finally, these values were also to ease the comparability of vARitouch and the method proposed by Tao et al., who also tested 70A and 90A in their staircase study.[69].
We employed the identical tactile augmentation system that was utilized during Study #1 and selected a granularity of 40 with 200 Hz tactile grains lasting 5ms.

Data collection and processing.
During the staircase procedure, the first author collected each participant's replies on a sheet of paper designed specially for the study.Please refer to Appendix A, which shows samples of staircase trials to illustrate the procedure.

Results.
For analysis, we used data from all twelve participants.Average results, their standard deviation, and 95% confidence intervals can be found in Fig. 6.As the confidence interval does not contain the actual hardness, we can say, with confidence, that the virtual compliance had a significant effect [41].Like Tao et al. [69], we also report the SD.We show that our results are similar to those of Tao et al. [69] in magnitude.However, the responses we received have much higher variability.These results show that the compliance illusion does indeed work robustly when we stimulate the back of the finger instead of the object the user is interacting with.The results also help us understand the nature of the experience, but they do not shed light on how this experience might fit into daily tasks.To gain a deeper understanding of how deploying such a system could be experienced, we present the findings of a qualitative study in the next section.In this study, we situated the proposed compliance illusion within a simplified design task.

Q2: USING VIRTUAL COMPLIANCE FOR PROTOTYPING
In this section, we use an early prototype to explore the use of vARitouch in a prototyping activity.While our first experiment shows that back-of-the-finger stimulation does indeed provide an experience of compliance, it remains unclear how users would experience this in a real-world setting.We conducted a participatory design study to deepen our understanding of (1) how nonexpert users perceive the compliance illusion, (2) how the device augments their interaction with tangibles, and (3) how the device could be integrated into real-world applications.We selected prototyping of an interactive interface as an application where vARitouch can be integrated.We envisioned vARitouch being used in a design process where low-fidelity prototypes made of 3D-printed rigid widgets can be augmented with the virtual sense of interactivity that the vARitouch's haptic illusion offers.
We designed a study to understand the potential of using vARitouch to augment non-compliant objects in a design task.Participants in this study were asked to design two smart home control interfaces with 3D-printed UI controls such as buttons, sliders, and knobs.These low-fi elements aimed to capture the essence of interaction without requiring intricate working elements.Meanwhile, the actuator attached to the participant's finger added a layer of interactivity and haptic feedback to the design process, transforming the static widgets into dynamic interactions.The haptic illusion augmented the controls by translating them from solid objects into interactive elements.
After the participants designed their interfaces, the first author conducted interviews to gather opinions on factors such as comfort, realism, and the effectiveness of the augmentation during the design process.

Setup and Apparatus
Figure 7 shows the setup used during the experiment.Participants were seated in front of an experiment desk and were provided with 3D-printed UI controls, such as buttons, sliders, knobs, toroids, and joysticks of different sizes and shapes.These objects were printed using a filament with a hardness of 70 Shore A. Additionally, there was a pressure-sensitive panel made of 4 FSRs8 attached to the underside of a thin black surface.These FSRs enabled sensing the applied pressure when participants placed the 3D-printed objects on it and were used to generate the motion-coupled vibrotactile feedback provided through the actuator attached to their fingernail.
We utilized the equipment detailed in Section 4.1.3,with the vibrotactile feedback parameters set at a granularity of 40, frequency of 200 Hz, and duration of 5 ms.
Before starting the experiment, the actuator was attached to the index finger of the participant's dominant hand using medicalgrade double-sided tape.Once attached, participants were able to interact with the 3D-printed objects to design their interfaces.To do so, they placed the objects on the black surface and touched them primarily with the fingerpad of the index finger, where the actuator was attached.The system sensed the pressure applied during the interaction and provided, through the actuator, a haptic response linked to the level of pressure applied.Participants were expected to feel vibrations in their index finger that simulated a virtual compliance effect as if the object they were interacting with was softer than it was.Thanks to this effect, for example, participants felt the rigid button was as "pressable" as an actual button.The results section (subsection 5.3) expands on the sensations that the feedback evoked.

Participants.
For the study, a total of 10 participants (five females, four males, one gender-fluid individual) aged 21-42 (M=25.4,SD=6.0) were recruited.We welcomed participants with any scientific background, resulting in a diverse group from HCI, graphic design, game development, human kinetics, and other fields.They received as compensation a $25 CAD gift card for their participation.Our research institution approved this study.

Data Collection. Data was collected through audio-and video-recorded interviews. Each participant participated in a 60minute participatory design session (including the prototyping activity and interview).
At the start of the sessions, participants signed a consent form and were instructed to wear the vibrator-actuator on their fingernails for the whole session.After the device was set up, participants began the prototyping phase (Figure 7).They were provided with rigid 3D-printed objects and tasked with designing two user interfaces to control a smart home, choosing the objects from the ones provided.While they were designing each interface, they used the think-aloud method to understand the role of the haptic illusion in the design process [25].After the participants designed the first interface, they were asked to explain each tangible object's use and the haptic illusion's role in interacting with it.The same process was repeated for the second interface.After the activity, participants were asked to answer open-ended questions, including questions about the experience of using haptic illusion to interact with the tangibles.
The data underwent anonymization for storage, achieved by assigning numerical participant IDs.All interviews were conducted entirely in English.They were then transcribed for the purpose of analysis.

Data Analysis.
The interview data was organized into preliminary codes and then organized into themes by the first author with support from the fifth author using reflexive thematic analysis [13,18].We began by familiarizing ourselves with the data, and then after a few iterations of categorizing the data into codes,

User
Perception.This section examines the participants' experiences during interaction with the compliance illusion.We explore the experiences users had, as they were engaging with the 3D prints.
Participants provided diverse descriptions of their experiences with the haptic illusion, including sensations like tapping, vibrations, and pressure sensitivity, each associated with different interactions and movement qualities/patterns.Some participants described the experience as someone tapping their fingers.Others cited the feedback as being subtle and feeling little vibrations or pulses on their fingers.P1 referred to it as "very soft but distinct."Half the participants also noted the effect of the applied pressure when the feedback was delivered; for example, P3 and P9 said that the vibration matched and is in response to the applied pressure, and P7 expanded this, stating, "I think the most notable feeling is the difference in pressure.It feels like more vibrations when I push harder." Also, participants reported a sense of depth perception caused by the illusion when pressing the tangibles.P5 described a "cushion effect" and a feeling of "depth in the material, " which contradicted the visual absence of penetration.One of the most cited movement qualities that users felt while experiencing the haptic illusion was pushing or clicking a button, mainly when interacting with the 3Dprinted buttons.However, it was not the only movement quality; even though the same vibrotactile feedback was delivered, participants also experienced bending when pushing a joystick's tip, ticks that indicate directionality when moving a slider handle, and rotation clicks when rotating a knob.In summary, most participants had an embodied experience of augmented materiality, though in some situations, users reported experiences such as tapping or vibration.We assume the experience of tapping or vibration was caused by instances where the users could not link their actions to the corresponding feedback they received.Overall, the haptic illusion made the interaction more immersive for the user by evoking a diverse range of tactile sensations that depended on the users' actions, the pressure applied during touch exploration, and the way the user touched the objects.The haptic illusion allowed the rigid elements used in the user interface prototype to have a degree of interactivity and responsiveness that they would not have had without the use of the vARitouch illusion.This provided the participant designers with a more embodied experience when interacting with the mock-up interfaces, as they provided a physical sense of interactivity rather than just consisting of 3D-printed dummy objects.

User
Reflections.Here, we collect reflections and judgments made by users.We include aspects such as the realism and effectiveness of the haptic feedback in conveying information or enhancing user interactions, as well as the overall satisfaction and engagement of the users.
Realism: Participants generally thought that the compliance illusion was accurate.For instance, P5 noted, "For bending, I have the feeling that I'm able to bend it quite well," and P6 stated, "I would say a very realistic because the vibration made me feel like [...] It's not just a piece of plastic."However, according to one participant, the vibration did not feel like "immersive feedback." Some participants also mentioned that the level of realism differs from tangible to tangible, so depending on what they are interacting with.
Significance: Participants were asked whether the haptic feedback provided comprehensible information or merely arbitrary vibrations.We received mostly favorable results; for example, P6 said, "feels very connected to what I'm doing." P1 noted that for the most part, it made sense, and P8 added that it was consistent and "it's not going to give me a random thing."We also asked the participants to describe in their own words what experience the feedback evoked; P1 said that it seemed to "replicate the feeling of touching different objects." P8 said, "digital device that attempts to emulate an analog experience" and "something has like a physical feedback that's reminiscent of those [analog] machines or technology and stuff." Satisfaction and engagement: Most participants expressed a positive impression of the vibrotactile feedback; P8, for example, described it as unobtrusive, while P9 mentioned that they wanted to keep playing with it.P10 expressed, "Feels really natural.Overall, it was not disturbing me.Even if I did some really different things.But it feels good, and I think I can live with that in 24 hours." On the other hand, some participants expressed mixed opinions.P5, for example, perceived it as being a bit weird, while P3 said, "it's kind of bothering me," and P9 mentioned that it "feels very vulnerable because it's a very small object.And we're used to our fingers being very sensitive." P3 noted a preference for using the feedback without tangibles, finding it more suitable for plain surfaces.
Effectiveness: Participants reported that the haptic feedback aligned with the mental imagery of their actions.P6 noted that the haptic sensations aligned with their expectations of movement and manipulation, reinforcing the link between feedback and intended action.P4 said that the feedback made them think that the widgets were operating and performing actions, while P5 stated that it provided a greater sense of control.P2 added that without the feedback, the object felt inert, whereas, with it, the object seemed responsive and alive.However, P3 noted that the feedback effect could fade if used daily.
Overall, the participants felt the feedback had a positive effect on the prototyping task and that it completely transformed their experience of it.However, both P5 and P3 were bothered by it at times.Here, we believe providing users with greater agency to the settings of vARitouch, as well as enabling them to selectively turn it on or off, would greatly benefit the experience.A major point of critique was also the perceived fragility of the system, which we believe to be a natural side-effect for an early prototype.Nevertheless, it is something we need to consider in future iterations.

Interaction with
Tangibles.During the study, participants interacted with a variety of tangible objects, such as 3D-printed rigid buttons, joysticks, sliders, toroids, and knobs.Upon augmentation with the haptic feedback, each object impaired distinct tactile experiences, as described below.
Buttons: Most participants experienced a strong sense of realism and authenticity interacting with the buttons (Figure 8, a).P6 felt like they were pressing an actual button, and P4 stated that it felt the most realistic of all the tangibles.Tactile sensations played a crucial role in users' experiences.P5 stressed the buttons' "squishy" and pleasant tactile responses.P6 emphasized that the feedback resembled a "1-click" sensation and solidified the feeling of pressing a button.P7's observation of a "bouncy" sensation while pressing a button indicated that the combination of physical movement and haptic feedback enhanced the tactile experience, contributing to an experience of interactivity.
Knobs: Participants reported feeling clicks/vibrations when interacting with the knobs (Figure 8, b), and some even noted distinct clicks when turning them, suggesting that the knobs provided a tactile sensation associated with their rotation.Most participants emphasized the realism and immersion provided by the feedback.They said that the sensation of turning a knob closely resembled the tactile experience of interacting with a real mechanical knob.This aspect contributed to a strengthened sense of control and engagement, as participants could feel adjustments happening in real time.
Joysticks: Participants noted that the rigid joystick (Figure 8, c), when augmented, gave them a sense of movement and bending.P4 stated that the sensation was similar to bending, which implies that the haptic feedback conveyed a physicality to the joystick's movement, creating a more immersive experience.P5 emphasized that they could feel the movement with the tip of their finger, describing the experience as if they were physically moving a joystick.P7 said they pushed the joystick despite knowing it would not move physically.This behavior indicates a strong desire for a seamless integration of physical movement with the augmented experience, emphasizing the potential for the haptic feedback to bridge the gap between physical and virtual interactions.
Sliders: Each slider had a small solid box and a movable slider handle (Figure 8, d), allowing the user to push it from side to side.As this object had a moving component, a few participants commented on the connection between the directional movement and the feedback.For instance, P3 mentioned a satisfying and noticeable movement pattern, like a "tik, tik, tik" when manipulating the handle inside the slider.A few of the participants also had a strong preference for this object over the others.For example, P6 noted that they strongly preferred the slider, citing the connection between physical actions and the haptic responses, enhancing their perception of interaction.
Some participants also interacted with the slider outside the sensing surface.P1 stressed the distinction between the tactile experiences with and without the augmentation.When augmented, they observed a smoother movement accompanied by vibrations simulating motion along a scale.P2 commented on an intriguing sense of inactivity when the slider lacked augmentation.This suggested an anticipation of responsive feedback during slider manipulation.
Toroids: The toroid (Figure 8, e) was less widely used than the other objects; only P7 used it.They noted that the tactile feedback varied depending on their finger's position on the toroid, perceiving a stronger vibration when their finger was closer to a specific area.This proximity-based variation in the sensation resulted in an intriguing effect, where the intensity of the vibration seemed to shift as they moved their finger around the toroid.Also, P7 drew an interesting link between the toroid's circular design and the concept of directionality, suggesting its use in designing new interactions.They proposed that the direction of vibration could convey indications of an object's location, like a compass, by associating different directions with varying vibration patterns.While the toroid was underutilized in our study, these insights can aid researchers exploring haptic feedback on toroids.
In summary, each 3D printed shape leads to distinct experiences, often experiences unexpected to us.The variety of experiences made it clear that while pressing against a rigid object provides an experience of virtual compliance, the geometries of the different tangibles also provided participants with additional contextual information, which changed their perception of compliance to be contextually meaningful within the interaction.For example, participants identified differences in the intensity and nature of feedback between knobs and buttons.Some noted that buttons produced clear on/off sensations, while knobs offered varying resistance and granularity even though the same feedback was delivered.This distinction in feedback contributed to the perception of different control functionalities.

Q2 Discussion
The study confirmed that vARitouch had a positive impact on the user experience for the prototyping exercise.Buttons were the subject of significant investigation by the participants, demonstrating the most compelling manifestation of the compliance illusion.Furthermore, utilizing knobs and sliders resulted in intriguing effects, as participants experienced the perception of rotational and directional motions facilitated by the vibrotactile augmentation.Overall,  the haptic augmentation made the prototyped system come alive in the minds of the participants, who felt that the haptic augmentation provided them with a greater sense of control.The positive impact of vARitouch on the task chosen was perhaps not surprising, as the exercise was geared to take advantage of vARitouch's strengths: making rigid items appear compliant and adding a sense of interactivity.Still, the overall positive reaction is encouraging.More surprising is the breadth of experiences participants reported on, including many that were unexpected even to us.Interacting withvARitouch unveiled a plurality of tactile experiences and characteristics of movement qualities, including bending, directionality, and rotation within rigid objects.A skeptic might identify this as a case of response bias, and while we cannot completely rule this out, we believe that there is in fact something more complex happening: There are multiple in prior work where similar simple setups led to a breadth of distinct experiences.This includes Pseudobend [31] which enabled users to experience rotation, bending, and shearing.Similarly, Haptic Servos [58], demonstrated that the same algorithm could create friction or compliance based on sensing modality.Finally, in an exploration of mid-air textures by Strohmeier et al. users reported experiencing virtual friction as weight or counter-force [62].
In line with the perspective by O'Regan and Noe [51], one can frame the experience as sense-making.When we encounter tactile feedback, we seek to make sense of it with respect to the action that caused it.We do so by comparing the sensory and motor pairing with material experiences we have had in the past and, subconsciously, seek an explanation for what we are experiencing.If the link is close enough, the sensory and motor pairing might then be interpreted as friction, bending, or compliance.If not, we experience it as vibration.
In conclusion, the qualitative study presents a strong case for our approach to tactile AR.Not only did the study show the utility of vARitouch for the example application case, it also showed that the approach chosen in the design of vARitouch has potential beyond a mere compliance-augmentation device.Both these positive results as well as feedback from the users themselves, also highlight the utility of finding technical solutions for complete encapsulation on the top of the finger.This will make the device more robust and remove the requirement to attach sensors to the objects the user wants to interact with.We present our approach to this problem in the next section.

Q3: FINGERNAIL PRESSURE SENSOR
While the psychophysical studies showed that the illusion is robust, and thematic analysis showed that such a system is desirable in a real-world scenario, there remains one crucial component to be addressed: We need a method to sense pressure without occluding the fingerpad.In this section, we outline our solution and present documentation of the sensor performance in a back-of-the-finger force-sensing configuration.As per our design rationale, our goal is to allow users to maintain a connection with the real-world tactile properties while benefiting from augmenting their perception with motion-couple vibrotactile feedback.
A potential method to achieve this is employing a thin film pressure sensor that is attached to the finger pad.This thin-film approach could allow users to largely perceive real-world tactile properties [49,74,79].Another option found in the literature is the use of a strain gauge attached to the fingernail [32].However, these approaches are typically single-use.For example, once removed, the strain gauge is broken permanently due to its fragility.Similarly, thin-film approaches remain experimental and are typically also designed for a single application.As we wanted our prototype to be both easily replicable and multi-use, these approaches were not considered.
Instead, we explored the use of an optical system to measure finger pressure.We recognized that the miniaturized LEDs and photodetectors available today were well-suited for integration into a nail-based solution.Among various documented optical systems in the literature, we initially tested one proposed by Mascaro [1].This system employed a green-light LED (525 nm) and a photodetector designed to detect changes in fingernail colorization caused by applied finger pressure.However, through practical experimentation with an early vARitouch prototype based on this design, we observed that using this wavelength made it behave similarly to a distance sensor.It primarily responded to changes in finger pad proximity to the sensor, rather than accurately measuring alterations in nail color.This finding led us to explore other wavelengths that can behave differently on human skin.
Upon reviewing Anderson's work on the optical behavior of human skin [4], it became evident that light in the near UV, visible, and near-infrared radiation spectrum penetrated the epidermis most effectively.Considering that our intended sensor placement was on the fingernail, we selected infrared light as the optimal wavelength.To simplify development, we opted to utilize an offthe-shelf solution capable of irradiating and measuring infrared light through the finger, with pulse oximetry emerging as the most suitable choice.

Implementation
We chose to adapt the Mascaro et al. design approach of photoplethysmograph (PPG) fingernail sensors because of their ability to be attached and removed from the fingernail easily, making them suitable in a non-permanent wearable scenario [44].However, we decided to use an infrared wavelength.PPG offers a simple and cost-effective way to monitor changes in blood volume within the microvascular tissue bed [16].
We opted for an integrated and off-the-shelf solution that can efficiently and compactly sense blood volume.Our choice was the MAXREFDES117 sensing module board 9 , which is centered around a MAX30102 Pulse Oximeter and Heart-Rate sensor 10 .This sensor comes equipped with integrated red (660nm) and IR (880nm) LEDs and a medium band photodetector (600-900nm).Utilized in a reflectance PPG configuration, it offers precise readings of blood volume fluctuations in the fingernail bed peripheral circulation when pressure is applied.Moreover, the sensor incorporates an ambient light cancellation system to reduce the impact of environmental sensing artifacts.
The sensing board is located in the lower layer of a 3D-printed case that is attached to the fingernail using make-up adhesive 11 .The sensor has to be placed as close to the middle of the nail as possible and tightly glued to prevent accidental displacements on the Z axis, which can lead to undesired sensing artifacts.The glue can be easily removed with alcohol or solvent for nails.The sensor's analog-todigital converter (ADC) configuration was adjusted to transmit an 18-bit data value via I2C to a Teensy 4.1 Micro Controller Unit (MCU).

Sensor Characteristics
We aimed to assess the performance of the proposed finger pressure sensor by comparing it with a TAL220 20 kg load cell 12 , as a baseline.To do so, we set up a test bed for assessing the capabilities of our finger pressure sensor as shown in Figure 9a.This setup comprises a flat plastic surface, along a load cell situated beneath the surface to capture force measurements, serving as a baseline.This testing setup enabled us to directly assess how well our sensor performed in comparison to the load cell under various levels of pressure applied to the surface.We recruited 3 participants with diverse skin colors to account for potential variability in sensor performance.Each participant performed three separate trials to ensure the reliability of our results.Participants were instructed to press their finger equipped with our sensor onto the test bed.They were asked to perform three actions: constant pressure, tapping, and variable pressure.
We recorded data from our finger pressure sensor and the load cell during each trial simultaneously.Video recordings of the participants interacting with both sensors concurrently can be found in the supplement of the paper.
We compared our sensor's pressure data with the load cell's force data to evaluate the sensor's performance.It performed well within the range of comfortable presses that we tested, equivalent to ∼1.2 newtons, as seen in Figure 9b.It should be noted that the data we show here is of instances where the sensor is performing under optimal conditions.In out-of-the-lab scenarios, we noted that artifacts can occur, primarily when the device is not mounted correctly.Also, the sensor readings can be affected by finger bending, which could be corrected using a finger-movement machine learning detection model [23].Additionally, given that we utilize a PPG sensor, the pressure signal is accompanied by a heart rate signal, which is mostly notable when the user does not apply pressure for a considerable period of time.Nevertheless, this issue can be effectively addressed by employing predictive filters such as adaptive notch filters (ANF) that have been used to extract heart rate from raw PPG signals [81].Moreover, user-specific calibration of minimum and maximum values is essential because individuals have varying blood volume and light translucence in the fingernail area.Nevertheless, in laboratory conditions, once the minimum and maximum values were identified and correctly scaled, the output of this sensor was used for creating the compliance illusion without any further data processing.

VARITOUCH IMPLEMENTATION AND APPLICATIONS
Having established that the concept is generally feasible, that it is desirable, and having presented a path towards making it technologically possible, we finally present vARitouch.Our Design approach has led us to develop a wearable device capable of changing the perceived compliance of objects without impairing the finger pad of the user, enabling compliance illusion experiences to virtually every surface.

System
The vARitouch system combines sensing and vibrotactile actuation within a convenient wearable form factor, consisting of a fingernail cap and an accompanying bracelet that houses the processing unit and power supply (Figure 10).The fingernail cap includes a vibrotactile actuator (LRA, Vybronics VLV101040A 13 ) on top of the sensing unit (subsection 6.1).This actuator delivers vibrations along the Z-axis, with a peak frequency response at 170Hz.Both components are integrated into a 180 × 160 × 150 mm casing, fabricated using flexible material (Formlabs Flexible V2 resin 14 ) to accommodate various finger sizes and fingernail shapes while minimizing the influence of ambient light on sensor readings.To ensure a secure fit and mitigate potential sensor noise caused by unintended movement, we glue the cap to the nail using make-up adhesive (Kryolian Mastix professional grade spirit gum 15 ).
Connecting the fingernail cap to the bracelet is a 5-lead cable, which is integrated into a custom-made expandable neoprene wristband.This bracelet houses the necessary components for generating the compliance illusion.It can include a power supply (LiPo or power bank capable of outputting a stable 5V DC) to make vARitouch a portable solution.
Within our system, we use an audio-based actuation approach.As such, a Microcontroller Unit (MCU) (Teensy 4.1 16 ) reads analog signals from the vARitouch finger-pressure sensor and subsequently generates grains, short sinusoidal impulses, in accordance with the user's applied pressure.This digital audio is then converted into an analog signal (utilizing a DAC, PT8211 17 ), which is further amplified (through an Adafruit PAM8302 2.5W mono amplifier 18 ) before being delivered to the actuator in the cap.

Applications
Here, we propose a range of tactile AR application scenarios using vARitouch.These applications extend into various domains, reflecting vARitouch's versatility and adaptability: 7.2.1 User Interfaces.As our user study showed, vARitouch can make rigid objects feel more interactive.This makes a natural application space of vARitouch for creating responsive UIs in tactile AR (Figure 11, a).
We see two areas that would significantly improve the user experience when using vARitouch.The first area is on-body interfaces, for which research has pursued three general approaches.The first are systems that project interfaces on the body, as seen in the work of Harrison et al. [28].The second approach uses thin-film temporary tattoos, as seen in the work by Weigel et al. [75,76].Third, some researchers have proposed omitting visual feedback altogether, relying solely on external cameras [27].None of these systems can provide any tactile confirmation of user input.With vARitouch, such systems could be augmented to provide satisfying tactile feedback to user input.The second area that would significantly improve the user experience when using vARitouch is in augmenting interactions with situated displays [50].These are displays, and often corresponding UIs that, through their positioning within social spaces, are provided with additional understandings of behavioral appropriateness or cultural expectations [29].With visual AR, situated displays can be dynamically deployed in human environments, which has renewed the interest of researchers in these contextual displays [15,26,42].Using vARitouch, such displays can similarly be augmented with a satisfying tactile component to the interaction by simulating force compliance-curves, compression, or clicks of tactile UI elements (Figure 11, a).
In an environment where systems such as vARitouch are widely used, it might also make sense for vARitouch to augment digital technologies such as smartwatches or tablets, as shown in Figure 1.
7.2.2Mediation of Information.vARitouch can serve as a tool for users to access invisible information.As discussed by Ihde [34] and Verbeek [72], such information can be provided in an embodied or hermeneutic symbolic manner.
Embodied.In embodied mediation, information is presented in a way that we understand pre-reflectively.As discussed in our design rationale, this is the default way of interacting with vARitouch.We also explored an example of such interaction in our qualitative user study.Designers might wish to 3D print the rigid shape of an object and then use vARitouch to explore different levels of softness; for example, one designer might create a 3D shape of a bicycle saddle and send it to multiple colleagues.Each colleague could then design the softness of the saddle.Softness profiles could be shared among all designers and tested on the same rigid 3D print.Here, the information that is mediated is the compliance itself (Figure 11,  b).
Hermeneutic.In contrast, compliance might also be used as a symbol to represent something else; for example, the same team of bicycle designers might also prototype different frame geometries and run a stress-strain analysis on these.The designers can then tactually explore the result of these models on the prototype frames with vARitouch, for example, by making the fragile places where strain is the highest feel softer than the places that are not under strain.In this scenario, the compliance is a symbol, a placeholder that the users interpret to represent other information (Figure 11,  b).

Sensorimotor Augmentation.
By providing information about the world around us, vARitouch also provides information about our actions.This means that vARitouch might also act as sensory augmentation, or, as per O'Regan and Noë [51], a sensorimotor augmentation, providing users with heightened sensitivity to fine motor activity.This can be used to support tasks where precise pressure levels are important, for example, it might support novices in learning how to use a potter's wheel (Figure 11, c).The un-occluded fingertip still allows the novice to feel the rich material properties of the clay, while at the same time the feedback from vARitouch helps them to keep the pressure at the desired level.
This feedback could also be collaboratively shaped; for example, a person offering professional massages might let their customer set maximum pressure levels for their comfort.The massage therapist could then feel, in real-time, the pressure level the customer wants, while at the same time having their fingers un-occluded for skin-toskin interaction.As the fingers are un-occluded, the person offering the massage can also use oils and creams without them interfering with the technology (Figure 11, c).
An extension of vARitouch as sensorimotor augmentation could be used as a rehabilitation tool (Figure 11, c).Stroke patients often require extensive rehabilitation exercises after paralysis, for example, using physical objects like balls to regain hand mobility [3].This process can be frustrating, especially since initial progress is slow [14].vARitouch can enhance this rehabilitation process by enabling a certain category and grade of stroke patients to feel their movements better.The added sensory feedback to even the smallest movement can provide patients with the encouragement that they are actually making progress [21,53].

DISCUSSION, LIMITATIONS & FUTURE WORK
In this paper, we build on related work on tactile rendering, taking inspiration from O'Regan and Noë's concept of sensorimotor contingencies, and present the first steps towards our vision of tactile AR.We intentionally structure the paper around exploring the assumptions behind our approach before presenting vARitouch, our current implementation.This approach stems from our interest in establishing a foundation for future devices that may differ in hardware from our prototype, rather than evaluating our specific device and its unique characteristics.Our findings show that backof-finger vibrotactile feedback is effective for this type of vibrotactile rendering.Additionally, our design approach, which focuses on creating material experiences by preserving sensorimotor contingencies, has proven promising.While our prototype was designed to induce an experience of compliance, in practice the experiences were much richer.There appears to be a plasticity to these types of sensorimotor illusions.However, we see the need for much future work to fully leverage these.
A technical limitation of this approach is that the positioning on the back of the finger means that the illusion happening is happening for the entire finger.The current prototype and the general approach we chose are unable to render information on a smaller scale than the fingertip.We believe this is an acceptable limitation, as our tactile senses are comparatively poor at spatial discrimination [9].On the other hand, our tactile senses are, particularly sensitive to temporal signal properties such as signal onset [35].This makes both system latency and jitter due to sampling rates particularly problematic [58].For our specific implementation, we found the sampling rate of our ADC (80 Hz) acceptable.However, it currently provides an upper bound to the fidelity of vARitouch and is likely one of the first aspects we would aim to improve for future iteration.
While we argue that vARitouch is self-contained, we recognize its need for integration into an AR ecosystem.Drawing from bARefoot's architecture [64], components needing high-frequency and low-latency updates, such as sensing and drive signal generation, are self-contained.Meanwhile, less time-sensitive tasks, like positioning or parameter updates, might be managed externally.Positioning could be achieved via optical tracking in AR glasses, and parameter selection through a Unity app.This approach enhances scalability: If there is a vARitouch device equipped to each user's finger, this architecture eliminates the need for 10 simultaneous high-frequency communication channels with a central server, as each unit independently computes its drive signal.
The current system is limited to making rigid objects softer.This is due to the choices we made when setting out to create a tactile AR system.Conceptually similar approaches could also have been chosen to target other experiences [31,58,62].However, our qualitative exploration has already shown that even this relatively simple setup is able to create experiences more broadly than mere compliance.A next step for vARitouch might be to add depth sensing to the fingertip, which would enable generating signals coupled to position, rather than force.Related work has shown that coupling tactile signals to force typically leads to a reduction in perceived counterforce [31,36], while coupling to position is typically experienced as added counterforce [62,65].This might provide a path towards not only making hard materials feel softer but also making soft materials feel harder.

CONCLUSION
This work presents vARitouch, a back-of-the-finger wearable designed to enhance the perceived material properties of the physical environment through vibrotactile feedback.By delivering tactile grains in response to pressure applied by the fingertip, vARitouch creates a sensation of variable compliance, thereby enriching the quality of interactions between users and their surroundings.Through a series of studies, we examined the perceptual and practical aspects of the proposed system.
Through a psychophysical scaling and a staircase study, we have demonstrated that the compliance illusion applied from the back of the finger is robust.We have also demonstrated that the compliance perception scales positively with the number of grains applied and that the reduction in hardness is within the magnitude of 30 Shore A levels.
The second phase, based on qualitative analysis of nonexpert users engaging in a prototyping study, provides insights into users' perceptions and interactions with virtual compliance across tangibles with different shapes.Thematic analysis of these interviews showed that, generally, the augmentation enriched the activity.It also highlighted that the sensorimotor contingencies created by the system can create a wider range of experience than mere compliance.
Finally, we present the technical implementation of our current vARitouch wearable.This sensing and actuation device can sense fingertip force pressure by measuring changes in fingernail vascularization and deliver haptic feedback based on the applied pressure through a small actuator integrated into the wearable device attached to the user's fingernail.
This work contributes towards realizing haptic augmented reality, envisioning a future where our interaction with the tactile world is as intuitive and effortless as our engagement with the visual realm today.The findings of these studies not only contribute to our fundamental understanding of tactile illusions, but also offer promising avenues for the development of new perceptual transparent systems.

A STAIRCASE SHEET
Figure 12: Example of staircase sheet.On the grid, the x-axis represents each repetition (time), and the y-axis represents the levels of stimuli (Shore A) of the reference object.If participants responded "yes" to the proposed question, we marked a filled dot on the corresponding repetition-stimuli pair and selected a softer reference for the next trial.If they responded "no," we marked an outlined dot on the corresponding repetition-stimuli pair and selected a harder reference for the next trial.The procedure is stopped when the participant reaches five reversals.

Figure 2 :
Figure 2: Different instances of virtual compliance.Kooboh (a) augments an object that can have custom compliance [37].Schorr et al.(b) present wearable delta robots that can present custom stiffness to users [61] (c) Our contribution, vARitouch, is the first to present such sensations in a wearable form factor that does not occlude the fingertip.

4. 1 . 1
Participants.A total of n=12 participants (three females, nine males) aged 24-35 (M=27.25,SD=3.46) were recruited through advertisements.They received a compensation of 12 EUR per hour for their participation in both studies.4.1.2Stimuli.The experiment used a factorial design in which four physical samples with different hardness were crossed with four levels of tactile stimulation.

Figure 6 :
Figure 6: Staircase results.Blue dots represent the actual hardness of test objects participants touch in each trial.The red dots indicate the average perceived hardness across all participants.

Figure 7 :
Figure 7: Participants designing the interfaces using the widgets provided along with the haptic augmentation

Figure 8 :
Figure 8: 3D-printed rigid tangibles used by participants during the prototyping activity

Figure 9 :
Figure 9: Sensor Characteristics: (a) Evaluation Set-Up.(b) Comparison between vARitouch Sensor Measurements and Load Cell

Figure 10 :
Figure 10: vARitouch Components.The 3D-printed fingernail cap contains an LRA and finger pressure sensors.The neoprene bracelet holds the microcontroller unit, the amplifier, and the power bank.

Figure 11 :
Figure 11: Application scenarios of vARitouch.User Interfaces (a) augmenting on body or situated displays with tactile interactivity.Information Mediation (b) providing users with symbolic or embodied information.Sensorimotor Augmentation (c) providing users with an augmented experience of the world around them as well as a more detailed experience of their own actions. 59]