Füpop: "Real Food" Flavor Delivery via Focused Ultrasound

Food and flavors are integral to our existence in the world. Nonetheless, taste remains an under-explored sense in interaction design. We present Füpop, a technical platform for delivering in-mouth flavors that leverages advances in electronics and molecular gastronomy. Füpop comprises a fully edible pouch placed inside the mouth against a cheek that programmatically releases different flavors when wirelessly triggered by a focused ultrasound transducer from outside the cheek. Füpop does not interfere with activities such as chewing and drinking, and its electronics may be integrated into devices already used near the cheek, such as mobile phones, audio headphones, and head-mounted displays. Füpop’s flavors are from “real foods,” not ones imitated with synthetic reagents, providing authentic, nutritive flavors. We envision that with Füpop, flavors may be synced to music, a phone call, or events in virtual reality to enhance a user’s experience of their food and the world.

Figure 1: Füpop comprises (1) a small focused ultrasound transducer placed outside the cheek that can be integrated into cell phones, audio headphones, or head-mounted displays; and (2) a 100% edible pouch, placed inside the cheek, that encloses calcium alginate spheres filled with different liquid foods.By varying the driving voltage of the transducer and the fabrication parameters of the spheres, spheres may be selectively "popped" to release their contained liquid.A Füpop designer may modulate the thicknesses of the sphere walls to choreograph an in-mouth flavor experience, delivering "real food," nutritive tastes to the user wirelessly.

INTRODUCTION
Food is not only a necessary component for our survival but also a centerpiece of personal, social, and cultural experiences [26,49].Accordingly, human-food interaction (HFI) has garnered great interest within the Human-Computer Interaction (HCI) community.HFI has been the subject of a Special Interest Group at CHI 2022 [9], as well as multiple workshops [14] at CHI [11,13], CHI PLAY [6], and DIS [8,12].Recent demonstrations of HFI technologies have spanned the integration of digital fabrication techniques into food preparation processes [19,34,38,39,47,48,87], the alteration of eating experiences through the stimulation of non-taste senses [4,52,77,80,81], and the creation of lickable "taste displays" that synthesize different flavors [45,46].Still, there are opportunities within HFI that remain under-explored.In particular, there is a dearth of technologies that enable dynamic interactions with the sense of taste after food leaves the plate and enters the mouth, with most demonstrations to date instead focusing either on the state of food before it enters the mouth or on the manipulation of senses other than taste to create illusory experiences.We still largely lack the means to manipulate, in real time, taste and texture in the mouth, one of our most important and densely innervated organs [7,24] that is rife with opportunities for interactive technologies.
Interestingly, the few systems that do manipulate taste via direct interaction with the tongue and mouth [3,45,46,57,[61][62][63] often do so without using items commonly considered to be food, instead using materials and techniques that do not provide a nutritive or caloric benefit to the user.This is in part by design, but it is also in part due to the fact that research in functional, edible materials and electronics is still in its infancy [25,64,86].Nonetheless, we believe that prioritizing the use of "real foods"1 over synthesized flavors is a promising direction to create authentic taste experiences that are not inherently limited by the drawbacks of imitation.Despite the wealth of recipes and techniques that chefs have perfected to serve up plates of food with delightful aromas, flavors, and textures, interaction designers have limited means to utilize "real foods" for in-mouth interactions and interfaces.Additionally, exploring edible ingredients and in-mouth interactions provides a unique opportunity to expand the space of sustainable and decomposable interactive systems.Edible materials are by definition easily degradable, generally possessing far lower carbon footprints than plastic alternatives or conventional electronics [72], and they are ideal candidates for designing unmaking experiences [66] or ephemeral user interfaces [70].
In this paper, we present Füpop2 , a technical platform that enables programmable, dynamic in-mouth flavor experiences with real food.We aim to utilize edible materials as much as possible and minimize the footprint of any non-degradable electronics.To do this, we relegate active electronic components to be completely outside the mouth -with the vision that they may be integrated into wearables or electronics already used on and around the face -and create a module existing entirely in the mouth with fully edible ingredients that are selectively responsive to external stimulation.In particular, we leverage the abilities of ultrasound energy to be transmitted wirelessly and to mechanically excite structures through the skin.Füpop comprises a completely edible pouch enclosing multiple different flavor capsules that unobtrusively rests inside the mouth against the cheek.The pouch does not readily dissolve or melt on its own in the mouth.We envision that future food designers may use Füpop to choreograph a taste experience for the eater whereby different flavors are sequentially released into the mouth and synced to different electronic stimuli, enabling more immersive musical or virtual reality experiences and allowing for novel interactions and communication, such as "taste messages, " that individuals may send to one another.
Füpop combines innovations in non-edible electronics along with those in molecular gastronomy and culinary science to create programmable flavor sequences with real, edible, and potentially nutritious ingredients that are not illusory or synthetic.Füpop does not require that a user's mouth remain in a fixed position and does not interfere with normal mouth activities such as chewing, drinking, swallowing, or talking.With low-profile electronics existing outside the face, dynamic experiences can be computationally triggered in real-time in response to stimuli such as music, gaming events, or bio-signals.In the subsequent sections, we present the following contributions: (1) The design and fabrication of Füpop (2) Ex vivo characterizations of design parameters that can be used to customize different flavor sequences, along with preliminary in vivo qualitative experiences (3) An ex vivo demonstration of a multi-flavor Füpop (4) Discussion around the applications, future embodiments, and significance of Füpop.

RELATED WORK
Füpop draws inspiration from a rich body of research in HFI and ultrasound technologies.Here, we summarize prior work in these areas and describe how this paper is situated with respect to existing approaches.

Human-Food Interaction
HFI spans a huge range of research that investigates the interplay between humans and food in nearly every aspect of our lives.Bertran et al. categorize HFI research into 6 domains: "Source" (obtaining food), "Store" (storing and disposing food), "Produce" (growing and manipulating food), "Track" (measurement of practices around food), "Eat" (consuming food), and "Speculate" (design fictions, meta-analyses, etc.) [1].Here, we present existing work in the "Produce" and "Eat" domains, focusing in particular on technologies that create new textures, aesthetics, and flavors during the food preparation and eating processes.
In the "Produce" domain, there have been numerous demonstrations of methods for crafting unique food textures and aesthetics, often integrating digital fabrication machinery, such as laser cutters and 3D printers [19,34,38,39,47,48,87], novel mechanical contraptions [89], or displays and other electronics [30] into the food preparation and serving experiences.Some of these technologies have been used to concoct dishes and flavors that reflect non-eating activities like physical exercise [34,35], while others have been used to investigate the relationship between taste and emotion both personally [20] and in intimate relationships [21].Some engineered characteristics during production also affect the subsequent eating experience.For example, FoodFab is a 3D printing system that allows a user to modulate infill pattern and density, in turn influencing an eater's chewing time and feeling of satiety [39].Shape-changing edible materials have also been developed and proposed as ingredients that enhance chefs' food preparation experiences and also open up possibilities for novel dining interactions [75,76].Researchers have indeed begun to design provocative systems that encourage and change the way diners can interact with and influence the qualities of their food before it comes to the plate.Logic Bonbon is a system in which users may squeeze flavored liquids into a bonbon that has internal fluidic chambers embodying digital logic gates.The combination of users' interactions with the bonbon and the bonbon's internal structure determines its resulting flavor [10].
In the "Eat" domain of HFI, technological approaches have largely focused on developing non-edible electronics, existing at least partially outside the mouth, that augment or simulate the experience of eating.One popular strategy to influence the taste of a food is to create multi-sensory experiences with other senses, particularly visual, audio, and tactile stimuli [4,52,53,77].Some systems employing this approach take the form factor of functional utensils that can interface directly with food.For example, Wang et al. presented ice cream cones [80] and drinking straws [81] that enhance users' eating and drinking experiences with sound to encourage playfulness and social intimacy.
While far less common, there have indeed been a few demonstrations of systems that manipulate the sense of taste itself, creating dynamic flavor interactions on the tongue.One method for this is to directly stimulate the tongue with electrical current.The tongue can readily perceive microamps of current, with reported thresholds as low as 5µA [69].Ranasinghe et al. presented an array of utensils and devices, such as a "Digital Flavor Synthesizer," that, in conjunction with heating and cooling elements, use 20-180µA of current to simulate sour, spicy, and minty tastes [61][62][63].Another approach that Miyashita pioneered is to simulate different flavors with lickable electrophoretic "taste displays" comprising a set of agar gels with various electrolytes that mimic 5 basis flavors (salty, acidic, bitter, umami, and sweet).Miyashita recreated certain tastes, as measured with an electronic taste sensor, by modulating the electric potential across each gel and thus the concentration of electrolytes at the surface of the tongue [45,46].Miyashita's Norimaki Synthesizer is a taste display in the form of a handheld stick that, while not designed to be eaten, is safe to lick [45].Finally, Brooks et al. presented a system for "taste retargeting" whereby liquid chemical modulators could be delivered into the mouth via tubes to alter the perceived taste of a certain food [3].
While greatly influenced by the above work, Füpop is fundamentally different in that it delivers flavors that can be experienced during the act of eating with real, edible, and nutritive foods that are not illusory or synthetic.Additionally, with low-profile, wireless electronics outside the mouth that can be integrated into existing electronics, Füpop does not require that a user's mouth remain open or closed or that special utensils are used, minimizing unnatural disruptions to normal eating and mouth-related practices.

Ultrasound in HCI
Füpop harnesses the power of ultrasound technology to induce inmouth experiences from outside the mouth with no physical wires between the two domains.Ultrasound is electromagnetic energy existing at frequencies above 20kHz, the upper threshold of human hearing.It is conventionally generated by exciting piezoelectric elements with an alternating electrical signal, causing them to vibrate at the stimulation frequency.Ultrasound is perhaps best known for its applications in the medical space, but it has attracted great interest in the HCI community as well.In medicine, ultrasound is commonly used for imaging [16] and several different active therapies, including tissue healing for physical rehabilitation [43,71], transdermal drug delivery [43,44], and targeted tumor ablation [33,43,44].It can be used non-invasively and is non-ionizing, unlike microwaves or x-rays, allowing it to be transmitted through healthy biological tissues with minimal risk [15].
In HCI, ultrasound has been instrumental in developing several novel interactive systems.UltraHaptics, reported in 2013, is one of the earliest of such systems; it comprises an array of 320 ultrasonic transducers that produce haptic sensations in mid-air [5].Since then, many other systems based on arrays of ultrasonic transducers have been demonstrated to create mid-air haptics, especially on the hands [42,55,68,79,83].This idea has also been used for haptic feedback on other body parts.Shen et al. demonstrated a system for generating haptic sensations in and around the mouth using an array of beamforming ultrasonic transducers attached to a virtual reality head-mounted display [65].Other demonstrations have created haptic sensations on the face [22] and on the lips [31].
Within the HFI domain, ultrasound has been used to levitate, mix, and move edible droplets of liquid [32,78].Furthermore, ultrasound has been used to levitate other small objects [17,18,41], selectively deliver audible sounds to a targeted audience [56], create "digital ventriloquism" illusory effects whereby observers perceive passive physical objects to be active sources of audio [28], and sense gestures and facial expressions [27,29].It has even been utilized to wirelessly power electronics [50].
Ultrasound waves quickly attenuate when passing through heterogeneous media, including biological tissue, and thus their existence in HCI research to date has been primarily restricted to usage in obstacle-free settings -even Shen et al. 's mouth haptics system requires that the mouth be open for a user to feel effects on the teeth or tongue [65].Researchers have reported a few techniques to overcome this challenge.SoundBender is a system that combines phased arrays of transducers and acoustic metamaterials to allow ultrasound beams to bend around small obstacles for mid-air levitation and haptic feedback [54].Using arrays of phased transducers and time-reversal signal processing, SkinHaptics is another system that allows perceivable haptics sensations generated on one side of the hand to be felt on the other [67].
Instead of an array, Füpop utilizes a single high-intensity focused ultrasound (HIFU) transducer.The transducer is functionally similar to those used in medical procedures to target malignant tumors beneath the skin without damaging surrounding tissue [33], but we operate it at much lower powers and for much shorter times.This simplifies the necessary driving electronics in comparison to phased arrays and also utilizes well established techniques for operating near and through the human body.

FÜPOP
Füpop is composed of 2 main components: (1) electronics existing entirely outside the mouth and (2) an edible pouch, made with Do It Yourself (DIY) friendly methods, existing entirely inside the mouth between the cheek and teeth.The pouch contains spheres of different liquid foods of the designer's choice that release their unique flavors sequentially when wirelessly triggered by the external electronics.In this section, we discuss the design and fabrication of each component.

External Electronics
A HIFU transducer with a resonant frequency of 4MHz, purchased from alibaba.com for $10 USD (Longzhichuang Co., Ltd.), is the core of Füpop's external electronics.The transducer is a cylinder 22mm in diameter and 10mm in height with a hemispherical piezoelectric ceramic element.Operating well above the frequency range of human hearing, the transducer is not audible when activated.Figure 2 shows an image of Füpop's transducer alongside a cross-sectional schematic illustrating how ultrasonic energy is focused through the skin.The main distinguishing characteristic of a focused transducer versus a normal ultrasonic transducer is that the focused transducer's vibrating piezoelectric element is shaped such that it forms a concave disc instead of a flat one, acting like a lens to direct ultrasonic energy towards a point in front of the transducer instead of being distributed and attenuated in multiple directions.This results in a concentration of acoustical pressure that can exist tens of millimeters deep in a heterogeneous medium while maintaining a safe and relatively low power density at the surface of the medium.It can, as we will show, create enough force to mechanically excite and rupture liquid spheres placed at or near the focal region.
The architecture of the driving electronics that we use for testing is shown in Figure 3.In comparison to the phased arrays described previously in the Related Work section, Füpop's single HIFU transducer may be simply driven by electronics that can generate a sine wave at the transducer's resonant frequency with sufficient amplitude.This can in theory be accomplished with a fairly small electronics package comprising (1) a microcontroller, connected to various signals and sensors of interest (audio, on-body sensors, etc.), that provides a trigger signal to (2) a driving circuit programmed to output the desired amplitude and frequency to activate the HIFU transducer.This package can be handily integrated into wearables and electronics that are already commonly used on or around the face, such as mobile phones, headphones, head-mounted displays, or face masks.For testing, however, we use a benchtop arbitrary waveform generator (SeeSii DDS-15MHz) amplified by a fixed-gain 55dB amplifier (E&I A150) to drive our transducer.The waveform generator is set to output a sine wave with a frequency of 4.3MHz, which we empirically found using a frequency scan to be the point of lowest impedance and thus most efficient power delivery.All input and outputs are impedance-matched to 50Ω.An Arduino provides a 5V signal to the waveform generator to trigger an excitation pulse of 0.25s.By adjusting the output level of the waveform generator, we deliver an excitation signal to the transducer of up to 80V P-P .This corresponds to a power of 24W/cm 2 at the surface of the skin.These conditions appear to be well within the regime of safe operation in and around the body; for reference, HIFU in medical applications is typically used at > 1000W/cm 2 for multiple seconds (even minutes and hours) at a time [43,84].

Edible Cheek Pouch
The edible component of Füpop, shown in Figure 4, is a pouch that is contained entirely inside the mouth and placed against the same cheek where the transducer rests on the outside.The pouch, made from an edible, flexible material like potato starch film or rice paper, encloses multiple spheres containing liquid foods of the designer's choice.The spheres are fabricated via reverse spherification, a well known technique in molecular gastronomy that results in delicate spheres of arbitrary liquids encased by a thin film of calcium alginate [58], which is an edible, tasteless material that is also frequently used in wound dressings [37].Reverse spherification is used to make fruit caviar, "popping boba, " and other culinary novelties.The process of reverse spherification utilizes completely edible ingredients and is safe and simple to make at home.The fabrication process is shown in Figure 5.The method of making Füpop's edible pouch is straightforward, utilizing DIY friendly materials and methods.First, an alginate bath is prepared by mixing 0.5% (by weight) sodium alginate in distilled water at room temperature.Sodium alginate initially forms visible clumps that slowly dissolve over ∼30 minutes.Once the alginate bath is prepared, it may be stored in a sealed container and reused over multiple days.Next, 1% (by weight) calcium lactate is added to a flavored liquid, such as juice, milk, coffee, or soda.To improve the shape of the resulting spheres, up to 1% (by weight) guar gum, a tasteless food thickener, is added to increase the viscosity of the liquid to be the consistency of a thin syrup.Next, a pipette with a tip opening of 0.15 mm in diameter is used to drop the flavored mixture into the sodium alginate bath.The alginate in the bath reacts with the calcium in the drop to form a calcium alginate film around the drop.After 5-45 seconds, the resulting sphere is removed from the sodium alginate bath with a slotted spoon and placed into a clean water bath to halt the thickening of the calcium alginate film.Drops of 0.4-1mL are the easiest to handle.Larger drops tend to form misshapen spheres (though techniques, such as freezing the drops first, can mitigate this) and are hard to pack into a reasonably sized cheek pouch, and smaller drops hold less flavor and are therefore more difficult to taste when popped.The spheres may be stored in their constituent liquid for up to 1 week without a loss of texture or flavor.The calcium alginate films enclosing the spheres are thin but robust and are not soluble in water or alcohol.The spheres may be handled by hand and do not melt or dissolve in the mouth.When a sphere is squeezed, the calcium alginate film pops, releasing the sphere's internal liquid (see Figure 6).
Multiple spheres are then packaged together using an edible material that does not readily dissolve in the mouth.Since the spheres themselves are resistant to saliva (remaining intact after soaking for >1 day), the wrapping films do not need to be impermeable to saliva, but they must stay intact in the mouth for a reasonable amount of time to hold the spheres in place.Meanwhile, the films must allow the inner liquid from burst spheres to escape into the mouth so that the user may taste them.We successfully made pouches with 0.28mm-thick grape leaves, 0.05mm-thick potato starch Oblate discs, which are digestible films for wrapping medicinal powders into a form to be swallowed, and 0.4mm-thick rice paper, which is commonly used to make the "summer rolls" of Vietnamese cuisine.We make 1mm-long perforations in the grape leaves and rice paper with a sharp knife to facilitate the release of liquid from popped spheres, but the Oblate discs are thin enough to render this step unnecessary.Both Oblate discs and rice paper are tasteless, selfadhering, and easy to cut and shape.They come in dried sheets, which we rehydrate in water, cut into pieces, and fold or roll into envelopes by hand to enclose the desired number of flavor spheres.Other wrapping options include perforated enteric-coated cellulose pill capsules, thinly sliced vegetables, or tea leaves.As we describe subsequently in our Characterization section, different wrapping films last for different amounts of time in the mouth, with some dissolving in a few minutes and others lasting for over 1 day.This, along with local availability, flavor, cost, and "mouthfeel" are factors that can influence the selection of one wrapping film over another.
Finally, edible glue, denture adhesives, or sticky foods may be used to help hold a finished pouch in place against the cheek, where it rests on the outside of the teeth.In this location, it does not interfere with activities such as chewing, drinking, and talking (see Figure 4).

CHARACTERIZATION
In this section, we present technical characterizations ex vivothat is, outside the mouth -so that we may offer better clarity of the design parameters and capabilities of our system.We have done our best to simulate in vivo conditions to give us confidence that our results can be replicated in the mouth.We also tested our system in our own mouths and report preliminary qualitative experiences in Section 4.5.For ex vivo characterization, we use a piece of hydrogel as a stand-in for a human cheek.We follow the recipe for MRI phantoms in [23] to create a hydrogel acoustically matched to human tissue that is composed of agar, NiCl 2 , MnCl 2 , NaCl, and water.We do not expect that the powers and pulse lengths we use here are intense enough to cause unwanted tissue heating, one of the known possible side effects of ultrasound treatments in medicine [43].Still, as a sanity check, we add orange thermochromic pigment that turns light green above 86 • F (SolarColorDust.com) to our hydrogel to visually ensure that we do not induce areas of dangerously high temperatures in the skin.We pour the hydrogel into molds that result in slices that are 6.7mm thick, corresponding to the average thickness of the human cheek cited in literature [36].We place the hydrogel layer on top of our ultrasonic transducer, which we encase in an acrylic box that has a circular cutout on its top surface for the emitting surface of the transducer's piezo element.A dab of ultrasound gel is used between the transducer and the hydrogel to improve coupling.

Effects of Sphere Parameters
The length of time that a flavor sphere spends soaking in the sodium alginate bath determines the thickness of its encasing calcium alginate film.We expect that thicker films are stiffer, requiring higher ultrasound excitation voltages to pop.Table 1 reports the average measurements of sphere popping threshold for different alginate bath soak times.Spheres are 0.8mL in volume for all soak times listed.Using a stopwatch, we measure soak time as the time between the drop hitting the surface of the bath and it leaving the surface of the bath on the slotted spoon.The popping threshold is determined by placing a sphere on the surface of the artificial skin, resting on the HIFU transducer, and then applying 0.25s pulses of increasing voltage, separated by 5s, until we observe a visible stream of liquid emitting from the sphere.Indeed, we observed that as soak time increases, the popping voltage threshold also increases, with voltages increasing from 32V P-P (14W RMS power) to 66V P-P (51W RMS power) for spheres soaked for 5-30 seconds.
Additionally, we measured the dependence of the force required to burst spheres on the sodium alginate soaking time.The bursting forces of spheres made for 7 different soaking times in the range of 5 to 60 seconds (10 spheres per condition) were measured by The results are plotted in Figure 7.The tongue can exert up to ∼16N [73], but the force of the tongue during swallowing is only 0.31N [74].Thus, spheres can likely indeed be burst with the tongue (or teeth) if there is intention to do so, but using sodium alginate soak times longer than 10s (resulting in spheres that require 0.39N to break), spheres are largely robust against accidental rupture from routine activities such as swallowing.
We also made spheres of different volumes between 0.25mL and 1mL soaked in the alginate bath for 10s, hypothesizing that larger spheres would burst at smaller voltages.Surprisingly, however, we did not observe a repeatable, systematic relationship between popping threshold and sphere size in this range, which we believe is a reasonable balance of having spheres large enough to taste and small enough to be held against the cheek without posing significant discomfort.This may be because popping threshold is more strongly related to alginate bath soak time, and the effect of small variances in soak time from sphere to sphere (from the nature of our manual fabrication and soak time measurement methods) dominate any effect of differences in sphere volume in the range of volumes we tested.
Some liquids, such as cranberry juice, require more guar gum than thicker liquids, such as mango pulp, to achieve the same consistency and result in comparably shaped spheres.Once normalizing for liquid consistency, however, we did not notice a dependence of popping threshold on the type of liquid enclosed in the spheres.We tested cranberry juice, bottled latte (coffee and milk), mango pulp, orange juice, broccoli juice, cola, and 2 different kinds of sports drinks.

Sensitivity of Sphere Placement
We envision Füpop as a system for which the "wearer" will have to place a pouch inside their own mouth.Thus, it is important to understand the tolerable misalignment of spheres with respect to the HIFU transducer.We conducted an experiment in which we placed a sheet of cheek-like hydrogel on an HIFU transducer.We then placed a sphere far away (>2cm) in x-y from the center of the HIFU transducer and incrementally pushed it towards the center of the transducer; meanwhile, we pulsed the transducer with the sphere's popping threshold voltage.We recorded the "maximum tolerable x-y offset" as the distance between the center of the sphere and the center of the HIFU transducer at the point when the sphere burst.We repeated this for 10 spheres of 3 different alginate soak times (5s, 15s, and 30s).Figure 8 plots these results.For 5s, 15s, and 30s soak times, the maximum tolerable offsets were 0.99±0.17cm,1.02±0.15cm,and 0.99±0.13cm,respectively.Thus, while this suggests that some care is needed when aligning the Füpop pouch inside a mouth to the HIFU transducer outside of the mouth, we believe that it is a reasonable misalignment tolerance that can be managed without the need for specialized jigs for reasonably sized spheres.For reference, spheres of 1mL in volume, which is our estimated maximum size that one might want to use to fit multi-flavor pouches comfortably in the mouth, have radii of approximately 0.62cm.Spheres of 0.6mL in volume have radii of approximately 0.5cm, so for a 3-flavor pouch with 0.6mL spheres (i.e.spheres vary only in soak time), the transducer could be almost completely centered over one of the spheres, and we would expect that the 3 spheres would burst sequentially as expected at their respective threshold voltages.Still, more testing, especially in vivo, could help confirm that spheres in a multi-flavor pouch indeed burst in the intended order across different misalignment cases.

Resistance to Dissolution
We tested our materials for their resistance to inadvertent dissolution by soaking them in a bath of real saliva.Spheres across all soak times remained intact for over 24 hours when stored in a bath of saliva at room temperature.We also soaked 3 different wrapping films -potato starch Oblate discs, rice paper, and grape leavesin saliva.Potato starch oblate discs took 15 min to break down, and rice paper took 50 min.Grape leaves did not show any sign of dissolving even after 24 hours.

Effect of Ultrasound Power
Driving the HIFU transducer with longer pulses and higher voltages may result in excessive power consumption from the driving electronics as well as risks of heating the skin to uncomfortable temperatures.We thus limit pulse lengths to 0.25s and excitation voltages to 80V P-P , corresponding to a RMS power of 90W and skin surface power density 24W/cm 2 .This in turn limits the soak time of spheres to 45s, after which point higher powers are needed to pop the spheres.Under these conditions, the artificial skin stays entirely orange, indicating that it does not heat to temperatures above 86 • F. The cheek pouch may also be touched with a finger without any discomfort or pain.As previously mentioned, HIFU is used in medical therapies for multiple seconds at a time with much higher powers [43,60].For tumor ablation, powers in the thousands of W/cm 2 are typically used [15].Even for therapeutic cosmetic treatments for lifting soft tissue, HIFU transducers with frequencies of 2-10MHz are typically operated at 50-400W [59] with no reports of adverse effects or severe pain [82].
Figure 9 shows a series of images of latte spheres with a 15s soak time under different excitation voltages.As voltage increases and approaches the popping threshold, the sphere becomes more misshapen by the ultrasonic excitation until the force is finally great enough to burst it.For the sphere in the figure, we expect that the threshold voltage (from Table 1) is between 47 and 57V P-P .For voltages applied significantly above the threshold voltage, the sphere explodes violently, spewing its liquid in a dramatic spout, as seen in the last image of Figure 9, when 60V P-P is applied.Below the threshold voltage, the ultrasonic energy pushes the liquid in the sphere towards the focal area away from the cheek, forcing the sphere into a pointed droplet shape.When we rest a fingertip on the sphere and activate the transducer below the popping threshold, we perceive the bulging of the sphere as a feeling akin to a light prick.Of course, more quantitative and qualitative data is needed to characterize these pressures and the resulting in-mouth sensations, but we are encouraged by this preliminary data and believe that we may be able to take advantage of the different pressures exerted by the spheres at voltages below the popping threshold to create different in-mouth textural sensations, which could potentially be reproduced repeatedly on the fly over the course of multiple minutes and even hours.

In-Mouth Testing
Here we report on preliminary testing that we conducted on ourselves to verify that Füpop can operate inside a real human mouth.The pouch was placed by opening the mouth, gently pressing on an HIFU transducer placed on the outside of the cheek, and using the resulting indentation on the inside of the mouth as a placement guide -while this likely did not result in exact centering of Figure 10: We burst a sphere of broccoli juice held against a cheek in our mouth with a HIFU transducer held on the opposite side of the cheek, excited with a 32V P-P 0.25s pulse.
the pouch with respect to the transducer, as noted in Section 4.2, spheres may be misaligned by up to 1cm, so this manual, "best guess" method was sufficient.Some care must be taken to ensure that the spheres do not prematurely pop during handling and insertion into the mouth, but once positioned in the mouth against the cheek, the edible portion of Füpop is fairly robust to most normal disturbances.One co-author kept a pouch containing 3 spheres inside their mouth against the cheek for 45 minutes, periodically drinking hot tea and cold water, swallowing, making chewing motions, and talking.The feeling of the pouch was foreign and somewhat uncomfortable at first, and the co-author reported feeling tempted to play with it with their tongue.However, they quickly became accustomed to the sensation within a few minutes and found the pouch largely easy to ignore.At the end of the 45 minutes, no spheres were ruptured, and the co-author did not report tasting leaking from any of the liquids contained in the spheres.This is unsurprising, as the calcium alginate film that forms each sphere is insoluble in water, does not melt at temperatures below 100 • C, and is mechanically robust, being known for its ability to last for days as a wound dressing [37].We made spheres containing cranberry juice, mango pulp, broccoli juice, cola, latte, and various sports drinks, and all co-authors were able to distinguish among the flavors and successfully identify them when they were popped inside the mouth, with only some confusion existing around the artificial flavor of the sports drinks.We again expected this, since the calcium lactate and guar gum added to the liquids are both tasteless and are added in very small quantities, and the liquids we tested are naturally distinctive from one another.
Finally, one co-author placed a single sphere filled with broccoli juice in their mouth against their cheek and centered a HIFU transducer on the other side, pressing the transducer against the outside of the cheek to help align it with the sphere on the inside (see Figure 10).This took some trial and error, suggesting that we may need a more robust alignment scheme in a future user study.Once the transducer was aligned, a 32V P-P 0.25s pulse was applied, resulting in the sphere bursting inside the mouth, as confirmed visually.The user reported tasting broccoli and did not notice any pain, noise, or sensation other than taste to indicate that the broccoli sphere had burst.

CHOREOGRAPHING FLAVOR SEQUENCES
As described in Section 4.1, spheres with distinct enough alginate soak times can be reliably sequentially burst with different HIFU voltages.A designer may thus "choreograph" a sequence of flavors to be delivered for a user by packaging together different flavor spheres each made with different alginate soak times.The spheres do not visually differ from one another besides being different in color, so a user cannot predict what order the spheres will burst in by merely looking at the edible pouch.
An ex vivo demonstration of this is illustrated in Figure 11.We create a 4-flavor pouch containing a mango pulp sphere soaked in alginate for 5s, a cola sphere soaked for 15s, a cranberry juice sphere soaked for 25s, and a broccoli juice sphere soaked for 35s.The spheres are wrapped in an potato starch Oblate disc, with the excess film cut away with a blade.Per the characterization presented previously, we expect the mango sphere to selective burst under excitation voltages higher than 32V P-P but lower than 47V P-P , the cola sphere to selectively burst with voltages 47V P-P to 57V P-P , the cranberry sphere to selectively burst with voltages 57V P-P to 72V P-P , and the broccoli sphere to burst above 72V P-P .Indeed, as seen in Figure 11, when the pouch is placed on our ex vivo testing setup, we use 4 pulses of increasing voltage to burst each sphere one at a time, with the expected order of mango, cola, broccoli, and cranberry flavors being released.At each voltage step, a single colored stream can be observed coming from the packet.Between pulses, we manually check remaining spheres by lightly pressing on them to ensure that they are indeed still intact.
To verify the selectivity of bursting spheres of different voltages and also test the effect of the presence of teeth against the backside of the spheres, we fabricated 20 pouches with Oblate potato starch discs, each containing 3 spheres with 3 different sodium alginate soak times -5s, 15s, and 30s.We placed the pouch on the inside of the cheek of a dental typodont model (an anatomical model mimicking the properties of human teeth and tissue that is used for practice in clinical settings for dental and hygienist students) with denture adhesive (Polident).For 10 pouches, we held the mouth of the model open, and for the other 10 pouches, we kept the mouth closed (see Figure 12).For each pouch, we placed the FUS tranducer on the outside of the cheek and applied pulses of increasing intensity until a sphere burst.We continued increasing the intensity until all 3 spheres in the pouch had burst.The results are plotted in Figure 12.For spheres soaked for 5s, the threshold voltage was 31.8±1.8Vwith the mouth open and 32.5±2.0Vwith the mouth closed.For spheres soaked for 15s, the threshold voltage was 47.1±2.4Vwith the mouth open and 46.6±3.1V with the mouth closed.For spheres soaked for 30s, the threshold voltage was 66.2±3.1V with the mouth open and 66.0±2.7V with the mouth closed.In all cases, the spheres burst in the order expected (i.e. in order of increasing soak time), and there was no significant difference in threshold voltages observed between the mouth open and mouth closed conditions.

Envisioned Applications
This paper focuses on the characterization and implementation of Füpop as a technological framework that can enable real-time interaction with taste and the delivery of food in the mouth.With future in-depth user studies, we believe that Füpop will be a valuable addition to the growing toolbox of technologies that the HFI community has developed to foster play and social connection in the experience of eating [2,51,80,81].We envision that Füpop can enable programmable multi-course experiences (albeit miniature ones) that can either be triggered on their own or along with a normal meal, enhancing or altering flavors.The timing of the ultrasound pulses that release each sphere may be regimented or programmed to be random.With many different variations possible, Füpop is a technological platform that eating experience designers can leverage.
6.1.1Tasty tangible telecommunication.Füpop opens up doors for technologies that stimulate our mouth along with, or in lieu of, the currently more common stimuli to our eyes, ears, hands, and nose.It may be seamlessly integrated with existing hardware commonly used for long-distance communication as part of a novel experience around taste interactions.An illustration of example potential integrations is shown in Figure 13.Headphones and cell phones, for example, are ideal candidates for Füpop, as they are already positioned on or near the cheek when conventionally used.We might imagine that Füpop's electronics can be readily integrated into such hardware or be offered as a clip-on accessory.To offer placement flexibility, the hardware extensions for Füpop may be made adjustable or conformal, and/or temporary adhesives, such as Vetbond Tissue Adhesive (3M), could be used when appropriate to further secure Füpop's transducer.However, since as discussed in Section 4.2, up to ∼1cm of misalignment between the spheres and the HIFU transducer is permitted, adhesives are likely unnecessary.
We envision that partners or close friends could exchange Füpop pouches and send one another choreographed flavor experiences for tangible telepresence interactions or surprising gifts.For example, Alice might gift her friend Bob a package containing a clip-on HIFU transducer for Bob's smartphone, a Füpop pouch, and a document illustrating how to place the pouch inside the mouth and instructing Bob to call a certain number on his phone.Bob secures the HIFU transducer onto his phone, places the pouch inside his mouth, calls the number, and is delighted to hear a voice message from Alice reminiscing on their shared experience during a recent cooking class.Alice has programmed the transducer to sequentially trigger the release of tomato soup, cheese fondue, and chocolate custard spheres, which she made herself with recipes from the cooking class, that are synced to her voice message as she describes each course.Such"taste messages" could be a way to share the sequential experience of exotic flavors and foods encountered during travel, evoke a certain mood, or create a new kind of suspenseful experience for entertainment that are difficult to replicate with non-technological means (i.e.simply telling someone to eat a series of snacks).
6.1.2Multimodality and immersive virtual reality.Similarly, Füpop may find use in virtual and/or augmented reality (VR/AR) applications.Aligning what the user sees with what they taste (or perceive through other senses) can increase their feeling of immersion and presence in a virtual world [85].Conversely, Füpop may even be used to intentionally create sensory misalignment, an effect where a user sees or feels one thing and tastes another.Sensory misalignment can create illusions across different senses that can be taken advantage to craft novel, unusual experiences [40].Whereas previous technologies often use organic materials that are not commonly "food, " even if non-toxic, we present a system which allows immersive technologies to leverage the rich diversity and complexity of the flavors and foods already in our world.While Füpop in its current embodiment does not support re-programming the order and content of flavors on the fly, we might imagine Füpop augmenting guided VR tours that take users through a pre-determined walkthrough of a space, with "real food"-based flavor deliveries via Füpop enhancing the visual experience along the way.
6.1.3Expanded embodiments.While we mostly discuss the possibility of sequential flavor delivery, spheres of the same size and film thickness may be packed into pouches to be popped simultaneously to create the mixing of flavors and potentially even new textures or auditory experiences.Spheres may be simply filled with carbonated beverages that fizz in the mouth, or they may be filled with liquids that react with one another to create dramatic foaming, temperature changes, or other sensations in the mouth that are not strictly based on the stimulation of taste buds on the tongue.With many different combinations of flavors, textures, and other in-mouth sensations possible, Füpop can be used to augment the experience of eating a dish, transforming it into a personalized, dynamic experience that changes with (or even within) each bite.With further development and experimentation, Füpop may also find applications for medicine and personal health.For instance, it could be integrated into taste assessment or retraining protocols Figure 11: A Füpop pouch containing 4 spheres with different calcium alginate film thicknesses.In order of increasing thickness: mango pulp (yellow), cola (brown), cranberry juice (pink), and broccoli juice (green).These spheres may be popped sequentially using increasing ultrasound pulse intensities.for patients who experience a loss of taste due to an infection or nutritional deficiency.
Finally, the technology of Füpop may also be used outside the mouth as well, both within the food "production" HFI domain and beyond food-related contexts.For instance, ultrasound transducers may be mounted to the bottom of plates beneath poppable spheres of interest, enabling playful interactions at the meal table that could even have multiple participants.Focused ultrasonic transducers, coupled with spherificated liquids, could also be used to induce surprising destruction events as part of the "unmaking" of a larger structure, encouraging viewers to reflect upon the temporality and degradability of the materials that they possess [66].

Safety
There are naturally practical safety considerations that need to be carefully characterized with a large-scale user study before the commercial or mass deployment of Füpop or a similar system.For example, the flavor spheres and edible pouch could potentially present choking hazards.Similarly, while we carefully limit the power of the FUS transducer that we use here in accordance with available clinical studies in literature, we ultimately use the transducer in a novel way that differs from documented medical uses.Thus, more rigorous clinical evaluation is needed to ensure that users are not subject to risks like burning and unwanted tissue ablation.Nonetheless, based on our technical characterization, available clinical studies in literature, and personal experience, all of which we discussed previously, we do not believe that Füpop presents more danger than existing research prototypes for influencing taste, smell, or touch.

Limitations and Future Work
There are notable limitations of Füpop and this paper, some of which future work can certainly address, and others of which are perhaps inherent to the approach and must be carefully taken into consideration when designing experiences with Füpop.This paper focuses on presenting a characterized novel technical platform for HFI designers to build upon and explore user experiences with.We acknowledge that a user study is needed to understand how Füpop is in fact perceived by people and how individual variables, such as cheek thickness, may affect the parameters of our system.Deploying Füpop in real human mouths will also be pivotal in testing our hypotheses, based on the ex vivo characterizations presented in this paper, that we can reliably create multi-flavor choreographed experiences with Füpop without specialized alignment jigs.In addition, design probes, trial deployments, and qualitative research could be undertaken to investigate how the interactions we introduce can fit into real contexts and practical situations.
Furthermore, one technical limitation of our system as presented is that while the timing of the bursting of each flavor sphere can be controlled by an external device on the fly, the order in which the spheres burst is pre-determined by the designer, since spheres with thinner alginate films will always burst before those with thicker alginate films.While this still enables the applications we describe above to be implemented to some extent, it constrains a designed application to be choreographed ahead of time and does not support arbitrary flavors being delivered in an arbitrary order.One possible future direction that could potentially overcome this limitation is to investigate how we might leverage different resonant frequencies corresponding to different sphere geometries.A wide-band piezoelectric transducer capable of emitting energy at multiple different frequencies could be used for this.A specific sphere may then be selectively burst with the transducer excited at the resonant frequency of that sphere, enabling spheres to be burst in an arbitrary order.Alternatively, we could make arrays of HIFU transducers that are each centered on specific spheres on the other side of the cheek, allowing specific spheres to be excited and popped simply by activating the corresponding transducer.Arrays, while complicating the electronics need to drive them and perhaps necessitating additional calibration schemes, could also help mitigate transducer-to-pouch alignment difficulties.We may draw inspiration from work in materials science to make such arrays soft and conformable [88].
Additionally, one characteristic of Füpop is that a limited number of flavors is available to a user for a given experience.There are only so many spheres that one can fit comfortably in the mouth and that can exist within a single transducer's focal area; moreover, a sphere cannot be revived once it is popped and its flavor is released into the mouth.In some ways, this is a limitation in contrast to approaches such as Miyashita's concept of electrophoretic "taste displays, " which offer lickable sticks of basis flavors that can theoretically blend to simulate a large spectrum of flavors across arbitrarily long periods of time [45,46].On the other hand, this is a unique characteristic that can alternatively be considered an asset.Unlike electrophoretic taste displays, the ingredients used in Füpop are fully edible and nutritive.This by default lends itself to more authentic flavor deliveries.If the designer wishes to deliver the taste of a cranberry, they can simply make a sphere filled with cranberry juice instead of perfecting and testing the right recipe of basis flavors to concoct a sufficiently convincing substitute.Füpop's flavors do not just offer the taste of food; they are food.Thus, Füpop may be useful in developing playful and positive associations with both the taste and consumption of healthy real vegetable juices, for instance.

CONCLUSION
While the interest in HFI is strong, there have been almost no demonstrations of technologies that can manipulate the sense of taste during the act of eating.This paper offers Füpop, a novel technology that opens the door for computational, "real food" flavor experiences.Füpop utilizes a small focused ultrasound transducer outside the mouth to release a programmed sequence of liquid food spheres that may rest unobtrusively inside the mouth.We have shared ex vivo characterization demonstrating a few of the design parameters of Füpop and also a preliminary in vivo test demonstrating elements of Füpop operating in a real human mouth.As researchers develop more edible, functional materials, we foresee the development of more capable systems that can even exist entirely inside the mouth without the need for external electronics.In the meantime, there are many opportunities to design hybrid systems that build upon advances in not only traditional electronics but also culinary science and gastronomy.We see Füpop as a union of these disparate worlds, and we hope that this work offers inspiration for future designers to develop more such systems to craft novel, more sustainable taste experiences with the rich food cultures and traditions already present in the world.

Figure 2 :
Figure 2: Left: Image of Füpop's focused ultrasonic transducer.Right: Cross-sectional schematic showing ultrasonic energy being focused at a point on the other side of the skin.

Figure 3 :
Figure 3: Füpop's HIFU transducer is driven by a 4.3MHz, 0.25s sine wave pulse.An Arduino provides the electronic trigger to activate a function generator, whose signal is amplified by an RF amplifier before being delivered to the transducer.

Figure 4 :
Figure 4: Füpop's edible cheek pouch encloses multiple spheres of flavored liquid (here, pink cranberry juice, yellow mango pulp, and blue sports drink).It does not readily melt or dissolve in the mouth.It is placed in the mouth between the teeth and cheek, where a HIFU transducer rests on the outside.Edible adhesives may be used to help keep the pouch in place against the cheek.

Figure 5 :
Figure 5: Fabrication process of flavor spheres.(a) A pipette is used to drop the sphere into a 0.5% sodium alginate bath.(b) The drop soaks in the bath for up to 45 seconds.(c) The resulting sphere is removed from the alginate bath with a slotted spoon.(d) The sphere is placed into a clean water bath for storage.

Figure 6 :
Figure 6: Left: Spheres after fabrication may be handled by hand.Right: When squeezed, the calcium alginate film encasing the sphere pops, releasing the internal liquid.

Figure 7 :
Figure 7: Plot of force required to burst spheres vs. soaking time in sodium alginate.

Figure 8 :
Figure 8: The maximum tolerable misalignment of spheres in x-y with respect to the HIFU transducer center is ∼1cm, regardless of sphere soak time.

Figure 9 :
Figure 9: A series of latte (coffee and milk) spheres under excitation with different peak-to-peak transducer voltages.All spheres are 0.8mL in volume with a 14s alginate bath soak time.

Figure 12 :
Figure 12: Bursting threshold voltage for spheres in a Füpop pouch containing spheres soaked in alginate for 5s, 15s, and 30s.Threshold voltage does not depend on whether the mouth is open or closed.

Figure 13 :
Figure 13: Mock-ups of Füpop electronics integrated into a cell phone, an audio headset, and a head-mounted display.

Table 1 :
Effect of Alginate Bath Soak Time on Ultrasound Voltage Popping Threshold