A Touch of the Future: The TOUCHLESS Hackathon 2022

Ultrasound haptics allows us to experience the sense of touch without contact with any physical surface. This novel "touchless" feedback can be used for various use cases but is not widely adopted nor incorporated in everyday products. The 2022 TOUCHLESS Hackathon aimed to enable novel practitioners to learn about touchless technology, generate new ideas, and implement prototypes. We invited participants to a 3-day hackathon in Copenhagen, Denmark, where we introduced touchless technology and provided novel touchless devices for prototyping use cases. Participants were joined by experts on ultrasound haptics, who helped them achieve their prototyping goals. Coming from various educational and national backgrounds, the participants approached the task in different ways and created four unique interactive prototypes. This event report introduces the TOUCHLESS Hackathon and reflects on the lessons learned.


INTRODUCTION
In the field of Human-Computer Interaction (HCI), novel computing devices are prototyped, and the interactions they enable are evaluated.Less common are the adoptions of these novel devices into our everyday life.One reason for the lack of adoption is the lack of access to the research prototypes, especially since a significant time and monetary investment is connected to acquiring, reproducing and implementing them into projects.Thus, it is important to give developers and designers access to novel technologies.
Hackathons have already shown their potential in facilitating the collaboration between technology creators and adopters.A hackathon is a themed, time-limited event (typically ranging from a few hours to a few days) where a group of people intensively collaborates to create a prototype that solves a problem within a particular theme [8,10].Initially thought as a way of bringing together coders and developers with designers and project managers [1], hackathons have developed into a tool for community engagement in biodiversity, future cities, and schools and more [8].Such events have the potential to generate excitement and enthusiasm around technological possibilities [11], which makes them ideal venues to introduce novel technologies.They create a context for toying around with technology which makes it easier for people to understand and think about potential use cases [5], as well as builds a bridge between technology creators and adopters.
One such technology is ultrasound vibrotactile haptics.These devices emit ultrasound waves with force strong enough to vibrate the human skin and thereby stimulate the mechanoreceptors situated in the skin.This type of haptics, or virtual stimulation of the sense of touch, is a relatively novel technology that is yet to be integrated into consumer products.For the scope of this paper, we will refer to this technology as "touchless" technology, as it allows for touch sensations in mid-air.The technology has extensive possibilities for research.It can be used to study the receptors in our skin [2], improve everyday interactions [7], and help us understand the experience of touch [3].
In April 2022, we organised the TOUCHLESS Hackathon 2022 in Copenhagen under the theme "A Touch of the Future"1 .The goal of the hackathon was to introduce touchless technology to a mix of interested people, such as developers, designers, and researchers, to generate ideas and potential use cases.For this, we set the following learning goals for the participants: • Learn about touchless technology • Generate new ideas on how to design and apply touchless interaction • Build, draw and test touchless prototypes in groups To facilitate these goals, we structured a program consisting of learning opportunities (i.e., expert talks and workshops), feedback sessions (i.e., participant presentations and expert assistance), and plenty of time to explore the technology.In this work, we provide an overview of the activities and reflect upon whether they supported the learning goals.

ORGANISATION
The hackathon was hosted at the University of Copenhagen.We invited participants from six universities located in or near Copenhagen (Denmark), London (UK), Krakow (Poland), and Navarre (Spain).In total, 26 participants signed up for the hackathon and were divided into four groups on the first day.We divided participants to foster idea-sharing and collaboration between people of various backgrounds.Each group had two or three people confident in coding in C++ or Unity (C#).The group configurations enabled people who were not confident in the coding to participate in the hackathon.Their backgrounds ranged from designers and musicians to computer scientists and engineers.Additionally, we invited eight academic and industrial experts to guide and assist the participants throughout the weekend.The experts were part of the EU-FET project TOUCHLESS, sponsoring this event.
During the hackathon, the participants were provided with the Ultrahaptics Evaluation Kit (UHEV1), a touchless haptic device (for an overview of touchless haptics, see Rakkolainen et al. [9]).Virtual Reality glasses and 3D printers were also made available in case some project ideas could involve their use.To inspire the participants on the kind of haptic experiences they could create, a box full of materials with different haptic properties was provided.The box contained fabrics, toys made of various materials, compact discs, cassettes, rubber bands, and further items with different haptic properties.
We assumed that the participants would not have in-depth knowledge about the technology at the beginning of the hackathon.Thus, when structuring the program, we gradually built a common understanding of the technology over the weekend.This was facilitated through a mix of research talks, workshops, and independent hacking.We aimed to leave the most time for hacking, as that is what we thought participants would get the most out of.As a means of setting milestones for the participants throughout the event, we asked the groups to present their work during the event, where they received constructive feedback from both experts and other groups.
Food for each meal and snacks were provided throughout the hackathon.We chose to serve vegetarian food for all dinners.Eating together served as a natural point of sharing ideas and progress.
In the following, we will provide a detailed overview of the hackathon.It started Friday mid-day and lasted until Sunday afternoon.

Friday: introduction and getting started
We planned Friday afternoon to be all about getting started with touchless technology.For this purpose, we organised a 25-minute talk motivating the use of touchless technology in everyday life, for instance, for social communication and interaction in automobiles.The talk was meant to inspire and give a high-level overview.
The opening talk was followed by a demonstration session, where the participants had the opportunity to feel the haptic feedback produced by the touchless technology on their own bodies.During this time, the participants asked questions and discussed their first ideas.
Next on the program was a session where participants could freely pick between two workshops.We recommended that the groups split their group members between the workshops, so knowledge from both sessions could inform the hacking process.The workshops ran simultaneously in different rooms.
The first workshop was called "How to Code in Mid-Air".It taught the participants how to do the technical setup of the touchless device and how to write code for it.The participants were provided with examples in C++ and Unity (C#), specifically made for the hackathon.This workshop session was meant for participants adept in coding.
The second workshop was called "The Hedonistic Value of Mid-Air Haptics" and was focused on the design of touchless haptic experiences.The workshop included a design session, where participants were encouraged to come up with pleasurable designs for these experiences.This workshop session was meant for participants wanting to focus on interaction design.
The rest of the day was reserved for a group session, in which we encouraged the groups to explore and brainstorm about the prototype they wished to create.

Saturday: hacking and inspiration
The focus of Saturday was on hacking and group work.Therefore, the only expert presentation of the day was a short 20-minute talk about the design of touchless devices.We had set a milestone for the participants at lunch, where they gave a five-minute talk about their project to receive feedback.During this session, both experts and other participants could engage in the discussion about the prototypes.Afterwards, the participants continued to hack away on their prototypes.
Throughout the day, participants had the option to participate in a user study for a research project using touchless technology.

Sunday: final hacking and conclusion
On Sunday, the final day, participants hacked away and put the final touches on their prototypes.At mid-day, they presented their prototypes to the experts and the other participants in a five-minute talk.After all talks were completed, groups would exhibit their prototype and show how they worked.In the afternoon, all participants received a prize they could take home.The prizes were inspired by haptic interactions, such as a wooden human statue or a fidget toy.

PROTOTYPES
During the hackathon, the four groups each created a prototype.All the prototypes used the UHEV1 device to induce touchless haptics.We here present the four prototypes:

Diddle Engine
We can communicate online through text, video and audio.But what about online touch communication?One group addressed this topic, known as Mediated Social Touch [4], by implementing the "Diddle Engine".The group was inspired by an article on our university's website, presenting a vision of being able to hug friends and family at a distance using touchless technology.The article was included as introductory material for all participants.However, as the intensity and interaction space of the touchless device is limited, the group scoped out other forms of greetings such as hugs, highfives and handshakes.They eventually settled on the lesser-known greeting -diddling.Diddling is a greeting between two people holding out their hand, one above the other, palms facing, and fingertips touching.The group implemented a prototype in which one person could diddle a virtual version of themselves.They used the touchless device, LeapMotion for tracking and implemented it using Unity.Additionally, they considered combining it with  Virtual Reality, but could not complete that due to time limitations.The Diddle Engine prototype is depicted in fig.1a.

Hapticolor
The aim of the Hapticolor project was to be able to differentiate colours using the sense of touch.This would allow users to experience colour in a way and could potentially give people with muted colour perception, due to colour vision deficiency or a visual impairment, a novel way of interacting with colours.The group mapped the colour range to the frequency of a haptic stimulus.
For their demo, they implemented a virtual flower with coloured petals, that users could interact with.They tracked the hand of the UHEV1 using a LeapMotion device and implemented it in Unity.The Hapticolor prototype is depicted in fig.1b, where users can be seen interacting with the virtual flower above the touchless device.

Mutics
The Mutics project aimed to create a music experience with haptics.Through the sense of touch, deaf people and people with other hearing impairments could experience music on their hands.Musical notes, pitch, and rhythm were mapped to different sections of the hand.The index, middle, ring, and pinky fingers were divided into three sections, each representing a different note, where haptic sensations were induced.The thumb represented the pitch on a continuous scale, while the palm was used to give a sense of the rhythm by rendering a haptic circle with varying intensity and size.Mutics could also be used as an additional sense to augment the experience of music listening.The hand was tracked by a LeapMotion device.The mapping design is depicted in fig.1c.

String
Another music-inspired project, the String project, aimed to allow guitar plucking in mid-air.The group created a virtual musical string instrument, where the touchless haptic device provides feedback upon plucking a virtual string.When plucked, the sound was generated through physical sound models, simulating realistic instruments.The sound model was imported as a plugin in Unity, and a LeapMotion tracked users' hands.
These prototypes show possible application areas of touchless haptics, addressing social, accessibility, and technical issues.There are similarities between projects, as both the Hapticolor and Mutics projects worked with the idea of sensory augmentation [6], augmenting the sense of touch to deliver information about colour and music.Both projects aimed to make colour and music accessible to a broader population and augment the user's experience with these two concepts.The projects were commonly inspired by the participants' free-time activities, such as surfing (Diddle Engine) and music (Mutics and String).

REFLECTIONS
In this section, we reflect upon the learnings of creating a hackathon based on a novel technology.We discuss the different initiatives we took to facilitate the learning goals of the event and what were the lessons we learned.

Learn about touchless technology
Our focus on devoting as much time as possible to hacking enabled learning by doing, rather than frontal lectures on technology.
Since the technology and its use cases were completely new to most participants, they appreciated how the workshops were structured.Instead of spending too much time in a classroom setting or having to study the technology on their own, participants could get their hands dirty immediately while having experts in touchless technology to guide their first interactions with it in the workshop.
During the hacking, they also always had experts available in case they had questions or felt stuck.Having multiple parallel workshops, which participants could choose between, allowed them to focus on their interests and learn from their teammates.It also saved time for hacking as we could transfer a lot of knowledge simultaneously to different members of the teams.
Participating in user studies allowed participants to try out expert demos that showcased the technology and inspired them about different domains it could be applied to.

Generate new ideas
The event had a strong focus on collaboration rather than competition to facilitate the sharing of ideas.Instead of declaring a "winner" of the hackathon and handing a 1st prize to that group, we gave all participants a participation gift.This allowed the event to have an atmosphere of working together, with participants openly discussing their ideas across groups and helping each other with minor technical issues.Splitting teams for the workshops and bringing different members together helped increase opportunities for cross-team discussions.Doing different activities together, listening to talks, coding programs for the touchless haptic device, brainstorming applications, and eating dinner together helped the participants exchange ideas and get creative.While this worked well for the event's goals, a drawback of this approach was that recruitment was more challenging without advertising a 1st prize.We believe more participants could have been recruited with a more desirable prize.
Along with communication amongst participants, we also facilitated interactions between experts and participants to inspire new ideas and discussions.This was done by creating opportunities to interact with them, such as participating in user studies, getting feedback on milestones and eating cake together.We believe that small events, such as our hackathon, require the organizers (experts) to be reachable by the participants and engaged with the projects.Engagement does not mean taking over the project, as the participants should remain the driving force, but rather that the organizer should help out when needed and inspire where they can.This ensures that the participants are motivated and feel that their work is valued.This helped participant engagement and their desire to create a prototype.It also enabled the organizers to track progress to see where participants excel or struggle when building touchless haptic prototypes.
An idea that did not work for inspiring participants was to provide a box full of materials with different haptic properties.We hoped participants would touch and play with items in the boxes to try to recreate or enhance the sensations they felt.However, very few participants came up to the box and none of the projects directly used the provided items.We imagine this could be because we needed a more diverse set of participants as most were from a computer science background and focused on exploring the technology rather than how it related to the real world.

Build, draw, and test touchless prototypes in groups
While planning the event, we set milestones for the groups.These were meant to help the participants organise their time and have continuous progress.We did not set specific goals for each milestone, rather we set headlines such as "Presentation of the main idea" or "Exhibition of prototype", leaving it to the groups how they would achieve the milestone.Thereby, we hoped to keep the presentations relaxed and informal while still making sure that the groups were on track.At the end of the hackathon, we organised an exhibition of the prototypes, where participants could discuss their designs with other participants, the experts, and guests from a research lab in Copenhagen.These discussions served as an inspiration for further designs for further research projects.The participants expressed cited this exhibition as a good final mark on the hackathon.

CONCLUSION
We believe in the potential of hackathons to give users access to novel technologies that have not yet found their way to consumer products.With a mix of talks, multiple workshops, and set milestones, our hackathon structure allowed novel users to engage with touchless technologies and produce meaningful prototypes.We also believe that we, as researchers within touchless haptics, learned a lot about the technology by seeing the novices engage with the technology.Thus, we encourage academia and industry to organise more hackathons in which they present their novel prototypes.
(a) Diddle Engine: Social interaction at a distance.(b) Hapticolor: Mapping colour perception to the sense of touch.(c) Mutics: Mapping musical notes to sections of the hand.(d) String: Touchless guitar plucking.

Figure 1 :
Figure 1: The four prototypes created at the hackathon.