Hey, What's Going On?

Highly Automated Vehicles offer a new level of independence to people who are blind or visually impaired. However, due to their limited vision, gaining knowledge of the surrounding traffic can be challenging. To address this issue, we conducted an interactive, participatory workshop (N=4) to develop an auditory interface and OnBoard- a tactile interface with expandable elements - to convey traffic information to visually impaired people. In a user study with N=14 participants, we explored usability, situation awareness, predictability, and engagement with OnBoard and the auditory interface. Our qualitative and quantitative results show that tactile cues, similar to auditory cues, are able to convey traffic information to users. In particular, there is a trend that participants with reduced visual acuity showed increased engagement with both interfaces. However, the diversity of visual impairments and individual information needs underscores the importance of a highly tailored multimodal approach as the ideal solution.


INTRODUCTION
Highly Automated Vehicles (HAVs) hold the potential to revolutionize future transportation, particularly for individuals with vision impairments seeking autonomy and independence [14].Globally, more than 270 million people are affected by severe visual impairments or blindness [7,61], making independent driving unattainable for many of them.HAVs can help these people as a step toward equality and accessibility, as it allows them to travel safely without requiring the assistance of others [5].
While blind and visually impaired people (BVIPs) are looking forward to the advantages of HAVs [5,34], unlike sighted individuals, BVIPs lack the visual cues required to gain knowledge about traffic information and their current situation.Nevertheless, research shows that gaining situation awareness (SA) as a passenger can increase trust and acceptance towards HAVs [18,42,77,80].Given that building trust and acceptance is crucial in automotive interface design [63], gaining sufficient SA and conveying traffic information becomes essential.However, previous research has primarily concentrated on conveying traffic information among sighted individuals.Therefore, these efforts predominantly rely on the visual modality [18,41,50,66,76], which remains inaccessible to BVIPs.
Giudice [37] observed that "the biggest challenges to blind spatial cognition are about insufficient information access" [37, p. 41].This aligns with the need for location verification among BVIPs traveling with HAVs [10,12].Therefore, developing non-visual interfaces that facilitate BVIPs' understanding of their surroundings and traffic information is necessary for achieving inclusive and accessible transportation in the future.
Numerous pedestrian navigation aid systems for BVIPs using tactile and auditory feedback have been investigated.The findings of these studies demonstrate that unobtrusive navigation information is preferred [15].Hence, within the field of auditory interfaces, BVIPs favored querying the system through a keyboard "rather than interacting via a menu or receiving a stream of continuous speech" [3, p. 118].Nonetheless, research found that the passengers' audio channel may be occupied due to non-driving related tasks, which might interfere with other auditory cues, such as radio, podcast, or conversations during the journey [15,78].Further, privacy concerns are raised when audio is played on speakers [23,64].One solution to this issue involves the incorporation of interfaces that leverage tactile information to convey traffic information [78].Brewer and Kameswaran [11] discovered that BVIPs saw "tactile solutions as most appropriate for self-reactiveness towards and self-reflection of their surrounding environment" [11, p. 193].Further, dynamic tactile maps have been investigated for visually impaired pedestrians, indicating that tactile interfaces can enhance their spatial abilities [43,44,79,81].Moreover, first-hand studies with BVIPs suggest tactile feedback to be less distracting and bothersome than alternative interfaces [6,27].
Conveying Traffic Information to BVIPs in HAVs: Introducing OnBoard • 67:3 However, little research has been conducted on tactile interfaces for BVIPs inside HAVs, although it has been shown that vibrotactile feedback can enhance SA during automated driving [32,55].
As most interfaces for BVIPs use tactile or auditory outputs [38], this work introduces OnBoard, an interface with tactile and extendable elements to convey current traffic information to BVIPs in HAVs.Embracing the principles of Participatory Design [58], we included the target group from the beginning of the design process.Hence, we organized a preliminary participatory design workshop (PDW) involving N=4 BVIPs, enabling us to gain valuable insights into the optimal design and functionality of OnBoard and an auditory-only interface, such as the importance of conveying information why the vehicle is stopping (e.g., due to a traffic jam).
Drawing from the insights obtained during the workshop, we constructed OnBoard using 3D printing technology to produce the tactile elements.These elements were combined with electronic actuators to facilitate their extendable functionality.For open-source additional material and fabrication files, see section 7.
Subsequently, we investigated the engagement and usability of OnBoard and the auditory interface in the following exploratory user study with N=14 participants, including individuals with legal blindness and participants with visual impairments.This enabled us to understand the individual needs and preferences dependent on their visual acuity.The experimental setup included a 2 Degrees of Freedom (DoF) motion chair and a 75" monitor to simulate automated driving.The descriptive findings from our study demonstrate that OnBoard showed comparable performance to the auditory interface regarding SA, predictability, and usability.However, OnBoard was rated slightly higher in terms of mental demand.Additionally, we observed a trend suggesting an increased engagement with both OnBoard and the auditory interface as the participants' visual acuity decreases.Nevertheless, individual behavior prevailed across all visual acuity levels, underscoring the importance of highly tailored interfaces for BVIPs in HAVs conveying traffic information.

Contribution Statement
• Empirical study that tells us about people.We conducted an interactive workshop with N=4 BVIPs to understand what traffic information is relevant to them and how to design a tactile and auditory interface that conveys this information.• Artifact.Based on the workshop, we built OnBoard and an auditory interface that conveys relevant traffic information.• Empirical study that tells us about how people use a system.We conducted a user study with N=14 BVIPs to explore usability and engagement with the auditory interface and OnBoard.

RELATED WORK
This research is situated within the context of existing research concerning BVIPs in the domain of HAVs.More broadly, a comprehensive body of literature includes research involving the target group in the development of navigation aids and the exploration of different techniques to convey traffic information in vehicles.

Navigation Aids for Visually Impaired People
Early research by Giudice and Legge [38] investigated the role of technological aids in visually impaired navigation.
They named four factors that should be considered: (1) The mapping of visual information through auditory or tactile output should be well specified and consider the cognitive requirements of users and the time they need to learn the new output.(2) Carefully select the presented information by reducing them to a minimum.(3) Every device has its own advantages and disadvantages to performing under specific situations.Hence, for successful navigation across various situations, it is likely necessary to combine multiple technological aids.(4) For a device to be aesthetically pleasing and usable, it should be minimally intrusive in terms of its form and function.
Approaching to support BVIPs' navigation, a large body of navigation aids was invented and assessed during the past years.Ducasse et al. [28] provided an overview of different tactile interactive maps for BVIPs.They classified them into Digital Interactive Maps, which are shown on a flat surface like a screen, and Hybrid Interactive Maps, which consist of both digital and physical displays.However, when it comes to planning a route, BVIPs perform better using dynamic tactile maps compared to a touchscreen-based map or swell paper [81].Based on this, Holloway et al. [44] investigated the understandability of tactile maps and haptic icons.They found that tactile maps are comprehensible without prior training or experience and can enhance BVIPS' SA for street crossing scenarios [43].Adding on this, BVIPs maintain better spatial relationships using handheld tactile maps with combined tactile and speech feedback, compared to speech feedback alone [79].
These findings highlight the effectiveness of tactile-based interfaces in improving the SA of BVIPs.While most existing strategies focus on pedestrian navigation via maps, the emerging market for HAVs [24] presents new opportunities.Most BVIPs are looking forward to the new autonomy that HAVs will bring and see benefits in traveling to previously inaccessible locations.However, little attention has been paid to accessible interfaces in the development of HAVs yet [12].Nevertheless, qualitative research suggests that interfaces for BVIPs should satisfy their requirements for SA and location verification inside HAVs.In response, Fink et al. [34] emphasized the need for innovative multimodal navigation tools for BVIPs that can be used throughout their journey.This need is consistent with the findings by Brewer and Kameswaran [11] and Brewer and Ellison [10], who involved BVIPs in the development of auditory and tactile interfaces for use in HAVs.

Conveying traffic information in Vehicles
Previous research has investigated augmentation strategies to convey traffic information to passengers' in (automated) vehicles through visual, auditory, and tactile cues.Most of this work, however, has been done in the context of sighted individuals.This section outlines these approaches.

Visual Cues.
Visual cues are frequently used to represent current traffic information in HAVs, such as via a windshield display [26].Research conducted by Colley et al. [18] demonstrated that the visualization of trafficrelevant objects can improve SA among participants.Additionally, it has been suggested that the presentation of future trajectories to passengers can enhance their trust levels [20,41,76].Hence, Schneider et al. [66] used peripherical light bands inside the vehicle to visualize objects and situations on the road.Similar approaches using ambient light showed increased effects on the participant's comprehension of the road situation [47,[50][51][52].

Auditory
Cues.Schoop et al. [67] demonstrated enhanced cyclists' awareness of nearby vehicles by utilizing auditory cues varying in direction, tempo, pitch, and timbre.These cues were mapped to correlate with the direction, distance, and vehicle type detected.In the context of HAVs, Gang et al. [36] utilized this modality to improve passengers' SA by incorporating earcons with 3D sound.Similarly, Nadri et al. [60] conducted a study that compared the efficacy of auditory and visual feedback and their combination in HAVs.They found that audio-visual feedback led to an increase in participants' SA compared to a visual-only representation of relevant traffic objects.This study also distinguished between the effectiveness of verbal audio and auditory icons, aligning with Glatz et al. [39], who found that auditory icons are effectively perceived for contextual information while verbal audio is more suitable for time-critical information.For BVIPs, Brinkley et al. [13] developed a prototype that enhances their SA inside HAVs through the use of audible location cues and spatial audio.Among others, they found that their system increased trust, SA, and perceived reliability.

Tactile Cues.
In contemporary vehicular settings, tactile cues are widely employed at the steering wheel to gain drivers' attention.For example, the study by Borojeni et al. [6] compared the effectiveness of shape-changing steering wheels and vibrations to convey takeover requests (TORs) of HAVs.Their results indicate that vibration is preferred for urgent requests, while shape-changing is preferred for less critical TORs, as they are perceived to be less annoying.Moreover, the authors recommend the use of on-body tactile cues to provide context information.This concept was examined by Chiossi et al. [16], who employed tactile cues affixed to the participants' bodies.They concluded that "tactile notifications can [..] be considered a promising and viable alternative to SA support in automated vehicles" [16, p. 6].Further, two studies conducted by Md.Yusof et al. [55] and Sonoda and Wada [70] measured an increase in SA and trust as they used vibrotactile wristbands to provide cues for the ego vehicle's turns and the presence of other vehicles approaching from the rear.Fink et al. [32] investigated SA through an ultra-sonic device in the context of HAVs.Their user study involved blindfolded people and found that the interface supported their SA.Hence, research implies using this system for BVIPs inside HAVs.In their follow-up research, Fink et al. [33] evaluated a multimodal interface (gestural-audio and haptic cues) among BVIPs and found that they were able to navigate driving scenarios effectively using gestural-audio, indicating points of interest via clock face position gestures.Further, haptic mid-air (ultra-sonic) was used to represent intersections and roads.However, they mentioned that "future work is necessary to explore how haptic cues can be successfully implemented in FAVs [fully automated vehicles] [...]" [33, p. 12].
Focusing on this statement and the need for non-visual interfaces [11], this research's goal is to develop a system that conveys traffic information to BVIPs inside HAVs.However, a significant issue with the design of most navigational interfaces for BVIPs is the insufficient attention given to the users' needs [15] and the limited focus on accessibility in HAV development [12].Therefore, prior to the development of such an interface, we conducted a PDW [58] with BVIPs to gain insights into their information needs regarding conveying traffic information inside HAVs.Based on the implications of this workshop, we then developed and explored an auditory interface and OnBoard-a tactile interface.
While Fink et al. [33] concentrated on controlling the HAV, such as altering the route and gaining knowledge about infrastructural points of interest (e.g., bakeries or coffee shops), this work aims to extend the application of tactile interfaces to convey traffic information.Within this work, we aim to understand how BVIPs would design tactile cues inside HAVs that convey traffic information.While related work showed that tactile cues help to increase SA inside HAVs [16,32,33,55,70], there is a research gap exploring how BVIPs would design tactile interfaces tailored to their individual needs, which is addressed by this research.

PARTICIPATORY DESIGN WORKSHOP
Following the participatory design approach [58], we conducted an interactive workshop with N=4 participants (male=2, female=2; Mean age=56.75,SD=8.47 range 43 to 66, see Appendix B for further details) to gain insights into information needs regarding conveying traffic information to BVIPs inside HAVs.Further, the preferred shape, position, and functionality of tactile elements for specific traffic information were discussed and explored during an interactive design session.

Procedure
The PDW, guided by four of the authors, was scheduled to last for three hours, with the planned agenda detailed in Table 1.After a short introduction of all attendees, a concise overview of automated driving capabilities was given to simulate the scenario of a journey with an HAV.Hereby, the participants were asked to imagine riding in an HAV where no intervention is needed to bring them to their desired destination, as suggested for HAVs with SAE levels 4 and 5 [65].
The workshop was divided into three main parts.First, the BVIPs were encouraged to start an open discussion regarding different traffic information relevant for them to have knowledge about (30min).Given that all participants were familiar with traveling as a passenger in a vehicle, subsequently, they were asked to rank these identified traffic information by relevance during group discussions (30min).Hence, the participants gave a score between 0 (not relevant at all) and 10 (highly relevant) to each identified traffic information.If there was Open Discussion about BVIPs' relevant traffic information to be conveyed 30 min Ranking of the proposed traffic information 90 min Interactive design session of tactile interfaces for the three highest-ranked traffic information disagreement among the participants about a score, we averaged it.Approaching a holistic view of all relevant information in the context of BVIPs, prior to the workshop, related literature [11,12,14,55,70] was consulted to identify additional relevant information that could potentially supplement those generated by the workshop participants.This traffic information from related work was ranked alongside those identified by the participants.
After ranking the information's relevance, the highest-ranked traffic information was (1) orientation at the destination, (2) distance/time to the destination, (3) reasons for delays, and (4) reasons for the vehicle stopping (refer to Table 2).Given the close similarities, reasons for delays was merged with the distance/time to the destination as the reason for a delay affects the time until the destination.
For the following interactive design session, these three traffic information were approached, revealing 30 minutes each for the participants to design and prototype their preferred tactile cues.The participants were divided into groups of two (group 1: P1 & P3, group 2: P2 & P4).Each group was given a wooden board (30cm x 21cm) covered with white velcro strips Figure 2a) and a set of 3D-printed tactile and stickable elements.These elements were shaped as follows: circles, triangles, squares, hexagons, and a semi-cylinder.Further, as a representation of the ego vehicle, we provided three different elements: a simple shape of a car, a tetrahedron, and a triangle with a knob at one corner (see Figure 2b).Moreover, the participants could choose whether to securely affix or stick the tactile elements in a rotatable manner on the board or position them onto a movable slider if desired.
For the three highest-ranked traffic information, each group was asked to design and prototype an interface using tactile elements to convey traffic information inside HAVs.During this session, participants were asked to explain their design decisions following the thinking-aloud method.
For the time they spent during the 3-hour PDW, the participants were compensated with 30 Euros.

Method
A reflexive inductive thematic analysis, similar to Braun and Clarke [8,9], was conducted by four authors directly based on the PDW's audio and video recordings.Subsequently, the codes generated from the analysis were collected and organized on a digital whiteboard.To increase reliability among the codes, the two researchers who supervised one group of participants coded the recording of the other group and vice versa, thereby facilitating cross-validation of the coding process.Following the coding phase, the authors convened in a meeting to cluster the codes.In cases of disagreement, they engaged in discussions to resolve any discrepancies.In total, 56 codes were generated, resulting in 9 clusters, which were grouped into three main themes, including the participants' opinion to tactile interfaces for BVIP inside HAVs, design preferences for tactile interfaces for BVIPs and their most relevant traffic information.

Results and Implications
Drawing on the clustering, the findings are organized into three sections: first, participants' views on tactile cues for traffic information in HAVs; second, general design preferences for these cues; and third, key traffic information and development of tactile representations.

Participant's Opinions on Tactile Interfaces in Highly Automated
Vehicles.Participants P3 and P4 appreciated the potential opportunities of a tactile interface as it empowers them to gain information on-demand via direct physical touch.P3 specifically highlighted the convenience of being able to touch the display when information is required while being able to refrain from touching it when no information is required.Unlike the auditory modality, which was associated with information overload, the tactile display was presumed to be less disruptive.
In particular, three participants imagined less critical information to be bothersome when conveyed through auditory cues.However, they also mentioned that when it comes to critical situations or important information, auditory verbal feedback is preferred over tactile feedback as audio was assumed to convey more context information, which aligns with Glatz et al. [39].According to the participants, important information includes that the vehicle arrived at its destination and notifications of dangerous situations, such as a bike lane next to the arrived vehicle.Further, any kind of emergency was preferred to be conveyed via audio.

Design
Preferences for the Tactile Interface.During the interactive design session, participants also discussed general design preferences for tactile interfaces inside HAVs.Although many public interfaces for BVIPs convey tactile information via braille, the participants explained that not everyone can read it.While related work is not able to identify the exact percentage of individuals who are able to read braille [68], the National Federation of the Blind estimated that less than 10% of people who are blind in the USA are able to read it [45].Hence, using braille is not optimal in terms of accessibility for all BVIPs.Further, P3 (who can see contours) mentioned that high contrasts are vital to see the elements on the interface roughly.Further, she explained that too many colors would be irritating.
Regarding the shape of the tactile elements, participants' preferences were ambiguous.While two of them favored a realistic shape, e.g., for the ego vehicle, the other two BVIPs expressed a preference for simple geometrical shapes as they are easier to comprehend.They raised concerns about the potential for information overload if multiple realistic shapes were presented simultaneously.Nonetheless, all participants concurred that OnBoard should minimize the complexity of the traffic information rather than attempting to provide a holistic overview, which aligns with Giudice and Legge [38].Therefore, they mentioned that the most important elements should be positioned in the interface's center, while less important elements should be positioned at the rim of the board.During the discussion on the orientation of the board, P3 and P4, who were legally blind, favored using the board with both hands.In contrast, P2 expressed a preference for using only one hand.In general, there was a preference to implement every tactile element as a button providing additional information via audio when pressing.

Relevant Traffic Information.
In terms of pertinent traffic information, the participants ranked orientation at destination, distance/time to destination, and reason for the vehicle stopping as the top three most relevant, excluding reason for delays due to merging (see subsection 3.1).While the traffic information ranked from 1-5 was independently identified by the workshop participants, and situations 6 and 7 were derived from prior research (refer to Table 2) and were included for ranking alongside the other situations.The ensuing paragraphs will delve into the participants' design recommendations for each of the three traffic information that obtained the highest rankings.
Table 2. Rating (1-10) of the traffic information by the participants.If there was disagreement among the participants, the score was averaged.(Individual scores in brackets).While the first five pieces of traffic information were mentioned by the workshop participants, the last three pieces of information were derived from related literature (indicated in the last column).
ranking rating traffic information source Orientation at the destination.In regard to the orientation at the destination, participants expressed the significance of being informed about their spatial orientation with respect to the destination.For their interface design, both groups decided on using tactile elements guiding the path toward the destination (refer to Figure 3a and Figure 3d).Further, Group 2 highly emphasized the importance of obstacles when exiting the vehicle, such as moving vehicles or bicycles.Consequently, a hexagonal button was used to indicate the presence of potential obstacles when exiting the vehicle (Figure 3a).In addition, the path to the destination was tactilely represented, accompanied by the use of a house-shaped icon (composed of a triangle and a square) to symbolize the designated endpoint.Further, they proposed an information button to present any kind of situation-specific information via audio (see Figure 3a).While the tactile elements were beneficial in enhancing destination orientation, all participants agreed on the necessity of conveying information regarding the vehicle reaching the destination via audio.
Distance/time to destination.For the distance/time to the destination, both groups used a slider element to indicate the process of the journey (Figure 3b and Figure 3e).P2 specifically mentioned that the tactile representation of the slider facilitated the perception of the estimated remaining distance until the slider reached its final position.Therefore, no additional audio would be required.However, Group 1 introduced a button representing the final destination, which, by activation, triggers an audio announcement providing information regarding the remaining distance and the estimated time of arrival (see Figure 3b).Further, independent of each other, both groups stated that delays should affect the progress slider by moving slower.In this regard, the participants prioritized receiving information about the time delay rather than the specific reasons for any detours taken by the vehicle.Reason for the vehicle stopping.P3 emphasized the importance of knowing the reasons for the vehicle stopping, e.g., if there is a traffic jam or cross-way.However, some other participants expressed their discomfort with receiving constant notifications for each stop, including traffic lights, particularly when conveyed audibly.Therefore, they suggested using situation-specific buttons that can be pressed to provide additional information about the specific reason for stopping.P2 expressed a preference for this button, as it would grant him the agency to decide whether he wishes to access the information.In terms of design, both groups used a moveable object blocking the way of the Vehicle; however, the chosen shape of this object varied as depicted in Figure 3c and Figure 3f.All participants, however, stated that if there is an emergency situation for stopping, such as a burst tire, the reason for the stop should be conveyed directly via audio as this information is urgent.

ONBOARD
Designing OnBoard followed the Participatory Design Approach [58] by taking into account the insights gained from the PDW's participants.This section will describe the final design decisions for OnBoard and introduce the interface implementation.

Design Requirements
In line with Giudice and Legge [38], an often-mentioned statement during the PDW was that the interface has to be simple and not overwhelming in terms of shapes.Building upon this, the most important information of the interface was preferred to be positioned in its center, while less pertinent information should be positioned towards the periphery.Additionally, one workshop participant mentioned that it is crucial for people with remaining visual acuity to have the tactile elements presented in highly contrasting colors, as also highlighted by Holloway et al. [44].Therefore, we kept the contrasting colors of the wooden boards and 3D-printed elements during the interactive design session of the workshop and designed OnBoard with a white background, while the tactile elements were black.

Conceptualization and Implementation
For each of the three highest-ranked traffic information, both groups created a design that we used to build the final version of OnBoard (see Figure 4).While the turns of the vehicle were mentioned as non-relevant for BVIPs, Md.Yusof et al. [55] and Karjanto et al. [47] showed that representing turns of the vehicle, either visually To represent the distance/time to the destination, we used a triangular element that moves along a progress slider as shown in Figure 4. Above the progress slider, we positioned a house-shaped element (as suggested by group 2 during the interactive design session) that incorporates a slightly raised, touchable "i" symbol, which serves as a button for progress information.To enhance the symbol's visibility beyond its tactile features, we applied white paint to provide a stark contrast against the black house-shaped element.As highlighted by Holloway et al. [44], the "i" symbol is the most intuitive for BVIPs as a symbol of "information." Upon pressing this button, a verbal auditory cue is triggered, providing the approximate distance in kilometers and estimated time in minutes until their arrival.
Conveying the vehicle's stopping action, we positioned a hexagon-shaped element in front of the vehicle as proposed by the workshop participants.This hexagon is designed to ascend and break through the surface of the interface board when the vehicle is approaching a stop.Once the hexagon is lifted and becomes touchable, it transitions into a functional button, offering a tactile element for user interaction.By pressing the element, it provides verbal information about the reason for stopping.When the vehicle accelerates again, the stop-sign moves back aligned with the surface.
The workshop participants expressed a preference for auditory cues to confirm that the vehicle had arrived at its destination.We incorporated this feedback into the final design, with a verbal audio announcement stating "you have reached your destination" triggered once the vehicle completes its journey.For orientation at the destination, we used the tactile approach inspired by the participants' designs.Hence, we used four elements that can ascend beyond the surface once the vehicle reaches its destination and transform into buttons.These elements were strategically positioned on the right and left sides of the vehicle's triangular representation (see Figure 4).The arrow-like shapes represent the recommended direction for exiting the vehicle.Meanwhile, the rectangular elements denote any potential obstacles on either side of the vehicle.Once reaching the destination, the left or right arrow ascends, depending on the recommended exit direction.Simultaneously one or both of the rectangular elements rise if there are potential obstacles alongside the vehicle.Pressing the arrow button triggers verbal audio to convey the recommended direction.Similarly, pressing one of the obstacle buttons activates audio feedback that notifies the passenger of any obstacles, such as passing cars, on either the right or left side.Additionally, upon reaching the destination, OnBoard's triangular representation of the vehicle is pointing toward the destination.
For the implementation, we 3D-printed the tactile elements, including the mechanism to lift each of the five tactile elements individually.Lifting these elements was achieved by five SG90 micro servo motors that could rotate to push each element upward.By rotating back, the elements moved back to their original position.Additionally, we used tactile push buttons attached to the lifting mechanism to detect if the tactile elements were pressed.By including a mechanical limit, the button functionality of the extendable elements was only active when extended.Further, for the vehicle's triangular representation, another servo motor enabled controlled angular rotation.The progression of the progress slider was achieved through the utilization of a stroke 80mm linear actuator, which smoothly moved the triangular shape along the stroke.In terms of providing verbal audio feedback, we used the freely available online software version of Text to Voice (2023) [72] to convert text into voice.The electronic components were controlled by an Arduino Mega microcontroller [4] and powered with an external power source.Furthermore, we embedded all mechatronics into a wooden box (30cm x 22cm x 22cm) to enhance durability.Further, the top face was laser-cut to accommodate the tactile elements.For a more comprehensive understanding and additional details, we invite readers to refer to the supplementary materials available in our publicly accessible repository, as indicated in section 7.

USER STUDY
To explore OnBoard with respect to conveyed traffic information, predictability, perceived safety, trust in automation, usability, and users' engagement, we conducted a within-subject user study with N=14 participants.
Auditory Interface.Chanana et al. [15] stated that auditory cues, besides tactile cues, are among the most used output modalities for BVIPs' interfaces.Therefore, along with OnBoard, we also explored an auditory-only interface conveying traffic information.The interface's verbal auditory cues were synchronized with the same key events as those triggering OnBoard.For instance, instead of the vehicle's triangular representation rotating on a turn, the verbal audio "turning right" is played.This ensured the exploration of both interface modalities based on the workshop results.According to Arditi and Tian [3], BVIPs prefer manually triggering audio cues instead of passively receiving a constant stream of information.Therefore, we implemented a button that switches on the auditory feedback when pressed.On releasing the button, the verbal auditory feedback stops.The button was embedded with a 3D-printed case to ensure the participants could hold it in their hands comfortably (see Figure 5).

Study setup
A visual stimulus was provided for the BVIPs using a 75" 4K monitor displaying a simulated journey with an HAV through an urban area from a passenger's ego perspective.For implementation, we used Unity version 2021.3.9f1[75] along with the Fantastic City Generator asset [54] and the Urban Traffic System [1].We ensured that this simulated traffic environment was familiar to what the participants experienced in their daily lives.During each simulated trip on the route, which lasted about 7 minutes on average, several key events were incorporated to simulate different traffic situations, including six turns, one stop at a traffic light, one roundabout, one stop at a crosswalk, one stop at a stop sign, and one instance of traffic congestion (see ??).The Unity simulation and OnBoard were connected via a serial USB interface.On the one hand, when the HAV in the simulation triggers one of the above-mentioned events, OnBoard reacts accordingly.On the other hand, when users press any of the tactile elements, the audio feedback is played through the computer's speakers, as OnBoard itself is not equipped with audio capabilities.
To enhance immersion (especially for participants with legal blindness), we positioned a 2 DoF motion chair [69] in front of the monitor.We used this chair to physically simulate the ride with the HAV using gentle roll movements to simulate the vehicle's turns and pitch movements to simulate breaking and acceleration.OnBoard was mounted between the participants' legs as depicted in Figure 1.
The further setup contained a camera-facing OnBoard to record the participants' touch interactions with it and a microphone recording qualitative feedback.

Procedure
For each study session, we described the setup in the study room and asked for their consent to record the study.We made sure they understood everything and could ask questions.Subsequently, we read the consent form aloud, which followed the research institute's ethical guidelines, emphasizing their right to stop the study at any time.The procedure further ensured privacy protection, anonymization of data, fair compensation, and risk aversion.Recognizing the special needs of our participants, we extended our ethical responsibilities beyond standard practice by providing personalized assistance, such as help with transport, to reflect our commitment to high ethical standards in research.
After their agreement, the BVIPs were introduced to the 2D motion chair and invited to take a seat.We asked them to imagine being passengers in an HAV, bringing them to their desired destination without any intervention needed [65].The scenes for OnBoard and auditory interface were then presented in counterbalanced order, each starting with a 2min introduction scene (including four turns and one crosswalk) to overcome potential novelty effects [74].The participants could redo the introduction scene as often as they liked until they felt confident they understood the functionality of OnBoard and the auditory interface.Subsequently, the participants were introduced to the 7 minutes' main scene.Following the main scenes of the auditory interface and OnBoard, we gathered demographic data such as age, gender, and visual acuity.Eventually, we engaged in a qualitative conversation regarding the participants' preferred interface and the reasons behind their choices.
The participants were compensated for the 1.5h session with 18 Euros.

Measurements
After each interface, the participants were asked to rate the auditory interface and OnBoard using the System Usability Scale (SUS) [46].Further, we measured the mental demand for each interface using the NASA-TLX scale [40].To evaluate the perceived quality of SA [29], we employed the Situation Awareness Rating Technique (SART) [71].An additional aspect examined was the participants' perceived safety, which was measured utilizing a set of four 7-point semantic differentials, ranging from -3 (anxious/agitated/unsafe/timid) to +3 (relaxed/calm/safe/confident) [31].Additionally, trust, which encompasses both psychological and physiological elements, was measured using the subscales "Predictability" and "Trust in Automation" from the "Trust in Automation" questionnaire by Körber [48].All questionnaires were read aloud to the participants.Further, we logged the engagement of both OnBoard and the auditory interface via Unity and a camera positioned above OnBoard filming the participant's interaction with the board.For OnBoard, we distinguished between sensing the tactile elements and pressing the functional buttons.For the auditory interface, we recorded how long the participants held the button to request auditory cues.After both interfaces, we used the Immersion subscale of the Technology Usage Inventory (TUI) [49] to ensure sufficient immersion was reached.
Eventually, the participants were asked to report their preference for either the auditory interface or OnBoard.Further, we encouraged them to state the reason for their preferences to collect qualitative data.

Participants
The participants were recruited from the prior participatory workshop, a local association of blind people, and the local self-help organization for visually impaired people.On average, they were M = 60.07 years old (SD = 12.46).The distribution between female and male participants was equal.No participant selected "non-binary" or "I prefer not to answer" for the gender question.Immersion [49] in the scenario was rated medium to high (M = 16.57,SD = 5.79; minimum possible: 4, maximum possible: 28).Among the participants, six individuals self-reported visual acuity of less than 2%, while three participants reported visual acuity of less than 5%.Appendix B shows details about the participants' demographics and their visual acuity.According to the WHO categories of visual acuity [35], these participants fall into the category of legally blind.In addition, one participant reported severe visual acuity (below 10%), while four participants reported moderate visual acuity (below 30%).In the case of different visual acuities for the right and left eye, we averaged the values.

Results
Considering the exploratory nature of the user study, we did not define any hypothesis.Moreover, because our study was centered around a highly specific and difficult-to-reach target group, we prioritized comprehensive participation over the recruitment of a large sample size.This was guided by the premise of understanding the experiences and unique challenges of these individuals rather than treating them as a homogeneous group.Hence, we did not perform hypothesis tests and focused on descriptive analyses and qualitative data.5.5.1 Questionnaires.The descriptive results (see Figure 6) indicate that the ratings for all taken measures are relatively similar between the auditory interface and OnBoard.However, the mental demand of OnBoard was rated higher than for the auditory interface, while the SA for the auditory interface was rated higher than for OnBoard.However, both interfaces yielded a low mental demand below 5.While both interfaces were rated high for predictability, perceived safety, and trust in automation, the SA, according to the SART scale, was rated medium to low.In contrast, the usability of both systems was rated high, whereas OnBoard yielded a slightly higher usability rating.All detailed descriptive data is available in Appendix A.

Participants' Engagement.
We tracked the participants' engagement with OnBoard and the auditory interface, such as sensing the tactile elements and pressing the buttons in relation to the simulated events during the ride.The participants' engagement is depicted in Figure 8.However, we did not analyze the engagement with the tactile elements used for orientation at the destination, as these buttons were only activated at the end of the ride.However, they are further assessed during subsubsection 5.5.3.We distinguish the interaction into sensing the tactile elements and pressing the buttons.For the auditory version, we tracked when the participants held the button to receive the audio (see Figure 8).
We fitted a linear model (estimated using Ordinary Least Squares) to predict the engagement with the visual acuity.The R 2 values for OnBoard are relatively low, predicting a negative effect on the visual acuity (progress slider: R 2 =0.02, button for progress information: R 2 =0.18, button reason for stopping: R 2 =0.24, vehicle representation: R 2 =0.19).The R 2 =0.41 for the auditory interface indicates a negative effect between visual acuity and engagement with the auditory interface.In general, the findings indicate a small trend that with decreased participants' visual acuity, there was increased engagement with both the auditory interface and OnBoard (see Figure 7).Whereas the visual acuity had a stronger effect on the engagement of the auditory interface than on OnBoard.However, this trend does not apply to all participants, as even individuals with higher visual acuity touched the tactile elements quite frequently (see P7 and P8, Figure 8).Individual behavior prevailed, and variations were observed among participants regardless of their visual acuity.Participants are sorted by their individual visual acuity.The total engagement assumes that with decreased participants' visual acuity, there was increased engagement with both the auditory interface and OnBoard.However, there is individual behavior, regardless of the participants' visual acuity 5.5.3Qualitative Data.After completing both interface scenes, participants were asked to indicate their preferred interface and were encouraged to provide feedback for their decision.
Overall, the results revealed that five out of 14 individuals preferred OnBoard over the auditory interface.According to the participants, this tendency is due to the variating visual acuity.For instance, P7 remarked, "since I can still see, I don't need OnBoard.I can rely on my sight.It is rather advantageous for those who are completely blind".This sentiment was shared by P3 and P5.Further, P10, who is legally blind, pointed out that OnBoard allows people who cannot see to perceive anything tangible.Due to her familiarity with reading braille and her preference for tactile information, she favored OnBoard.
In general, both interfaces received highly positive responses from the participants.All participants expressed their ability to comprehend the relevant information conveyed by both interfaces.P6, who favored the auditory version, highlighted the convenience of leaning back and simply pressing one button rather than having to search for the elements on OnBoard.This preference aligns with the feedback provided by four participants, who criticized the size of OnBoard.In particular, P8 pointed out that "it would be better to have everything (all tactile elements) in hand.Otherwise, I have to search for them".Furthermore, P4 suggested the rectangular tactile elements conveying obstacles upon reaching the destination should be triggered earlier for better preparation.Other participants emphasized the need for more detailed auditory descriptions of the obstacles' locations, expressing a desire to know exactly where to pay attention when exiting the vehicle.
However, P12, who preferred OnBoard, expressed appreciation for the selective accessibility of information, stating, "I am not bombarded with information, but I can access it when I want to".P4 shared a similar sentiment, appreciating the flexibility to access information according to their own preferences with OnBoard.This is in line with P7, who criticized that for the auditory interface, he could not select information like with OnBoard.Instead, he had to wait until the relevant information was played.
Conversely, participants P3, P8, and P11-14 found the auditory version more user-friendly and less demanding, as it only required using a single button to switch the audio on or off.P12 and P14 expressed their comfort in being able to activate the audio when needed and deactivate it when not necessary, which is in contrast to the statement of P4, who disliked the auditory information being predefined, lacking the option to choose which specific information to receive at a given time.

DISCUSSION
This research was driven by the need for non-visual interfaces for BVIPs in HAVs to convey traffic information [11,15].Its primary objective was to enable these individuals to receive relevant information regarding ongoing events during their journey, as suggested by Brinkley et al. [12] and Brewer and Ellison [10].We adopted the participatory design approach [58] by including the target group at the beginning of the design process.Hence, we conducted an initial interactive workshop with N=4 BVIPs to understand their needs and design preferences for a tactile interface to convey traffic information inside HAVs.
The workshop's insights suggested an interface with simple-shaped tactile elements adapting to the current traffic Based on the workshop's implications and the participants' preferences, we built OnBoard to convey traffic information to BVIPs inside HAVs.In a within-subject design user study with N=14 participants, we explored OnBoard and an auditory-only interface and investigated their effects on participants who are blind or visually impaired.For the user study, we simulated a ride with an HAV using a 2 DoF motion chair [69] and a 75" monitor.While the quantitative data reveal similar effects of both interfaces in terms of perceived safety, situation awareness, prediction, trust in automation, and usability, OnBoard yielded a higher mental demand than the auditory interface.

Tactile and Auditory Cues to Convey Information in Highly Automated Vehicles
The results of the user study imply that there is a trend that the engagement for non-visual information increases with a decrease in visual acuity.This finding is in line with Mahadevan et al. [53] and Colley et al. [22], indicating that visual cues are still important for BVIPs if they have remaining visual acuity.We observed that participants with lower visual acuity showed more engagement with the interactive board and the auditory interface than BVIPs with higher visual acuity.However, while the overall participant engagement in Figure 7 and Figure 8 and the qualitative feedback supports this trend, it is important to consider the possibility of novelty effects in the user study [74], wherein participants explored both systems regardless of their specific needs or preferences.In this regard, we also assume that over time, as users adapt to the interfaces [56], these effects decrease, allowing for a more accurate assessment of the interfaces, including potential learning effects.
The low R 2 values observed in the linear regression models predicting users' engagement based on their visual acuity highlight the likelihood of variations in this trend highly depending on individual information needs and preferences.Several participants a higher need for understanding surrounding traffic information than others, irrespective of their visual acuity.This observation is due to the complexity of visual acuity, as participants reported individual visual impairments, including differences in acuity between the eyes and differences between foveal and peripheral vision.For instance, P8 mentioned that he has less than 2% foveal acuity but can still distinguish between light and dark, which aids in orientation.In addition, factors such as whether participants were born visually impaired or acquired their impairment later in adulthood may also influence their specific information needs.Thus, the individual information needs of BVIPs in HAVs must be considered for future highly tailored interfaces.

Gained Situation Awareness
While the scores for perceived safety, predictability, trust in automation, and usability point to the effective usage of the interfaces, the SART questionnaire's low ratings for SA in both OnBoard and the auditory interface present a contrasting picture.Nonetheless, the qualitative feedback suggests that the participants comprehended all relevant traffic information conveyed by the auditory interface and OnBoard.This inconsistency was also observed by Md.Yusof et al. [55], where participants were able to understand the tactile cues indicating future turns of the HAV, yet their SART rating remained comparably low.On the one hand, Endsley et al. [30] mentioned that low ratings for the SART questionnaire are due to participants' inability to accurately self-report their SA, as they may not be aware of the specific information they are missing.This is particularly true for BVIPs, as they poorly receive visual information.On the one hand, the observed discrepancy may indicate that the BVIPs can comprehend the conveyed tactile and auditory cues; however, they need more information to gain holistic SA of the traffic.

Balancing Control and Information Accessibility
Fink et al. [33] found that BVIPs desired access control for manipulating driving tasks inside HAVs, such as start/stop behavior and pulling over.Contrary to this, our research implied that passengers inside HAVs may not be able to take control of driving tasks.While Fink et al. [33] also investigated how BVIPs could gain SA in HAVs through a multimodal interface, its focus was primarily on infrastructure-related information, such as the number of roads ahead or nearby facilities, leaving traffic information unaddressed.
Whether BVIPs should control driving tasks is controversial, as our results show comparatively low ratings of SA with the use of the OnBoard and the auditory interface.We did not investigate whether the conveyed traffic information would be sufficient for BVIPs to control driving tasks.However, even existing HAV designs for sighted individuals exclude direct interaction with the vehicle [17, 19-21, 59, 82], underscoring the concern that allowing BVIPs to take control of driving tasks may be imprudent.

Practical Implications and Future Work
The diverse preferences expressed by participants regarding both the auditory interface and OnBoard indicate the potential integration of both systems into future HAVs.Based on this finding, we see potential in combining the OnBoard's tactile cues with the auditory interface, as Yatani et al. [79] already suggested combining tactile and audio modalities to enhance navigation for BVIPs.
Participants particularly valued the selective on-demand functionality of OnBoard.However, previous research indicates that BVIPs might have reservations about auditory cues, as their auditory channel may be occupied with non-driving activities such as conversations, and privacy issues may arise when played over speakers.[15,23,64,78].Therefore, we make the following recommendations for future design: By incorporating multiple buttons with different auditory information channels (such as future turns, the reason for the vehicle stopping, etc.) that can be selectively activated or deactivated, the combination would leverage the advantages of both systems.Furthermore, during the PDW, participants emphasized a high need for interfaces that support gaining knowledge about potential obstacles when exiting the HAV.Therefore, we recommend that future accessible interfaces inside HAVs consider this scenario; future research should delve deeper into the effective communication of this information.

Limitations
The PDW involved only four BVIPs.While this does not necessarily compromise validity [73], it is essential to acknowledge that the opinions expressed by these participants represent only a small subset of the target group.Moreover, the design of OnBoard relies on the subjective opinions of these four participants.Although we also considered related work to build both the auditory interface and OnBoard, it is important to consider both interfaces cautiously.Despite our efforts, other interface designs might be even more suitable, such as combining auditory and haptic interfaces as suggested for future work (see subsection 6.4).Furthermore, although the motion chair enhanced immersion, the user study lacks external validity as we did not explore the interfaces in a real vehicle.The user study also did not include non-driving-related tasks that are expected in highly automated driving scenarios.Yet, existing research indicates that such tasks affect the behavior of sighted passengers [2,19,57].Consequently, it is reasonable to assume that non-driving-related activities would similarly affect the study results.Nonetheless, other research has found that the most common non-driving-related task is idling and looking out the window [25,62].Another consideration is that our study assessed only a limited set of dependent variables.The inclusion of a broader range of measures might have provided deeper insights into participants' preferences and interactions with the interfaces.Further, considering the specialized nature of the target group, our user study was conducted with a limited sample size of N=14.Therefore, we refrained from conducting significance tests.Instead, we evaluated the data descriptively, introducing the possibility that the observed results might be attributed to random chance.

CONCLUSION
This paper presents OnBoard, an interface that uses tactile cues to convey traffic information to BVIPs in HAVs.We adopted the participatory design approach by conducting an initial interactive workshop with N=4 BVIPs.During this workshop, participants identified three relevant pieces of traffic information that were highly relevant to them -orientation at the destination, distance/time to destination, and reason for the vehicle stopping.In a subsequent interactive session, the participants aimed to design tactile cues for the aforementioned traffic information, suggesting reduced tactile shapes and high contrast, especially catering to those with remaining visual acuity.These workshop outcomes and existing literature informed the creation of OnBoard and an auditory interface.Both interfaces were explored in the following within-subject user study with N=14 participants who are blind or visually impaired.Our qualitative and quantitative results indicate that tactile cues have the potential to convey traffic information to BVIPs.Further, we found a trend that with decreasing visual acuity, participants' engagement with both interfaces increased.However, this trend did not hold universally for all participants, as individuals with higher visual acuity also showed frequent engagement with both interfaces.Therefore, based on individual information needs and preferences regardless of visual acuity, we suggest that highly tailored interfaces should be developed to promote future accessible HAVs.
Our work represents a significant step towards improving the conveyance of traffic information to BVIPs inside HAVs through the use of OnBoard.However, achieving a comprehensive and highly tailored solution will require further research and refinement, with a strong emphasis on the active involvement of the target group in the design process.

OPEN SCIENCE
The source code for OnBoard, including all relevant blueprints, 3D-printing files, and laser-cutting files, has been made publicly available.They can be accessed via the following link: https://github.com/luca-maxim/hey_whats_going_on.In addition, the entire Unity scenes, including installation instructions of the required 3rd party Unity assets, are available upon request.
A DESCRIPTIVE DATA OF THE USER STUDY

Fig. 2 .
Fig. 2. Artifacts for the interactive design session during the participatory design workshop.(a) Wooden boards covered with white velcro strips.(b) 3D-printed tactile elements

Fig. 3 .
Fig.3.Participants' interface designs of the tactile interfaces for each traffic information.3D-printed stickable tactile elements were utilized in the participatory design workshop.For space reasons, illustrations are in portrait mode.

Fig. 4 .
Fig. 4. Interface design of OnBoard (30cm x 22cm x 22cm) with extendable elements that ascend /descend according to the simulated traffic information.The figure depicts the tactile elements with their functionality as either functional buttons and/or extendable components

Fig. 5 .
Fig. 5. Auditory interface featuring a button that activates verbal audio when pressed and deactivates it upon release.

Fig. 8 .
Fig.8.Scarf plot of the participant's engagement with OnBoard auditory interface in relation to the simulated traffic situations during the ride.OnBoard's elements include the Progress slider, Progress Information (button), the reason for stopping (button), and triangular vehicle's representation.Participants are sorted by their individual visual acuity.The total engagement assumes that with decreased participants' visual acuity, there was increased engagement with both the auditory interface and OnBoard.However, there is individual behavior, regardless of the participants' visual acuity

Table 1 .
Scheduled agenda for the three hours participatory design workshop

Table 3 .
Table of scores of the questionnaires of the user study

Table 4 .
Table of participants' demographic data for both workshop and user study