An Exploration of Mock Scenarios as a Prospective Method for Informing Law and Policy Thinking on Service Robotics

Modern service robots (such as delivery robots) are becoming more commonplace, and it is important to bridge the gap between legal and engineering thinking to be able to propose and implement effective policy. As a step in this direction, our presented work aims to inform lawyers and policy-makers on useful policy thinking surrounding service robots. Specifically, we use a mock robotic technology ordinance and accompanying mock scenarios to assess whether different types of shared context and information seem to influence robot law and policy experts' analysis of ordinances related to robotic technology. The presented work provides preliminary insights on how perceptions and interpretations of robot policy might change across different levels of background grounding and robotics understanding. Our findings can contribute to the ongoing discourse on responsible development and regulation of service robotic systems.


INTRODUCTION
Robots -such as the modern wave of autonomous delivery robots -are entering day-to-day human-populated settings, with relevant policy often trailing one step behind and sometimes struggling to connect to the realities of robotic systems.Bridging the gap between legal and technical perspectives is challenging but crucial for efective communication and policy creation [2,3].As one step toward this productive interchange, we considered the prospective learning intervention of mock ordinances and cases relevant to service robotics.These were loosely drawn from real ordinances and events that are already happening with delivery robots.
The broader context of this work has to do with artifcial intelligence (AI) literacy, which refers to an individual's ability to understand, interpret, and engage with AI concepts, technologies, and applications.Past work has focused on developing and evaluating AI literacy programs for university education based on a comprehensive conceptual framework [8].The rich domain of explainable AI (XAI) also represents a type of AI literacy, focusing on creating AI systems that can provide clear, human-interpretable explanations for decisions [1,5,6].A related body of work shows that framing information is essential for thinking about service robot policy-relevant matters from privacy [7] to trust calibration [9].Eforts in the domain of teaching law and policy experts about the essential technical intricacies needed for shaping reasonable frameworks around service robots (e.g., a past keynote presentation encouraging the bridging across these domains [4]), however, are a less common, yet essential, topic of study.We focus on this developing space in the work presented here.
Our central research question in this paper is whether the presentation of diferent mock service robot scenarios, along with a common base mock ordinance, infuence thinking when it comes to evaluating the base robotic technology-relevant ordinance.We conducted a pilot between-subjects investigation with these scenarios, the full text of which is included in the paper's Appendices.The scenarios specifcally focus on autonomous service robots falling under the category umbrella of food delivery or last-mile delivery, as these types of systems are quickly beginning to proliferate day-to-day life in many regions.The main contributions of this work include the scenario text itself, as well as early insights for hypothesis generation as to the future application and impact of this type of intervention.

METHODS
In this pilot between-subject investigation, participants in the 2023 We Robot conference (including individuals with a range of experience from law, ethics, and policy to technical robotics) responded to questions about our mock ordinances and scenarios, as further described below.This work was conducted with approval from our university ethics board.

Pilot Evaluation Design
Participants in our pilot investigation saw one of three written cases, as described below.The scenarios were written to exploit ambiguities in the base ordinance.
• Base Ordinance (Control Condition): Participants were presented with the base ordinance without an additional overlying scenario.This served to establish a baseline understanding of the ordinance alone.See Appendix A for the complete ordinance text.• MoonBot (Exploratory Condition 1): Participants received the base ordinance, followed by a hypothetical description of a robot that a company, MoonBot, is preparing to deploy.See Appendix B for the complete scenario text.• Scooter (Exploratory Condition 2): Participants were presented with the base ordinance, followed by a mock scenario in which a motorized scooter collided with a delivery robot.See Appendix C for the complete scenario text.

Procedure
Participants were recruited at the We Robot conference, an interdisciplinary conference on robot law and policy, as representative parties with interest in shaping better policies around real-world robots.Prospective participants were approached during short breaks between conference sessions and invited to participate.Willing participants received one of the three above-described cases to read.
After reading the provided text, participants completed a set of questions about the case, as further described below.The scenario and questions were presented on paper to facilitate rapid deployment and data collection.Aside from one question about robotics experience, no demographic information was collected due to the brief and public nature of the study.

Measurement
The question set following the cases consisted of the following: • (For MoonBot and Scooter Conditions) Opening Questions about Whether the Ordinance Was Violated: Participants responded yes/no/maybe, and completed a free-response question about their underlying reasoning.• (For All Conditions) Likert Scale Questions: Participants rated the likelihood of the ordinance accomplishing each of the following on a 5-pt Likert-type scale from "Not Likely" (1) to "Very Likely" (5): -Improve the safety performance of autonomous delivery robots.-Preserve the privacy of residents.
-Reduce the likelihood that a company will deploy an autonomous delivery system.
• (For All Conditions) Closing Free-Response Question: Participants shared feedback on what (if anything) they would change about the ordinance.• (For All Conditions) Robotics Experience Self-Report: Participants reported their familiarity with robotic technology on a 5-pt Likert-type scale.

Participants
A total of 41 attendees of the 2023 We Robot conference completed the pilot.Demographics-wise, a substantial number of participants had a relatively high level of experience with robotic technology ( = 4.0 out of 5, = 1.2).

Analysis
For scale-wise questions, we used descriptive statistics to assess trending between diferent condition experiences, as well as analysis of variance (ANOVA) tests with an = 0.05 signifcance level to assess statistically signifcant diferences across conditions.We used thematic analysis to analyze the free-response data.

RESULTS
All 41 participants completed the full questionnaire.Of the 41 participants, 11 experienced the base ordinance condition, 14 the Moon-Bot condition, and 16 the scooter condition.A synopsis of the categorical, scale-wise, and free-response results follows.

Ordinance Violation Verdict Results
For the MoonBot and scooter conditions, participants determined whether the ordinance had been violated.In the MoonBot case, no participants thought the ordinance had been violated, three responded maybe, and 11 responded no.For the scooter condition, perceptions were more mixed.Four participants stated that the ordinance was violated, eight responded maybe, and four responded no.Reasoning for these responses ranged widely, from "no uploads to the cloud allowed" as a reason for a clear violation for many MoonBot participants to "the ordinance isn't clear enough on what the contact info needs to include" as a reason behind a response of maybe for the scooter case.

Scale-Wise Question Results
Trends in the descriptive statistics appear in Table 1.Overall, ratings for the base ordinance tended to be more optimistic than ratings for one or both conditions when it came to preserving safety and privacy of citizens and reducing levels of robot deployment (the latter of which could be seen as good or bad, depending on the context).Once an accompanying scenario was introduced, ratings in all areas tended to decrease, aside from the one case of MoonBot safety ratings (which increased relative to the base ordinance condition .Compared to the scooter condition, the MoonBot condition tended to yield higher perceptions of safety and likelihood of the ordinance to reduce deployment.The scooter condition tended to yield higher ratings for privacy preservation. Statistical tests indicated that within the current sample, none of these trends corresponded to an associated signifcant diference.There were no signifcant diferences in the privacy, safety, or deployment reduction ratings across condition (all > 0.191).

Free-Response Question Results
For the closing question about changes that participants would make to the presented ordinance, key emergent themes were as follows: vehicle characteristics, data upload to the cloud, contact information requirements, privacy concerns, safety regulations, mapping and recording restrictions, and emergency stop mechanisms, as further shown in Table 2.One participant in the MoonBot condition and three in the scooter condition left this feld blank; accordingly, we divide the values in the table by 11 base ordinance condition participants, 13 MoonBot participants, and 13 scooter participants when computing percentages.
Occurrence rates of diferent codes across condition shows some potential for the presented scenarios to guide thinking about robotics policy.For example, without the scenarios overlaid, participants thought more about vehicle characteristics (73% of cases) and mapping/recording restrictions (36% of cases), compared in other conditions.The MoonBot condition led to relatively more thinking about data upload (occurrence in 54% of responses) and privacy concerns (46% of responses) compared to the other conditions.On the other hand, the scooter condition seemed to elicit the most thinking about safety and emergency stop functions (both 38% occurrence), at least by a small margin compared to other conditions.Once a scenario was laid over the core ordinance, participants seemed to think more about contact information requirements (38% of cases for both MoonBot and scooter, compared to 18% in the base ordinance condition).

DISCUSSION
Our fndings can be contextualized within the broader landscape of research on human-robot interaction, autonomous systems, and urban mobility.Although the pilot focused on a small set of beginning mock scenarios, future work in related domains can potentially use a similar approach to provide additional insights and contribute to a more comprehensive understanding of user perceptions of and preferences for service robots (and beyond).The ANOVA results showed no statistically signifcant diferences across scenarios in terms of safety performance, preservation of privacy, and autonomous delivery system deployment likelihood, but trending in the data might inform future scenario design and evaluation.For example, having additional scenario context tended to raise concern levels for safety and privacy.
Several common themes emerged in the critique of the current ordinance, such as restrictions on vehicle characteristics, the importance of clear contact information, and concerns about data uploading and data privacy.These themes have signifcant design implications for the development of autonomous systems and related regulations.Our results hint that service robot designers should prioritize ensuring robust data privacy measures, providing explicit contact information to address user concerns and enhance overall system safety, and incorporating clear and accessible emergency stop mechanisms (as a few examples).The results also highlight the need for standardized language and clear defnitions in city ordinances related to autonomous systems.We noticed while parsing the qualitative data that ambiguities in terminology, such as the interpretation of "the cloud," can lead to confusion and misinterpretation of regulations.
There are clear limitations to the work, both in the small sample size and the relative simplicity of the questions and responses.However, as indicated by Table 2, a concrete ordinance and scenario still can elicit specifc and ftting areas for discussion and critique.This demonstrates the promise of even such relatively simple ordinances and scenarios to serve as a potential mechanism for initiating focused and apt discussions in the coming waves of important robot policymaking tasks.In future steps, we plan to generate ordinances and scenarios with closer ties to real-world laws and cases, in addition to collecting data from a larger sample with less background training in robotics.
Overall, this work ofers valuable insights on perceptions and regulation of autonomous systems in human-populated settings such as dense urban environments.By examining responses to diferent scenarios, we shed light on the factors that infuence public attitudes towards safety, privacy, and deployment surrounding autonomous delivery systems.

Table 1 :
User responses to scale-wise questions across condition.Results are reported as mean ± standard deviation.The highest mean value in each category is bolded for emphasis.

Table 2 :
Overview of key emergent themes in the qualitative analysis of changes respondents would make to the ordinance, with frequencies for each condition and example quotes.Themes are ordered from highest to lowest frequency of occurrence.The vehicle needs a clearly marked & accessible emergency stop that a person physically next to [the robot] can trigger"