A Robot Dialogue Authoring Interface with Smart Capabilities

Researchers have called for the development of dedicated authoring interfaces that can support caregivers in authoring socially assistive robot content. In this paper, we present an authoring interface specifically designed for authoring robot dialogue for reading a book with a child. Our interface incorporates past research on Socially Assistive Robot dialogue and introduces some automatic capabilities aimed at speeding content authoring. We discuss key challenges that arose while working on this interface and describe future work to evaluate, improve, and generalize this interface.

While SARs seem highly successful in a research setting, there are several barriers to their wide deployment [5,12,14].Access to robots is a fairly salient obstacle, but the obstacles only start there.Several contextual, technical, and domain-specifc barriers need to be overcome before robots can be widely adopted.Contextual barriers include the technical design of context-aware tools and organizational adoption and support for robots [4,14].Technical barriers include technical difculties with connecting to a robot or understanding how to efectively control the robot [5].Domainspecifc barriers include the availability of and access to assistive content delivered using the robot [5].In our work, our focus is on overcoming the barrier associated with content development.
Caregivers using SARs to deliver care have suggested that the assistive content itself is a central component of using SARs [5].SAR content is often perceived as a pre-condition to using these robots.However, this content is often authored through invisible labor [7].SAR content authors typically develop this content using the teleoperation interface itself, rely on make-shift authoring interfaces in the form of spreadsheet software, or even rely on technologists to hard-code content into the teleoperation interface.While those approaches may be reasonable in a research setting, they are clear barriers to overcome when wanting to increase access and usability of SARs.
Socially Assistive Robot researchers have called for the development of dedicated authoring interfaces [5,8].While a caregiver may perform both tasks of authoring content and teleoperating robots, the context in which they do so, and their needs in accomplishing each are fairly distinct [4].Authors currently rely on teleoperation interfaces to author content; interfaces that may provide the capabilities to perform these tasks but are not designed to do so efectively.
Importantly, researchers have analyzed how teleoperators use SARs and identifed several content themes that typical socially assistive interaction may follow [8].Teleoperators engage in content focused on rapport building at the beginning and end of an interaction.The core of the interaction focuses on the assistive content to deliver.Throughout the interaction, teleoperators may use feedback dialogue after questions are prompted and may also include ignorance statements, likely when the content needed was not already available.Importantly, teleoperators need to maintain the pace of conversation by the robot [4]; dialogue utterances ought to be short to account for potential interruptions by interlocutors in conversation.These patterns ought to be considered when designing authoring interfaces for this content.
Our focus is on teleoperated SARs, since in domains where socially assistive robots are used, there ought to be a human in the loop and more teleoperation is encouraged [6].However, while the interactions of these robots are teleoperated, there may be room for autonomous support tools to help in authoring content.

THE AUTHORING INTERFACE
In this paper, we present a dedicated authoring interface for teleoperated socially assistive robot dialogue that (1) incorporates content themes of SAR dialogue, and (2) includes automation capabilities that aim to simplify and speed content authoring.Our authoring interface targets content for reading a book with children using the Peerbots teleoperation interface [15]; a free application that can be used to control Misty robots [17] and Peerbots agents.

Simple Author Input
The authoring interface is shown in Figure 1 and includes a single textbox for authors to enter content within and then select a submit button.The goal of this design choice is to provide a simple interface where authors can enter all their content and the automation provided by the interface can transform that content into robot dialogue.This design choice aims to abstract away any understanding of how the robot teleoperation interface works and minimize the learning curve to start authoring content.

Automated Capabilities
After an author submits their content, the interface performs three automated processes and returns the output in a format usable by the teleoperation interface.

Automated Inclusion of Rapport-Building Content.
Since teleoperators often begin and conclude with rapport-building content, the authoring interface will automatically include several rapportbuilding dialogue actions at the beginning and end of the content.These are randomly selected from a long list of pre-set options.

Long Dialogue Uterance Spliting.
When converting the author's content into robot dialogue, our interface will split long dialogue into several dialogue actions.When a dialogue action is initiated, the robot typically continues to speak until all content is verbalized from that dialogue action.By having shorter dialogue actions, teleoperators can better maintain the pace of conversation.
Our authoring system parses the author's content using the spaCy library [11] and splits sections at punctuation marks and when detecting long sentences.Dialogue actions are created for each sentence and based on new lines.The system then splits sentences based on a maximum number of words and/or characters to be included in a given dialogue action.When splitting long dialogue, the dialogue actions are split at subordinating conjunctions when detected.

Automated Feedback & Ignorance Content Afer Qestions.
When authors include questions in their content, denoted by a question mark, the authoring system will automatically add three dialogue actions.Specifcally, the authoring system will include one dialogue action for (1) positive feedback, (2) constructive feedback, and (3) an ignorance response.Each of those is randomly selected from a preset list of content for each of those themes.

Customizability
While our authoring system includes some opinionated defaults, it also allows authors to over-ride several of the decisions being made.Directly following the text box for entering the content, the authoring interface presents settings to customize automatically appended dialogue, see Figure 2. Authors can enter their preset lists for rapport building: greetings and farewells, feedback: positive and constructive, and ignorance responses.Even when authors decide to spend time modifying these choices, highlighting these themes separately in the interface may serve as a reminder of the content teleoperators may need.If authors are not satisfed with the way the automated algorithm has split their long dialogue actions, they may modify the settings for sentence splitting under advanced settings.

Preview Capabilities
Before downloading and leaving the authoring interface, we found it crucial for authors to be able to preview how the content they authored would be converted to teleoperation utterances.The preview serves two purposes: (1) reviewing the automatically produced and edited content and (2) mentally placing the author in the state of a teleoperator in case that leads to desired changes.To do so, we included two diferent ways in which authors would be able to review their authoring.

Table Preview.
We included a table listing the dialogue as it appears in a comma-separated values (CSV) fle to be ingested by the teleoperation interface.This includes metadata about the dialogue for review by the author.Additionally, a table aims to provide a fairly concise way to review the large amount of content that may be authored.An example of the preview table is shown in Figure 3.

Teleoperation Interface Preview.
We included a preview of the generated content as it would appear in the teleoperation interface to shift the author's mindset to that of the teleoperator.The context experienced by authors and teleoperators varies greatly.Authors ought to understand the result of their work and try to imagine how a teleoperator would use the authored content.We suspect this design choice encourages that behavior.An example of this preview is shown in Figure 4.

KEY CHALLENGES
While developing this authoring interface, we came across several key challenges that we would like to highlight.

Standard Robot Dialogue Format
When developing content for SAR dialogue, we had to choose a single platform for which to develop content.We chose the Peerbots interface due to familiarity with the interface and since it is generally available as a free application.However, all content developed using our authoring interface currently only works with the Peerbots interface, unless others decide to implement the Peerbots format.We were not able to fnd a standard format that is used by multiple interfaces.We would recommend that researchers identify common patterns across dialogue teleoperation interfaces and develop a standard for dialogue that can be shared across these interfaces.Given how important content is to deploying robots [4,5], and that it is often authored through invisible labor [7], we fnd it imperative that content be transferable across platforms to minimize any extra work performed by authors.

Integrating with Teleoperation Interface
While we found the dedicated authoring interface valuable for focusing on authoring content, based on our personal use, we found that it distanced us from the experience of teleoperators.We found it important to make the connection between how this content was authored and how it would be consumed in teleoperation.This is precisely why we included previews of the content as it would appear in the teleoperation interface (see Section 2.4.2).However, this required substantial development efort and would likely not have been feasible had the teleoperation interface been substantially more complex.While dedicated authoring interfaces seem highly benefcial, we would recommend that they be embedded within teleoperation systems or be closely integrated.

FUTURE WORK 4.1 Experimental Evaluation
We plan to experimentally evaluate the authoring interface's performance compared to other interfaces.Specifcally, we think it is important to evaluate the interface compared to the teleoperation interface and compared to spreadsheet-based interfaces that authors may use.We suspect that using a dedicated authoring interface, either the spreadsheet software or our developed interface, would be rated higher by authors compared to the teleoperation interface and would be faster at creating the same dialogue.We also suspect that our authoring interface's automation capabilities will be rated highly by authors and that our interface will result in faster authoring compared to spreadsheet software.

Additional Features
While our authoring interface implements several automated features, there is likely room for additional features and further improvement on existing ones.We present several examples that could improve the efcacy of this authoring interface.
In future work, we recommend studying the development of context-aware automated suggestions.Our current automatically generated content is context-agnostic, and simply randomly selected from a user-defnable list.While this may not provide the best results, it gives authors a sense of consistency in knowing what to expect.There is potential for improvement of our auto-generated dialogue actions using context-aware algorithms.However, this may also result in more review time needed.
We recommend future interfaces support the inline editing of individual automatically-generated content.This can be achieved through close integration with other interfaces such as the teleoperation interface itself or spreadsheet software.However, ideally, the authoring interface itself would allow authors to make edits directly.

Generalizing To Other Domains
While our focus was on dialogue authored for a subset of socially assistive interactions, there is likely much to learn from developing similar tools for other applications.For future work, we would recommend identifying more patterns that are used in authoring content and developing automated features to support those.We suspect that this tool may be generalizable to all teleoperated social robot dialogue authoring and would encourage research to investigate that possibility.

CONCLUSION
Researchers have highlighted the importance of dedicated authoring interfaces for teleoperated Socially Assistive Robot Dialogue.Authors have diferent needs and work in diferent contexts compared to teleoperators [4].Authoring content often happens through invisible labor [7].Robot dialogue often follows consistent patterns that ought to be accounted for in authoring to ease content authoring [8].While SARs should include substantial teleoperation [6], there may be room for automated capabilities to ease content authoring.
We developed an authoring interface with automated capabilities to ease and support the authoring of teleoperated socially assistive robot dialogue.Our interface accepts an open-ended text response from authors and parses it to be converted to short robot dialogue actions.The interface automatically accounts for dialogue content themes and appends additional content that may be useful in teleoperation to minimize the burden of authoring that additional content.While our dedicated authoring interface is separate from the teleoperation interface, it provides a preview of how content would appear in the teleoperation interface to situate authors in the experience of teleoperators.
While developing this interface, we identifed two key challenges; a lack of standard formats for robot dialogue content, and difculty integrating with teleoperation interfaces.We were not able to fnd a standard format for teleoperated robot dialogue that can be used across teleoperation interfaces.We encourage researchers to investigate the feasibility of developing such a standard so content can be easily ported from one tool to another.Developing a dedicated authoring interface, separate from the teleoperation interface, provided substantial fexibility in what we were able to create.However, we found that it distanced authors from the experience of teleoperators.While we were able to create a replica view of the teleoperation interface to preview the authored content, we suspect this would not be feasible and/or reasonable as teleoperation interfaces become more complicated.While we encourage further development of authoring interfaces, we recommend they directly integrate with teleoperation interfaces where possible.
For future work, we plan to experimentally evaluate the efcacy of our authoring interface, develop additional automation and review capabilities, and generalize our authoring interface to other domains.We plan to evaluate our interface based on the time taken to author content and author-reported usability compared to existing teleoperation and authoring interfaces.We may also introduce additional capabilities such as inline editing of authored content during review and context-aware content generation.Finally, we plan to investigate patterns in social robot dialogue use to generalize our interface beyond dialogue for socially assistive robots for children.

Figure 2 :
Figure 2: A section to customize the automatically generated dialogue for each content theme and modify sentence splitting settings.

Figure 3 :
Figure 3: A preview of the output to download and preview resulting dialogue actions.

Figure 4 :
Figure 4: A preview of the dialogue utterances in a replica of the teleoperation interface.