GARRY: The Gait Rehabilitation Robotic System

Gait rehabilitation is a critical aspect of post-stroke recovery, and emerging technologies such as virtual reality and wearables are playing a pivotal role in facilitating this process. However, despite the potential benefits, there is a significant gap in robot-based rehabilitative systems that facilitate repeated use by maintaining users' attention long-term. Our research aims to bridge this gap by creating a comprehensive system that utilizes different feedback types and robotic assistance to support users' gait rehabilitation outcomes. In this paper, we introduce GARRY (Gait Rehabilitation Robotic System), a new robotic system that provides interactive feedback during locomotor training. It promotes engagement by gamifying the rehabilitation process, offering a fun means for the user to meet their rehabilitation goals defined and set by physical therapists. GARRY also incorporates behavioral feedback to introduce a sense of companionship during a session. We make GARRY open-source to other researchers in hopes of encouraging accessibility and to promote research in the field. Our code can be found here: https://github.com/UCSD-RHC-Lab/GARRY


INTRODUCTION
Despite recent advancements to reduce the risk of stroke, the incidence continues to rise globally, with over 12.2 million new cases every year [4].At an individual level, of the patients who survive a stroke, roughly 73% will never recover to pre-stroke levels of mobility and stamina, and will retain a residual disability [4].Stroke survivors and their families often experience high treatment costs, changes in social and economic roles, and reduced ability to perform activities of daily living (ADLs) which can signifcantly compromise their independence and quality of life [20,24].
In particular, the ability to walk independently is a strong indicator of quality of life and overall autonomy for stroke survivors and is a major goal of post-stroke rehabilitation [25,26].Many have explored robotic systems to support gait rehabilitation [1,9,19], as well as to support stroke rehabilitation more broadly (e.g., upper limb rehab, ADL training) [3,5,10,[13][14][15]21].However, one open question is how to enable domain experts (e.g., physical therapists (PTs)) to control specifc variables such as feedback valence (i.e., positive, negative, binary, explicit) during robot-delivered locomotor training to improve intervention efcacy.To our knowledge, no existing systems allow PTs to easily investigate this question experimentally, which is what our work aims to address.
We address this gap by exploring the efectiveness of locomotor training via a robotic system that delivers visual and physical feedback to users during gait rehabilitation.We introduce new means for robots to take on the role of a "rehabilitation partner", and automatically adjust their behavior in response to a person during training sessions.Our system leverages locomotor learning practices to explore the efcacy of feedback valence as delivered by a robotic system, and its efect on patients' recovery.
In this paper, we introduce GARRY (see Fig. 1), a new robotic system that provides real-time, multimodal feedback during locomotor training.It promotes engagement by gamifying rehabilitation, ofering a fun means for users to meet their rehabilitation goals defned and set by PTs.We defne multimodal feedback as providing feedback to users through multiple modalities, such as a tablet display and robot movements.GARRY also incorporates behavioral feedback to introduce a sense of companionship during a session.We tested GARRY's functionality with data from real users.
Ultimately, using our system, we hope to increase user engagement with rehabilitative interventions and reduce the impact of residual impairment.With little research into this combination of stimuli (visual and physical), we also hope to promote research into more novel combinations of accessible and engaging therapeutic strategies.Given the wide range of applications of robot-assisted rehabilitation and the growing body of research in developing these systems [14], GARRY is applicable to many contexts and populations that are of interest to the HRI community.The current implementation is thoroughly documented to allow PTs to determine the long-term efcacy of the individual styles involved in rehabilitation and focus on emerging ideas in the rehabilitation space.

THE GARRY SYSTEM
All aspects of GARRY are customizable, including the robot platform, rehabilitative tasks, software, feedback personalization, and the gamifed feedback mechanism.
The robot platform interacts with the user throughout a training session, responding with diferent feedback based on training performance.Our gamifed system provides users with personalized feedback, rewarding or subtracting points from a user's overall score during a session.Furthermore, the robot performs diferent gestures based on the feedback valence and user performance.We defne a gesture as a sequence of robot movements specifc to the robot's capabilities.It can also incorporate data such as the current amount of steps taken to infuence its actions.The modular nature of the system enables PTs to integrate other applicable user behaviors to their work, such as facial expression and gait style.
Our system supports four feedback valences: positive, negative, binary, and explicit.This is inspired by Banh et al. [1] who suggest that difering robot-delivered feedback types can enhance motor learning and retention [6].Further, research suggests that gamifed rehabilitation systems reduce boredom from repetitive motions and result in higher user engagement rates [18].The variety of feedback types will allow us to explore which approaches are generally more efective at achieving certain rehabilitation goals.

Rehabilitative Content
We designed GARRY to be extensible to diferent rehabilitative contexts.Depending on the context, a PT can specify a set of tasks for GARRY to deliver, focused on the user's area of interest and goals, and dictated by their mobility level.In this article, we discuss a specifc example: gait rehabilitation training.Our focus is on ankle power and the peroneus muscles' performance during plantarfexion, characterized by the A2 value.The PT might specify that the user's goal is to achieve a certain A2 value with each step for 60% of the steps they take over the course of ten minutes.
The PT can then set the specifc type of feedback the user interacts with.GARRY supports four valences of feedback.Positive feedback rewards the user with points when they perform at or above the goal.Negative subtracts points from the user when they perform below the goal.Binary integrates both positive and negative feedback.Explicit simply displays to the user their real-time A2 value; it does not utilize the point system.
GARRY provides a database that stores essential components of a session, enabling PTsto review user data and adjust training as needed.This might include isolating and identifying the efcacy of specifc types of feedback.The database also allows PTs to view specifc dates, values, and other session data.

Robot Platform
GARRY can work on any mobile or tabletop robot platform.In this article, we used a TurtleBot 2 since it can display a screen at human height and is low cost, providing great accessibility.We used the Turtlebot as a proof of concept, and implemented gestures such as moving back and forth and turning left and right.GARRY can also integrate with other robots such as the Stretch or Quori, which can move their arms.Our system can be easily extended to support interactive responses during interventions, including movements, gestures, and sounds depending on the robot's capability.The frequency of these responses is based on the user's progress during the intervention and can be customized by the PT or roboticist.
We implemented GARRY's ROS component in ROS Melodic, Ubuntu version 18.04.The nodes within this environment work together to determine the gestures and movement of the robot during a session.The robot carries out these actions based on the last fve seconds of session data to avoid overstimulating the user.

Tablet Feedback System
After starting the intervention, GARRY plays an interactive game with users.The games are based on the feedback mode chosen by the PT, which determines how the system rewards users with points that constitute the score of a session.The scores are tracked and presented in a leaderboard style to promote self-competition [23].
Throughout a session, users may be encouraged in various ways when they meet or exceed session goals.This can come in the form of visuals like confetti when a value hits a certain level, depending on the feedback valence.The app also displays text feedbackdiferent phrases that correlate to levels of performance.GARRY also displays a line chart-a historical visualization of a user's performance in relation to their session goal.This graph allows the user to see their progress over time, which may implicitly encourage them.Additionally, users or PTs can view past sessions by feedback type to view their data and see how they have progressed.

SOFTWARE ARCHITECTURE AND IMPLEMENTATION
Our system utilizes various technologies, including a tablet app, data management via a Flask server and SQLite database, MATLAB, and ROS.In our context, PTs use wearable sensors to record a person's gait patterns during locomotor training.GARRY's MATLAB component receives and processes the data to determine the A2 value (and other measures).Based on this A2 value, GARRY adjusts a) the robot's gestures and b) the feedback displayed on the tablet.In addition, the tablet application makes API calls to our Flask server and stores session and user data in our database for analysis.

Tablet Application
We implemented our tablet application using Flutter, a cross-platform mobile application development library.This is deployed as a web application, to improve extensibility and accessibility across operating systems and platforms.Developers thus have the fexibility to deploy it as an application for other platforms and operating systems to match the needs of their particular hardware setup.Its front end is separated into three main categories: setup pages, an in-session page, and score pages.The setup pages include the main page, the session page, and the feedback selection page.The PT starts from the main page and enters the ID of the user or creates a new one as they see ft.The session page allows the PT to start a new session or view previous sessions associated with the current user.The feedback selection page allows the PT to choose the session's feedback type, if the PT decides to start a new one.
The in-session page is the main screen displayed during each session, with a set of widgets that provide real-time feedback differently according to each feedback type (see Fig. 3).The widgets include a custom thermometer that flls to the percentage value of each step (where 100% represents the goal value), text feedback to the user based on the value of each step, and a line plot showing their session progress relative to the goal value.
The score pages are leaderboards.After each session, the app shows the leaderboard, scrolls to, and highlights the most recent score in blue.This board includes the scores of every type of session.

Data Management
Our system manages data via a back-end Flask server and an SQLite database.The data allows PTs to monitor user performance over time and analyze the efectiveness of each feedback type across all users.All data is stored locally to support user privacy and security.
The database is split into the users table and the sessions table.The scheme for the users table is simple and only contains the User ID (Primary key) and the Name attributes.On the other hand, each entry in the session table consists of the Session ID (Primary key), User ID (Foreign key), Year, Month, Day, Start Time, Duration, Score, Feedback Type, and Threshold (Goal Value).This distinction provides room for scalability as other PTs may be interested in integrating other information about users and/or sessions.
We manage our database via REST API calls to our Flask server, which accepts requests and makes SQLite queries correspondingly.For example, a POST call to the route //1 will create a user in our database, while a GET call to the same route will retrieve that user's data.In addition, after each session, a POST call to the route //< > will be made to create a new session with session ID < > linked to the user.The application could then make GET calls to the same route to display sessions sorted by non-ascending score on the leaderboard pages.

MATLAB Implementation
GARRY utilizes MATLAB to process raw sensor data in real-time and simulate sessions for testing.The MATLAB code sends realtime, correctly formatted data from users to both our tablet application and the robot via websockets.Our code interfaces with the rest of GARRY to facilitate the rehabilitation session.
Our implementation streams preexisting session data provided by our clinical collaborators into the app and the robot.This data is easily replaceable with real-time data received from sensors.
The current MATLAB code begins the data exchange protocol upon receiving a keyword from our app.It then continuously sends the A2 and goal value to both the app and the ROS component.

ROS Implementation
Our ROS code receives data from the rest of our system and moves the robot accordingly.It is characterized by 2 nodes, the Data Processing Node (DPN) and the Gesture Generation Node (GGN).
The DPN subscribes to /feedback_type and /data, determines if the user has taken a "good step" (see Sec. 3.4.1)from the A2 and goal values, and sends the result to the /good_step topic.The GGN subscribes to the /feedback_type and /good_step topics to direct robot movement.If the robot receives a "true" message from /good_step and fve seconds have passed since the start of the last gesture, it will perform a new gesture.

Robot Algorithm.
The robot gives feedback based on the user's step performance, which we determine by dividing the A2 value by the goal value for each step.The DPN calculates the average performance over the last fve seconds using a sliding window with no overlap.If this is above the activation threshold, then the robot gives feedback.We experimentally determined that fve seconds gives the robot ample time to physically respond while minimizing user disorientation.In addition, while the default activation threshold is set to 90% (also experimentally determined), the PT can adjust this value depending on the user's performance.
The robot's response is then informed by the current feedback type.For example, in our work, the robot might express celebration by turning left and right in the positive feedback type.Those using GARRY can implement and experiment with diferent responses based on their application.

USAGE NOTES
Here is GARRY's basic workfow: (1) PT enters user information via the tablet application, chooses "start a new session", and selects that session's feedback type.(2) Once the researcher starts the session through the app, it sends a message to PT's MATLAB program to signal that the session is starting, and the feedback type to /feedback_type.(3) MATLAB then begins streaming the user's step data and threshold to the app, which updates and animates the screen based on the user's performance, as well as the /data topic.(4) When the DPN receives data from the /data topic, it calculates the proportional A2 values, determines the average of the last fve seconds, and sends that data to the /good_step topic, which directs the robot's actions appropriately.Steps 3 and 4 repeat until the PT manually ends the session.(5) Throughout the program, the app calls the Flask server's REST API endpoints and stores, retrieves, and updates both session and user data in our SQL database as necessary.
In the provided repository, we provided a README and source code containing detailed instructions to start and test the interface.Our code supports simulation using pre-existing data in addition to the real-time system, so both sets of instructions are provided.Researchers can make adjustments to the feedback type, the point system, and the amount allocated.They can also customize the data source as long as it can connect to our system via websocket.

FUTURE WORK AND ETHICAL CONSIDERATIONS
While GARRY was designed for our context of delivering poststroke gait rehabilitation, there are additions we hope to include that could increase its efcacy and impact on the HRI community beyond the needs of our clinical collaborators.One next step is to conduct longitudinal studies with post-stroke survivors to ensure that GARRY can engage and motivate these users during rehabilitation, as well as explore the efcacy of each feedback type.
Other future work will consist of interface and user experience design improvements.We aim to have the robot respond directly to additional user behaviors such as physiological responses and unique gait styles to provide more personalized and efective feedback.We also plan to design more interactive visual and aural experiences (while not being too distracting), to support user engagement.We aim to support more functionality for users, with improvements like adjusting the goal and text feedback mid-session and adding fltering to the leaderboard.
Our work raises some ethical considerations, including accessibility, health equity, and privacy.Gait rehabilitation has been linked to increased independence and quality of life for stroke survivors [20,24], and robot-delivered locomotor training has the potential to signifcantly improve rehabilitation outcomes [11,16].However, it is important to note that rehabilitation outcomes often depend heavily on factors such as a person's severity of stroke and their pre-stroke abilities [8,17], so roboticists may need to work with clinical experts to help users set realistic expectations for their treatment outcomes.In addition, these systems may not be accessible or culturally contexualized most in need of these interventions [27].Therefore, more work is required to determine how these systems can be made afordable and accessible to these populations.
It is also important to consider data privacy, and potentially contributing to worker displacement with the use of rehabilitative robotic systems [12].With regards to privacy, we aimed to engage in data minimization practices as well as local storage of data.With regards to worker displacement, if systems like GARRY are efective, it raises the risk of displacing the rehabilitative workforce [7,22], which necessitates careful participatory-centered design with stakeholders to avoid such outcomes.
Finally, with any robot-delivered health intervention, it is important to plan for its eventual end [2].We are continuing our work with PTs to explore how to best design GARRY to facilitate its exit.

CONCLUSION
In this work, we present GARRY, an open-source robotic system that provides real-time, multimodal feedback during locomotor training.
We make GARRY open-source to others in hopes of encouraging accessibility and to promote research in the feld.Our personalized robotic system contributes to research related to locomotor rehabilitation as well as other applications like telehealth.Thus, this system will ultimately extend the efcacy and accessibility of robot-delivered rehabilitation interventions.

Figure 1 :
Figure 1: A user interacting with GARRY, an interactive robotic system that provides real-time feedback to users during locomotor training.

Figure 2 :
Figure 2: An overview of the GARRY software architecture.

Figure 3 :
Figure 3: The in-session page of GARRY's tablet application in the Positive feedback type.