An Accessible, Three-Axis Plotter for Enhancing Calligraphy Learning through Generated Motion

Learning a motor skill is essential for many aspects of our lives. The complexity of some of these activities makes it hard for novices to understand through observation. Calligraphy writing is one such artistic practice where learners compare the visual differences between their writing and expert manuscripts and adjust until they have achieved similar results. We propose an accessible plotter-based system that guides the learner’s arm and hand in three directions with an actuated brush. It converts static Chinese calligraphy manuscripts to G-code that reproduces the calligrapher’s movement. Through a user study with twelve novice calligraphy learners, we validated the efficacy of our system as a learning tool that allows novices to gain an intuition of nuanced skills such as depth variation more effectively compared to watching a video recording of the same movement.


INTRODUCTION
CHI '24, May 11-16, 2024, Honolulu, HI, USA Learning motor skills is integral to our everyday lives, from performing simple daily tasks to playing sports to participating in artistic practices.Unlike learning languages or math, learning these motor skills requires using the body.Completing bodily gestures involves movements that are difcult to capture symbolically and thus hard to learn through written representations alone [15].Furthermore, the movements are tacit and difcult to learn through observation alone.For fne motor skills, it often takes strenuous practice to achieve mastery.Many prior works have been developed to facilitate motor skill learning, from exoskeleton to visual guidance systems.Methods for supporting motor learning range from assistive tools (e.g., chopsticks helpers and bike training wheels) to externalizing inconspicuous information through e.g., data visualization [28], and to sharing bodily experiences directly between a skilled and a novice user [25].
This paper explores the design of systems that provide augmented feedback -feedback or information that is extrinsic and needs to be demonstrated by an expert or an external interface -to support novices to learn complex, fne motor skills [41].Specifcally, we use learning Chinese calligraphy as an example of a real-world motor skill, fush with tacit expressions.Although at times the teacher would demonstrate or hold the learner's hand directly, they are not always physically present to provide feedback.Traditionally, learners develop muscle memory by repeatedly observing and copying expert manuscripts.This practice requires the learner to envision the expert's gestures from a bygone calligraphic performance, using the static output as a reference for the movement that produced the outcome.A few systems have been developed to replicate calligraphy scripts or a calligraphy writer's movements.These systems typically involve a multi-degree-of-freedom robot [45] or complex mechanical structures [22], which are impractical for hobbyists.
To that end, we sought to design an accessible system that augments the process of learning how to write Chinese calligraphy.Specifcally, the system ought to supplement learning in the absence of a teacher, for example, in between in-person demonstration sessions.In addition, the system should be reasonably available and intuitive to use, which led us to the idea of modifying an xy-plotter.An xy-plotter is typically used to create drawings and writings by mounting a pen or maker at the end efector of the plotter.Diferent from print writing, Chinese calligraphy writing has dynamic stroke thicknesses, a feature that is at the core of its art form.As such, diferent from a pen, the calligraphy brush body is compliant, and the pressure applied to the brush and the direction of the brush tip create intricate variations in the written output.We adapted a standard xy-plotter with vertical brush movement and added a button for user control.Chinese calligraphy requires one to hold the brush with a fairly rigid wrist position where the brush body should remain roughly perpendicular to the paper, thus a three degrees-of-freedom (DoF) system allows the learner to focus on the change in the brush's movement while maintaining the specifc brush holding pose.To the best of our knowledge, as there is no current database of the stroke patterns of Chinese calligraphy scripts, we developed and open-sourced a calligraphy toolpath generator that converts static Chinese calligraphic scripts into movement paths.The toolpath generator takes into account the compliant characteristics of the calligraphy brush.
We ran a study with twelve participants with limited prior experience with Chinese calligraphy, which showed the system's efectiveness in enhancing learning.Participants found the plotter intuitive in conveying depth changes and considered it a more engaging learning tool compared to watching a recorded video of an expert writing.Some felt less self-conscious and consequently more confdent in their writing.Based on the study feedback and results and our design process, we conclude with design guidelines and considerations for designing efective augmented feedback systems to support learners in developing fne motor skills.
In this paper, we frst start by describing the background of calligraphy learning, drawing insights from calligraphy experts and the frst author's experience learning calligraphy as a child, which motivates the design goals of our system.We then describe the systems' hardware design and software pipeline.We demonstrate our system's performance at successfully replicating the nuanced aspects of Chinese calligraphy writing.We also present our user study with twelve novice learners, from which we derive insights on how to design systems for supporting the learning of complex, fne motor skills like calligraphy writing.
To summarize, the contributions of the paper are: • An accessible, plotter-based mechanism for learning calligraphy writing.• An open-source design and pipeline that creates calligraphy scripts into tool paths.• Insights from expert interviews and the design process of the assistive calligraphy learning system.• A twelve-person, within-subject user study that demonstrates the system's efcacy in teaching calligraphy to novice learners.• Insights on how to design external force feedback systems could help novices learn a complex, fne motor skill.where individuals fgure out how to achieve it; and fnally through experience, individuals will reach the Autonomous Stage where they can simply accomplish the task without consciously thinking and even add personally expressions [7].Augmented feedback, also known as extrinsic feedback, is information that needs an external source for further explanation.Usually, it is provided by a trainer or a display.This is diferent from intrinsic or internal feedback (sensory aference), which is always present during motor learning [41].It is generally accepted that augmented feedback is important, especially during the beginning two stages (cognitive and associative) of motor learning.This is because, during these two stages, the learner has minimal experience and requires a lot of attention and cognitive load [7].Although, many prior studies use rather simple motor tasks -tasks that "have only one degree of freedom, can be mastered in a single practice session, and appear to be artifcial" [47] -to demonstrate the efcacy of augmented motor feedback, such as aiming [33] and simple movement pattern reproduction [4].This work contributes to the investigation of the efcacy of augmented feedback for complex, fne motor skills.

RELATED WORK 2.1 Augmented Feedback for Motor Learning
In Human-Computer Interaction (HCI), many developed systems that support the transfer of motor skills.Most relevant to this work are ones that require hand dexterity to perform fne motor tasks.Many created devices using either projection mapping systems [14,48] or head-mounted displays [1,11,12] that allow one to view another person's hand movements from the frst person's perspective.For example, MirrorFugue is an interface that visualizes a remote teacher's bodily movement and hand gestures on a piano keyboard, which has been shown to improve remote piano learning.While these approaches are relatively easily scalable, the rich tactile information in hand movement and manipulation is absent.
Many systems were built to provide haptic feedback to facilitate motor learning.For example, Seim et al. used a set of vibrotactile motors that vibrate the learner's fngers while performing other tasks to gain "muscle memory" -a technique called passive haptic learning.They used this technique to teach individuals rhythmic activities like playing the piano [36], learning morse code [37], and braille [35].Instead of these rhythmic activities, our system serves a diferent kind of learning task that focuses on spatiotemporal movements, and thus simple, discrete points of vibrations are insuffcient to convey the full information.Furthermore, the writer must execute a series of procedural movements while simultaneously responding to the visual output on paper.This requires hand-eye coordination and a combination of visual and haptic feedback.
Most related to our work is the DigituSync system, developed by Nishita et al., which is a passive exoskeletal glove that helps, for example, learn the piano.DigituSync physically connects the hands of a teacher and a student so that the student feels the teacher's fnger movement with near-zero latency [25].Similar to playing the piano, the mastery of calligraphy involves precise control of the hand's pressure on the brush and speed of movement.However, the target use case of our system is to support novice learners outside of an in-person learning session; DigituSync focuses on hand-over-hand learning between two people.

Adaptive Hand-held Tools
The rise of digital tools has changed the practices of artists and designers, supporting human creativity with varying levels of precision and efciency [29,30,40,44,53,54].
More related to this paper are works that focus on tasks that involve the user holding a pen or brush.One way to augment the writing experience is by digitally augmenting the visual output, such as prediction of the next stroke [3,42] or repainting captured video output [34].Lopes et al. directly augmented the user with Electric Muscle Stimulation [20].Systems that combine haptic and audio feedback had been used to teach visually impaired users hand-writing [31,32].
A simple pen plotter is a common tool that people use to create precise drawings.Several prior works further augment the writing environment by modifying the platform (e.g., a table).Most of these leverage magnetic force feedback to attract or repulse the writing tool as a form of guidance.[18,19,27,49].Other systems directly augment the writing tool itself by adding direct force feedback or constraints to the writing tool [13,24,50].These allow the users to not be tethered to the specialized tabletop.
These systems are especially benefcial for complimenting freehand drawing with machine precision, and precision is important for tasks like creating a CAD model, graphs of data, and copying and pasting.Specifcally, few works focus on learning as the goal.In this paper, we focus on calligraphy writing as an exemplary complex motor skill that involves a degree of artistic expression.In the next section, we discuss learning practices for Chinese calligraphy in detail.

BACKGROUND ON CHINESE CALLIGRAPHY LEARNING AND EXPERT INTERVIEW
This section is a brief overview of the art of Chinese calligraphy as well as the teaching practices and learning experience.We consulted two expert calligraphers who have been practicing and teaching calligraphy for a decade.The frst is a calligraphy instructor at his local community center, and the second is a graduate student studying Chinese calligraphy.We also drew from the frst author's frst-person experience learning calligraphy as a child from her grandfather.
Calligraphy writing is centered around the way one controls the brush and the brush tip.Diferent from writing with a pencil or a pen, individuals hold the brush almost perpendicular to the paper, which can be a difcult posture to perform steadily and consistently.Calligraphy as an art form has evolved diferent stylistic fonts such as cursive font (草書), and semi-cursive font (行書), but novice learners start by learning the standard font (楷書).Learning Chinese calligraphy breaks down into a few foundational skills that require fne motor movement For example, traditional calligraphic techniques of "exposed tip" (露鋒) and "concealed tip" (藏鋒) requires intricate change in the direction of the brush movement and pressure applied to the brush tip.From an experiential perspective, calligraphy involves an appreciation of both the structure ("form") of the writing and the momentum ("force") inherent in the calligraphic form [39].Note that writing calligraphy also requires the knowledge of the stroke's order and the spatial structure of Chinese characters, which requires some level of semantic understanding of the Chinese characters and is beyond motor learning.Therefore, this paper primarily focuses on teaching novice learners fne motor movement skills.
A common practice typically used with novices is "hand-overhand" teaching, where the calligraphy teacher holds the learner's hand and guides the learner through their movement.This is not unique to calligraphy.For example, this methodology is also used with people with visual impairment [43] and preschool children with disability [2,46].However, a teacher is not always available.Thus, calligraphy learning traditionally involves observing and copying static expert manuscripts, and they rely on comparing the visual diferences in the output and adjusting until they have achieved similar results.When looking at a character, students are encouraged to observe the character as a whole instead of copying stroke by stroke.This is important to grasp the character's holistic structure and to preserve the fuidity in writing.Students may start by tracing over characters, but it is recommended that students write on a blank piece of paper to develop intuition.To improve, students need to be persistent.Thus, motivation plays an important role in learning calligraphy, especially for novice learners.
Besides learning from a skilled user directly, a few groups have developed technologies to teach novice users with the presence of a virtual tutor [9,51].These approaches require a human instructor to provide live feedback or fully capture the instructor's bodily movement, which is only sometimes available.Calligraphy writing is not only difcult to learn for humans but also tricky for machines to replicate.The existing movement paths generation processes are often optimized for the machine's capabilities [5,52] or for optimizing motion capture and replication [22,26].In addition, these systems require complex motor systems or a multi-DoF robotic arm, which are inaccessible to learners outside of these research labs.
In summary, mastering calligraphy writing necessitates individuals to closely observe and adapt to the varying outcomes produced by diferent brush deformations and movements in real-time, which calls for fne motor skills and hand-eye coordination.The posture of holding the brush perpendicular to the paper plays a crucial role in the outcome of the writing.For beginners, it can be a bit tricky to get used to.Through hand-over-hand teaching, learners can beneft from immediate, co-located feedback on controlling the brush, especially when it comes to pressure on the brush and other nuanced control.Unfortunately, not everyone has access to a teacher, and the conventional method of rote training through copying static traces without proper reference and feedback can lead to diminished motivation.
Drawing from insights from expert interviews and existing systems for motor learning, we designed our system for replicating and teaching calligraphy.

SYSTEM DESIGN AND IMPLEMENTATION
We set out to design an accessible system for novice calligraphy learners to gain additional feedback while practicing on their own.Specifcally, we have the following design goals: First, our system should be accessible to users (i.e., using of-the-shelf components), and the efort should be akin to completing an Instructable project.Second, the system should help learners form good writing habits, such as maintaining a steady wrist and holding the brush perpendicular to the paper.This also means that it should not only capture the essence of the characters but also efectively convey the correct depth and movement speed to the learner.Third, users should be able to intuitively use our system and prefer to learn with it over rote practice.Learning with our system should be engaging, as motivation and positive reinforcement may indirectly improve learning outcomes.Finally, we are avid supporters of open source, and we make available the hardware component design and toolpath generator so that people can remake and build upon our system.
Our fnal system consists of two parts: a software pipeline that converts static images of Chinese calligraphic scripts into movement trajectories, and a three Degree-of-Freedom (DoF) gantry modifed from an xy-plotter that follows the calculated trajectory.Next, we describe each component of the system in detail.

Hardware Design
The hardware construction of this prototype repurposes an xy plotter [21] and the grbl protocol for controlling the movement (Figure 2).Using a plotter keeps the design low-cost, allows the user to see the output from above, and does not obstruct the user's body movement.To add movement in the z-direction, a rack-andpinion mechanism is added to the servo motor (MG90S).Typically, a pen plotter has a pen up/down function with only two discrete states, but our design needs to achieve a range of z movement.Thus, we modifed the spindle's speed control, which uses pulse width modulation (PWM) signals, to control the servo horn position.In this case, we corresponded the min and max duty cycle (1-2 microseconds) to the min and maximum angle (0-180 degrees).
Each degree of servo rotation corresponds to 12.9 degrees of pinion rotation or 3.2mm of rack travel in the z-direction.The total length of the rack is 82.25mm, which leaves ample travel for the brush to move down and up.Resolution and dimension in the x and y axes inherit those of the plotter kit: precision in the x and y is 0.1mm, and the available writing area is 310mm by 390mm.The modular design of the brush holder allows the user to swap for diferent sizes or types of calligraphy brushes easily.The brush is mounted perpendicular to the paper as it is the correct posture for Chinese calligraphy writing.
During our prototyping process, we found while users received ample information from the movements, they needed to experience more agency rather than simply following the plotter.Therefore, we added a button to the brush handle to give the user a basic form of control.The button is connected to the hold and resume functionality of the grbl frmware: the machine resumes its movement when the user "squeezes" the handle (efectively pressing and holding the button) and stops when they let go.

Movement Generation
We developed a series of algorithms to generate toolpaths for the plotter to reproduce given characters or samples of calligraphy (Figure 3).The toolpaths are encoded in G-code and sent to the plotter via a serial connection.We open-source our toolpath generator at https://github.com/mitmedialab/collography to enable anyone to adapt their plotter to write Chinese calligraphy.
One important feature of calligraphy lacking in most graphical digital representations of Chinese characters is the separation of the strokes and their order.The stroke order is not just a pragmatic rule but also afects the fuidic aesthetics of the character.To address this, we used stroke orders from the open-source database "Make Me a Hanzi" [17] for a standard font.However, for other font styles like cursive and semi-cursive, we devised a semi-automatic method to derive this information from any image of characters or calligraphy.A semi-automatic process is necessary because the characters written in non-standard font styles may have a diferent stroke order, and thus manual labeling is then needed to trace each stroke in the correct order.We hope our work and open-sourced tool-path generator can encourage people to contribute to the labeling of the stroke order of non-standard fonts.
After the stroke order of a character is obtained, we apply adaptive thresholding and de-noising and compute the morphological skeleton of the binary image.The stroke thickness is generated by ftting a maximum inscribed circle centered at each point on the medians that touches the outline.The circle's radius becomes an estimate for the stroke thickness at the given point.Finally, we resample the labeled data points and apply smoothing at stroke intersections to mitigate the jerk in the radius of the inscribed circles.After obtaining the rough movement in three axes, in the following sections, we describe the heuristics that enable us to optimize the tool path for a modifed plotter.
We note that our system makes sure at the beginning of each writing session, there is a step that dips into the ink reservoir by going up and down a couple of times, and then it removes excess ink and smoothes the tip of the brush by performing a swirling motion clockwise and counterclockwise three times each.This is to make sure there is enough ink in the brush as the amount of ink afects the thickness of the stroke during writing.
To facilitate the fne-tuning of the plotter's movement and to accommodate diferent brush sizes, we also created an interface that allows users the adjust the parameters based on the output (Figure 5).Next, we describe the heuristics that enable us to reproduce most features of traditional Chinese calligraphy.

Compensation Algorithm.
When plotting with a rigid pen, the tip's position directly refects the handle's position on the paper.This results in marks being reproduced exactly as the toolpath.Conversely, a calligraphy brush's tip is compliant and "lags" behind the handle of the brush (Figure 4).The amount of lag depends on the softness of the brush, the length of the hairs, and the amount of pressure.Based on the target thickness of the stroke at a certain moment, we empirically devise a function that models the distance between the brush tip and the axis of the handle at a given moment based on the target stroke thickness.Then, we add this distance to the original path in the direction of motion at that point.Intuitively, this "exaggerates" the shape of the original path, which compensates for the "smoothing" of the shape introduced by the lag of the brush tip. Figure 4 is a simplifed case with a constant target thickness.Limitations arise due to the difculty of modeling brush deformation, and adjustments may be required to account for environmental variables.

Algorithm for Creating Tips at the Start and End of Strokes.
At the start and end of each stroke, the tips are not always stylized the same.We simulate the traditional calligraphic techniques of 露鋒 (revealed tip) and 藏鋒 (concealed tip).A gradual lowering motion in the same direction of the stroke is added before the start or end of each stroke to prevent the brush bristles from splitting randomly; this produces the look of 露鋒, or the "revealed tip" (Figure 6 middle).For "concealed tip" 藏鋒, a backward motion is used, i.e., a motion in the opposite direction of the stroke (Figure 6 right).An empirical threshold based on overall thickness is used to determine which technique to use: if the radius of the frst point of the stroke is above this threshold, we surmise that the stroke might be best drawn with a concealed tip and vice versa.

4.2.3
Algorithm for computing the speed of brush movement.It is well known in the sketching community that human drawers tend to change the speed of pen movement as they draw diferent shapes [38].In traditional calligraphy, the speed of the brush movement also varies along each stroke, often creating a certain rhythmic pattern.Our observation reveals that strokes are slow at both ends and fast in the middle, which we model with a section of a sinusoidal curve.We also noticed that thinner parts are faster, which we model with a linear function.To get the fnal speed, we multiply the complement of the frst function by the second: we use the formula 1 − 0 (1 − sin()) • ( 1 + 2 ), where t is the percentage progress in the stroke, z is the thickness, and c0, c1, c2 are constants dependent on hardware and setup.For example, if the rhythmic pattern should have a larger efect on the speed, one could increase 0 ; if speed should respond to z height in a more sensitive manner, one could increase 1 and decrease 2 .These could all be tweaked according to the hardness of the brush, raw-ness of the rice paper (how fast ink spreads), as well as personal taste.The output of this expression is between the normalized range of 0.0 to 1.0, where 1 means very fast, and 0 means very slow.This value is further mapped into the feed rate of the particular machine being used.This simplistic heuristic can be improved upon through additional labeling of data of, for example, actual calligrapher's movement.

PERFORMANCE EVALUATION
Before evaluating how well the system is able to support learners, it is important to frst understand how well the system is able to recreate the static strokes.We developed a method to compute the distance between two given renderings of the same Chinese character.We use this same method for comparing the plotter outputs against the originals and evaluating participants' learning results in our user study, detailed in later sections.Our system quantitatively measures the similarity in position and thickness of each point on each stroke, resulting in a lower error score for more similar renderings.

Evaluation Method
We frst pre-processed the input images by applying an image threshold to separate the characters from the background, and then we extracted the polyline skeleton from the binary image using the trace-skeleton library [8].Next, we match the data points from two images by taking the median point of each polyline and using the Jonker-Volgenant algorithm [10] to compute a linear assignment.The linear assignment problem is an optimization problem where a set of things need to be matched to another set of things.In our case, the "things" to be matched are the median point of each polyline.Each assignment has an associated cost, and the cost is the distance between matched points.The goal is to minimize the total cost.To match the strokes, we again use Jonker-Volgenant on only the endpoints of the strokes since we know that the points on a polyline are ordered along the polyline.Every data point in an image is now matched to those in another image (i.e., a bijection).Since the scale and image translation of the characters might be inconsistent across photographs, we align the data points by computing a transformation using a least squares approximation.Next, we compute the Euclidean distance transform [23] of the binary image to give us the distance values for the position distance and thickness diference, where a perfect match would be a distance of zero.

Evaluation Results
We chose Yan Zhenqing's (顏真卿, 709-785) calligraphic masterpiece Duobaota Bei (多寶塔碑), as Yan's writing is typically used as a template of the standard font for learners.The full script consists of thousands of characters, and we semi-randomly selected ten characters that have a certain level of complexity in terms of stroke count and variations of the stroke thickness.Besides subjective evaluation, we used the algorithm described in the previous section to quantify the distance between the script and the system's output.Figure 7 shows the plotter's output compared to the original script.The brighter the overlaid color indicates a bigger error.Note that there is no upper bound of error as a character could theoretically written completely diferently.While we would not claim that our outputs are nearly as good as the original masterpiece, the output looks visually pleasing.More importantly, it reproduced many key features in this calligrapher's work, such as the embellishment at the stroke endings and the stroke thickness.Of course, a more stable system (e.g., with a more precise digitalization of the stroke data and prediction of tip features) would enable even better results.

USER STUDY
The study's main goal is to evaluate our system's efcacy in teaching novice users calligraphy writing.The hypothesis is that by observing the characters being written by the plotter while simultaneously feeling the change in movement through the plotter-driven brush in their hand, the learner's ability to perceive and understand how to produce the depth variation can be improved.Our method is compared with watching a video instruction as a visual-only control condition.

Procedure
We conducted a within-subject study with twelve participants.Figure 8 shows the study protocol.The participants were recruited with the condition that they are right-handed and have little to no exposure to calligraphy writing.The study consisted of two rounds of 10-minute demonstration-followed-by-rewrite tasks.During the 10-minute tasks, participants were given the freedom to learn and practice however many times without a defned goal of character count.This is to mimic how learners typically learn at their own pace.There were two demonstration conditions: watching video playback and following along with the plotter by holding the brush.For the video playback condition, the participants watched a recording of a calligrapher writing the character flmed from the top.For the plotter condition, participants were instructed frst to watch the plotter move and write, and then they were given a chance to familiarize themselves with the device and how to hold the brush and move with the plotter.Then, they were asked to hold the brush during the plotter's movement.
The chosen characters, "十" (shi) and "人" (ren), each having merely two strokes, combined cover the frst four of the most important fundamental stroke types (一 the horizontal, the vertical, and the diagonals).Starting from these building blocks, the learner can construct almost all Chinese characters through various combinations thereof.Additionally, more complex characters would require memorization of the stroke order and some baseline understanding of Chinese characters, abilities that must be gradually acquired over time.In fact, it is a common sentiment among calligraphers, that simple characters are harder to write well: subtle faws become more apparent when one could not dazzle the audience with a furry of strokes.The same is true for both the human and the computer "eye": by having fewer overlapping and intersecting strokes, we could also obtain a more accurate algorithmic analysis of how well each stroke is performed.
In a demonstration phase, participants were instructed to learn with the goal of repeating the style of the demonstrated writing.Immediately after the demonstration, the participant was asked to rewrite without intervention.Instead of giving a target character count, we blocked ten minutes for each demonstration-followedby-rewrite task.We gave the participants the freedom to learn at their own pace as this is the closest to how learning takes place outside of a study setting, where no instructor is present to guide the learner.Participants learned each character with either demonstration method.The two characters and two conditions were presented using a balanced Latin square across participants.The results were compared within subjects.After each condition, participants completed a questionnaire about their perceived sense of successful acquisition of skills and cognitive load.At the end of the study, participants went through a semi-structured interview.

Analysis and Results
We performed statistical analysis on the quantitative questionnaire data, and we also coded and thematically clustered participants' think-aloud and post-study interview statements to extract trends.6.2.1 Replicability.Firstly, we were interested in how well the participants were able to recreate the characters.During each 10minute demonstration-followed-by-rewrite task, each participant wrote a diferent number of characters, and thus we did not use all characters written for evaluation.Instead, we chose the best character from each condition for this evaluation based on the criteria defned in Section 3. Briefy again, these criteria include the similarity in stroke thickness and the overall composition and positioning of the strokes.One category of criteria that is hard to tell solely based on the visual output is the criteria that involve the process taken during writing.These include the order in which the strokes were written and the poster of the hand and wrist.Thus, taking into account all of these criteria, we employed a manual screening of the recording (from a top-down view) of the participants during writing, and the best overall character of each session is used for the later evaluation.
The same method was used in section 5.1 to compare the similarity between demo characters and participants' writing.We computed the amount of error for each participant, where the lower the error score, the more similar the character is to the example, hence better.Figure 9 shows the participant's writing, where a brighter color indicates a bigger error.We performed an ANOVA on the data as the scores had a normal distribution (Shapiro-Wilk normality test p<.05).The plotter condition had a statistically signifcant lower score compared to the video condition (F=8.96,p<.01).
In the questionnaire, participants were asked to rate "Q1 -how well you were able to replicate the movement" (1 -not at all, 7exactly), and "Q2 -how much you felt in control of your movements when you rewrite the character" (1 -not in control, 7 -in control).For each question, we performed an ANOVA on the data as the scores of the questions had a normal distribution (Shapiro-Wilk normality test p<.05).As shown in fgure 10, for both questions, the plotter method was rated slightly higher (although not statistically signifcant) than the video method (Q1 video: AV=3.58,SE=.40; plotter: AV=4.00 SE=.35 Q2 video: AV=4.25,SE=.48; plotter: AV=4.67,SE=.40).
Most participants commented on the importance and usefulness of being able to not only see but also feel the change in depth in the brush stroke with the use of the plotter.Especially for the "人" character, the change in thickness was shown more saliently through the plotter's movement.Participants commented that with the video demonstration, they focused on and overanalyzed the subtle movement but then realized the pressure matters.In contrast, experiencing the change in depth with the plotter helped participants "not only emulate and but also remember the movement" (P9) better.The plotter also provided "a sense of the scale of the movement" (P5) and demonstrated close-ups of how the brush tip moves due to the movement.In addition, participants made fewer mistakes in the stroke order when learning with the plotter as one participant noted "I felt it was easier to learn the stroke order on the plotter because I was practicing the actual motion more." (P4) 6.2.2 Cognitive load and agency.In addition to evaluating how well participants performed, we were also interested in the cognitive load and the reported sense of agency of the participants to assess the efcacy of the learning method.We followed the original 1 to 7 scale of the NASA TLX questionnaire, where a lower score means less task load and more desirable.For each question, we performed an ANOVA on the data as the scores of the questions had a normal distribution (Shapiro-Wilk normality test p<.05).The overall task load for the two methods is similar (video: AV=3.33,SE=.26; plotter: AV=3.38,SE=.20) (Figure 10).The plotter method did not reduce participants' cognitive load during the task, which is somewhat expected.Introducing an additional modality of sensory input (i.e., kinesthetic feedback) on top of the visual cues requires the participants to consider and consolidate both sources of information.
Most participants were frustrated by the discrepancy between how easy the task seemed to be (the characters appeared to be simple) and how difcult it was to write using a brush.In the interview, some participants mentioned the beneft of using a plotter as a way to experience the movement from the "perspective" of the teacher: "I found it hard to map to other people, so I try to overlay myself with the teacher" (P3).Participants noted the plotter's movement was "surprisingly smooth" and felt that the machine was guiding them: "it wasn't pushing me, and I'm kind of in control" (P6).

Engagement and Afect.
Another important aspect of learning is one's afect.We also asked participants to rate "Q3 -This was an efective way to learn", "Q4 -This was an engaging experience", and "Q5 -This was a fun experience" (1 -strongly disagree, 7 -strongly agree).The ANOVA results showed that the plotter method was rated signifcantly higher and thus more efective, engaging, and fun as a learning method (efective: F=4.57, p<.05; engaging: F=14.72, p<.01; fun: F=10.05, p<.01) (Figure 10).Specifcally, participants commented that learning with the plotter was "intuitive and fun" (P1).Participants mentioned that learning from the plotter made them feel less judged than having a human mentor.However, one participant felt the plotter was "harder to relate to" (P8) compared to human demonstration and had "a fear of breaking the machine" (P2).Some participants commented on the efect of comparing themselves against the plotter or the video demonstration.One participant noted: "I feel less frustrated comparing myself to the plotter" (P10), and another participant mentioned they felt nervous and hesitated a lot more when being "confronted" with the video: "I also constantly compared myself to the video so I stopped a lot more in the writing" (P11).These comments point to the nuanced "judgment-free" aspect of using a machine for instructions.

DISCUSSION, LIMITATIONS, AND FUTURE WORK
Overall, compared to the video-based learning method, the plotterbased method, which involves physically engaging with the movements required for calligraphy, provides a more efective learning experience than watching instructional videos.The hands-on approach allows learners to better understand stroke dynamics, increases their confdence, and reduces self-consciousness.Additionally, the plotter condition helps learners grasp nuanced skills such as depth variation more intuitively.
We also now share additional fndings directly elucidated using the prototype and calligraphy writing as the case study of a fne motor skill.Then, we broaden the scope of recommendations to design systems that support motor skill learning.
Figure 10: Survey results of how participants rated the two conditions.Overall, the plotter method was rated as a signifcantly more efective and engaging method to learn while allowing the learner to feel they are better able to replicate the character and have more control.

Device Limitations
Writing calligraphy is a complex, fne motor skill that involves many intricate hand and arm movements and muscle control.The plotter method is inherently limited by the device's three degrees of freedom (DoF); thus it was chosen to focus on helping learners better perceive the change in depth and pressure.A comparison between the system and a multi-DoF system like a robotic arm would be valuable.
Currently, the plotter follows pre-determined movements that the learner passively follows.Even though the current system has a buttoned input that indicates the user's intent to start and stop following, an ideal system would allow the learner to have more agency in their movement.From the hardware design, one way to modify the current hardware to provide adaptive feedback is to use DC motors with encoders instead of stepper motors.This would enable adaptive force feedback using proportional-integral-derivative (PID) control, where instead of moving along a predetermined path, users could freely move on the canvas when the DC motors are turned of.In addition, using our analysis algorithm that was built to score the participants' writing, we could build an adaptive haptic guidance system based on the learner's performance as they progress.
Learners might want to practice one stroke repeatedly before moving on to the next.One can do so by choosing a specifc stroke as the full character.In addition, the button allows the user to pause in the middle of the character.But it would be nice to repeat only a portion of the character.However, it is worth noting that calligraphy writing emphasizes the overall fow of fnishing one character in one go.Furthermore, learners might also want to monitor the progress over time, and future work can implement a computer vision program that continuously calculates the diferences between each character and the model character.
The current toolpath generator is not intended as a comprehensive tool.As of now, it is used for previewing the plotter path and for easier adjustment of the plotter parameters.Future work can make the tool path generator more user-friendly.Future work can also modify the current system to have more degrees of freedom for more complex motor tasks by, for example, giving the end efector the ability to rotate.

Generalizability Beyond Calligraphy Learning
The frst and the most obvious extension of this work is learning other types of calligraphy (e.g., Arabic Calligraphy).One can adapt the approach of translating static traces of diferent languages into a time series of movement paths.We hope that our path generation tool makes learning other types of calligraphy and writing more accessible.
Beyond learning calligraphy, the brush can be replaced with other tools; skills and experiences that require a focus on depth perception can beneft from a similar setup as our system.For example, a drilling simulation where the user learns the right pressure and depth for diferent materials; a dental training program for learning how to perform periodontal probing; coupled with a Virtual Reality display, one can simulate the haptic feedback of feeling the surface relief of a painting or a piece of sculpture that is otherwise inaccessible.
Besides motor skill learning, our system can be used to represent force feedback across distance and time.An early inspiration for this work is T.M. Riddle's Diary in Harry Potter, where words appear magically on blank pieces of paper and one can communicate with a ghostwriter.Similarly, our device can mimic the presence of a remote or an absent person.
From a system design perspective, we show the importance of focusing on a specifc aspect when teaching a complex, fne motor skill.In the next paragraphs, we share insights that are also generalizable to other fne motor tasks and multi-modal learning.

Identifying the Learning Goal to Determine the System's Capabilities and Constraints
For fne motor skills like calligraphy writing, there are many aspects that a learner needs to master.However, having too many foci can overload the learner.For novices especially, building up foundational skills frst is important.Our approach is to break down the complex task into smaller components of the skill and each system focuses on one particular aspect.The constraints of the system help isolate particular aspects of the motor skill for training.Our system limits the learner to follow along with the demonstration, and it helps the learner to build associations between when the change in the brush's height (i.e., due to downward pressure) happens during the writing.To design systems that support learning fne motor skills, it is important frst to defne the learning goals and success metrics, thereby quantifying the error or areas for improvement.The learning goals then guide the design of appropriate machine capabilities and constraints.

Designing the Spatiotemporal Aspects of System Intervention
Once the system's capabilities are determined based on the learning goals, one should consider its interaction with the learner.Systems would require diferent interactions to elicit diferent learning experiences.Here, we mainly discuss the interaction design space in the spatiotemporal dimension as it is the most generalizable beyond the aspects specifc to the type of motor tasks.Spatial colocation is an important aspect of motor learning, especially as a shift in the spatial orientation introduces additional cognitive load.For example, Kirsh et al. prove that in the game of Tetris, players perform epistemic actions "to uncover information that is hidden or hard to compute mentally" [16].In calligraphy writing, learners typically observe the teacher from a third-person point of view.However, when the machine provides spatially colocated feedback, it allows the learner to perceive the feedback in the same coordinate frame, and thus no mental rotations are needed.For example, participants commented that having direct feedback in their hands helped, and they often referred back to the machine by overlapping themselves with the machine's movement as a reference.
The temporal coincidence of the system feedback and the change in outcome allows learners to build an association between the feedback and the outcome.The plotter provides real-time feedback to the learner's hand while writing the strokes.The temporal coincidence of the haptic and visual feedback allows learners to connect the change in stroke with the change in z-depth more intuitively.
On the other hand, there is a contention about when the machine intervention should happen.A turn-taking approach separates feedback from the machine during training from practicing.However, if feedback happens while the user is practicing, a misalignment or mismatch between the machine's and the human's intended action can lead to confusion and disruption in learning.In the following sections, we will delve into the impact of the dissimilarities between the user and machine's behavior and model on the user's perception of the machine's role in human-machine interaction.

Preserving Learner's Sense of Agency
During motor learning, the learner's mental model of the task adapts based on the feedback they receive from the machine's interventions, as well as by adjusting their movement after comparing the expected outcome to the actual outcome.The Sense of Agency (SoA) that the learner experiences is a refection of the contingency between their action and the resulting efects.The machine's intervention during training can afect this contingency, and the hope is that the learner would perform better in the absence of the augmented feedback system.Thus, preserving the learner's SoA is an important aspect to consider when designing augmented feedback systems.
The sense of agency becomes even more important when there is tight integration with the user's body.In the case of this system, the learner has full control over their hand (e.g., deciding how and when to grip).However, the presence of the machine's intervention during training does not allow the learner to exert much efort.Therefore, the lack of participation does not allow the learner to identify the consequences of their actions clearly.Once the machine is "removed" during practice, as we found during the user study, the learner still needed to recall and refer to the training experience.
Ideally, an augmented feedback system is like a "disappearing pen" [6] that enhances the user's skills without getting in the way.As the learner adapts to the machine's model, they calibrate their movement and update their internal model.With exposure to the machine's behavior, the learner also adapts to their expectations of the machine's action over time.Throughout this process, the machine can also sense and learn the user's capabilities, gradually shifting to a more behind-the-scenes role.By prioritizing the user's agency and adapting to their needs, the machine can allow the user to be in control of the learning experience.

CONCLUSION
This paper presents a system that recreates calligraphers' movement from static scripts using a three-axis brush plotter, enabling hobbyists to learn calligraphy writing.We showed that the system is able to reproduce calligraphy characters with high accuracy.We conducted a user study to evaluate the benefts and challenges of our system in teaching novice learners.We found that the system was able to efectively help learners perceive and reproduce the variations in stroke thickness and overall shape.Learning using the system was also much more engaging compared to learning from a video demonstration.Insights from the design process and evaluation of the system suggest the potential of augmented feedback for learning a complex motor skill.

Fitts
and Posner describe the behavioral process of acquiring motor skills in three stages: Starting with the Cognitive Stage, where an individual is trying to fgure out what to do; the Associative Stage,

Figure 2 :
Figure 2: Left: The system's hardware consists of a three-axis plotter.Right: The modifed z-axis gantry allows the user to experience continuous movement in the z-direction.

Figure 3 :
Figure 3: Pipeline for generating movement paths from static characters.The top and bottom characters are examples of the standard and cursive font, respectively.

Figure 5 :
Figure 5: A graphical user interface for previewing the fnal toolpath of the plotter and adjusting diferent parameter values to accommodate diferent brush types.

Figure 6 :
Figure 6: Intricate techniques in Chinese calligraphy that reveal by landing the brush tip gradually (middle diagram) or conceal the sharp edge of the brush tip by moving in the opposite direction of the stroke's path (right diagram).

Figure 4 :
Figure 4: The compensation algorithm takes into account the "lag" in the toolpath due to the compliance of the brush tip.

Figure 7 :
Figure 7: Comparison between the example script and the plotter's output.The frst row is the original characters from the calligraphy masterpiece.The second row is characters written by the system.The third row overlays the calculated error on the plotter's output, where the brighter the color, the bigger the error.

Figure 8 :
Figure 8: The within-subject user study protocol where each row represents one of the procedures for a participant.

Figure 9 :
Figure 9: Results from the user study.Each character is the best attempt from the participant.The lighter the overlaid circle, the greater the error.The box plots show the plotter condition has a signifcantly lower error rate compared with the video condition.