Griffith: A Storyboarding Tool Designed with Japanese Animation Professionals

The “E-conte,” storyboard in English, is commonly referred to as the “blueprint” in Japanese animation (anime) production, consisting of scene illustrations, timing information, and textual descriptions. This paper introduces “Griffith,” a digital system for creating these storyboards. Due to its highly cultural and domain-specific nature, the tool design entailed an in-depth study of the E-conte process and a longitudinal collaboration with an experienced anime director and producers. The resulting system contributes not only domain knowledge, but also generalizable insights into a creativity support environment for visual storytelling, including the importance of vertical timelines and discrete yet integrated tools. To reflect on the interaction design, we presented Griffith to professionals with diverse roles in anime production. Our findings highlight the benefits of the Griffith user interface and the need for a socio-technical focus in designing creativity support tools.


INTRODUCTION
The global popularity of Japanese animation (anime) has grown immensely [49].With its market size having doubled over the past decade (2011-2021), reaching 2.74 trillion yen in 2021, and its overseas sales surging from 0.27 to 1.31 trillion yen, its cultural significance is recognized worldwide.Despite its popularity, the anime production process has remained on the fringes of computer science research.The production process was modeled after industry innovators such as The Walt Disney Company (Disney) in the 1950s and was entirely analog.The production process has been gradually transformed to take advantage of digital technologies, but its progress has not been widely shared in the academic community, in contrast to how Pixar and Disney have developed their 3D computer graphics (CG) animation production process with active engagement in the CG research community.
Nevertheless, the importance of hand-drawn 2D illustrations in animation production remains unchanged, not only in the anime industry, but also in Western studios and those focusing on 3D CG animation, such as Disney, which rely heavily on hand-drawn sketches to prototype lively movements of characters and objects.We believe that by reporting on the efforts to innovate the 2D animation production process in the underrepresented non-Western world, this paper serves as a first step toward a diverse landscape of the future of animation production, allowing the research community to compare cultural differences and understand fundamental challenges in creativity support for 2D animation.
In particular, this paper sheds light on the overlooked but paramount aspect of anime production: the "E-conte, " pronounced like "Eeh-con-te, " and translated as "storyboard" in English.We propose a digital storyboarding tool, Griffith (Figure 1), for anime directors and storyboard artists.It aims not merely to substitute a pen and sheets of paper in the E-conte format, but to enhance the entire storyboarding experience.Studying such a creativity support tool is intriguing for three reasons (detailed in Section 2): First, the E-conte Figure 1: The main view of Griffith.It shows the vertical timeline on the left and the infinite drawing canvas on the right, which can be optionally switched to the script with the toggle button to resemble the actual E-conte format. is considered the "blueprint" of the actual anime film in a highly standardized format and serves as a communication medium with a prominent role in anime production.Second, while most facets of anime production have transitioned to digital, E-conte and its tools remain analog despite their importance, posing the research question of "Why not?" Third, although E-conte translates to storyboard in English, its authoring process has differed considerably from Western animation studios like Disney, contributing to a diverse cultural landscape of creativity support tools research.
We recognized the need to gain significant domain-specific knowledge to design and develop Griffith, and, therefore, closely collaborated with anime industry professionals.The process of this longitudinal study is described chronologically in Appendix A. To reflect on the interaction design, we showcased Griffith to professionals with diverse roles in anime production and relevant fields.Their qualitative feedback highlights the benefits of the Griffith user interface and the need for a socio-technical focus in designing creativity support tools.The longitudinal development process and the resulting system together represent design and artifact contributions in the underexplored domain of anime production, and more interestingly, we found generalizable insights into a creativity support environment for visual storytelling, including the importance of vertical timelines and discrete yet integrated tools.
It should be noted that the authors have presented some work on anime storyboarding in an academic context [25,[27][28][29][30], but this paper is the very first academic and archival work 1 .The next section builds on these previous efforts and provides a first in-depth study of the practice of E-conte.It concludes with the design principles for the digital anime storyboarding tool.The following sections present the development process of an anime storyboarding tool and report unique insights from the HCI research, including the detailed user interface design and its evaluation.

E-CONTE: A MEDIUM FOR ANIME STORYBOARDING
While storyboarding in general is well covered in Pallant's seminal work [51], we could find no archival reference that thoroughly explains the role of E-conte in the overall anime production process and how the tools and environment for authoring E-conte have evolved over time.This section reports insights from the existing Japanese literature and our interviews with anime industry professionals, whose process is disclosed in Appendix A. A rough comparison table of storyboard-related aspects in Japanese and Western animation production processes, summarizing the discussions in this section, is provided for convenience (Table 1).

Role of E-conte in Anime Production
The anime production process consists of pre-production, production, and post-production (Figure 2).While pre-production involves around ten people, production and post-production involve over a hundred in most cases [58].The E-conte is authored based on 1 An oral presentation at the annual conference of the Society for Animation Studies [28] was accepted by submitting a 500-word abstract under a very different academic discipline in the humanities and the conference did not produce proceedings.An invited article for ACM XRDS [25] was an essay discussing the challenges and benefits of HCI research "in the wild" that only scratched the surface of Griffith's development process.The other antecedents were poster presentations without papers.scenario script, character design sheet, and other settings information and is considered the "blueprint" of the actual anime filmit is the final output of the pre-production phase and serves as a directional document for animators in the production phase.The role of E-conte as a blueprint is shared with storyboards in the Western animation production process.Unlike live-action films, which often rely on editing after several versions of a scene have been shot, each element in every frame of an animation is drawn intentionally, emphasizing the importance of careful planning in pre-production.
However, what distinguishes E-conte from general animation storyboards is its highly standardized format as a strict specification document.Its format has become unified under the collaborative nature of the production phase in the Japanese animation industry, where many small studios and, in typical cases, many freelance animators work together to create animations.It is usually prepared as sheets of paper in a specific format.It consists of scene illustrations, timing information, and text descriptions aligned vertically from top to bottom (Figure 3).The scene illustrations provide visual explanations of the characters' actions, the situation around them, and how the scene should be captured by a camera.For example, the second row of illustrations shows the boy coming out of the narrow space, and the arrow represents his forward movement as he is surprised and interested in the situation.The script explains not only what happens and who says what (speech script), but also how the characters should be animated (stage directions).
Authoring an effective E-conte requires a precise and rich imagination to bridge the gulf between the abstract specification and the final animated result.To help precisely plan the timing, a director often places a stopwatch next to E-conte sheets of paper and uses it to measure time.The popularity of stopwatches among directors and storyboard artists is evidenced by a stopwatch designed especially for them, with a dial that measured six seconds per lap, with each second divided into 24 equal parts, which is said to have been made for around two hundred [50].They also sometimes speak aloud to check if the time frame is sufficient for the character to say a few words.To aid their imagination, a director and other creators often manually prepare a "previsualization" from E-conte, which turns over scene illustrations in the specified timing.It helps them to understand the rough idea of the animated result, reducing their cognitive workload.
These authoring practices imply that a creativity support tool for E-conte should not only serve as an alternative to pen and paper, but should also include auxiliary features of relevant tools (e.g., stopwatch) and surrounding activities (e.g., previsualization).
When the E-conte authoring is complete, the sheets of paper are copied extensively and distributed to the animators.These provide enough information for the animators to draw the animated scene within the specified time frame.The tangible nature of paper offers the advantage of accessibility; animators and other staff can refer to the content at any point without the need for specialized equipment.However, this analog replication process poses a significant challenge: ensuring everyone, from the director to animators and others, stays updated with the latest version.

Historical and Cultural Perspectives on Storyboarding
With the advancement of digital technology, most parts of the anime production process have gradually transitioned from analog to digital.Examples include scriptwriting, which benefits from versatile tools like word processors and mind maps, and keyframe animation, which benefits from illustration drawing tools.Among them, E-conte was an exception.The graphical and exploratory nature of the E-conte authoring process set a challenging goal for interaction design, calling for a specialized creativity support tool.Existing storyboarding tools, such as TVPaint [11] and Storyboard Pro [22], can export content in the E-conte format.However, when asked to compare these tools with the E-conte format, the director pointed out that their authoring interfaces do not accurately capture the essence of E-conte: the term "E-conte" merges "e" (drawing) with "conte" (continuity).The tool should actively prevent the user from being distracted from the continuous nature of visual storytelling.It should allow easily sketching multiple illustrations, seeing them as a series of frames of continuous scenes rather than independent illustrations, and quickly navigating to the frame of interest.
The characteristic vertical timeline in the E-conte format, where the frames are stacked vertically without much spaces, is seemingly contributing to the improved continuity.When navigating within the temporal domain with a horizontal time axis, the hand can occlude and thereby obscure parts of the preceding frames for left-handed users or the subsequent ones for right-handed individuals.Such occlusion disrupts the continuity of perception between frames.Contrastingly, the E-conte format embraces a vertical time axis, where frames seamlessly flow from the top downward.This orientation ensures that neither the preceding nor the subsequent frames are obscured.
Furthermore, although E-conte translates to storyboard, its authoring process differs significantly from that of Western animation studios such as Disney.The former is a relatively isolated process by a director and a few storyboard artists.Historically, E-conte was developed to prepare for a TV animation series, where episodes had to be created in a hurry compared to Western long animated films.This may have influenced the E-conte authoring process to be very personal and detailed.The whole process of anime production was established like software development in the waterfall model.The E-conte is usually only checked by the director and producers, but not by the scenario writer.The production process takes the Econte as a strict specification document and produces the animation as instructed, so the E-conte remains mostly unchanged, except for minor local changes that may be made in meetings between the director and animators.In contrast, the latter is collaborative and open, as denoted by the term "story" + "board, " which literally means a wall-mounted board to which creators paste scene illustrations and brainstorm on possible storylines [51].The introduction of digital technologies has changed the practice of pre-production in Western studios, although they vary in size (the previous work discusses Pixar, DreamWorks, and Disney as examples of large studios, LAIKA as one of medium-sized studios, and A+C in UK as a small one), but in each case the storyboard remains an important document for collaborative visual story-making.This cultural difference creates a distinct use case of casual and private tool use in the anime industry, not necessarily supported by existing tools.
In addition to these technical and cultural difficulties, developing a novel E-conte authoring tool is also challenging from an economic point of view.This is because the number of tool users, namely directors and storyboard artists in the pre-production phase, is way smaller than that of animators and other roles in the production phase.As such, economies of scale in tool development are less achievable.
Despite these challenges, the director emphasized the benefits of established software features of digital tools, such as undo and redo, cut, copy and paste, and efficient management of design alternatives, which would augment the pen and paper experience in the E-conte authoring tool.These findings are consistent with the previous study of how visual artists use software in their practice [41]: artists value manual creation, love low-level automation features like undo and redo, and are wary of high-level automation features that may disrupt their manual creation workflow.

Design Principles for Griffith
The study on the E-conte as a medium for anime storyboarding revealed the technical, cultural, and economic challenges and the importance of developing a creativity support tool.It informed the design principles for the initial implementation of Griffith: (1) A system that could be used in casual and private settings, not limited to a strict work environment, preferably with a feature to share the latest version with others.(2) A user interface (UI) that allows authoring storyboards interactively while respecting the continuous nature of visual storytelling; with respect to the conventional E-conte format, the following aspects would seem promising and interesting to explore their digital alternatives: (a) A vertical timeline to see frames as a series of continuous scenes, which could prevent hand occlusion (b) A navigational interface to grasp the overview of the storyline and quickly open the frame of interest (3) Auxiliary features to support storyboarding, each implemented as a discrete tool, that comprise a creativity support environment; examples include, but are not limited to: (a) A stopwatch tool to measure time and its integration to apply timing to existing frames in the E-conte.(b) A feature to create "previsualizations" on the fly to bridge the gap between abstract specifications and the final animated result.
Following these design principles, the remainder of this paper focuses on system design to address the technical and cultural hurdles.We consider the economic concerns to be a topic for future work, as outlined in Section 6.3.

RELATED WORK
Before introducing the resulting system Griffith (Section 4), we touch on a theoretical foundation to design the system for artists and related work on authoring animations.

Theoretical Foundations for Interaction Design for Artists
We adopted the user-centered design approach to crafting a tool tailored for anime directors and storyboard artists.A prior survey on creativity support tools research [15] issued a call to move past purely in-vitro studies.Echoing this sentiment, our strategy was an in-vivo study, partnering with industry professionals to create a tool seamlessly integrating into their current workflow.A previous study of how visual artists use, adapt, and develop software [41] suggests that artists can guide technical implementations of domain-specific program representations.Given the long history of anime production, the design of our system was inspired not only by the artist interviews (to reflect current needs) but also by the traditional paper-based format of E-conte (to reflect the accumulated knowledge embedded in the medium, e.g., vertical timelines).
When collaborating closely with designers, software developers (and researchers) often recurrently face so-called "design breakdowns" [46] that can arise from gaps in information, unforeseen edge cases, or unrecognized technical constraints.Four principles to address these issues were proposed [40]: 1) offering multiple perspectives on desired interaction designs, 2) maintaining a single source of truth for the design specification, 3) revealing the invisible constraints, and 4) design by enaction to find more edge cases by experiencing the design in context as soon as possible.In our journey, we did encounter these breakdowns.To address them, we employed multimedia (like screen recordings) to detail interaction designs, centralized discussions in Slack, convened regular online meetings to clarify details of user requests, and set up continuous integration to deploy the latest system to ensure timely and relevant user feedback.
Instrumental interaction [2] is a theory that treats an interaction instrument as a mediator between the user and the manipulated object.Our study (Section 2) revealed the importance to design the E-conte authoring tool as a set of various interaction instruments.Such a complex and realistic setting allows us to experiment and generate a preliminary yet novel theory on how to relate different interaction instruments in a single tool (Section 6.2).This way, our domain-specific system design contributes to generative theories of interactions [3].
During our longitudinal tool development process, there have been times when we have implemented features based on theories of effective interaction design.Conversely, there have also been times when we realized after the fact that the features we implemented were consistent with theoretical principles.For instance, we consider the previsualization feature (Section 4.3.2) to be a good example of an effort to address the well-known "gulf of execution and evaluation" [48].The undo, redo, and copy features reflect the importance of near-term experimentation and exploration of design alternatives [55] (Section 4.3.3).We allow users to write notes in the out-of-frame area of each illustration (Section 4.2.2) and also provide the overview interface (Section 4.2.3), the design of which turned out to be consistent with the creative strategies of professional artists of "margins and view-shifts" [14].Our system design can be seen as additional evidence in support of these theories.

Creativity Support Tools for Authoring Animations
As mentioned in Section 2.2, the digitalization of the 2D animation production process has been almost complete, and TicTacToon [12] was one of the successful early examples of digital animation authoring tools.It was developed in France, acquired by a Canadian company, Toon Boom Technologies, and then renamed to Toon Boom Harmony [23].As the original paper [12] discussed, computer graphics tools are so difficult to apply to 2D animation.The creation process is essentially a production line involving over a hundred people working together to produce tens of thousands of drawings.After almost twenty-five years, this is still true in anime production.
Story Pad [54] is an in-house storyboarding tool developed by Disney for their storyboard artists with a focus on casual use.While the detailed interaction design is not disclosed, in their introductory video they explicitly mention that casual use is beneficial for collaboration because it makes it easier to show the current storyboard to colleagues.Our design principle overlaps significantly with Story Pad in its support for casual use, but with more of a focus on personal use.As such, the storyboarding user interface (UI) could be designed purely for authoring, assuming the user has a working memory for the current content.The content sharing feature does not need to be integrated into the main storyboarding UI and can be implemented in a separate UI.
In the research context, there has been a continuous effort to lower the barriers to author animations.However, most prior work provides user interfaces for directly drawing or manipulating animations, not involving a storyboarding process.For instance, given a single image, multiple control points, and their motions, shape deformation methods such as as-rigid-as-possible deformation [19] could generate animations.When naively applied, these methods cannot handle topological changes or self-occlusion, and commercial tools such as Adobe Character Animator [1] and Live2D [21] utilize multi-layered structured images and achieve so-called 2.5D animations.K-Sketch [8] is an informal tool for prototyping animations with stylus input, applying predefined transformation algorithms to the objects on the canvas.Another work aimed at utilizing offline painted media to stylize objects [47], which are then scanned and animated with digital authoring software.Draco [33] proposes a technique called kinetic textures that enables interactive editing of motion algorithms applied to a collection of objects.Together with its subsequent research, the work was shipped as Autodesk SketchBook Motion.Prior work on live programming allows interactive edits and parameter tuning of programs to generate motion graphics [31].Recent work aims to add interactivity to animating visual content [26,32].
As storyboards are known to convey visual stories instantly, novel applications utilizing the storyboard format have been proposed.For instance, prior work proposed an automated tool to generate storyboards from video clips [44] and a more sophisticated tool to transform video clips into schematic storyboards, which are static visualizations using the visual language of storyboards [17].The recent proliferation of deep learning techniques enabled an end-to-end system that automatically generates storyboards from scenario scripts [6].With the development of media technologies, storyboarding interfaces for novel temporal arts have become possible, such as a tool to author stereoscopic storyboards for stereo 3D films [18] and augmented reality-based storyboards [53].Our work derived its unique design principles from the anime industry, contributing novel insights on storyboarding user interfaces complementing the prior work.
Storyboarding is a popular method for prototyping interaction design [57], and many tools have been proposed [34,39].Storyboarding activities in interaction design and anime production share particular requirements, such as supporting their visual nature and casual use for exploring ideas.However, they significantly differ in the outcome: interaction design prototypes vs. detailed specifications for compelling visual and auditory storytelling.We valued the domain specificity and have yet to learn actively from the interaction design prototyping process.We will briefly touch on this topic as our future work (Section 6.4.1).The research contributions of this paper would be better interpreted as creativity support research for domain-specific professionals, such as a study and novel tool proposal regarding expert designers' practice of interacting with colors [24] and clay artists' practice for novel manual-digital fabrication workflow [4].

Anime Studies
Research on Japanese animation within academia has predominantly originated from the humanities.Researchers in academic communities like The Society for Animation Studies [13] often pose foundational questions, such as "What is anime?"[9] examining anime content through the lens of media studies.Ian Condry's work, "The Soul of Anime: Collaborative Creativity and Japan's Media Success Story" [7], marked a significant shift by adopting an ethnographic approach to explore "Who makes anime?", delving into fieldwork at anime studios and related locales.Our study seeks to answer the "How" and is distinctly shaped by the interventional approach of HCI, aiming to discern "How to (more effectively) make anime?" In computer science, conventional research on anime primarily focuses on achieving an anime-like aesthetic in 3DCG, such as direct manipulation techniques for stylized lighting and shading [56] or view-dependent control of physical simulations to preserve a characteristic appearance [37].The rise of machine learning research also recognizes anime as a promising application domain, as evidenced by the GitHub repository dedicated to "awesome anime research" in signal processing and machine learning [52].
Anime storyboarding remains relatively unexplored.In 2017, a prototype of a digital storyboard authoring tool was proposed [38]; however, it was not substantiated with empirical evidence nor subjected to user study evaluations.As of 2023, to our knowledge, only Storyboard Pro [22] and e-Conte Board [43] are developed explicitly for authoring anime storyboards and utilized in professional anime production.Storyboard Pro's default user interface is reminiscent of standard video authoring tools, featuring a horizontal timeline and a drawing canvas.Similarly, TVPaint Animation, primarily designed for the "Genga" and "Douga" stages, offers an analogous interface.Although both can output to the E-conte format, their experience diverges from traditional paper-based storyboarding.
In a distinct approach, the e-Conte Board has been developed simultaneously but independently of our efforts in the anime industry and was released in 2020 as an iPad application.Its goal is to reproduce the user experience of the paper-based storyboard on the digital platform, presenting an interface that closely mirrors the analog layout with a vertical timeline.While it serves as the most comparable tool to our project, key differences in our approach include the introduction of an infinite canvas that provides "margins" for ideation and annotations (Section 4.2.2), an overview interface designed to aid "view-shifts" (Section 4.2.3), and instrumental interactions [2] of discrete instruments for auxiliary activities (Section 4.3).Additionally, ours can share the latest version of the storyboard on the web while theirs can only export the current version.Aside from these specific differences in features, we believe the critical difference is that ours is a research testbed designed with the aforementioned principles to learn and report findings, while theirs is a final product designed with a paper-based experience in mind.We look forward to seeing more examples of creativity support tools for anime production and beyond that build on our work.

GRIFFITH: A STORYBOARDING TOOL
The long-term development of Griffith, as outlined in Appendix A, has yielded a wealth of features.However, for clarity and relevance, the remainder of this section will only cover those directly related to our design principles.The following overview illustrates the development journey and provides context for this selective focus.

Web-based System for Casual Use on
Tablets, PCs, and Smartphones Current tools for storyboarding mostly work on personal computers whose users need to sit in front of a desk.In contrast, to achieve the design principle (1), it is desirable that Griffith users can use the system in a more casual and private setting.The anime director mentions that ideas can come up at any moment, and instant access to E-conte is critical not to forget them.A combination of pen and sheets of E-conte format paper naturally provides instant access, and our system must provide a comparable experience.
A prior prominent work with an instant access feature is the e-Conte Board, which was designed after the analog E-conte format and implemented as an iPad application.While native applications can inherently provide instant access, we decided to implement Griffith as a web application in the early stage of development, primarily because of the ubiquity of web browsers.The director can author the E-conte on their iPad, Android or Windows Tablet, and other devices with web browsers.Recent browsers implement Web Standard specifications for Progressive Web Apps such as Web Storage API and Web Push API, which aid the realization of socalled "local-first software" [35] and support our decision to focus on personal use of the tool.
The design princple (1) also emphasizes the feature to share the latest E-conte information with others, making a web-based implementation the best choice.Our system shows the E-conte information with the authoring features on tablet and large screen devices and without them when accessed from a smartphone.The producers in the development team were particularly supportive of the sharing feature, as they often need to monitor the progress of the E-conte authoring, which tends to be a secretive process handled solely by the director.

User Interface for Graphical and Exploratory Storyboarding
To meet the design principle (2) of supporting graphical and exploratory storyboarding without distracting from the continuity of visual storytelling, we have implemented interaction features informed by the director interview, observations of the E-conte sheets in the wild, and previously proposed theories for creativity support.

4.2.1
Main view with a vertical timeline (Figure 1).Many creativity support tools, particularly those designed for editing temporal art media, typically present the time axis in a horizontal layout.Such a design choice intuitively aligns with the conventional representation of time as seen in a standard seek bar, where the progression extends from left to right.However, as hinted by the adoption of the vertical timeline in the paper-based E-conte format (Section 2.2), we identified problems with this horizontal orientation, especially when users do not engage with the interface via a hand proxy like a mouse cursor.The fingertip or pen can occlude and thereby obscure parts of the preceding frames for left-handed users or the subsequent ones for right-handed individuals.
In contrast, adopting the vertical orientation in the Griffith main view brought some evident advantages.This orientation ensures that neither the preceding nor the subsequent frames are obscured, irrespective of whether a user operates with a fingertip or pen.Furthermore, the view provides a smooth infinite scrolling experience, navigating from the start to the end of the E-conte information.This is better than navigating between sheets of paper by flipping through many sheets.
Additionally, and perhaps even more saliently, we noticed that vertically aligned frames offer an uncomplicated comparison of the left-right balance within frames.Typically, anime films are projected on screens in landscape orientation.They also depict characters engaged in actions influenced by gravitational forces that move primarily horizontally.These constraints result in dramatic visual storytelling that often relies on positioning within the horizontal frame.Given these multifaceted reasons, we ardently advocate for the vertical placement of the time axis, a principle we have upheld in the main view and across other interface elements that involve multiple frame presentations.

Infinite canvas with vector-based sketching.
While the traditional E-conte format shows textual information next to the vertical timeline of scene illustrations, our interface can utilize that space to show a larger canvas for drawing a scene illustration.The user can optionally drag the border between the vertical timeline and canvas to resize frames in the timeline and the canvas.
Scene illustrations in the E-conte format are the backbone of anime storyboarding.These illustrations convey the visual direction of a specific scene, and their uniqueness is not necessarily in their artistic detail but in their expressiveness.Directors and storyboard artists have their own styles and tastes, ranging from simplistic stick figures and abstract symbols to more detailed sketches.Regardless of the style, these illustrations generally embed emotional cues, amplified motions, and deformations, represented through diverse brush stroke widths.
Contrary to software like Photoshop and CLIP STUDIO [5], which offer extensive features to draw artistic illustrations, the canvas for E-conte authoring should be minimalistic.This is supported by the observation that E-conte sheets typically use a limited color palette, often ranging from a single color (black) to several colors (e.g., black for primary stroke, red for framing and notes, light blue for shades, and yellow for transparent and bright regions).While customization of brushes offers flexibility, the priority resides in the need to swiftly switch between tools, especially dynamically adjusting brush stroke widths.The need for varied brush strokes underscores the importance of a pressure-sensitive brush tool.
An advantage of the digital medium is its ability to introduce features impossible in the analog world.The director pointed out a common challenge in analog storyboarding -the inability to reframe a scene illustration without completely redrawing it.This process of scaling the scene to fit within or focus on a particular frame can be easily managed digitally.Consequently, we integrated a reframing tool, enabling drag-and-drop adjustments within a fixed red rectangle, symbolizing the frame.The presence of the red frame in our digital canvas, indicating the final viewing frame for the anime film, does not restrict artists from sketching outside its confines, realizing an infinite canvas.Drawing parallels to a prior study [14], utilizing "margins" around the primary workspace for external scribbles and notes would enhance creativity.
To further optimize the digital canvas, features such as copypasting frame content were introduced.In order to retain the quality of illustrations even after numerous reframing and editing actions, each brush stroke is stored as a vectorized path as opposed to rasterized pixels.While some vector-based tools erase entire paths with an eraser tool, our tool simulates a traditional eraser by creating clipping masks in areas where the eraser has been used.
Informed by the E-conte paper format showcasing continuity by vertically arranging illustrations, our canvas interface also has an optional mode called "in-canvas frames" to show neighboring frames just as in the E-conte format.While the director appreciated the feature, the digital canvas can elevate this with the ability to superimpose preceding and succeeding frames using specific transparencies.Our tool also implements this technique, known as the "onion skin" feature, prevalent in animation tools like TVPaint Animation.The user can switch between in-canvas frames and onion skin features (or turn both off) to match their taste.4).The exploratory E-conte authoring process inherently necessitates recurrent reflections on the overarching narrative.Traditionally, directors and storyboard  artists, like our consulted director, often rely on flipping through hundreds of paper sheets to achieve this holistic understanding.

Overview interface (Figure
Recognizing this, our director envisioned a distinctive user interface (UI) that would streamline this process, allowing him to grasp the broader picture quickly, benefiting from the working memory of the user.His conceptual sketch of this desired UI (Figure 5) visualizes frames in sizes that vary but not randomly.Each column has frames of different sizes.The frame of current interest is displayed in the centermost column in the largest scale.The size progressively shrinks as the distance from this focal frame increases, with the immediately adjacent frames being slightly smaller, and this shrinking effect continues outward.This arrangement can be considered a slight modification of the "Fish-Eye" visualization [16], where each row maintains a constant size, providing a dynamic and visually intuitive perspective.
However, implementing such a dynamic visualization poses technical challenges.It demands intricate calculations to determine each frame's size and necessitates constant layout adjustments whenever a user's focus shifts to a different frame.Also, showing large frames at the screen center would make the distinction between that interface and the main view vague.Explicitly switching from the main view to the interface showing the overview allows the user to check the recent changes they made in a larger picture, a frequently-used strategy of creative professionals known as "view-shifts" [14].
Given these pragmatic and theoretical backgrounds, we opted for a straightforward approach in our initial implementation (Figure 4).We provided an overview interface that presents a multitude of frames of the same size.The display's capacity is contingent on the device's screen size, but it typically can accommodate upwards of 400 frames.For extensive storyboards surpassing this number (for instance, a thirty-minute TV animation episode might span around 200 E-conte sheets, translating to over 1000 frames), the interface furnishes a scrolling feature, ensuring left and right navigation.

Discrete Instruments to Support Storyboarding
We incorporated auxiliary features to support storyboarding to meet the design principle (3).Here, we highlight four notable "interaction instruments" [2] that mediate interactions between the user and the object of interest.The first two instruments are explicitly listed as subgoals (3a, 3b) and would extend or strengthen the theory of instrumental interactions.The third was an interesting small interaction design study whose result aligns with the prior work [55].The fourth was an excellent example of a pragmatic effort to keep the bond between the conventional analog tool and its digital counterpart as much as possible when using an analog tool is inevitable.For every instrument, we start by detailing the needs informed by the expert interviews, followed by our actual implementation.
4.3.1 Stopwatch (Figure 6).Need: As the design principle (3a) specified, using a stopwatch is a pivotal part of the E-conte authoring process.After drawing a scene illustration, the user may use the stopwatch to gauge a character's speaking duration.They might also navigate scenes to identify and correct untimed frames, ensuring the E-conte details are production-ready.They often employ the stopwatch merely to measure time, sometimes without associating it with specific frames or scenes.This can be because of timing mistakes or because they envision the needed scene illustrations and their durations before the actual drawing.Implementation: Recognizing its diverse use cases, we designed the stopwatch as a discrete tool that operates independently yet can interact with the main storyboarding interface.It exists as a draggable overlay on the main interface, enabling quick time measurement and, if needed, assigning that time to a specific frame.When manually entering timing data via a modal dialog on the main interface, users can still access the stopwatch.Clicking its button inputs the measured time into the dialog.This natural interaction results from a chain reaction: the clicking action assigns the time to the frame of interest, and then the reactive component in the modal dialog shows the updated time.With this example, we showcased that a natural interaction between the instruments can be achieved by appropriately setting the reactive connections between each instrument and the manipulated object.7).Need: As the design principle (3b) specified, the feature to generate and play previsualizations on the fly is important as it bridges the gap between the abstract specification of the E-conte information and the final animation, reducing the user's mental workload.The user may want frequent previsualization checks for sequences of continuous frames grouped by the same camera work, often referred to as a "cut."Moreover, they might also want previsualizations for multiple cuts or over a custom range.

Previsualization (Figure
Implementation: To support the typical use case, we embedded a button to trigger per-cut previsualization at the beginning of every E-conte cut in the vertical timeline of the main view.Additionally, we implemented a tool to specify a custom range on the overview interface that triggers tailored previsualization.These are two kinds of reification of the same interaction instrument for showing the previsualization player.The former was a one-click button, as the per-cut header is already bound to frames in a specific range.The latter needed a range-selecting interaction, as the overview interface shows all frames and needs filtering.

Local undo and redo. Need:
The E-conte consists of many frames.The user repetitively selects a frame through the vertical timeline in the main view or via the overview interface and edits its content.As the established software feature, the user would expect undo and redo to be implemented, but how?For instance, the user draws a stroke on frame A, then switches to frame B and draws another stroke.If they return to frame A and trigger undo, should the stroke in frame A be removed, or should the one in frame B be deleted?Alternatively, should the system update the frame of interest back to frame B without removing any strokes?
Implementation: Through user testing, we discerned a preference for local undo-redo actions tied to the currently visible frame, which removes the stroke in frame A in the prior example.This decision acknowledges user assumptions that prioritize frame edits over navigation and ties undo-redo actions closely with visible data.Such a design aligns with the creative need known as "near-term experimentation" [55], allowing the user to iteratively assess the quality of the edits on their scene illustration.4.3.4Script import and authoring.Need: The E-conte is drawn based on the scenario script.In the conventional E-conte format, each frame contains script, which extends the scenario script to include further textual direction for animators.The storyboarding tool should also support writing such script based on the scenario script.
Implementation: Informed by the director's interview, we implemented a scenario script import tool that takes a plain text script and inserts blank frames, each corresponding to a line in the script.Since the script is highly standardized to represent one message in one line, such a simple import tool seemed already useful.There is a toggle switch at the top of the vertical timeline that overlays or hides the script on top of each frame.The user can freely edit the text for each frame.

4.3.
5 Conventional E-conte export.Need: Despite the digital nature of our tool, there is an inevitable pull towards traditional methods, particularly with senior animators who lean towards paper mediums.
Implementation: The E-conte export feature outputs the traditional E-conte format with an added modern touch: a QR code on each page.Scanning this code redirects to a digital version of the respective frames, ensuring a bridge from the tangible to the digital realm.

QUALITATIVE EVALUATIONS WITH ANIME INDUSTRY PROFESSIONALS
The domain experts have continuously informed our study and longitudinal development process as outlined in the Appendix.As a measure to enhance objectivity and investigate overlooked aspects, we conducted additional casual interviews by demonstrating Griffith to a broader pool of anime industry professionals in February 2022.Moreover, in February 2023, we interviewed the anime director who helped us throughout this work to validate the system design holistically.To aid the purpose, we asked him to use the system to author an E-conte for 90 seconds of an anime film before the interview.

Casual Interviews with Professionals
In February 2022, we completed implementation of the essential features for E-conte authoring and introduced Griffith to a select group of professionals with diverse experience in anime production.Each professional was given a demonstration, followed by a onehour interview.Our participants were: • P1) A director of an anime studio, having working experience as a producer, scenario writer, and storyboarding artist • P2) A director of a different studio who is an experienced anime producer • P3) An animator and storyboarding artist • P4) A storyboarding artist • P5) An anime producer who participated in the interview study and also serves as the director of an anime studio All of these professionals in the anime industry appreciated the novel storyboarding tool developed within the industry.P1 appreciated the support for the casual use case and commented that the previsualization is particularly appreciated from the producer's point of view, as its manual creation in their conventional workflow required heavy desktop applications like Adobe Premiere Pro.He also suggested that novices would appreciate the casual use, and the tool would lower the barrier to learning storyboarding.P2 considered the overview interface a critical feature and appreciated the main view that automatically calculates the sum of each frame duration and the canvas that can accommodate freeform notes.P3 and P4 had prior experience drawing storyboards with desktop applications (ProCreate, an illustration drawing tool, and Storyboard Pro) and valued Griffith's capability for casually sketching rough ideas with a pressure-sensitive brush tool.P3 also appreciated Griffith's built-in support for scenario script (Section 4.3.4),which was missing from ProCreate.P4 and P5 particularly favored the feature of sharing the latest E-conte with others.
There were also expectations and comments on the missing features.P1 felt the need to set the aspect ratio to the portrait mode of smartphones, which would require another user interface (UI) layout.P2 requested a feature to import images and videos, such as scanning data of the manga (Japanese comics; many anime films are made after existing mangas), character design sheets, and other reference materials, which would contribute to a more holistic creativity support environment for storyboarding.P2 and P3 desired a tool to specify camera work (e.g., pan up and pan down), which would consequently require the extension of the previsualization feature to simulate such camera motions.P4 requested detailed canvas features, including shortcut keys and layers in each scene illustration (to ease copy and paste of background illustration, excluding the foreground characters).
Despite their appreciation and expectation of the Griffith UI, a recurring topic of discussion was the potential adoption of Griffith throughout the production process by various roles of creators, from directors to animators, which will be detailed in Section 6.3.

Authoring an E-conte with Griffith
In February 2023, we asked the anime director to author an E-conte with Griffith.He could draw over 700 frames, resulting in 148 cuts, within a week.Then, we interviewed him for an hour.It is worth noting that the stopwatch feature was not implemented at this point, and while he could manually assign timing with textual input, many frames were left unspecified.
He generally appreciated individual features such as inserting new frames between existing frames (which is literally impossible or requires physically cutting and pasting frames in paper), the overview interface to grasp the holistic view, "onion skin," reframing, and support for local undo and redo.Additionally, he offered suggestions for improvement, including the stopwatch feature (already implemented to date), layers in each scene illustration (aligned with P4, Section 5.1), and a feature to import reference materials and specify camera work (aligned with P2 and P3, Section 5.1; discussed again in Section 6.4.2).
In August 2023, we followed up to get feedback on the vertical timeline and stopwatch features, whose results are discussed next.

DISCUSSIONS
Based on the qualitative evaluations, we discuss how the proposed system worked and report insights for future work.

Vertical Timelines
To enable a qualitative comparison of the benefits of vertical timelines against horizontal ones, we developed modes for both orientations in the main view and overview interface (Figure 8).
6.1.1Vertical timeline in main view.Within the main view, we confirmed with the anime director the benefits we considered in Section 4.2.1 : smooth infinite scrolling without hand occlusion and uncomplicated comparison of left-right balance within frames.In addition, we noticed that the division of the screen space into two parts, the vertical timeline on the left and the canvas on the right, did not disrupt the perceived continuity within the e-conte.The user scrolls along the vertical timeline when they want to navigate and switch between frames, without paying much attention to the canvas.Therefore, hand occlusion on the canvas, which occurs for right-handed users, did not seem to be a problem.When the user draws on the canvas, they are typically only interested in adjacent frames, which are always visible on the vertical timeline for right-handed users, or can be made visible on the canvas with the "in-canvas frames" mode.
An incidental observation was that, due to the landscape aspect ratio dominant in contemporary anime films, horizontal timeline scrolling per frame was lengthier, making the vertical timeline even more beneficial.
6.1.2Vertical timeline in overview interface.The overview interface resulted in a more nuanced assessment.Unlike the main view, frames in this interface are arranged in both one and two dimensions.With the vertical timeline, if the screen height does not suffice to display all frames (which usually is the case), a line break is introduced.For exhaustive frame counts, this means frames exceed the screen's bounds, necessitating horizontal scrolling.The director showed a clear preference for the vertical timeline with horizontal scrolling.This choice was influenced by the consistency in frame alignment with the main view's vertical timeline, ensuring a seamless transition between both views.Conversely, the first author of the paper, lacking E-conte authoring experience, found the horizontal timeline approach feels natural.He drew parallels with presentation software like Microsoft PowerPoint, where the main authoring view utilizes a vertical timeline while the overview adopts a predominantly horizontal one.In the overview interface, the distinct advantage of hand-occlusionfree scrolling disappears; therefore, the preference seemingly leans more on one's prior exposure to E-conte crafting.
6.1.3Potentially cultural choice: line break to left or right.Another facet worth noting concerns the direction of the line break, which we chose to shift left.We anchor the starting frame at the top-right corner, cascading down to the last frame on the furthest left.This arrangement resonated with the director and the paper's authors, but we hypothesize that this inclination might be tethered to our familiarity with vertical writing conventions.In Japanese vertical writing, line breaks occur to the left, a practice deeply rooted in cultural norms.This suggests that individuals accustomed to different writing systems may have divergent inclinations.

Discrete yet Integrated Instruments
The original idea of instrumental interaction [2] primarily discussed how software tools can mediate user-to-object relationships.Guided by this principle, we designed several discrete tools for Griffith: a pressure-sensitive brush, eraser, undo and redo tools to manipulate canvas content, the reframing tool to manipulate canvas clipping, the previsualization tool to specify a group of frames and generate their previsualizations, and the stopwatch tool to measure time.
In a complex system like Griffith, multiple instruments co-exist; therefore, its system design necessitates the consideration of toolto-tool relationships.For instance, the brush and eraser tools may appear distinct, but they both manipulate the same object (canvas frame data).The user seldom wants to use both simultaneously.Similarly, the undo and redo tools reference the same user action stack, which, in our case, is stored as part of the frame data, facilitating the local undo and redo actions.
These connections between the manipulated objects are usually statically defined at the development time.In other words, how an interaction instrument integrates with another instrument cannot be modified at run time.This inflexibility is against the philosophical goal of instrumental interaction that appreciates the capability of analog tools to be combined with others and be easily repurposed.
Among all auxiliary instruments in Griffith, we found the stopwatch most interesting from the perspective of flexibility to connect to another instrument or even to disconnect from the others.The user can use it to assign timing to the current frame of interest in the main view, the overview interface, or to input timing to a textual input in the modal dialog.They can also use it merely to measure time.The current implementation is very primitive: the tool can write the currently measured time to the frame of interest, and the observed flexibility is just the difference of whether the user calls the write function.However, the notion of multiple interaction modes around an interaction instrument that allow optional integration with compatible instruments could be generalized.
Existing research efforts such as Morphic [45] and Webstrates [36] provide a platform for malleable software, where the user can edit the software to meet their personal needs, providing maximum flexibility given that they have prior programming knowledge.Visual programming environments for artists such as MaxMSP [59] and TouchDesigner [10] could be considered prior work, but the integration between the components is only defined as dataflows in specific types, not interactions.Our future work would investigate a software architecture that supports application-level malleability between interaction instruments, allowing the end-user to add, remove, combine, and repurpose existing instruments without programming knowledge.

Socio-technical Tool Design for Collaborative Creativity
From our casual interviews (Section 5.1), we noticed a recurring topic concerning Griffith's potential adoption throughout the production process.These professionals did not perceive Griffith as merely a research prototype; instead, they seriously considered how it could improve the production workflow and saw the potential to be used by the creators of the various roles.We previously discussed that developing an E-conte authoring tool poses economic challenges (Section 2.2).Deploying it and ensuring its sustained use within the industry present even more significant obstacles.To address these issues, a shared sentiment among the interviewees was that Griffith's web-based nature could be a game-changer.Unlike other tools for E-conte authoring, Griffith has the potential to enhance the E-conte information utilization during the production phase, substantially increasing the number of users.As highlighted by P4 and P5, Griffith allows users to stay updated with the latest information without being confused by physical paper copies.
Griffith, in this light, is more than just a creativity support tool.It represents a socio-technical system wherein individuals involved in the pre-production and production phases collaborate seamlessly to create an anime film.This idealized vision sounds undeniably promising, but it is essential to consider sociocultural perspectives when developing such a system.
The first consideration involves the E-conte authoring process, which used to be a secretive process by directors and storyboard artists.As a producer noted in Section 4.1, revealing this process might offer managerial benefits such as providing a more precise timeline for E-conte completion.However, many directors might resist external interventions.Pivoting from this secretive model to one resembling Disney's, where storyboard details are universally accessible on the "story board, " may be challenging.
The second consideration involves the utilization of E-conte.Traditionally, E-conte, printed on physical paper sheets, would be replicated multiple times and distributed to individual animators.While this method made staying updated with the most recent versions challenging, it also localized communications among staff members.With the digitization of these processes through Griffith, implementing a discussion board becomes feasible.We already prototyped it, but reactions were mixed.While the ability to communicate based on the latest E-conte version was appreciated, there have traditionally been various sensitive exchanges among directors, storyboard artists, producers, animators, and other staff.As such, crafting any discussion feature to balance privacy and maintaining convenience is essential.

Future Work
In addition to the two directions of future work mentioned above, exploring instrumental interactions in complex software and considering social relationships in a broader scope of the anime production process, this subsection discusses underexplored areas in tool design.It also discusses the generalizability of the findings in this paper.
6.4.1 Active Support for Prototyping Storylines.The professionals in the anime industry appreciated the merit of Griffith's support for casual sketching (Section 5.1) and interactions to add, remove, or rearrange frames, as well as seamlessly split a cut into two or merge multiple cuts into one.Such features effectively aid their ideation processes.
It would form an interesting future work to investigate further active support for prototyping storylines.Even though the outcome is different (interaction design vs. static anime film), the approach to designing user interfaces (UIs) for prototyping storylines could gain insights from established practices within the interaction design domain.A particularly promising avenue to explore is to design a UI that accommodates branching storylines, allowing creators to visualize and navigate multiple narrative paths.The anime director kindly sketched an example UI Figure 9.

Leveraging a Wider Variety of Structured
Information in Econte.While the tool was designed with feedback from professionals, looking back at the E-conte format, we can easily point to interesting future work.Currently, the illustration is a static image of a particular scene.Meanwhile, it has some hidden structured information that could be handled in a more systematic way, including camera work and visual directions.The script is currently treated only as plain text, but can be categorized into speech scripts, stage directions, messages to animators, etc.Once this type of information becomes available, it would enrich user interactions for e-conte authoring, navigation, and previsualization.While anime production appears to be a niche from a WEIRD [42] perspective, the insights of animation storyboarding tool design in WEIRD culture have never been pervasive either, almost ignored in non-WEIRD culture, including the anime industry.This paper has often compared our workflow to that of major Western studios, but there must be more studios we are not aware of.We would like to reiterate that sharing culturally bound creative processes contributes to increasing diversity in the landscape of creativity support tool research, which would be an important challenge for the academic community.
Nevertheless, the findings in this paper may not be representative of the anime industry as a whole, due to the limited number of professionals within our reach.The E-conte format is definitely universal within the current industry, but the tools surrounding it may vary slightly from one director to another storyboarding artist.As we expand the user base, we expect to find more useful "auxiliary instruments" (Section 6.2) , allowing them to create a more customized work environment.In addition, it would be interesting to study how the tool design, such as the vertical timeline (Section 6.1), is perceived by users outside of the Japanese culture.
From the perspective of the generalizability of the storyboarding tool design, interesting future work would be to investigate how well our tool can be adapted for different storytelling applications: storyboarding live-action films, sketching presentation slides, and prototyping comics.

CONCLUSION
This paper marks the first attempt to introduce the E-conte (storyboards for Japanese animation) authoring process in anime production in the academic context.Primarily informed by anime industry professionals, it revealed how it remained analog to date.It introduced the user-centered approach to setting the design principles of the digital E-conte authoring tool and developing a system called Griffith.The qualitative user study thoroughly evaluated the system.Along with the design and artifact contributions in the underexplored domain of anime production, this study generated insights regarding the use of vertical timelines in authoring visual storytelling, an extended model of instrumental interaction incorporating tool-to-tool relationships, and socio-technical tool design for collaborative creativity.

Figure 2 :
Figure 2: Overview of the typical anime production process, roughly divided into pre-production, production, and postproduction phases.

Figure 3 :
Figure 3: An example sheet of paper in the E-conte format from Starlight Promises, courtesy of MIXI, Inc.

Figure 4 :
Figure 4: The overview interface of Griffith.

Figure 5 :
Figure 5: The director's sketch of the proposed overview interface, showing frames of different sizes in each column.

Figure 6 :
Figure 6: The stopwatch, when enabled, is overlayed on the other instruments and can be always manipulated.The measured time can be optionally assigned to the current frame.

Figure 7 :
Figure 7: The previsualization player can be opened from the per-cut header in the main view (or by selecting a custom range in the overview interface).

Figure 8 :
Figure 8: The main view of Griffith with the horizontal timeline, implemented for the qualitative comparison.

Figure 9 :
Figure 9: The director's sketch of the updated overview interface, showing multiple narrative paths and floating frames.

Table 1 :
A rough comparison of storyboard-related aspects in Japanese and Western animation production processes.