Physical-Digital Programming

Scientists, artists, and engineers are innovating with digital fabrication machines, yet they lack effective tools to program machines for unconventional tasks. We argue for programming language foundations to empower these practitioners to build bespoke fabrication workflows for themselves.

D igital fabrication machines, like 3D printers, laser cutters, and CNC mills, can be programmed to manipulate physical matter. Digital fabrication has become an increasingly popular topic in human-computer interaction (HCI) research. Before being used in homes and workplaces, digital fabrication evolved in the mid-20 th century for use in high-volume manufacturing environments. As a result, the way people program these machines today reflects computing constraints that are more than 50 years old: batch programming, separate applications for different parts of the fabrication pipeline, and limited ability to deviate from the norms.
To bring digital fabrication technology into the 21 st century, HCI has explored several conceptual approaches for people to use machines and their associated software in new and emerging domains. For example, interactive fabrication has explored how nontraditional input like voice data can control machines interactively [1]. Compositional fabrication gives users ways to add adjustments to a job while the machine is still working [2]. Personal fabrication calls for HCI researchers to build end-to-end pipelines that have domain knowledge built into the system, thus lowering the barrier for users without experience in that domain [3]. In particular, the personal fabrication lens has shaped a large body of work in HCI where researchers build endto-end systems for non-experts in domains as diverse as hydraulics, metamaterials, and on-skin electronics.

EMERGING CASES OF EXPLORATORY DIGITAL FABRICATION
Despite this focus in HCI research, digital fabrication practitioners from science, art, and engineering prioritize safe experimentation, rather than end-to-end solutions. These unconventional applications might include 3D printing batteries at the micron scale, using cellular automata to make pottery, or using machine learning to test and design 3D printed lattices for withstanding impacts. In these cases, practitioners need to write their own code for custom ma-chine control. Relying on researchers to build all-in-one tools for these tasks is not feasible.
In these contexts, end-user fabricators can neither rely on pre-made pipelines nor established computeraided design (CAD) and computeraided manufacturing (CAM) tools for getting the job done. In the case of established tools, a fabricator would typically have a CAD editor for producing geometry, a separate application for translating geometry into computer numerical control (CNC) code, perhaps some post-processing after that, and then another application for deploying the CNC code to the machine.
However, when deviating from the delineated steps in CAD and CAM software to make something that the Scientists, artists, and engineers are innovating with digital fabrication machines, yet they lack effective tools to program machines for unconventional tasks. We argue for programming language foundations to empower these practitioners to build bespoke fabrication workflows for themselves.

Physical-Digital Programming
software doesn't support, end-users have no choice but to program functionality themselves. We wish to highlight, rather than gloss over, emerging end-user programming of machines.
Most of the time, programming a digital fabrication machine from scratch means programming directly in CNC code, which is quite low-level. For example, in the popular G-Code language, the instruction G0 X100 Y100 Z20 F1000 moves the machine's tool to the location at 100mm on the xaxis, 100mm on the y-axis, and 20mm on the z-axis, all at a speed of 1000mm per minute. Having to think at this level of granularity makes programming a machine with CNC code similar to programming in assembly language: It works for low-level, well-defined tasks that are common in machine shops, but it is cumbersome to use for more unconventional fabrication.
For example, consider an artist who wants to replicate a 3D printing technique shown in WirePrint by Mueller et al. [4], where the authors printed a wireframe of a model rather than the entire object. The artist would need to do a great amount of experimentation to get the technique to work on their machine. They would need to try different rates of extruding filament from the print head, different machine speeds, and different model dimensions just to get the wireframe's triangles to stand up at all. All of these factors also vary based on the artist's current 3D printer, such as how it moves (machine kinematics) or the material behavior of the printing filament.

PHYSICAL-DIGITAL PROGRAMMING
Given that scientists, artists, and engineers are constantly having to program new functionality, we should rethink how to best build tools for unconventional fabrication. Currently, conceptual approaches for digital fabrication in HCI largely focus on usability, or "lowering the floor," rather than on experimentation. While this is helpful in many cases, it also risks reinforcing the idea that digital fabrication must be like program execution, where the end user is hands-off and the system does all the work.
However, this idea contrasts with existing observations where practitioners are constantly reshaping the process to their liking. Studies of exploratory digital fabrication in the wild like #PlotterTwitter by Twigg-Smith et al. have already shown us that, rather than shying away from programming, artists who work with drawing machines called plotters are already writing, borrowing, and remixing code themselves [5].
Perhaps this drive to program at the edge of the physical and digital worlds is not so surprising. In an article about the benefits of combining HCI and programming languages research, Chasins et al. stated "while both GUIs and languages ... [make] it easy to say common things, a language empowers users to say uncommon things too" [6]. End-user fabricators are interested in uncommon tasks by definition, so we should consider how to view explor-The way people program these machines today reflects computing constraints that are more than 50 years old.
available to Krieg. The only solution available to him was to write his own code in tandem with the behavior of the milling arm.
This constant creation of ad-hoc physical-digital code is commonplace across domains, but it is difficult for other designers, artists, scientists, and engineers to share, understand, and extend each other's code.

IMPRIMER: A COMPUTATIONAL NOTEBOOK PROTOTYPE FOR PHYSICAL-DIGITAL PROGRAMMING
One way to make physical-digital programming more accessible is through literate programming. With literate programming, a program comprises natural language, illustrations, and interactive elements interleaved with source code. Computational notebooks are a chief example of literate programming and are commonly used in scientific computing, machine quickly use custom code to generate toolpaths-which is where the machine's tool moves-then quickly visualize, experiment, and iterate on them just like a programmer would test and iterate on their code.
To motivate this new style of programming, we'll discuss an example, AESTUS, which is a series of wooden vases created by designer Oliver David Krieg [7]. The vases feature delicate grooves on their surface, which were made using a 7-axis milling arm. To design the grooves, Krieg wrote a custom program to control the robotic arm in a precise way. The grooves could not be 3D modeled and therefore required a different software solution than what was commercially atory fabrication from a programming language lens.
On the other hand, while prepackaged, end-to-end pipelines are often too high-level and inflexible for experimental fabricators, G-Code's low-level control alone is not enough. The common practice of writing small snippets of G-Code lacks many features of programming languages that we take for granted, for example: functions, visualization, testing, previewing in-situ, interoperating with outside libraries, and adapting to different machines.
Here, we contemplate a third option: physical-digital programming. A physical digital programming environment would help practitioners

Figure 2. Surface milling notebook.
Left: Users choose a mathematical function of two variables and use graphical elements to experiment with function parameters and see the toolpath visualization update in real time. Middle: To ensure feasible milling, users define variables to offset and clip the function within a desirable bound as described in a hand-drawn diagram. Right: Users specify tooling parameters to calculate feeds and speeds (top) before dispatching the job to the CNC mill (bottom).

Figure 3. Tile molds milled with Imprimer.
Left: A block of molds generated from several mathematical functions of two variables (sinc, sine/ cosine, Goldstein-Price) milled in insulation foam. Right: The corresponding tiles cast in plaster.
tions of two variables, sketch-based milling with augmented reality, and a parametric shelf generator with associated debugging tools. For the first demonstration, a user can mill the surface of a material based on mathematical functions without having to do CAD first. This notebook (see Figure 2) includes a visualization for the surface of a function of two variables alongside a derived toolpath that can be dispatched to a CNC mill. For example, we implemented the two-variable unnormalized sinc function: where f is the frequency or "waviness'" of the ripple that the function resembles, x and y are 2D coordinates on the milling surface, and z is the depth of the cut. Notebook users can use graphical input elements or edit code to experiment with the resulting machine toolpath in real time. We also provided a function that sums sine and cosine terms as well as the Goldstein-Price function commonly used to test optimization algorithms.
learning, and related fields that deal with digital data. Properly extended, computational notebooks could provide a flexible programming environment for programs that cross into the physical world and back.
For instance, a designer inspired by Krieg might want to explore the possibilities of CNC milling in an open-ended way. With computational notebooks, it becomes possible for the artist to program toolpaths directly, therefore allowing them to engage with the aesthetic possibilities of machine behavior. Computational notebooks allow direct machine programming in a way that supports exploration of the machine's aesthetic language, rather than treating it as the executor of digital geometry. With Imprimer, the artist could easily document, extend, and share their code with a growing community of exploratory makers and artists.
To prototype literate programming for machines, we built Imprimer, where the end user works from a computational notebook to control a CNC mill with an AR projection guide (see Figure 1). In our implementation, we used the Observable Notebook tool and a large CNC mill for wood called a Shopbot whose cutting space measures 2.4 by 1.5 meters, or in the Shopbot's native units, 96 by 60 inches.
In particular, we chose to build Imprimer for CNC milling, rather than for other types of digital fabrication. This is because of the steep learning curve it presents to those without prior experience, which makes experimentation difficult. To explore the versatility of Imprimer, we implemented three demonstration notebooks: surface milling molds by sampling func-Scientists, artists, and engineers are constantly having to program new functionality, we should rethink how to best build tools for unconventional fabrication. Despite some tensions with using CNC notebooks for exploratory fabrication, participants expressed that the notebook lowered the barriers both to getting started with a machine as well as for reconfiguring existing workflows.
"The concept is neat, and I see a lot of applications. It seems if you can swap out the toolpath generation part to match an application-as long as you could write [code for] toolpath generation-then there's a lot of real-world use. Using a notebook feels like a more natural way to interact with a machine." In particular, one participant from our user study, who is an engineering educator, highlighted Imprimer's potential as an educational tool to scaffold machine use.
"I really like that you can write code that creates buttons and sliders. This way you could limit the notebook to specific functionalities, like you can adjust and play with these numbers, but not these other ones that might be more dangerous to mill ... most students would stick to the controls you give them." To experiment with the milling topology, users can modify these functions or write their own functions in code as needed.
To support the library, we developed a network architecture to connect the Shopbot with the user's notebook, as well as an augmented reality overlay for visualizing movements on the Shopbot's physical bed. Code executed in the notebook directly affects both the Shopbot and the overlay. In the end, we explored a range of different functions and different parameters, milled the molds out of the insulation foam, and cast the tiles in plaster (see Figure 3). Figures 4 and 5 showcase the second and third example notebooks, respectively.

EXPERIMENTS WITH USERS
To better understand how users conceptualize literate programming as a technique for CNC milling, we conducted a user study with six participants who had varying levels of experience in CNC milling and in programming. Our initial experiments underscored the need for careful scaffolding of domain concepts if users were to understand and feel comfortable manipulating views or changing code.
In particular, one participant who was not familiar with CNC milling found it challenging to choose tooling parameters in the notebook cells; he was especially concerned with how precise measurements needed to be. The physical-digital programming paradigm encourages rich visual and linguistic documentation alongside source code. Left: Notebook users can experiment with different shelf dimensions and see a 3D rendering of the assembled shelf update immediately. Middle: When any shelf parameters are changed, the notebook generates new planar geometries of the shelf parts as defined in notebook cells which users can edit if desired. Right: A generated 3D toolpath (top) that can then be dispatched to a CNC mill (middle); rabbet joint parameters can be adjusted to achieve a tight fit (bottom). cation will come not by abstracting machine operations for users, nor by relying on low-level machine CNC code alone, but by letting makers have better control and knowledge of their machines so that they can explore a fuller range of human-machine cocreations. Through physical-digital programming, we have the opportunity to unite diverse populations around a cross-disciplinary infrastructure of accessible machine operation, understanding, and collaboration.