PaperTouch: Tangible Interfaces through Paper Craft and Touchscreen Devices

Paper and touchscreen devices are two common objects found around us, and we investigated the potential of their intersection for tangible interface design. In this research, we developed PaperTouch, an approach to design paper based mechanisms that translate a variety of physical interactions to touch events on a capacitive touchscreen. These mechanisms act as switches that close during interaction, connecting the touchscreen to the device’s ground bus. To develop PaperTouch, we explored different types of paper along with the making process around them. We also built a range of applications to showcase different tangible interfaces facilitated with PaperTouch, including music instruments, educational dioramas, and playful products. By reflecting on this exploration, we uncovered the emerging design dimensions that considers the interactions, materiality, and embodiment of PaperTouch interfaces. We also surfaced the tacit know-how that we gained through our design process through annotations for others to refer to.


INTRODUCTION
Paper and touchscreen devices are two types of objects found all around us today.They are common-yet disconnected-platforms used in the design of interactive systems.Paper is physical material with a long history.It is used as a tangible carrier for information in books and print, as a medium for craft and decorative arts, as components for architectural structures, and as a material for everyday goods and packaging.With such diverse traditions, paper offers a wide range of material expressions and making practices for interaction design, including electronic pop-up book [38], deformable inputs [13], and shape-changing interfaces [47].Touchscreen devices, on the other hand, mainly serve as a two-dimensional interface that receives simple finger gestures as input-taps, swipes, pinches.The richness of our interactions with these devices lies beneath the glass via the gamut of digital applications that they access, rendered by dynamic pixels on the screen.
In this research, we investigated the unlikely intersection of paper and touchscreen devices as ubiquitous materials for tangible interaction.We took reference from earlier research [25,28,51] that demonstrated extending the sensing capabilities of touch screens through conductive traces.Instead of bare skin contact, we explored using the ground wire of the device itself to trigger touch events on its screen (Figure 2).This works reliably on all unmodified touchscreen devices (e.g.iPhones, Android Phones), and it also decouples the trigger of the device's "touch" event from actual human touch.From there, we developed PaperTouch, an approach to trigger touch events on touchscreen devices through paper mechanisms.These paper mechanisms contain simple conductive traces and function as switches that close a circuit between the device's ground and its screen during interaction (Figure 5).We designed a range of mechanisms that serve to translate different physical interactions to the simple act of closing this switch-therefore enabling a variety of physical interactions that are all detected by activating touch events on the screen.This lets us build hybrid interactive systems [43] comprising paper-based interactive structures that activate touch events on an electronic device, which in turn responds with digital feedback (e.g.visual, audio) programmed in software (Figure 1).
In this paper, we detail the working principle of PaperTouch mechanisms.We also outline our material exploration and the interactive switches we developed that served as fundamental building blocks for further design, as well as annotate the tacit design knowledge that emerged from our investigations.We then demonstrate a series of applications that we built with this approach to showcase the breadth of design possibilities it facilitates.We reflect on these applications to surface emerging design dimensions of interactive systems built with PaperTouch for other HCI designers and researchers to consider.

Contribution
Through PaperTouch and our exploration findings, we hope to add one more tool to the design toolbox for prototyping and building interactive systems.We offer the following contributions to HCI, specifically addressing tangible interface design and materiality of interactive system: (1) We propose PaperTouch, an approach we developed that integrates paper and touchscreen devices for tangible interface design.
PaperTouch mechanisms are electronics-free and translate physical interaction into touchscreen events on a digital device.We detail the working principle of PaperTouch mechanisms and surfaced the design know-how we gathered from our research through design process with annotations for others to refer to.
(2) We made a range of applications for a variety of contexts to demonstrate what can be built with PaperTouch.
(3) We outline emerging design dimensions that considers the interactions, materiality, and embodiment of tangible interfaces built with PaperTouch, as well as how PaperTouch aligns with existing digital prototyping practices.

BACKGROUND AND RELATED WORK 2.1 Tangible Interfaces as Hybrid Physical-Digital Systems
Tangible interfaces are "hybrid physical and computational systems that awaken both existing and novel physical artifacts through the mediating power of computation", as broadly defined by Ullmer et al. [43].On one hand, tangible interfaces may facilitate interactions entirely mediated by bespoke physical matter, keeping digital devices (e.g.commercial computers) hidden in the background for handling computation.This includes interfaces like shape-changing platforms [20] and interactive robot swarms [32].On the other hand, tangible interfaces may facilitate hybrid physical-digital interactions; interfaces that are saliently manipulated through interactions with bespoke physical media, as well as interactions with digital devices such as touchscreens and other conventional input hardware.The interactive systems enabled by PaperTouch fall into the latter category-systems that weave physical and digital media to support hybrid interactions.Related to our work, other researchers have demonstrated tangible interfaces that use touchscreen devices to detect interactions with physical tokens [41], as well as interactions beyond the device itself [30].In building these hybrid systems, designers have to consider the role and connections between physical and digital media.We elaborate on the related work that inspired us in the following sections across two domains-paper as physical material for HCI and touchscreen devices as a platform for tangible interfaces.We also outline the overarching approach we developed to weave physical and digital media into a coherent tangible interface in section 5.1.

Paper for Tangible Computing
Paper is a versatile substrate.Traditionally-and till today-paper is used as a medium for written information.Within HCI, researchers used paper as a flexible and approachable platform for circuits, such as with inkjet printed conductive ink [35], copper tape traces [39], or conductive coatings [53].Such circuits can be combined with other electronic components for physical computing.Researchers have also incorporated shape-memory alloys and polymers [47], [38] as well as ferro-magnetic elements [33] onto paper for delicate shape changing and actuated interfaces.Paper encompasses a broad range of materials, each offering a different set of potentials [5] for tangible interaction design.For example, structural paper such as construction paper and cardboard can be made into mechanical structures capable of different types of movement [34].Researchers have also used specialized papers such as swell paper for tactile interfaces [16], or carbon-coated paper [54] for deformable paper sensors.Interactive composites can even be fashioned directly into paper by embedding electronics during the paper-making process [17].There are many crafting and making practices surrounding paper, and it can be shaped into a myriad of structures for tangible interaction.For instance, kirigami (cutting and folding paper) can be used to make compliant sensing structures [54], haptic inputs [13], and interactive pop-up books [38].Paper with printed circuits can be cut and layered into 3D objects with embedded electronics [35].Paper can also be woven with other materials into wearable sensors and displays [27].
In this research, we investigated how paper could be shaped into tangible inputs that trigger touch screen sensors.We took a broad interest in this material and explored different types of paper (printing paper, tissue paper, cardboard etc.), along with both manual and computational fabrication approaches.This includes adding conductive traces onto paper, as well as shaping paper into different structures and mechanisms.We then consolidated the insights surfaced from the wide material-driven exploration into a series of design patterns for others to adopt and extend when developing tangible interfaces with paper.

Extending Computing Devices for Sensing Physical Tangible Interactions
At the core of a PaperTouch interface is a contemporary touchscreen device.Such devices offer a powerful computing platform for everyday activities and host a range of sensors and input systems within a small package.Beyond interacting with the device itself, these electronic features can also be used to facilitate tangible interfaces built on-top-of and around the device.By leveraging an existing computing device, such tangible interactive systems reduce the electronics investment required as compared to building the entire system from scratch.It is also convenient to (re)program the interactive behavior of such systems as software (e.g.web applications) is easily deployed onto touchscreen platforms.Previous research demonstrates using smartphones as a platform for tangible interfaces.For example, 3D printed channels can distort the acoustic signals between the phone's speaker and microphone to detect interaction [30].The device's camera can also be used with computer vision trackers to read the state of physical inputs around the device [52].The accelerometer and magnetometer can also be used to parse vibrations [50] or magnetic fields [7,24] generated by different input widgets placed around the device.
For this research, we leverage the mobile device's touchscreen as a platform for sensing interactions beyond touching the screen itself.There is a sizable body of research that demonstrates extending the touchscreen's ability to detect bareskin contact for different applications.For example, the touchscreen can be extended by a great distance to other surfaces [48] or to the back of the device [18,51].More complex tangible interfaces such as keyboards, controllers, and wearables can also be composed through touchscreen extensions [25].In our work, we focus on activating touch events on the touchscreen with paper mechanisms that do not require bare skin contact.We do so by devising a switch that mediates the conductive connection between the screen and the device's ground bus.Touchscreen devices commonly use projected capacitive touch (PCAP) sensing via electrodes under the protective glass to detect bare-skin contact [6].Essentially, bare skin contact, such as with a human finger, adds additional capacitance to "ground" which results in a transfer of charges at the sensor electrode that the device detects.PaperTouch draws reference from prior research [25,28,51] that demonstrated extending such touch screen sensing to bare skin interaction on external tangible interfaces connected via conductive traces (Figure 2A).

DEVELOPING THE PAPERTOUCH SWITCH
We faced two main challenges in attempting to replicate such related work.First, touchscreen extensions are sensitive to the design parameters of the conductive trace (e.g.length, resistance) and are prone to both false positive and negative triggers on an unmodified device.Other systems [12,25,28,51] used modified touchscreen devices installed with custom firmware to directly access the raw capacitive sensor measurements.This enabled the researchers to develop applications that rely on custom sensor calibration.Second, touchscreen extensions require bareskin contact with the sensing electrode.This complicates the design of both extension traces and tangible interface structure to facilitate this contact.With these limitations in mind, we explored other means of triggering the touch screen.We learnt from prior work [25] and explored using a more reliable source of grounding than the human body.For example, a large body of conductive material (like a copper sheet) can serve as a grounding agent.In this research we conveniently made use of the device's internal ground bus to trigger touch events on the screen (Figure 3A).This approach establishes a consistent ground point for the device, ensuring detectable changes in capacitance.Moreover, we found this approach to work with many unmodified touchscreen devices, including iOS and Android devices.
Building on this working principle, the core unit of PaperTouch (Figure 2B) is a switch that divides a conductive trace into two segments: (1) the screen electrode is held in contact with the touchscreen with double-sided tape, while (2) the ground electrode is connected to the device's ground bus (Figure 3B).We accessed the device's ground bus by fabricating a plug for the device's charging port that exposes only the ground connection (Figure 3A).Closing a PaperTouch switch connects these two segments and triggers a touch event on the touchscreen device.(3) Configuration C: The screen and ground electrodes are positioned on the same side but separated.A conductive element is positioned on the opposite side.The switch closes when these two sides come into contact.(4) Configuration D: This configuration adopts a sandwich-like design.A conductive layer is nestled between the opposing screen and ground electrodes.The switch is closed when all three layers establish contact.Figure 5 demonstrates simple real-life examples of each configuration.With these PaperTouch switch configurations, we explored a series of mechanisms that translate different tangible interactions to closing the switch.We detail the mechanisms we developed, as well as the tangible interfaces we built with PaperTouch in sections 4 and 5 of this paper.Conductive traces are the key component of PaperTouch switches as they enable the extension of a touch screen's capacitive sensing capabilities.We surveyed different approaches to apply conductive traces on paper to inform our research.A common approach is to apply conductive ink with inkjet printing [14,29,44] however, this works on sheet materials with plastic coatings or specialized surfaces that the printed ink can bind to.Alternatively, conductive inks can be applied via screen printing [42,53], or plotting [15].Besides inks, conductive sheet material such as copper tape can be cut manually [39], or with a plotting machine [40], and transferred onto the sheet material.

Fabricating Conductive Traces on Paper
We were attracted to using a tabletop plotter to fabricate conductive traces on paper.The plotter offers a few advantages in our case.It offers the flexibility of drawing conductive ink traces with a pen, or cutting conductive sheet materials like copper tape.It can also simultaneously use different tools and work on a variety of other sheet materials, such as cutting, scoring, and perforating paper and cardboard.Tabletop plotters are also inexpensive and easy to set up.Furthermore, plotters are open-ended machines that encourage manual craft alongside digital fabrication; for instance laying copper tape traces by hand onto a paper sheet before cutting an outline with the plotter.We used the Cricut Maker plotter in this research and adopted two main workflows for applying conductive traces onto paper: (1) Direct drawing with a conductive pen: As shown in (Fig- ure 6D), conductive traces can be manually drawn onto paper with a conductive pen, or attached to the plotter and computationally controlled.We used the Circuit Scribe pen 1 in this research and attached it to the plotter with a custom 3D printed holder.To ensure good ink flow during drawing, we fixed a pressurized syringe on top of the pen's ink tube (Figure 6A).This increased air pressure ensures continuous and consistent ink from the conductive pen, preventing any drawing interruptions.(2) Transferring copper tape traces: We first use the plotter to cut the trace pattern onto copper tape.We then use transfer tape to lift the trace pattern and apply it to the paper substrate (Figure 6C).We use this approach for papers that repel or absorb conductive ink, such as plastic-coated paper or cardboard.
Conductive ink is prone to scraping off, while thin copper tape traces can easily break when folded.To address this, we apply a layer of cellophane tape on top of the conductive traces to secure the conductive materials to paper and reduce wear and tear.

EXPLORING PAPER
Paper can function as a substrate for conductive traces that can be further assembled into basic physical inputs that trigger the touchscreen.However, as shown by previous work [12,27,38,54], paper offers a far broader range of tangible interactions and material expressions than the simple switch configurations we developed.And, this space widens when considering more forms of paper like cardboard and materials that reflect paper's essential attributes: sheet materials that demonstrate similar flexibility, foldability, and 1 https://circuitscribe.com/collections/pens-inksurface qualities (like adjacent materials such polyester sheets.)We therefore explored many different types of paper materials and the making processes around them.Our aim was to harness these different qualities of paper in combination with the different switch designs to develop a richer set of mechanisms that translate different tangibles interactions to activating the touch screen.

Weight and Structural Affordances
Paper materials come in a variety of weights, indicated by their gsm (grams per square meter) value.Printing paper is usually around 70 to 80 gsm.It is arguably the most familiar type of paper to many people, and one that we used for many of our prototypes and experiments in this research.The weight of paper is proportional to the thickness of the sheet.This in turn has a significant impact on the types of structures that we can build with it, and the tactile experiences that the structures facilitate (Figure 7A).Extra lightweight paper (less than 50 gsm) such as tissue or crepe paper tears and crumples easily.Such paper can be easily cut into delicate structures that respond to gentle forces.Lightweight paper (around 70 to 150 gsm) bends and creases easily.While light paper offers reasonable surface stiffness (i.e.flatness), it still buckles relatively easily-like crushing a piece of paper with our hands.It is easy to compound multiple folds on light paper to make complicated 3D structures, such as the iconic origami crane.This quality also enables compliant structures like paper springs.Heavyweight paper (from 150 to 300 gsm) resists folding or bending, and can therefore be made into sturdier structures.Such paper is often used for product packaging.However, it is also difficult to compound multiple folds on heavy paper, and is more appropriate for simpler structures, such as rigid 3D components.Compliant structures made with heavier paper deliver more force feedback compared to lighter paper (Figure 7A).In addition, composite paper materials multiply and amplify the strength of otherwise flimsy paper.For example, corrugated cardboard is typically made from sandwiching layers of light kraft paper, and the resultant material can be cut, folded, and assembled into furniture [4] and even architecture [31].

Visual and Surface Qualities
Paper offers a variety of visual and surface properties for interaction design.One aspect of paper is its optical abilities; i.e. its ability to transmit, diffuse, or block light.Many lightweight papers work as soft light diffusers, with varying degrees of translucency.This can be used to "blur" or soften a lighting source (Figure 7C).Some materials, like clear PVC sheets, are transparent.Such materials can provide support or structure while allowing underlying visuals to be seen.Thicker paper, or colored paper, can serve to block light completely.These materials can be used in tandem with paper folding and cutting techniques to mediate the formation of light and shadows, such as in a shadow puppet show.
Paper is colorful and it is easy to apply graphics onto paper through manual marking of printing machines.Colors or visual patterns can be used to signal interactions [46].Colors and visual marks on paper can also serve decorative purposes that enrich interactive experiences (Figure 7B)-just like how illustrations bring a storybook to life.

Paper Crafting Techniques
We explored a variety of crafting and fabrication techniques for paper.These processes transform paper into interface components that leverage the different qualities of paper.
We laser cut heavier papers (above 300gsm) and cardboard to create enclosures or mechanical structures.The precision of laser cutting makes it ideal for intricate designs where fit is critical.Regular print paper as well as light to heavyweight papers were typically cut with the plotter.Plotting achieves a similar resolution to laser cutting, with the added advantage of its ability to combine cutting with other operations such as scoring fold lines or applying conductive ink (Figure 7D).Most of our interactive switches are crafted with this process.For delicate materials, such as tissue paper and origami paper, we opted for manual cutting with a paper knife or scissors to preserve their fragile structures.
We also experimented with how traditional paper crafting methods might be used with PaperTouch switches.For instance, we used pop-ups as well as paper springs to create different paper mechanisms.Paper crafting techniques like quilling-tightly coiling thin paper-led to structures that can function as sturdy perpendicular supports while maintaining flexibility across its plane.We used this to craft elements that require a mix of rigidity and flexibility, such as the handle of a controller (Figure 7E).
In terms of assembling multiple paper components, we incorporated slots and integrated joints for more rigid materials like cardboard and thicker paper.For more pliable papers, we used adhesives like double-sided tape or glue.

PAPERTOUCH IMPLEMENTATION
In essence, PaperTouch (Figure 8) employs paper mechanisms that, when combined with the trace configurations (section 3.1), become interactive switches that translate different physical interactions to a touch event on the screen.In this section, we outline our approach to implement PaperTouch interfaces.First, we introduce a set of interconnected concerns as a framework to conceptualize, organize, and iterate upon different elements within a PaperTouch interface (Figure 8).We further provide three examples that demonstrate how we applied the design concerns to simple PaperTouch interfaces.Second, we elaborate on our strategy of building reliable PaperTouch switches; a key design challenge that surfaced from our experience in making such artifacts.We present four types of mechanisms for reliable switches that we developed as a response to this challenge.Finally we discuss the technical design constraints that are critical in the implementation of PaperTouch interfaces.PaperTouch interfaces are systems that facilitate hybrid physicaldigital interactions.With that in mind, our approach to designing a PaperTouch interface can be categorized into four broad and interconnected concerns organized as a matrix depicted by Figure 8.On one axis, these concerns can be split across physical aspects of the interface that are made from physical paper media, and digital aspects defined by software and rendered on the touchscreen device.On the other axis, these concerns can be split between frontstage elements that end-users interact with, and backstage elements that make the interface function but are hidden from end-users.

PaperTouch Design Concerns
Physical Frontstage: As demonstrated in our exploration (section 4), we view paper as a diverse family of materials that can be shaped into a variety of interactive structures.This expands the scope of tangible interactions possible with PaperTouch beyond the simple "push" afforded by the switch.Our concerns with regard to designing the physical frontstage revolves around identifying physical interactions and crafting the paper structures that afford these actions; such as rotating a knob, brushing paper bristles, or pressing a button.
Physical Backstage: When designing the physical backstage, we are concerned with developing paper mechanisms that translate the variety of actions in the physical frontstage to the singular goal of activating the corresponding PaperTouch switch (Figure 4).In turn, the PaperTouch switch is where physical actions are converted into digital touch events picked up by the touchscreen device.To ensure reliable contact between the screen electrodes of the Paper-Touch switch with the touchscreen, we attached the contact points to the screen's surface with thin double-sided tape.We further reinforced any paper "wires" (Figure 9) with cellophane tape to reduce wear and tear during repeated attachment and detachment.Multiple traces can be secured onto a thin plastic sheet to create a multi-connector that maintains the relative positions of each contact.
Digital Backstage: We identify different touch events and the physical interactions that are associated with them through software programs that we develop in the digital backstage.These programs can be developed through custom code, as well as software prototyping and wireframing tools such as Figma, ProtoPie, and even PowerPoint or Keynote.Touch events are identified through their position (i.e.x and y coordinates) on the screen, and these events can be captured in code, or through software elements such as hyperlinks and digital buttons.
Digital Frontstage: At the digital frontstage, we develop the audio and visual feedback that is rendered on the device programmatically based on the touch events that happen.We are especially concerned with how these audio-visual elements work alongside the physical frontstage elements.Overall, we pay attention to developing these frontstage elements to work harmoniously as a coherent hybrid interface.
We illustrate how we navigate these design concerns through three examples below.These examples demonstrate three simple hybrid interface components that each afford a distinct physical interaction with accompanying digital feedback.We further demonstrate more complicated examples that we designed for a variety of application scenarios in section 6.

5.1.1
Example: Push Button UI (Figure 10A).We begin the design of this push button UI by conceptualizing the physical and digital frontstage.First, we built a push button mechanism using paper's flexibility to provide a spring-like recoil for tactile feedback.We then designed the corresponding UI element on the screen-a blank rectangle that fills up when the button is pressed.With these frontstage elements in place, we attended to their connections in the backstage.We integrated conductive traces into the base and spring of the push button to close a PaperTouch switch when the button is fully pressed.We then secured the screen electrode to the touchscreen.In terms of software, we used ProtoPie to detect the touch events at the particular coordinates where the screen electrode is placed, and programmed the corresponding visual feedback to show when a touch event is detected.10B).We designed a rotating paper knob that is mapped to a digital dial on the touchscreen.We used a sandwich structure to build the paper knob that is constrained to rotate around a post.For the physical backstage, we split the knob's rotation into two arrays of parallel screen electrodes that extend from the knob's base to two vertical regions attached to the touchscreen.As the knob rotates, it turns a copper tape pad that creates a connection between ground and a subset of screen electrodes.Moving towards the digital aspects-we used ProtoPie to detect the position of the touch event along the two vertical regions and calculated the corresponding rotation of the knob.We used this calculation to update the rotation of the digital dial as well as the reading on the screen (0-100%).10C).We designed a physical token identifier that displays the ID of the detected token on the touchscreen.The physical frontstage consists of a paper platform with a socket that receives different token blocks.Each token has a unique conductive footprint that selectively closes a two by two matrix of PaperTouch switches on the platform (physical backstage).These switches are attached to four locations on the touchscreen.At the digital backstage, we used ProtoPie to listen for touch events at these four locations, and programmed the software to return the corresponding token ID based on the combination of touch events detected.The shape of identified token is then displayed on the screen.

Contact Mechanisms
The biggest challenge we encountered in designing PaperTouch mechanisms was reliably closing the contact of a PaperTouch switch during interaction (physical frontstage to backstage).In particular, paper is prone to creases and bulges, and the conductive contacts on a simple paper switch might not fully touch during interaction, leading to false negative events.To address this critical issue of reliable contact, we developed a series of different contact mechanisms.These mechanisms address the common types of forces at play for the interfaces that we explored: (1) Parallel switch mechanism (Figure 11A): Parallel forces are where the switch's contacts slide along each other's surface, either through linear translation or rotation.We used  a sandwich-like structure and configuration D (Figure 4) to keep the layers pressed together.We also used foam tape to pad the conductive center to ensure good contact between the screen and ground electrode.(2) Pinching switch mechanism (Figure 11B): A pinch is a symmetrical and perpendicular force applied to both sides of the switch.This mechanism is simple to construct as the pinching force ensures good contact between both surfaces of the switch.The screen and ground electrode traces should be designed to maximize overlap during pinching.(3) Heavy perpendicular switch mechanism (Figure 11C): A heavy perpendicular force pushes a switch's moving contact against its static contact.The heavy force ensures good contact.We also used foam tape on one side to ensure greater surface to surface contact when this mechanism is pressed.
(4) Light perpendicular switch mechanism (Figure 11D): Good contact is challenging for low force interactions (e.g.gently tapping a surface).We used a U-shaped channel on one side of this switch mechanism that compresses to ensure a larger surface area of contact when a light perpendicular force is applied.

Design Constraints and Limitations
Similar to prior work on interactive touchscreen extensions [25], the dimensions of conductive traces on a PaperTouch mechanism are also constrained by certain parameters.Specifically, conductive traces that contain too much conductive material will automatically trigger a touch event when attached to the touchscreen.We tested these dimensions on traces drawn with conductive ink (circuit scribe pen) or cut from copper tape to understand their operational range.We took reference from ShiftTouch [25] and mapped similar parameters 2 onto the PaperTouch schematic (Figure 12A), resulting in four parameters to test: touch point diameter (P), screen electrode length (L), and screen electrode width (W).In addition, we tested the number of repeated passes drawn by the conductive pen (screen electrode layers, Y) to account for the amount of conductive ink deposited on the trace.Figure 12C shows an example of the sample we fabricated to test these parameters.We made these samples on 120gsm black construction paper.We tested each variable with three identical samples on a variety of common touchscreen devices, including the Huawei Nova 3i Android phone, iPhone 12 mini, iPad Pro, Duet 3 Chromebook, and Duet 5 Chromebook (Figure 12D).We concluded that a set of parameters was operational when all three identical samples reliably triggered a touch event repeatedly on the device.To determine the variables to test for each parameter, we first conducted a pilot test to probe the extents of each parameter.Based on the results of this pilot, we then systematically broke down each parameter into a list of variables and tested each variable, while keeping the other dimensions constant.We present our findings in Figure 12B, highlighting the variables that worked reliably.We arrive at the following set of heuristics for the design of conductive traces in PaperTouch mechanisms based on this test.It is important to note that these recommendations might change depending on the specific paper substrate or conductive material used.
(1) The touch point diameter (P) is best kept between 4.5mm to 8mm, similar to the findings reported in [25].Small touch points might not register touch events, while large touch points might create false positive triggers.(2) The screen electrode trace length (L) should not exceed 180mm.Longer lengths introduce too much conductive material to the system, resulting in false positive triggers.For screen electrode width (W), we recommend trace widths between 0.75mm to 1.5mm.For screen electrode layer (Y), we recommend a maximum of three overlaid strokes.This avoids introducing too much conductive material that leads to unintentional triggers as well.

Device Limitations.
Besides the constraints surrounding the design of conductive traces, PaperTouch is also limited by the capabilities of the touchscreen device.Notably, contemporary touchscreen devices are limited to a maximum of 10 simultaneous touch events.

APPLICATIONS
We designed and built a range of applications with PaperTouch.Through these applications, we demonstrate how different mechanisms might be integrated into a cohesive tangible interface for a specific context.We provide the fabrication files of these applications in the supplementary materials accompanying this paper. 2In ShiftTouch [25], touchpoints are formed from four parallel conductive lines measuring 8-10mm in length and 0.6mm in width with 0.5mm spacing between each line (overall height 4mm).ShiftTouch also recommends a maximum screen electrode length of 180mm.

A C B
a U-shape made from 140gsm paper allows for better contact to the trace layer.

Paper Band-Tangible Music Instruments
In this application, we built a series of tangible interfaces that control the digital music instruments on an existing web application 3 .Paper band includes three paper-based instruments: a keyboard, a recorder, and a set of bongo drums.
6.1.1Keyboard.(Figure 13) The keyboard comprises two main parts.The top part houses the keys, designed to be pressed like a piano key.The bottom layer is a flat sheet of paper with 12 switches and electrodes that serve to translate pressing the paper key to activating the digital key on the screen.We used the "light perpendicular" contact mechanism (Figure 11D) for this interface.Pressing a paper key depresses a U-shaped conductive segment, which closes the corresponding switch on the bottom layer.
For the physical construction of the keyboard, we used a range of paper weights for different degrees of stiffness and flexibility.Each key and its spring is folded from a piece of 220gsm paper, while the  U-shape segments are made with 140gsm paper.The keyboard's casing is constructed from 400gsm paper to provide a robust frame to keep the keys in place.
6.1.2Recorder.The recorder presents two interactions: blowing air into a mouthpiece and pressing a button.The mouthpiece mechanism consists of a flap that pushes against another surface when there is sufficient air pressure.This closes the contacts on both surfaces.The button mechanism on the recorder has a similar construction to the keyboard key.We placed the mouthpiece switch in series with the button switches (Figure 14) such that both interactions have to happen before a note is played on the digital instrument.
6.1.3Bongo Drums.(Figure 15) Each drum consists of an inner structure and an outer sleeve.The inner structure contains the switch, positioned at the center of the drumhead.The top surface of the outer sleeve is elevated by 4mm from the inner structure.A pliable pattern is cut into this face, and we lined its underlying surface with copper tape.When players strike the bongo, the pliable pattern depresses and makes contact with the inner structure-closing the switch.
We used 160gsm paper for the inner structure and 220gsm paper for the outer sleeve.Each drumhead is cut with a different pliable pattern, resulting in slightly different haptic feedback when interacting with each drum.

Lucky Bear: Prototyping a Playful Coin Bank
We explore PaperTouch as an approach for rapidly prototyping interactive products.Lucky Bear (Figure 16B) is a coin bank that turns into a slot machine when fed with coins.We used corrugated cardboard and a mobile phone to prototype this interactive product.
We designed an enclosure out of cardboard and inserted the phone into the front, using part of the screen as an animated display for the product.For interactions, we embedded three mechanisms around the cardboard enclosure.Pinching the bear's ears closes a direct touch switch (Figure 4A) to trigger a happy expression, inserting  a coin pushes down a paper flap (Figure 4B) and activates the slot machine mode, and pulling the lever arm closes a switch at the rotation stopper (Figure 4C), starting the slot machine.
We programmed an interactive webpage that responds to these tangible interactions.The webpage is divided into two sections: the exposed portion serves as the bear's face for displaying animations, while the covered portion has three buttons corresponding to the three interactions.We attached electrodes from each mechanism to these buttons.

"The Life of a Frog": Educational Diorama
"The Life of a Frog" (Figure 17) is an interactive educational diorama designed to showcase the journey of a frog from hatching to maturity.People interact with the diorama through a series of paper props placed on top of a tablet device: The first interaction involves lifting a paper flower to reveal hidden frog eggs on the screen underneath.The second interaction involves brushing a clump of paper grass, which triggers a tadpole to swim out on the screen.The third interaction involves touching the paper lotus, resulting in the screen playing an animation of a dragonfly flying out to be caught and eaten by a frog.The last interaction involves pushing a paper lilypad , which triggers a fully grown frog to leap out from beneath a stone.We used the a mixture of direct touch switches (grass, lotus, Figure 4A) and pinch mechanisms (flower, lilypad, Figure 4B) to detect these interactions.The visuals displayed were developed in Microsoft PowerPoint as an interactive slideshow.

Pop-up birthday card
We made an interactive birthday card (Figure 18) with an embedded mobile phone that reveals a pop-up birthday cake when opened.Opening the card closes a switch (Figure 4B) and triggers the phone to play a song.We designed animated visuals to play on the phone screen to resemble flickering candle light shining through the perforated birthday cake structure.Blowing on the "candles" atop the cake pushes a flap to close a switch (Figure 4B), which triggers a cheering sound to play.We used ProtoPie to develop the digital interactions in this application.

EMERGING DESIGN DIMENSIONS
As we reflected on the applications we built, we observed that beyond representing what we would build with PaperTouch, they also reveal new insight for future research and design [19].We grounded our reflections with the research through design process, which emphasizes that design knowledge is encapsulated within the artifacts itself and such artifacts can be instrumental in revealing unforseen dimensions that contribute to a design space [22].With this in mind, we used the artifacts we made to probe the emerging design dimensions of tangible interfaces built with Paper-Touch.These dimensions are not meant to be exhaustive but rather represent our perception of the possibilities within the PaperTouch approach [21].These dimensions can serve as conceptual handles that other designers and researchers can use to explore and extend this work.We discuss four design dimensions that surfaced from our reflection (Figure 19).

PaperTouch Interactions
There were a total of 13 distinct tangible interactions across the applications we built (Figure 19).While far from an exhaustive list of possible interactions, we reflected on them to understand how PaperTouch biased our actions as designers.
7.1.1Continuous vs. Event based.As mentioned in section 5.1, PaperTouch mechanisms can be organized into continuous and event-based inputs.Surprisingly, the applications we built all used event-based mechanisms.This was in part due to the constraints imposed by working with paper as a material.Unlike earlier work that focused on direct touch interfaces with touch screen extensions (e.g.[25,28]), PaperTouch mechanisms had to ensure robust contact between conductive traces.This proved to be more challenging to achieve with paper, even though we earlier demonstrated rotary continuous mechanisms with PaperTouch (Figure 10B).Instead, we found ourselves gravitating towards leveraging the qualities of different types of paper and paper crafting techniques to support a variety of interactions that all translate to closing a switch via eventbased mechanisms.This reveals our tendency as designers to lean on the materiality of paper (Section 7.2) for tangible interactions.
7.1.2Direct Touch vs. Indirect Interaction.PaperTouch switch configurations support both direct touch (Figure 4A) and indirect interactions (Figure 4B,C,D).Direct touch interactions include pinching Lucky Bear's ear (Figure 16C), or brushing the grass in The Life of a Frog (Figure 17C).In these examples, we made use of physical form and paper's inherent structural qualities (rigid cardboard and thin flexible paper strips) to enrich the interaction beyond closing a switch with one's finger.Majority of the application interactions fall into the indirect interaction category.These interactions made use of mechanisms that redirected different physical interactions and phenomena to close the PaperTouch switch; for example, pushing a button (Figure 14B), turning a lever (Figure 16E), or blowing onto a flap (Figure 14A).

Paper's Materiality
Paper played many roles in the applications we designed, from facilitating physical interaction through different mechanisms, to supporting the physical structure of the tangible interface.We reflect on the different facets of paper's materiality and how they contribute to the overall interface design.7.2.1 Mechanical to Material Mechanisms.We adopt many different actions when interacting with the physical objects around us.Similarly, we were able to develop paper into a diverse set of mechanisms that facilitated interactions that were forceful as well as gentle.We observed that these mechanisms can be organized on a spectrum from mechanical to material.Mechanical mechanisms rely on paper as a construction material that can be made into solid parts.In these mechanisms, paper is shaped into components such as boxes and cylinders, and assembled into tangible input devices like knobs, sliders, and buttons.Material mechanisms directly incorporate paper's inherent material properties in the mechanism's behavior.For example, leveraging the flexibility and lightness of printing paper for switches activated by blowing (Figure 14D) or brushing (Figure 17C).PaperTouch mechanisms in between mechanical and material integrate these two aspects of paper-such as a paper push button that incorporates compliant paper springs to trigger the switch with a light touch (Figure 13D).7.2.2Paper as a backstage and frontstage material.Paper serves two important roles in the applications we built.As a frontstage material, we used the expressive qualities of paper to communicate the interaction affordances of the tangible interface, as well as convey its aesthetics for different contexts.As a backstage material, we used paper as a substrate for conductive traces, as well as a construction material for internal mechanisms and structures.Just like in packaging design where a single piece of paper serves both exterior appearance and interior structure-many applications we built made use of a single part to address both frontstage and backstage concerns (Figure 16A).This introduced additional complexity into our process as we had to simultaneously resolve the routing of traces, physical structure, mechanical behavior, and visual appearance when designing a single part.However, it also demonstrates the economy of material use that PaperTouch supports as a design approach for building tangible interfaces.

Embodiment of PaperTouch Systems
PaperTouch systems are composed of paper structures and touchscreen devices.These two things come together in different ways to give computational composites [45] with different structures, textures, and appearances [49].Beyond sensing interaction through touch events, touchscreen devices play a significant role in the materiality of the applications we built.They provide audio, visual, and haptic feedback.Touchscreen devices also have a monolithic physical presence that needs to be incorporated into the overall system.We reflect on the different types of embodiment of paper and touchscreen devices for PaperTouch systems: 7.3.1 Side-by-Side.For the Paper Band application, the music instruments were placed next to the touchscreen device, with paper electrodes extending like feelers from the instruments and attached to the touch screen.In such a side-by-side embodiment, paper interface and touchscreen device are standalone entities minimally connected together.We find such embodiment appropriate for contexts where the paper artifact serves as an alternative interface for software applications (e.g. the music web applications in the Paper Band), while preserving the device's original interactive and hardware features (e.g.touchscreen gestures, hardware buttons, ports).

Encapsulate.
For the Lucky Bear and Birthday Card applications, the touchscreen device is embedded within a larger paper structure.In such encapsulated embodiments, the rigid device becomes an integral part of the interface's physical structure.For the Lucky Bear, we framed a portion of the device's screen as the bear's face, synchronizing the animated expressions with the interactions around the physical object.For the Birthday Card, we hid the entire touchscreen device at the base of the card, and used it to playback audio messages and sound effects.The paper card also diffused the visuals displayed on the screen, adding color to the background.

Blend.
In the "The Life of a Frog" application, paper props and inputs were affixed onto the touchscreen device as a diorama of a pond ecosystem.Such blended embodiments emphasize the interplay of paper and touchscreen elements, and they mutually enrich each other's materiality to achieve a cohesive whole.For instance, we positioned animated visuals on the device's display to correspond with physical elements, such as water ripples radiating from the paper lily pads, or a dragonfly emerging from a paper flower.The display also illuminated the paper elements, enhancing the three dimensional appearance of the diorama through colors, light, and shadow.

Programming PaperTouch
PaperTouch systems work by triggering touch events on the touchscreen device.This opens up opportunities for PaperTouch to interact with the wide range of software platforms designed for such devices.In our applications, we programmed PaperTouch's software interactions in three main ways: 7.4.1 Connecting to existing software applications.PaperTouch interfaces can serve as alternative control for existing software applications.In Paper Band, we built a range of paper music instruments that control an existing music web application.This echoes related work [25,28,51] that demonstrates using touchscreen extensions to control keyboard input or scroll through a webpage.
7.4.2Rapid prototyping.Tangible interfaces built with PaperTouch can be rapidly programmed with software prototyping tools.For example, "The Life of a Frog" visualizations were developed as a Microsoft PowerPoint slideshow.We positioned hyperlinks on the screen to be triggered by the different paper inputs, moving the slideshow to the slide with corresponding visuals.In addition, contemporary software wireframing tools like Sketch, Figma, and ProtoPie, offer designers a wealth of interactive features and media playback capabilities; and they can similarly serve as a platform to rapidly create tangible user interfaces together with PaperTouch.7.4.3Developing new software.Custom software can be programmed from scratch for tangible interfaces that demand a high software fidelity or customization.Unlike other electronics-based physical computing approaches, we experienced that software can be developed separately from hardware with PaperTouch.For Lucky Bear, we developed a web application to define the system's interactions and feedback.Each interaction (insert coin, pull lever, pinch ear) was defined by a button on the webpage, and we were able to test our software by clicking on these buttons-before integrating the touchscreen device into the cardboard enclosure.

DISCUSSION 8.1 Paper Prototyping Tangible Experiences
Prototypes help designers express and test different aspects of a product or system [23], and paper prototypes are a common method to rapidly mock-up digital user interfaces (UI) for others to experience [3].While this traditionally meant sketching out UI wireframes on actual sheets of paper, it has evolved to include using digital wireframing tools (e.g.Figma) to rapidly visualize and deploy UI ideas.Such digital platforms support responsive interaction logic that have to be manually simulated in analog paper prototypes.Similarly, tangible experiences can be rapidly mocked-up with physical materials and props, and simulated using "Wizard of Oz" style facilitation [2,9].Responsive tangible interactions can also be programmed by embedding physical computing into the prototype [36].However, physical computing is challenging and requires significant electronics and software development overhead [8].
We see the potential of PaperTouch as an alternative approach for prototyping tangible experiences; occupying the space between static mock-ups and physical computing.PaperTouch interfaces can be rapidly constructed from paper, an inexpensive material that can be shaped with a variety of manual crafting and digital fabrication processes.Furthermore, the interactive logic and behavior of these tangible interfaces can be rapidly developed in "no-code" platforms like slideshow builders (e.g.PowerPoint, Keynote) and digital wireframing tools.We believe that this opens PaperTouch up to a wide audience with diverse backgrounds and different areas of technical expertise-such as industrial designers, museum curators, educators, and parents.

Materializing Interactions with Paper and Touchscreen Devices
The transition from physical to digital media is perhaps most evident in our move from paper-based formats like books, magazines, and newspapers, to electronic platforms such as mobile phones, e-readers, and tablet computers.This evolution implies a shift from tangible to digital information, altering the intrinsic qualities of physical objects in favor of digital attributes [10].The trend towards dematerialization has sparked discussions about the role of materiality in HCI, including calls to re-center interaction design on the experience of physical materials, rather than just leveraging their functional properties [26].
In this research, the dematerialization of paper prompts us to reevaluate its materiality, especially when placed alongside contemporary digital device.Through PaperTouch, we offer our perspective on how these two materials might by synergistically incorporated to support tangible interface.We view touchscreen devices as another material with its own set of physical features and design constraints for designing physical interfaces; and paper as a platform that augments and extends the interactive capabilities of the touchscreen further into the real world.
Our perspective offers an instance of designing tangible interfaces by optimistically embracing hybrid interactions.As a future research agenda, we see an opportunity to extend our approach, such as the matrix of design concerns (Figure 8) we outlined, to other physical and digital media.For instance, we are inspired to explore the potential of other media-such as textiles, ceramics, wood-along with touchscreen devices.Just like paper in this research, we believe that these materials are not passive, but are active components with diverse properties and rich histories of making practices that shape the design process that they apply to [11].We look forward to the new design contexts that might open up to such hybrid tangible interfaces with each new material explored.

Annotating Design Research
PaperTouch switches are conceptually simple, however, we-as designers-had to grapple with working with the richness and messiness of paper as a physical material in the real world.Throughout this project, we uncovered many details about working with paper and touchscreen devices that collectively informed the mechanisms, interactions, and prototypes we built.Articulating these implicit design know-how, such as the thickness or texture of paper and the nuances in using them, proved challenging with the traditional research paper format.We observe that many papers communicating new approaches for designing interactive systems focus on sharing the generalizable concepts around the work-rather than the design details that are also critical in enabling others to specifically replicate the research [37].As such, alongside outlining the key concepts around PaperTouch as an approach for building tangible interfaces-we also annotated specific design details, "tips and tricks", and hacks that we employed to not only realize functional tangible interfaces, but also make them work well [1].With these annotations, we hope to practically support other designers and researchers who are interested in using and extending Paper-Touch.Through this research practice, we also hope to highlight the significance of articulating practical material knowledge in design research within HCI, and provide one example of how such knowledge might be disseminated.

CONCLUSION
In this paper, we presented PaperTouch-an approach that incorporates paper and touchscreen devices for tangible interface design.We took a step by step approach, starting with developing switches that triggered touch events by connecting devices to its ground bus, followed by exploring mechanisms that translate physical interactions to touch screen events through paper's materiality, and finally demonstrating a variety of interactive applications.By reflecting on this process, we discussed emerging design dimensions that consider the interactions, materiality, embodiment, and programming of PaperTouch interfaces.Through PaperTouch, we hope to cast paper and touchscreen devices in a new light-as mutually beneficial materials that contribute hybrid materiality that can be directed towards designing and building tangible interfaces.

Figure 2 :
Figure 2: Diagrams of touchscreen extensions A: Existing approach of extending touchscreens via bare-skin contact on traces.B: Working principle of PaperTouch.C: PaperTouch mechanisms take the role of a switch to close contact between screen and ground.

Figure 3 :
Figure 3: A: USB-C connector with a crocodile clip soldered to the ground connection.B: Example of a PaperTouch switch unit, connected to the screen and the device ground bus.

3. 1
Switch ConfigurationsWe defined four distinct PaperTouch switch configurations (Figure 4): (1) Configuration A: In the simplest form, a switch can be made by creating a thin separation between the screen electrode and ground electrode.Directly touching this gap closes this switch due to the slight conductivity of bare skin.(2) Configuration B: The screen and ground electrodes are positioned on opposite sides.An external force brings them into contact, closing the switch.

Figure 6 :
Figure 6: A, B: Circuit Scribe pen fitted with a syringe for consistent ink flow mounted in a Cricut Maker.C: Transferring copper traces to paper with transfer tape.D: Plotter cutting follows the previously draw conductive traces.
optical abilities: transmit, diffuse, or block light.tissue paper is much softer and can be used to create delicate structures that require gentle force.quilling can create sturdy paper components, such as the knob of a slider.

Figure 7 :
Figure 7: Exploring paper's qualities A: Papers of varying GSM demonstrate different support strengths; B: Colors and patterns of paper provide both decorative and interactive design options; C: Paper's optical qualities allow for creative light manipulation; D, E: A variety of fabrication and crafting techniques for paper.

Figure 8 :
Figure 8: Matrix of interconnected design concerns in the PaperTouch interface.
trace can be printed on 120gsm paper, easy to bend and flexible.use glue dots and double-sided tape to attach the trace to the screen.transparent PVC can be used to secure the different traces together.

Figure 9 :
Figure 9: PaperTouch contacts secured onto a touchscreen with double-sided tape.Multiple traces can be secured to another sheet for ease of use and positioning.

Figure 10 :
Figure 10: A: Push button UI with paper button press and on-screen visual feedback; B: Knob UI with paper dial rotation and digital percentage indicator; C: Token identifier with interchangeable tokens and corresponding shape identification on-screen.

Figure 12 :
Figure 12: A: Schematic of PaperTouch switch components; B: Variables tested for touch pint size, screen electrode, length, width and layering; C: Conductive trace sample used for testing (left: cooper tape, right: conductive pen); D: Evaluation across various touchscreen devices, from left to right: Huawei Nova 3i, iPhone 12 mini, iPad Pro, and Chromebook models Duet 3 and Duet 5.

Figure 13 :Figure 14 :
Figure 13: Paper Band single octave piano.A: The piano keys are pressed to play sounds.B:The structure and contact mechanism of a key-the folded spring provides tactility and resistance.The U-shape conductive segment allows the switch to close when a light force is applied.C: Connection of the PaperTouch switches to the touch screen and ground, the keyboard is peripheral to the device.
patterns on 220gsm paper can result in varying amounts of deformation to provide subtle differences in haptic feedback.

Figure 15 :
Figure 15: Paper Band bongo set.A: Interacting with the drum.B: The drum inner structure and outer sleeve exposed.The different patterns cut on the router sleeves result in varied feedback.C: Layout of the interface and touch screen device.
is not limited by length.All switches can share the same ground trace for cleaner design.foam tape backing on each electrode provides more pressure to secure it to the screen.

Figure 16 :
Figure 16: The Lucky Bear cardboard coin bank.A: Unfolded cardboard and paper net of the Bear.The internal conductive traces are laid out before being folded.B: Bear with phone enclosed.C,D,E: Interactions with the Bear, pinching the ear, inserting a coin, and pulling a lever.
50gsm can be used for creating gentle interaction.

Figure 17 :
Figure 17: "The Life of a Frog" diorama.A: The paper structures are overlaid on the screen.B,C,D,E: Interactions with the garden by pinching, brushing, touching, and pushing.

Figure 18 :
Figure 18: Birthday card with pop-up cake.A: The card over the device screen with light diffusing through the paper.B: Opening the card to trigger the lights and song.

Figure 19 :
Figure 19: The design dimensions of PaperTouch, reaching into interactions, materiality, embodiment, and software platforms.