Drawn onto a Skybox: An invitation to collaborative immersive drawing using the Spheri platform

We describe an installation that invites the audience to collaborate on the performance of immersive drawings that will be visualized on-the-fly using the custom hardware-software platform Spheri. These drawings are made on a flat surface using traditional physical materials and following a modular tracing process that helps the user conform to the rules of cubical spherical perspective. Concurrently with the drawing performance, the Spheri platform captures the drawing and converts it into an immersive VR visualization on-the-fly, while tracking the drawing motion and choosing the camera viewpoint accordingly. The installation explores the new media of immersive handmade perspectives as well as new forms of intuitive and performative visualization. Countering the expectations of digital media as instruments for efficiency and ease of production, it aims instead at leading the user into slow and pondered geometrical thinking through handmade immersive drawing.

spherical perspectives and a modular tracing method that facilitates the drawing process, through a custom-made platform that enables on-the-fly visualization of VR panoramas.In all of this, it is a fundamental matter that all drawing is handmade with pencil on paper (or marker on glass) and that the platform is strictly used to translate the handmade drawing into a virtual environment and to help visualize it seamlessly.The machine is not to take center stage but to be as invisible as possible.
The present work must be understood as invitation to immersive drawing, in the context of a multi-front effort by the authors to disseminate the use of handmade immersive drawing methods.The authors have been involved for some years in the creation of these methods and in their teaching at multiple levels and to multiple audiences [3], [4], [6].At the fundamental level, the authors have created both geometrical methods and drawing schemes, as well as software and hardware platforms to draw, visualize, interact, and teach spherical perspectives.This was mostly directed at audiences that have a reason to invest some time in learning spherical perspective drawing.The present installation is aimed at a different audience: general audiences that may not have an incentive to go through long explanations or analytic dissections of perspective, or that may not have the background to understand them, but that may still gain some insights from a more casual, playful approach to the subject.The approach also may have applications to more formal teaching applications, as will be discussed ahead.

SPHERICAL PERSPECTIVES AND IMMERSIVE VISUALIZATIONS
Spherical perspectives are flat handmade drawings that create an immersive visualization through VR software and hardware.They have applications in the visual arts [5], [13], architecture, and cultural heritage documentation [18] as well as in visual education [9].The technical aspects of these drawings have been extensively addressed and developed elsewhere [4].For our purposes here the following suffices: a spherical perspective is a handmade drawing that is constructed on a flat surface following a set of perspective rules; if these rules are followed then the drawing can be scanned and digitally folded around a sphere.Then the viewers, standing virtually at the center of the sphere, have the visual illusion (through anamorphosis [3]) of looking at an immersive environment that surrounds them on all sides.There are many different spherical perspectives, each with their own set of rules for drawing.Each such set of rules defines a different drawing language with its own aesthetics and graphical expression possibilities [5] but all of these flat drawings, though looking very different, carry the same information and generate the same immersive environment when seen in VR.The reason there are many spherical perspectives is that a spherical perspective is a two-step process: in step one we project the 3D environment radially onto a "visual sphere" around the eye; in step two, we flatten the sphere onto a plane using a cartographic map.Of course, there are many cartographic maps to flatten the sphere, and for each such map, we get a different perspective.
Two of these are especially well known: these are the azimuthal equidistant (or 360-degree fisheye) and the equirectangular perspectives (Figure 1), the former having characteristics familiar from wide-angle photography and the latter being a common projection for storage and processing of digital 360-degree photographs.Both are what we expect from "curvilinear perspectives" in the sense that they transform spatial lines into smooth curves on the perspective plane.Ahead we will deal with another variant which is less common and somewhat peculiar but useful for our purposes.

WHY DO WE CARE ABOUT SPHERICAL PERSPECTIVES
Drawing can be seen as a utilitarian skill whose purpose it is to create works of art for consumption by an audience.But drawing is also a mode of thinking -both a tool of introspection, and of analysis of the external world.As a way of producing objects for consumption, handmade drawing has been challenged by digital art.For instance, 2D hand-drawn animation has declined in market relevance and been replaced by what is essentially 3D sculpting and puppetry.Yet, handmade drawing remains essential in the process of the artists who create the digital sculpts and their actions.Similarly, architects create their designs using CAD and BIM, but ideation still often starts as a handmade sketch [19].In fact the premature employment of digital tools is sometimes criticized as leading to cookie-cutter designs or to forcing the architect to think inside the limiting parameters of the software [7].Right now, AI creates images mostly from text prompts, but an effort is being made to create them from sketches, which can better transmit the artists intentions.This too may push handmade drawing to a functional adaptation.The death of handmade drawing is constantly announced, but what is seen instead is a concentration of drawing on its role of conceptual activity and mode of thought.Digital art has not led to the destruction of handmade drawing, but it has displaced it to new grounds and forced it to repeated transformations.This is an ongoing process of challenge and adaptation, and it is in this context that we see the importance of spherical perspective.It is a new way to extend handmade drawing in one of its most conceptual aspects, that tradition of rational drawing that includes linear perspective and that intersects so well with computer graphics.It was argued in [2] that spherical perspective is a concept with educational value for artists, especially those who work in digital media.It is there argued, in much the same vein as in [14], that digital artists tend to adapt their thought processes to the interfaces they are presented with rather than the other way around.These interfaces abstract and encapsulate away the fundamental concepts (geometrical, mathematical, algorithmic) that make the machine work, leaving only a black box with an interface.Each choice of interface creates a language to deal with the fundamental object, which enables some interactions and blocks others, but more than blocking them, hides them from consideration.This abstraction and encapsulation is necessary and is itself a creative act; nevertheless, it is beneficial to cyclically peer again inside the box and rethink its interfaces.It was argued in a previous work [2] that spherical perspectives can be used as an educational device to expose a surprising number of the inner workings of these digital black boxes through the discipline of technical drawing.This has been tested both with art students at the university level and with young students being introduced to perspective [9].With the advent of AI, however, the role of black boxes and their interfaces in shaping the very language of art creation has increased beyond what was predicted even a few years ago.AI image generators are profoundly opaque black boxes that cannot be "opened" by the methods proposed in [2], but this has only made the act of drawing more urgent as a way of grounding human artists as more than button pushing clients.
We argue for the artist's need to focus on activities that directly create rather than request creation.If in [2] it was claimed that digital art pushed a process that was bureaucratic in nature, forcing the artist to jump through unnatural hoops to talk to the machine, learning interfaces to make high level requests rather than dealing directly with fundamental algorithmic thinking, AI has now created quite the opposite -the artist now literally talks to the machine but there is no clear map of what to do; instead of the stifling but predictable steps of a techno-bureaucracy there are just the verbal incantations of a hopeful mendicant in front of an oracle.Immersive drawing through spherical perspectives is an activity that is grounded in physicality and at the same time pushes the brain to understand vision and space.It is a deeply contemplative act that requires hard thinking and physical activity, that is slow and deliberate and in which each mark carries meaning.It is also extremely visual rather than verbal.Consider what a peculiar predicament it is that the visual artist who wishes to embrace AIassisted processes must deal with the fact that AI art is generated by a verbal rather than visual process -and through stilted prose, no less.This is not just learning a new tool; it is switching to a completely distinct -even antagonistic -thought process.The artist risks taking the role of patron or of art director, making requests and giving feedback rather than directly creating.In our view, immersive drawing is just what the human brain, eye and hand are aching for in the present juncture: to exercise natural intelligence rather than begging for favors from the artificial one; to make images with the visual brain rather than stilted prose.
There is a need for this grounding, even and especially if one wishes to explore AI art while avoiding its pitfalls, just like the 3D animation artist regularly needs to re-ground himself in pure drawing.

THE PROBLEM WITH PERSPECTIVE
There is however a problem blocking this goal of spherical perspective drawing.The problem is that these drawings can be difficult to visualize in the head of the untrained, and even harder to explain.Spherical perspective drawings look very deformed in their flat state.This is required to cover the sphere but presents a problem.The experienced draughtsman can "read" (and "write") these drawings even in the flat.The casual observer cannot.The draughtsman is required to draw using the shapes of "geodesics" [4], which can be quite complex and require technical drawing abilities to be rendered and geometrical thinking to be understood.This effort is precisely what makes them such an interesting exercise, but it presents a barrier to entry; the casual observer will enjoy the contemplation but mostly will give up on understanding them and therefore will not draw in them.The authors have performed workshops on these perspectives and the minimum time requirement for a basic understanding is around 90 minutes, which is much more than a gallery visitor will spend on a piece.
One way to solve this is through interfaces that allow the user, as much as possible, to avoid the need for a knowledge of the perspective rules [12], [21].There is a place for that approach, but it carries its own difficulties, and in our case, it is the knowledge of perspective -and its possibilities as a thinking mode -that we wish to explore for its own sake, so it would defeat our purpose to go that way.
In this installation we instead aim to prop the user over the initial barrier in three ways: first, by the choice of the cubical perspective, as an especially intuitive type of spherical perspective; second, through the use of a custom on-the-fly visualization platform called Spheri; third, through a set of modular pre-drawn examples that guide the audience without explicit instruction.Let us address these points separately.

CUBICAL PERSPECTIVE
As we mentioned there are many spherical perspectives, as each choice of flattening creates one.Cubical perspective is a rather peculiar sub-type of spherical perspective.
Its flattening is obtained in two steps: first we project radially from the sphere to a concentric cube, and then we cut the cube open and flatten it onto six squares (Figure 2).We get a cross-shaped figure composed of a horizontal row of four squares intersecting a vertical row of three squares.Polyhedral spherical perspectives like this one project spatial lines not onto smooth curves like the equirectangular or fisheye cases but onto sets of line segments that can change direction or even break at the edges of the faces.
From the viewpoint of computer graphics, cubical perspectives are the same as skyboxes [11].See also tetraconic perspective [1] for another example of a polyhedral-based perspective.

Advantages of the Cubical Perspective
Cubical perspectives are just spherical perspectives with peculiar characteristics.These can be summarized thus: locally (i.e., on each square of the cross-shaped projection) they are simply linear perspectives.Globally, on the other hand, they are like the other spherical perspectives: each spatial line has two vanishing points (always located in distinct cube faces).
Take the example of Figure 3.For reference, call "front face" to the square where the vertical column of faces intersects the horizontal row.Notice that the green-white wall is seen straight on in this face but is seen to vanish to two different vanishing points at the faces to its left and right.Similarly, the red and white column is seen straight on in the front face but is seen to vanish to the zenith in the square above it (top face).
Locally, each line projects as a line segment; globally, a line's perspective image will be made up of several such segments, scattered around the picture in peculiar ways.In Figure 3 we see three red columns on the top face that appear seemingly out of nowhere; they are coming from the back view, that is, from the face on the furthest right of the cross-shape.See also how the ramp on the front view changes angle subtly as it crosses the edge to the right view; it takes some theory to know by how much.These transitions between faces of the cube are better understood in terms of the geodesics of the sphere which project onto the cube.The full classification of these geodesics cube has been done in [6].This is a complicated casuistic that can be more confusing that the equirectangular case for instance.This is the tradeoff of cubic perspective.But in our present work we do not aim for such exact (and exacting) methods.The audience will be invited to draw in an exploratory way by being presented with examples that can be traced, combined, and extended.
Consider the example of Figure 3, composed of simple structures like walls, columns, and ramps.This example was designed by the authors to deliver a short workshop on cubical perspective.In that context the participants would draw up to this scene from nothing, using fundamental rules.By contrast, in the present installation, such simple examples are instead used to set a global stage with a scaffolding that can be grasped intuitively: you can see which side is up, you can guess how big things are relative to one another and how their size changes with distance.Even if you do not understand exactly where each line will go because you have not been taught the fundamental rules, you can still follow simple transitions from contextual cues: a wall that in one cube face is seen straight on and then abruptly changes direction to go to a vanishing point is a typical example.Intuition is enough to grasp that the green-white wall is a straight wall and only the point of view has changed when crossing from the front to the right view; this allows one to intuit how further features -say, a sketch of a human figure -should be fitted in in the faces of the cube to be consistent with the local perspective of the scaffolding.And this gives the audience -duly coaxed and helped along -the confidence to try and think about the global perspective and how it integrates the six views.In this way a fun collaborative drawing leads into the first steps in understanding a non-trivial perspective.Ahead we will explore such simple constructions in a modular way as a stimulus for the drawing instinct to lead into an unstructured exploration of spherical perspective drawing.

THE SPHERI PLATFORM
The second ingredient in our mix is the Spheri platform (https://www.spheri.art).Spheri is a prototype hardware and software platform (originally programmed on top of TouchDesigner [20], and more recently on a mix of JavaScript, MediaPipe [22], and Three.js[23]).The platform consists of: • a drawing surface, • a computer, • an image capture system that can involve multiple cameras, and which captures both the gestures of the performers and the drawing being made, • a program that interprets hand gestures through bodytracking, • a program that converts the drawn images onto immersive visualizations on-the-fly.
Spheri was created to enable on-the-fly visualization of the VR environment created by a spherical perspective [16].The spherical perspective can be captured live while being created with physical or digital media, or it can also be pre-existing media stored in the computer [17].One of the latest features of Spheri is the control of the VR camera through hand gestures, such as pinching in the air for zoom or moving the hand for panning the camera.These gestures in midair are captured through a camera and control the immersive display in a very seamless way.Not only is this very practical, avoiding the breaking of flow for the person doing the drawing or distractions from using mouse or keyboard-based interfaces, but it also actually lends a rather dramatic flair to a drawing performance.
The platform can be run in two ways, depending on how many cameras are being used, which we can call collaborative and performative: In collaborative mode, the setup requires two cameras: the first one for capturing the drawing, and the second one for capturing hand gestures.The functioning implies a user (e.g., an artist, or an instructor) drawing a spherical perspective on a flat surface (e.g., pencil over paper or marker over glass) while the platform displays the VR visualization of the perspective in real time.The same user can also control the virtual camera's viewpoint with hand gestures, for example drawing with the right hand and operating the platform with the left hand through the second camera.Although the user can do both drawing and controlling of the camera, it is not difficult to see that this is not the most practical workflow: it is usually better to assign the camera control to a second user, hence the term collaborative.
In Figure 4 we see an example of this mode.The presenter on the left is drawing while the one on the right is controlling the camera with hand gestures.Behind them, we see a live feed of the flat drawing while the VR view was being projected on another wall (not shown in the photo).The camera control may also be assigned to a member of the audience.
In performative mode the setup includes only one camera, and the hand gestures are simplified.In fact, the platform uses the hand of the person who is drawing to orient the camera within the VR scene.To be more specific, the VR camera follows the position of the tip of the index finger (Figure 5, bottom).In this case, the drawing must be made over a transparent surface (e.g., a glass table), which in turn must be between the hand and the camera.In Figure 5 (top) we see a user drawing on glass under which is the camera, Spheri is detecting the hand of the draughtsman, and simultaneously assembling and displaying the VR scene to the audience.
In this modality, there is no need for a "cameraman", and the focus is fully on the drawing being performed.This removes a cognitive burden from the user, who can then concentrate on the drawing.In the present installation this is essential, as we want the invitation to collaborate on the drawing to be as simple to take up as possible, and the focus to be on the drawing process itself rather than on learning the camera gestures.We have found in the past that the pinch and pan functions are so amusing to the audience that they might lose focus from the drawing [15].In this new mode the user needs to touch the drawing with the index finger to even pan through the picture, as hovering it over the surface is the only way to get a new view.This serves as a ruse -once the user is so close to the picture, the tendency will be to actually engage with the surface and draw something.

MODULAR TRACING
As we stated above, the authors have a long experience of teaching spherical perspectives.But this teaching is slow and analytical.With the proposed installation we are instead aiming at a playful way to draw the casual visitor into immersive drawing not through careful instruction but through a playful, puzzle-like activity.
Among the spherical perspectives, the cubical one has a puzzlelike nature all of itself.On each face of the cube the image is perfectly understandable, and the viewer only struggles on how to fit together the various views that contain a feature in order for the whole to make sense.This leads us to play on the notion of jigsaw puzzle as a way to construct drawings in cubical perspective without explicit instruction.We offer visitors a set of premade sketches on transparent paper, as well as clear sheets of the same media, on top of which users may assemble their own drawings.The simple local character of cubical perspective makes it easier to make drawings through layering of pre-built elements.We prepared drawings of floor grids, walls, arches, stairs, etc., made so as to fit together in several configurations.Examples of these modules include: a floor grid that extends towards infinity in every direction; a wall seen straight on the front view of the cube that then crosses to the left and right faces at appropriate angles; a stairway that starts on the The visitors may for instance layer the sheet of the wall on top of the floor grid, and, finding a stairway that happens (by design) to fit the wall, they may then layer this on top.On each layering step users will use each layer as a guide and trace the desired element onto their own drawing sheet.At each step the tracing will be only partial and will involve interpretation and selection.For instance, if a wall is layered on top of the floor grid then only the tiles that are not obscured by the wall will be copied (Figure 6, Figure 7).This act of tracing, forces users to make sense of the scene in spherical perspective, to interpret visually what is happening on the global scene being constructed.All while the user can count on the Spheri platform to visually check on the immersive scene being constructed.The layering and drawing are done on top of a glass table with a camera underneath.The camera captures the layered scene in real time and the Spheri platform transforms it into a VR scene that is projected onto a monitor on-the-fly.Users may control the view by hovering their pen (in fact their index finger) over the drawing and check the result of their experiments as they are being done.This live visualization creates a feedback loop of trial and error that naturally "teaches" an intuitive notion of the spatial relations in a cubical perspective drawing.At some point a basic environment will have been constructed by this layering and tracing.Then, the users may have a notion of the overall space and of the transition rules, with which they can start creating their own original elements and adding them to the picture alongside the tracings.For instance, we encourage users to add human characters and try to see how their apparent size and orientation should change according to their position on the scene, seeing characters from below when drawing them on the top face of the cube.
All this layering requires the drawing media to be erasable.The installation can work with several different materials, as long as they permit enough transparency for both users (to see what they are doing) and for the camera (to capture a meaningful rendering).The installation may be done in several ways: with layers of tracing paper, acetate or mylar (polyester drafting film), using graphite pencil or appropriate erasable markers.The camera may be placed under a glass table or on top of a transparent box (e.g., made of acrylic) placed on top of any regular table.Several setups are feasible as long as the position of camera, drawing and hand are preserved in sequence, to avoid occlusion of the drawing by the body of the user and allow the camera tracking to follow the drawing hand.
At the end of their participation, visitors are encouraged to pin their drawings to a growing gallery on the walls of the installation.This gallery is initially seeded by examples drawn by the authors but will grow with the visitor's work and gradually create a collaborative exhibit, cyclically curated by the authors.

INSTALLATION SCHEMA AND RIDER
The following schema is appropriate for the installation (Figure 8): • A transparent or translucid table of size 1.2 x 0.8 m., where layering and drawing will take place.• Another table to hold the set of modular transparencies.
• A projection screen.
• A computer to run Spheri.
• An external camera placed under the table, with a minimum HD resolution (1920x1080px).This will track the drawing hand through the transparent paper.• A support to hold the camera under the table.
The total space necessary for setting up the installation is around 3x3 meters, although if available, a deeper space would be useful for obtaining a larger projection.

APPLICATIONS TO FORMAL TEACHING
Recent works have reported on an experiment to bring immersive perspectives to the classroom at the Portuguese 9 th grade level [8], [9].
These experiments had very positive results but one of the difficulties was exactly with the fact that the precise rules of spherical perspective are inadequate for what is intended in these 9 th grade classes.Instead of formal methods, a scheme of trial-and-error  With Spheri, this verification can be done in real time, greatly facilitating a trial-and-error approach with an instant feedback loop.Also, the "jigsaw puzzle" approach using modular tracings may be another useful contribution to develop a quick intuitive understanding of perspective drawing basics with playful activities adequate for the target audience.These are avenues to pursue in future work.

CONCLUSION
We consider that drawing, as a mode of thought, is in some ways being neglected even in the creative disciplines.We aim at using the same technology that its often taken as a threat to drawing to instead stimulate it and bring it innovation and new modalities of expression and enjoyment.
We have described an installation that draws the visitor gently and playfully into a new form of immersive drawing.Our aims are primarily didactical -we aim at an aesthetical experience that is at the same time a dissemination of the idea of spherical perspective as an emerging sub-discipline of drawing.
Furthermore, this experimental installation -using modular transparencies together with Spheri -can be presented in several different ways, some of which seem adaptable to more formal education activities aimed at younger students.
We count on refining its presentation over several upcoming exhibitions and classroom tests to make it a tool for both the enjoyment and the learning of spherical perspective drawing.
Finally, Spheri is being converted from a local prototype to a web resource available to all users.Its final version will be located at https://www.spheri.art.This open access to the platform as a web-based application will ensure greater compatibility between devices, and expand the options to learn about spherical perspectives anytime, anywhere.

Figure 1 :
Figure 1: Two examples of spherical perspectives: On top we have an Azimuthal Equidistant spherical perspective, colloquially known as a 360-degree fisheye.On the bottom we have the same view but in equirectangular spherical perspective.Original artworks by A. B. Araújo.

Figure 2 :
Figure 2: The same view as in Figure 1 but now in cubical perspective.Drawing by A. B. Araújo.

Figure 3 :
Figure 3: Example of a simple cubical perspective.Drawing by L. F. Olivero of a collaborative design made by the authors for a Bridges Conference workshop at Aalto University, Finland.

Figure 4 :
Figure 4: live cubical drawing in a workshop in Aalto University, Finland. A. B. Araújo (left) is drawing on paper while L. F. Olivero (right) controls the VR camera through hand gestures.

Figure 5 :
Figure 5: User draws cubical perspective while Spheri detects the hand and displays the VR view (note how in the VR view the top face -where the pen is touching -is seen joined with the four faces adjacent to it on the cube).

Figure 6 :
Figure 6: Architectural fantasy mounted by superposition of several modular layers: floor tiling, stairs, ramps and arches.Drawing by A. B. Araújo.

Figure 7 :
Figure 7: Two tracing modules (floor grid and stairway) used for the construction of the picture in Figure 6.

Figure 8 :
Figure 8: Scheme of the installation