Instruments of Vision: Eye-Tracking and Robotics as an Embodied Interface

Abstract

In the age of ubiquitous visual technologies and systems, our perceptive apparatuses are constantly challenged, adapted, and shaped by instruments and machines, rendering the observing body as an active site of knowledge. Your Eye's Motion by Luna is an interactive installation that uses real-time eye-tracking to control a robotic creature named Luna (Figure 1). Materializing eye movements through a wondrous spectacle of light, motion, and color, the observer becomes conscious of her gaze enacted and extended by a robotic counterpart. Building on a diverse set of theories and understandings of vision from the fields of cybernetics, visual studies, embodied mind, and more, the project explores how our perceptual apparatuses and bodies are reconfigured in relation to machines and the environment to afford new ways of seeing. Once we see how observing bodies accommodate feedback from actions to cognition, we can uncover the embodied and affective potential of eye movement as an interface for robotics. The curiosity of Luna invests in this potential, articulating a unity between our embodied percepts and machinic environments to create a "vision machine."

References

1. Jonathan Crary. 1988. Techniques of the Observer. October 45 (1988), 3. DOI: https://doi.org/10.2307/779041
2. Jonathan Crary. 2012. Techniques of the Observer: On Vision and Modernity in the Nineteenth Century. MIT Press, Cambridge, MA.
3. Andy Clark. 2019. Surfing Uncertainty: Prediction, Action, and the Embodied Mind. Oxford University Press, Oxford, UK, 208--216.
4. Anne-Heloise Dautel. n.d. Perceptual moments at Anamorphic Waves exhibition, Ugly Duck, 2019.
5. Marcel Duchamp. 1946--1966. Étant donnés. Philadelphia Museum of Art. Retrieved March 30, 2021 from https://www.philamuseum.org/exhibitions/324.html?page=2
6. Karl Friston, Rick A. Adams, Laurent Perrinet, and Michael Breakspear. 2012. Perceptions as hypotheses: saccades as experiments. Frontiers in Psychology 3 (2012), 151.
7. Mark B. N. Hansen. 2006. New Philosophy for New Media. MIT Press, Cambridge, MA.
8. Jerome Lettvin, Humberto Maturana, Warren McCulloch, and Walter Pitts. 1959. What the frog's eye tells the frog's brain. Proceedings of the IRE 47, 11 (1959), 1950. DOI: https://doi.org/10.1109/JRPROC.1959.287207
9. David C. Lindberg. 1996. Theories of Vision from Al-Kindi to Kepler. University of Chicago Press, Chicago, 5.
10. Lev Manovich. 2001. The Language of New Media. MIT Press, Cambridge, MA, 282.
11. J. Kevin O'Regan and Alva Noë. 2001. A sensorimotor account of vision and visual consciousness. Behavioral and Brain Sciences 24, 5 (2001), 939.
12. Jeffrey Shaw. 1995. PLACE. Jeffrey Shaw Compendium. https://www.jeffreyshawcompendium.com/platform/place/
13. Francisco J. Varela, Eleanor Rosch, and Evan Thompson. 1991. The Embodied Mind: Cognitive Science and Human Experience. MIT Press, Cambridge, MA, 205.
14. Paul Virilio. 2007. The Vision Machine. Indiana University Press, Bloomington, 4.
15. Heinz von Foerster. 2010. Understanding Understanding: Essays on Cybernetics and Cognition. Springer, New York, NY, 283.

Index Terms

• Applied computing

  • Arts and humanities

    • Media arts

• Computer systems organization

  • Embedded and cyber-physical systems

    • Robotics

• Hardware

  • Communication hardware, interfaces and storage

    • Sensor applications and deployments

• Human-centered computing

  • Interaction design

    • Interaction design theory, concepts and paradigms