Mar Canet Sola
Varvara Guljajeva
Abstract
This paper describes an interactive robotic art installation Dream Painter by the artistic duo Varvara & Mar that deploys artificial intelligence (AI), a KUKA industrial robot and interaction technology in order to offer the audience an artistic interpretation of their past dreams, which are then turned into a collective painting. The installation is composed of four larger parts: audience interaction design, AI-driven multicoloured drawing software, communication with an arm robot, and a kinetic part that is the automatic paper progression following each completed dream drawing. All these interconnected parts are orchestrated into an interactive and autonomous system in the form of an art installation that occupies two floors of a cultural centre. In the article, we document the technical and conceptual frameworks of the project, and the experience gained through the creation and exhibition of the interactive robotic art installation. In addition, the paper explores the creative potential of speech-to-AI-drawing transformation, which is a translation of different semiotic spaces performed by a robot as a method for audience interaction in the art exhibition context.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Dream Painter: Exploring creative possibilities of AI-aided speech-to-image synthesis in the interactive art context
- Roland Barthes. 1972. Mythologies, trans. Annette Lavers (New York: Hill and Wang, 1972) 151 (1972), 82.Google Scholar
- Vanessa Julia Carpenter and Elisa D. Mekler. 2019. Towards metrics of meaningfulness for tech practitioners. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1--8.Google Scholar
- Paul Cohen. 2016. Harold Cohen and AARON. Ai Magazine 37, 4 (2016), 63--66.Google ScholarDigital Library
- Kevin Frans, Lisa Soros, and Olaf Witkowski. 2021. Clipdraw: Exploring text-to-drawing synthesis through language-image encoders. arXiv preprint arXiv:2106.14843 (2021).Google Scholar
- Madeline Gannon. 2017. Human-centered Interfaces for autonomous fabrication machines.Google Scholar
- Benjamin Grosser. 2011. Interactive Robotic Painting Machine. Retrieved April 2, 2022 from https://bengrosser.com/projects/interactive-robotic-painting-machine/Google Scholar
- Varvara Guljajeva. 2018. From Interaction to Post-participation: the Disappearing Role of the Active Participant. Ph.D. Dissertation.Google Scholar
- Varvara Guljajeva. 2021. Synthetic Books. In 10th International Conference on Digital and Interactive Arts. 1--7.Google Scholar
- Aaron Hertzmann. 2019. Aesthetics of neural network art. arXiv preprint arXiv:1903.05696 (2019).Google Scholar
- Aaron Hertzmann. 2020. Visual indeterminacy in GAN art. Leonardo 53, 4 (2020), 424--428.Google ScholarCross Ref
- Indrek Ibrus and Maarja Ojamaa. 2020. The Creativity of Digital (Audiovisual) Archives: A Dialogue Between Media Archaeology and Cultural Semiotics. Theory, Culture & Society 37, 3 (2020), 49--70.Google Scholar
- Tzu-Mao Li, Michal Lukáč, Michaël Gharbi, and Jonathan Ragan-Kelley. 2020. Differentiable vector graphics rasterization for editing and learning. ACM Transactions on Graphics (TOG) 39, 6 (2020), 1--15.Google ScholarDigital Library
- Yuri M Lotman. 1990. Universe of the mind: A semiotic theory of culture. Indiana University Press. 143 pages.Google Scholar
- Daniel Michelis and Jörg Müller. 2011. The audience funnel: Observations of gesture based interaction with multiple large displays in a city center. Intl. Journal of Human-Computer Interaction 27, 6 (2011), 562--579.Google ScholarCross Ref
- Simone Natale and Andrea Ballatore. 2020. Imagining the thinking machine: Technological myths and the rise of artificial intelligence. Convergence 26, 1 (2020), 3--18.Google ScholarCross Ref
- Changhoon Oh, Jungwoo Song, Jinhan Choi, Seonghyeon Kim, Sungwoo Lee, and Bongwon Suh. 2018. I lead, you help but only with enough details: Understanding user experience of co-creation with artificial intelligence. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1--13.Google ScholarDigital Library
- Fred Plaut. 2000. FREUD, SIGMUND. The Interpretation of Dreams. A New Translation by Joyce Crick. Oxford & New York: Oxford University Press, 1999. Journal of Analytical Psychology 45, 4 (2000), 649--651.Google Scholar
- Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning. PMLR, 8748--8763.Google Scholar
- Eleanor Sandry. 2017. Creative collaborations with machines. Philosophy & Technology 30, 3 (2017), 305--319.Google ScholarCross Ref
- Peter Schaldenbrand, Zhixuan Liu, and Jean Oh. 2021. StyleCLIPDraw: Coupling Content and Style in Text-to-Drawing Synthesis. arXiv preprint arXiv:2111.03133 (2021).Google Scholar
- Uwe Seifert and Jin Hyun Kim. 2008. Towards a conceptual framework and an empirical methodology in research on artistic human-computer and human-robot interaction. In Human Computer Interaction. Citeseer.Google Scholar
- Patrick Tresset and Oliver Deussen. 2014. Artistically skilled embodied agents. In AISB.Google Scholar
- Patrick Tresset and Frederic Fol Leymarie. 2012. Sketches by Paul the robot. In Proceedings of the eighth annual symposium on computational aesthetics in graphics, visualization, and imaging. 17--24.Google ScholarDigital Library
- Josh Urban Davis, Fraser Anderson, Merten Stroetzel, Tovi Grossman, and George Fitzmaurice. 2021. Designing Co-Creative AI for Virtual Environments. In Creativity and Cognition. 1--11.Google Scholar
- Joseph Weizenbaum. 1976. Computer power and human reason: From judgment to calculation. (1976).Google Scholar
Index Terms
Dream Painter: Exploring creative possibilities of AI-aided speech-to-image synthesis in the interactive art context
Recommendations
Dream Painter: An Interactive Art Installation Bridging Audience Interaction, Robotics, and Creative AI
MM '22: Proceedings of the 30th ACM International Conference on MultimediaDream Painter is an interactive robotic art installation that turns the audience's spoken dreams into a collective painting. By telling one's past dream, a participant guides the interactive robotic system in the latent space of the AI model that ...
From rituals to magic: Interactive art and HCI of the past, present, and future
Highlights- Interactive art and HCI have contributed to one another.
- Interaction, ...
AbstractThe connection between art and technology is much tighter than is commonly recognized. The emergence of aesthetic computing in the early 2000s has brought renewed focus on this relationship. In this article, we articulate how art and ...
A Robot for Artistic Painting in Authentic Colors
AbstractArtistic robotic painting automates the process of creating an artwork. This complex and challenging task includes several aspects: creating algorithms for rendering brushstrokes, reproducing the exact shape of a brushstroke, and developing the ...
Comments