Ozone: continuous state-based media choreography system for live performance

ABSTRACT

This paper describes Ozone, a new media choreography system based on layered, continuous physical models, designed for building a diverse range of interactive spaces that coordinate arbitrary streams of video and audio synthesized in real-time response to continuous, concurrent activity by people in a live event. We aim to build rich responsive spaces that sustain the free improvisation of collectively or individually meaningful non-linguistic gesture. Ozone provides an expressive way to compose the potential "landscape" of an event evolving according to the designer's intent as well as contingent activity. A potential-energy engine evolves superposed states over simplicial complexes modeling the topological space of metaphorical states.

References

1. Aaron Bobick, S. Intille, J. Davis, F. Baird, Claudio Pinhanez, L. Campbell, Y. Ivanov, A. Schötte, and A. Wilson. The kidsroom: A perceptually-based interactive and immersive story environment. Presence 8: Teleoperators and Virtual Environments, pages 367--391, November 1996.
2. I. I. Bukvic, J. Kim, D. Trueman, and T. Grill. munger1: Towards a cross-platform swiss-army knife of real-time granular synthesis. In ICMC Proceedings, 2007.
3. Lucy Bullivant. Responsive Environments: Architecture, Art and Design. Victoria and Albert Museum, 2006.
4. Center for New Music and Audio Technologies. Open sound control. Technical report, CNMAT, University of California, Berkeley, 2002.
5. M. Garelli and M. Gribaudo. Performance Analysis of the ARIA Adaptive Media Processing Workflows using Colored Petri Nets. Electronic Notes in Theoretical Computer Science (ENTCS), 232:55--73, 2009.
6. Neil A. Gershenfeld. When Things Start To Think. Henry Hold and Company Inc., 1999.
7. Usman Haque. The architectural relevance of gordon pask. In Lucy Bullivant, editor, 4dsocial: Interactive Design Environments, pages 54--61. London: John Wiley and Sons Ltd., 2007.
8. S. E. Hudson and G. L. Newell. Probabilistic state machines: dialog management for inputs with uncertainty. In Proceedings of UIST 1992, pages 199--208, 1992.
9. L. Kovar and M. Gleicher. Simplicial families of drawings. In Proceedings of UIST 2001, pages 163--172, 2001.
10. Myron W. Krueger, T. Gionffrido, and K. Hinrichsen. Video place, an artificial reality. In Proc. CHI85 Human Factors on Computing Systems, pages 35--40. ACM Press, 1985.
11. Tom Ngo, Doug Cutrell, Jenny Dana, Bruce Donald, Lorie Loeb, and Shunhui Zhu. Accessible animation and customizable graphics via simplicial configuration modeling. In Proceedings ACM SIGGRAPH Conference, pages 403--410. ACM Press, 2000.
12. M. Ouimet and K. Lundqvist. Timed abstract state machines: An executable specification language for reactive real-time systems. Technical report, May 2006. Technical Report ESL-TIK-000193, Embedded Systems Laboratory, Massachusetts Institute of Technology.
13. S. G. Parker, D. M. Weinstein, and C. R. Johnson. The scirun computational steering software system. Modern Software Tools in Scientific Computing, pages 1--40, 1997.
14. L. Peng, G. Kwon, Y. Chen, K. S. Candan, H. Sundaram, K. Chatha, and M. L. Sapino. Modular Design of Media Retrieval Workflows Using ARIA. Lecture Notes in Computer Science, 4071:491, 2006.
15. Lina Peng, Gisik Kwon, K. Selçuk Candan, Kyung Ryu, Karam Chatha, Hari Sundaram, and Yinpeng Chen. Media processing workflow design and execution with aria. In MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia, pages 800--801, New York, NY, USA, 2005. ACM.
16. Claudio Pinhanez and Aaron Bobick. It/i: A theate rplay featuring an autonomous computer graphics character. In Proceedings of the ACM Multimedia 98 Workshop on Technologies for Interactive Movies, pages 22--29. ACM Press, 1998.
17. K. Ryall, J. Marks, and S. Shieber. án interactive constraint-based system for drawing graphs. In Proceedings of UIST 1997, pages 97--104. ACM Press, 1997.
18. Xin Wei Sha, Michael Fortin, and Jean-Sébastien Rousseau. Calligraphic video: a phenomenological approach to dense visual interaction. In MM '09: Proceedings of the seventeen ACM international conference on Multimedia, MM '09: Proceedings of the seventeen ACM international conference on Multimedia, pages 1091--1100, New York, NY, USA, 2009. ACM, ACM.
19. Flavia Sparacino, Glorianna Davenport, and Alexander Pentland. Media in performance: interactive spaces for dance, theater, circus, and museum exhibits. IBM Systems Journal, Vol. 39, Nos. 3 and 4, 2000.
20. R. van Liere, J. D. Mulder, and J. J. van Wijk. Computationa lsteering. In HPCN96, pages 35--40, 1996.
21. Sha Xin Wei. The TGarden Performance Research Project. Modern Drama, 48:585--608, 2005.
22. Sha Xin Wei. Poetics of performative space. AI and Society, April, 2007..
23. Sha Xin Wei, Chris Salter, and Laura Farabough. Sponge: Unstable surfaces. ec/artS, Issue 2, 2000, 2000.
24. Sha Xin Wei, Tim Sutton, Jean-Sébastien Rousseau, Emmanuel Thivierge, Olfa Driss, Josée-Anne Drolet, and Michael Fortin. Cosmicomics responsive video installation 2007.
25. Sha Xin Wei, Tim Sutton, Jean-Sébastien Rousseau, Emmanuel Thivierge, Josée-Anne Drolet, and Michael Fortin. Meteor shower responsive video installation 2006.
26. Jan C. Schacher, Philippe Kocher. Ambisonics Spatialization Tools for Max/MSP. in Proceedings of the IICMC, New Orleans, ICMC, 2006.
27. Jot, J.-M. and Warusfel, O. A real-time spatial sound processor for music and virtual reality applications. in Proceedings of the ICMC, San Francisco, Pages 294--95, ICMC, 1995

Index Terms

• Applied computing

  • Arts and humanities

    • Fine arts

    • Performing arts

• Computer systems organization

  • Embedded and cyber-physical systems

  • Real-time systems

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

• Human-centered computing

  • Human computer interaction (HCI)

• Software and its engineering

  • Software notations and tools

    • Development frameworks and environments