Multimodal video annotation for contemporary dance creation

ABSTRACT

This paper presents a video annotator that supports multimodal annotation and is applied to contemporary dance as a creation tool. The prototype, developed for Tablet PCs, explores bimanual interaction, using pen and touch input interfaces. This combination can be more natural and familiar than the traditional input interfaces (keyboard or mouse). Contemporary dance is a domain where this type of flexible interaction with video material is relevant in order to augment and improve the rehearsal and creative processes. Motion tracking is used to define the dynamic behavior of the annotations and voice input complements the other modalities. The paper describes the design decisions done by the multidisciplinary development team and the current status of the tool.

References

1. Bradski, G. R. Computer Vision Face Tracking For Use in a Perceptual User Interface. Intel Technology Journal 2, 2 (1998), 12--21.
2. Brandl, P., Forlines, C., Wigdor, D., Haller, M. and Shen, C. Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces. In Proc. AVI 2008. ACM Press (2008), 154--161.
3. Cabral, D. and Correia, N. Pen-Based Video Annotations: A Proposal and a Prototype for Tablet PCs. In Proc. INTERACT 2009, LNCS 5727, Springer-Verlag (2009), 17--20.
4. Cabral, D. and Valente, J. Programmer's Guide for QT Gui + openFrameworks (OF) in C++ (Visual Studio 2008). Technical Report. CITI and DI, FCT/UNL (2011).
5. Hinckley, K., Yatani, K., Pahud, M., Coddington, N., Rodenhouse, J., Wilson, A., Benko, H. and Buxton, B. Pen + touch = new tools. In Proc. UIST 2010. ACM Press (2010), 7--36.
6. iPhone Human Interface Guidelines for Web Applications: User Experience. Technical Report. Apple Inc. (2010).
7. Li, Y., Hinckley, K., Guan, Z. and Landay, J. A. Experimental analysis of mode switching techniques in pen-based user interfaces. In Proc. CHI 2005. ACM Press (2005), 461--470.
8. Sellen, A. J. and Harper, R. H.R. The Myth of the Paperless Office. MIT Press, Cambridge, MA, USA, 2003.
9. Zivkovic, Z., and van der Heijden, F. Efficient Adaptive Density Estimation per Image Pixel for the Task of Background Subtraction. Pattern Recognition Letters 27, 7 (2006), 773--780.

Index Terms

• Information systems

  • Information systems applications

    • Multimedia information systems