skip to main content
research-article

Watch-and-comment as a paradigm toward ubiquitous interactive video editing

Published:06 November 2008Publication History
Skip Abstract Section

Abstract

The literature reports research efforts allowing the editing of interactive TV multimedia documents by end-users. In this article we propose complementary contributions relative to end-user generated interactive video, video tagging, and collaboration. In earlier work we proposed the watch-and-comment (WaC) paradigm as the seamless capture of an individual's comments so that corresponding annotated interactive videos be automatically generated. As a proof of concept, we implemented a prototype application, the WaCTool, that supports the capture of digital ink and voice comments over individual frames and segments of the video, producing a declarative document that specifies both: different media stream structure and synchronization.

In this article, we extend the WaC paradigm in two ways. First, user-video interactions are associated with edit commands and digital ink operations. Second, focusing on collaboration and distribution issues, we employ annotations as simple containers for context information by using them as tags in order to organize, store and distribute information in a P2P-based multimedia capture platform. We highlight the design principles of the watch-and-comment paradigm, and demonstrate related results including the current version of the WaCTool and its architecture. We also illustrate how an interactive video produced by the WaCTool can be rendered in an interactive video environment, the Ginga-NCL player, and include results from a preliminary evaluation.

References

  1. Abowd, G. D., Mynatt, E. D., and Rodden, T. 2002. The human experience. IEEE Pervasive Comput. 1, 1, 48--57. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Belmont, V. 2006. Create your own video blog. CNet Reviews, http://reviews.cnet.com/Create_your_own_video_blog4660-10165_7-6634979.html.Google ScholarGoogle Scholar
  3. Bulterman, D. C. A. 2003. Using smil to encode interactive, peer-level multimedia annotations. In Proceedings of the ACM Symposium on Document Engineering (DocEng'03). ACM, New York, 32--41. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bulterman, D. C. A., César, P., and Jansen, A. J. 2006. An architecture for viewer-side enrichment of tv content. In Proceedings of the 14th Annual ACM International Conference on Multimedia (MULTIMEDIA'06). ACM, New York, 651--654. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Butkus, A. and Petersen, M. 2007. Semantic modelling using tv-anytime genre metadata. In Proceedings of the European Interactive Television Conference. 226--234. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Cattelan, R. and Pimentel, M. 2008. Supporting multimedia capture in mobile computing environments through a peer-to-peer platform. In Proceedings of the ACM Symposium on Applied Computing (SAC'08). ACM, New York, 1246--1251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Cattelan, R. G., Teixeira, C., Ribas, H., Munson, E., and Pimentel, M. 2008. Inkteractors: interacting with digital ink. In Proceedings of the ACM Symposium on Applied Computing (SAC'08). ACM, New York, 1246--1251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. César, P., Bulterman, D. C. A., and Jansen, A. J. 2006a. The ambulant annotator: empowering viewer-side enrichment of multimedia content. In Proceedings of the ACM Symposium on Document Engineering (DocEng'06). ACM, New York, 186--187. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. César, P., Bulterman, D. C. A., and Jansen, A. J. 2006b. Benefits of structured multimedia documents in idtv: the end-user enrichment system. In Proceedings of the 2006 ACM Symposium on Document Engineering (DocEng'06). ACM, New York, 176--178. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. César, P., Bulterman, D. C. A., and Jansen, A. J. 2008. Usages of the secondary screen in an interactive television environment: Control, enrich, share, and transfer television content. In Proceedings of the European Interactive Television Conference. 168--177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. César, P., Bulterman, D. C. A., Obrenovic, Z., Ducret, J., and Cruz-Lara, S. 2007. An architecture for non-intrusive user interfaces for interactive digital television. In Proceedings of the European Interactive Television Conference. 11--20. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Coppens, T., Trappeniers, L., and Godon, M. 2004. Amigotv: towards a social tv experience. In Proceedings of the European Interactive Television Conference.Google ScholarGoogle Scholar
  13. de Resende Costa, R. M., Moreno, M. F., Rodrigues, R. F., and Soares, L. F. G. 2006. Live editing of hypermedia documents. In Proceedings of the ACM Symposium on Document Engineering (DocEng'06). ACM, New York, 165--172. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. den Ende, N. V., de Hesselle, H., and Meesters, L. 2007. Towards content-aware coding: User study. In Proceedings of the European Interactive Television Conference. 185--194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Geerts, D. 2006. Comparing voice chat and text chat in a communication tool for interactive television. In Proceedings of the 4th Nordic Conference on Human-Computer Interaction (NordiCHI'06). ACM, New York, 461--464. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Girgensohn, A., Boreczky, J., Chiu, P., Doherty, J., Foote, J., Golovchinsky, G., Uchihashi, S., and Wilcox, L. 2000. A semi-automatic approach to home video editing. In Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology (UIST'00). ACM, New York, 81--89. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Goularte, R., Cattelan, R. G., Camacho-Guerrero, J. A., Valter R. Inácio, J., and da Graça C. Pimentel, M. 2004. Interactive multimedia annotations: enriching and extending content. In Proceedings of the ACM Symposium on Document Engineering (DocEng'04). ACM, New York, 84--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Guimarães, R. L., Costa, R. M. R., and Soares, L. F. G. 2008. Composer: Authoring tool for itv programs. In Proceedings of the European Interactive Television Conference. 61--71. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Harboe, G., Massey, N., Metcalf, C., Wheatley, D., and Romano, G. 2008. The uses of social television. Comput. Entertain. 6, 1, 1--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Hemmeryckx-Deleersnijder, B. and Thorne, J. M. 2007. Awareness and conversational context sharing to enrich tv based communication. In Proceedings of the European Interactive Television Conference. 1--10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Hua, X.-S. and Li, S. 2006. Interactive video authoring and sharing based on two-layer templates. In Proceedings of the 1st ACM International Workshop on Human-Centered Multimedia (HCM'06). ACM, New York, 65--74. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Hua, X.-S., Lu, L., and Zhang, H.-J. 2004. P-karaoke: personalized karaoke system. In Proceedings of the 12th Annual ACM International Conference on Multimedia (MULTIMEDIA'04). ACM, New York, 172--173. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Kindberg, T., Spasojevic, M., Fleck, R., and Sellen, A. 2005. I saw this and thought of you: some social uses of camera phones. In Proceedings of the Extended Abstracts Conference on Human Factors in Computing Systems (CHI'05). ACM, New York, 1545--1548. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Kirk, D., Sellen, A., Harper, R., and Wood, K. 2007. Understanding videowork. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'07). ACM, New York, 61--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Liu, K.-Y. and Chen, H.-Y. 2005. Exploring media correlation and synchronization for navigated hypermedia documents. In Proceedings of the 13th Annual ACM International Conference on Multimedia (MULTIMEDIA'05). ACM, New York, 61--70. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Luyten, K., Thys, K., Huypens, S., and Coninx, K. 2006. Telebuddies: social stitching with interactive television. In Proceedings of the Extended Abstracts Conference on Human Factors in Computing Systems (CHI'06). ACM, New York, 1049--1054. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Madhwacharyula, C. L., Davis, M., Mulhem, P., and Kankanhalli, M. S. 2006. Metadata handling: A video perspective. ACM Trans. Multimed. Comput. Comm. Appl. 2, 4, 358--388. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Nielsen, J. 1992. Finding usability problems through heuristic evaluation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'92). ACM, New York, 373--380. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Pimentel, M., Baldochi, L., and Cattelan, R. 2007. Prototyping applications to document human experiences. IEEE Pervasive Comput. 6, 2, 93--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Pimentel, M., Goularte, R., Cattelan, R., Santos, F., and Teixeira, C. 2007. Enhancing multimodal annotations with pen-based information. In Proceedings of the Workshop on New Techniques for Consuming, Managing, and Manipulating Interactive Digital Media at Home. IEEE Computer Society, Los Alamitos, CA, 207--213. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Pimentel, M., Goularte, R., Cattelan, R., Santos, F., and Teixeira, C. 2008. Ubiquitous interactive video editing via multimodal annotations. In Proceedings of the European Interactive Television Conference. 72--81. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Ramos, G. and Balakrishnan, R. 2003. Fluid interaction techniques for the control and annotation of digital video. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology (UIST'03). ACM, New York, NY, 105--114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Regan, T. and Todd, I. 2004. Media center buddies: instant messaging around a media center. In Proceedings of the 3rd Nordic Conference on Human-Computer Interaction (NordiCHI'04). ACM, New York, NY, 141--144. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Ross, J. M. and Ross, K. R. 2005. Developing web-based video training modules to aid students learning multimedia skills. J. Comput. Small Coll. 21, 2, 281--287. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Sadka, A. H. 2004. Visnet: Noe on networked audiovisual media technologies. In Proceedings of the Workshop on Image Analysis for Multimedia Interactive Services.Google ScholarGoogle Scholar
  36. Shamma, D. A., Shaw, R., Shafton, P. L., and Liu, Y. 2007. Watch what i watch: using community activity to understand content. In Proceedings of the International Workshop on Multimedia Information Retrieval (MIR'07). ACM, New York, 275--284. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Shaw, R. and Schmitz, P. 2006. Community annotation and remix: a research platform and pilot deployment. In Proceedings of the 1st ACM International Workshop on Human-Centered Multimedia (HCM'06). ACM, New York, 89--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Truong, B. T. and Venkatesh, S. 2007. Video abstraction: A systematic review and classification. ACM Trans. Multimed. Comput. Comm. Appl. 3, 1, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Tsekleves, E., Cruickshank, L., Hill, A., Kondo, K., and Whitham, R. 2007. Interacting with digital media at home via a second screen. In Proceedings of the Workshop on New Techniques for Consuming, Managing, and Manipulating Interactive Digital Media at Home. IEEE Computer Society, Los Alamitos, CA, 201--206. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Ullmer, B., Ishii, H., and Glas, D. 1998. mediablocks: physical containers, transports, and controls for online media. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'98). ACM, New York, 379--386. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Weiser, M. 1991. The computer for the 21st century. Scientific American 265, 3, 94--104.Google ScholarGoogle ScholarCross RefCross Ref
  42. Weisz, J. D., Kiesler, S., Zhang, H., Ren, Y., Kraut, R. E., and Konstan, J. A. 2007. Watching together: integrating text chat with video. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'07). ACM, New York, 877--886. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Wright, P. C. and Monk, A. F. 1991. The use of think-aloud evaluation methods in design. SIGCHI Bull. 23, 1, 55--57. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Zigelbaum, J., Horn, M. S., Shaer, O., and Jacob, R. J. K. 2007. The tangible video editor: collaborative video editing with active tokens. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI'07). ACM, New York, 43--46. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Watch-and-comment as a paradigm toward ubiquitous interactive video editing

    Recommendations

    Reviews

    Gerald Friedland

    Whether it is an exciting sports game, a suspenseful thriller, or a mind-twisting crime series, it is more fun to watch one of these in a group of people, where you can comment on the foul that wasn't really one, or try to come up with theories on who the murderer actually is. Now imagine that instead of gathering up with people in your living room or at a local pub, you share your thoughts and emotions with the entire world. This paper describes a system that does exactly this: it enables group interaction and commenting on video broadcasts on the Internet. Given an arbitrary video document on the Internet, the system allows for user-video interaction of different types: one can add textual comments, digital ink operations, or associate edit commands, like pause or loop, with a video. The annotations can be stored, or distributed on a peer-to-peer (P2P) based multimedia platform. The paper presents the system, including its various design principles and underlying conceptual ideas. It also presents preliminary evaluation efforts that provide some evidence for the usefulness of some of the features of the system. The paper focuses on the technological aspects of the system. While I found the described conceptual ideas useful and innovative, they are not breathtakingly original. Given that this work focuses on human-computer interaction, I would have liked to see a more elaborate evaluation of how people liked the system when they were forced to use it. Also, issues like split attention-for example, watching and typing at the same time-are not addressed. Overall, the paper is a nice summary of the available technology in this field, underpinned by an elaborate related-work section and bibliography. Online Computing Reviews Service

    Access critical reviews of Computing literature here

    Become a reviewer for Computing Reviews.

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 4, Issue 4
      October 2008
      117 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/1412196
      Issue’s Table of Contents

      Copyright © 2008 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 6 November 2008
      • Received: 1 August 2008
      • Accepted: 1 August 2008
      Published in tomm Volume 4, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!