skip to main content
article

A narrative-based abstraction framework for story-oriented video

Authors Info & Claims
Published:01 May 2007Publication History
Skip Abstract Section

Abstract

This article proposes a novel video abstraction framework for online review services of story-oriented videos such as dramas. Among the many genres of TV programs, a drama is one of the most popularly watched on the Web. The abstracts generated by the proposed framework not only give a summary of a video but also effectively help viewers understand the overall story. In addition, our method is duration-flexible. We get clues about human understanding of a story from scenario writing rules and editorial techniques that are popularly used in the process of video production to explicitly express a narrative, and propose a new video abstraction model, called a Narrative Abstraction Model. The model effectively captures the narrative structure embedded in a story-oriented video and articulates the progress of the story in a weighted directed graph, called a Narrative Structure Graph (NSG). The model provides a basis for a flexible framework for abstract generation using the NSG as the intermediary representation of a video. Different abstracts can be appropriately generated based upon different user requirements. To show the effectiveness of the proposed model and method, we developed a video abstraction system realizing the framework, and successfully applied it to large volumes of TV dramas. The evaluation results show that the proposed framework is a feasible solution for online review services.

References

  1. Adams, B., Dorai, C., and Venkatesh, S. 2002. Toward automatic extraction of expressive elements from motion pictures: Tempo. IEEE Trans. Multimedia 4, 4 (Dec.), 472--481. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Arai, H. 1987. Fundamental Techniques for Scenario Writing. Da-Bo Munhwa (in Korean).Google ScholarGoogle Scholar
  3. Arijon, D. 1991. Grammar of The Film Language. Silman-James Press.Google ScholarGoogle Scholar
  4. Babaguchi, N., Kawai, Y., and Kitahashi, T. 2002. Event based indexing of broadcasted sports video by intermodal collaboration. IEEE Trans. Multimedia 4, 1 (Dec.), 68--75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Bordwell, D. and Thompson, K. 1996. Film Art: An Introduction. McGraw-Hill Companies.Google ScholarGoogle Scholar
  6. Boreczky, J. S. and Rowe, L. A. 1996. Comparison of video shot boundary detection techniques. In Proceedings of SPIE Storage and Retrieval for Still Image and Video Databases IV (Feb.). San Jose, CA, 170--179.Google ScholarGoogle Scholar
  7. Branigan, E. 1992. Narrative Comprehension and Film (Sightlines Series). Routledge.Google ScholarGoogle Scholar
  8. Brooks, K. M. 1996. Do story agents use rocking chairs? The theory and implementation of one model for computational narrative. In Proceedings of the ACM International Conference on Multimedia. Boston, MA, 317--328. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chatman, S. 1980. Story and Discourse---Narrative Structure in Fiction and Film. Cornell University Press.Google ScholarGoogle Scholar
  10. Hanjalic, A., Kakes, G., Lagendijk, R. L., and Biemond, J. 2001. Indexing and retrieval of broadcast news programs using DANCERS. SPIE Journal of Electronic Imaging, Special Issue on Storage, Processing and Retrieval of Digital Media, 10, 4 (Oct.), 871--882.Google ScholarGoogle Scholar
  11. Hanjalic, A., Lagendijk, R. L., and Biemond, J. 1999. Automated high-level movie segmentation for advanced video-retrieval systems. IEEE Trans. Circuits Syst. Video Tech. 9, 4 (June), 580--588. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Hanjalic, A. and Zhang, H. J. 1999. An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE Trans. Circuits Syst. Video Tech. 9, 8 (Dec.), 1280--1289. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. He, L., Sanocki, E., Gupta, A., and Grudin, J. 1999. Auto-summarization of audio-video presentations. In Proceedings of ACM Multimedia. Orlando, Florida, 489--498. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Jung, B., Ha, M., Kim, H., Park, K., Lee, H., and Kim, W. 2004a. A component-based DCT/LDA face recognition method for character retrieval in TV programs. In Proceedings of the International Workshop on Image Analysis for Multimedia Interactive Services. WedAmPo1 (April).Google ScholarGoogle Scholar
  15. Jung, B., Kwak, T., Song, J., and Lee, Y. 2004b. Narrative abstraction model for story-oriented video. In Proceedings of the ACM International Conference on Multimedia (Oct.). New York, NY, 828--835. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. KBS Access Statistics. http://access.kbs.co.kr/.Google ScholarGoogle Scholar
  17. Kender, J. R. and Yeo, B. L. 1998. Video scene segmentation via continuous video coherence. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (June). Santa Barbara, CA, 367--373. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kupiec, J., Pedersen, J., and Chen, F. 1995. A trainable document summarizer. In Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (July). Seattle, Washington, 68--73. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Lienhart, R., Pfeiffer, S., and Effelsberg, W. 1999. Scene determination based on video and audio features. In Proceedings of IEEE International Conference on Multimedia Computing and Systems (June). Florence, Italy, 685--690. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Meng, J., Juan, Y., and Chang, S. F. 1995. Scene change detection in a MPEG compressed video sequences. In Proceedings of IS&T/SPIE Symposium Electronic Imaging (Feb.). San Jose, CA, 14--25.Google ScholarGoogle Scholar
  21. Nang, J., Jeong, J., Ha, M., Jung, B., and Kim, K. 2002. An authoring tool generating various video abstractions semi-automatically. In Proceedings of the Third IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing (Dec.). Hsinchu, Taiwan, 311--318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Nam, J. and Tewfik, A. H. 1999. Video Abstract of Video. In Proceedings of the Third IEEE Workshop on Multimedia Signal Processing (Sep.). Copenhagen, Denmark, 117--122.Google ScholarGoogle Scholar
  23. Omoigui, N., He, L., Gupta, A., Grudin, J. and Sanocki, E. 1999. Time-compression: Systems concerns, usage, and benefits. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (May). Pittsburgh, 136--143. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Patel, N. V. and Sethi, I. K. 1996. Audio characterization for video indexing. In Proceedings of IS&T SPIE on Storage and Retrieval for Still Image and Video Databases IV (Mar.). San Jose, CA, 373--384.Google ScholarGoogle Scholar
  25. Pfeiffer, S., Lienhart, R., Fischer, S., and Effelsberg, W. 1996. Abstracting digital movies automatically. J. Visual Comm. Image Repres. 7, 4 (July), 345--353.Google ScholarGoogle ScholarCross RefCross Ref
  26. Sharff, S. 1982. The Elements of Cinema: Towards a Theory of Cinesthetic Impact. Columbia University Press.Google ScholarGoogle Scholar
  27. Smith, M. A. and Kanade, T. 1997. Video skimming and characterization through the combination of image and language understanding techniques. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (June). Puerto Rico, 775--781. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Song, J. and Yeo, B. L. 1999. Fast extraction of spatially reduced image sequences from MPEG-2 compressed video. IEEE Trans. Circuits Syst. Video Tech. 9, 7 (Oct.). 1100--1114. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Song, J. and Yeo, B. L. 2000. A fast algorithm for DCT-domain inverse motion compensation based on shared information in a macroblock. IEEE Trans. Circuits Syst. Video Tech. 10, 5 (Aug.), 767--775. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Sundaram, H. and Chang, S. F. 2002. Computable scenes and structures in films. IEEE Trans. Multimedia 4, 4 (Dec.), 482--491. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Sundaram, H., Xie, L., and Chang, S. F. 2002. A Utility framework for the automatic generation of audio-visual skims. In Proceedings of the ACM International Conference on Multimedia (Dec.). Juan-les-Pins, France, 189--198. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Syeda-Mahmood, T. and Ponceleon, D. 2001. Learning video browsing behavior and its application in the generation of video previews. In Proceedings of the ACM International Conference on Multimedia (Sept.). Ottawa, Canada, 119--128. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Uchihashi, S. and Foote, J. 1999. Summarizing video using a shot importance measure and a frame-packing algorithm. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (Mar.). Phoenix, AZ, 3041--3044. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Vasconcelos, N. and Lippman, A. 1998. A spatiotemporal motion model for video summarization. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (June). Santa Barbara, CA, 361--366. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Yeo, B. L. and Liu, B. 1995a. A unified approach to temporal segmentation of motion JPEG and MPEG compressed video. In Proceedings of the International Conference on Multimedia Computing and Systems (May). Washington, DC, 81--88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Yeo, B. L. and Liu, B. 1995b. Rapid scene analysis on compressed video. IEEE Trans. Circuits Syst. Video Tech. 5, 6 (Dec.), 533--544. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Yeo, B. L. and Liu, B. 1995c. On the extraction of DC sequence from MPEG compressed video. In Proceedings of the IEEE International Conference on Image Processing, vol II (Oct.). Washington, DC, 260--263. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Yeung, M. M. and Yeo, B. L. 1997. Video visualization for compact presentation and fast browsing of pictorial content. IEEE Trans. Circuits Syst. Video Tech. 7, 5 (Oct.), 771--785. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Yeung, M. M., Yeo, B. L., and Liu, B. 1996. Extracting story units from long programs for video browsing and navigation. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems (June). Hiroshima, Japan, 296--305. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Yeung, M. M., Yeo, B. L., Wolf, W., and Liu, B. 1995. Video browsing using clustering and scene transitions on compressed sequences. In Proceedings of IS&T SPIE Multimedia Computing and Networking (Feb.). San Jose, CA, 399--413.Google ScholarGoogle Scholar
  41. Yoshitaka, A., Ishii, T., Hirakawa, M., and Ichikawa, T. 1997. Content-based retrieval of video data by the grammar of film. In Proceedings of the IEEE Symposium on Visual Languages (Sept.). Capri, Italy, 310--317. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Zhang, H. J., Low, C. Y., Smoliar, S. W., and Wu, J. H. 1995. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proceedings of the ACM International Conference on Multimedia (Nov.). New York, NY, 15--24. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A narrative-based abstraction framework for story-oriented video

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!