Abstract
This article proposes a novel video abstraction framework for online review services of story-oriented videos such as dramas. Among the many genres of TV programs, a drama is one of the most popularly watched on the Web. The abstracts generated by the proposed framework not only give a summary of a video but also effectively help viewers understand the overall story. In addition, our method is duration-flexible. We get clues about human understanding of a story from scenario writing rules and editorial techniques that are popularly used in the process of video production to explicitly express a narrative, and propose a new video abstraction model, called a Narrative Abstraction Model. The model effectively captures the narrative structure embedded in a story-oriented video and articulates the progress of the story in a weighted directed graph, called a Narrative Structure Graph (NSG). The model provides a basis for a flexible framework for abstract generation using the NSG as the intermediary representation of a video. Different abstracts can be appropriately generated based upon different user requirements. To show the effectiveness of the proposed model and method, we developed a video abstraction system realizing the framework, and successfully applied it to large volumes of TV dramas. The evaluation results show that the proposed framework is a feasible solution for online review services.
- Adams, B., Dorai, C., and Venkatesh, S. 2002. Toward automatic extraction of expressive elements from motion pictures: Tempo. IEEE Trans. Multimedia 4, 4 (Dec.), 472--481. Google Scholar
Digital Library
- Arai, H. 1987. Fundamental Techniques for Scenario Writing. Da-Bo Munhwa (in Korean).Google Scholar
- Arijon, D. 1991. Grammar of The Film Language. Silman-James Press.Google Scholar
- Babaguchi, N., Kawai, Y., and Kitahashi, T. 2002. Event based indexing of broadcasted sports video by intermodal collaboration. IEEE Trans. Multimedia 4, 1 (Dec.), 68--75. Google Scholar
Digital Library
- Bordwell, D. and Thompson, K. 1996. Film Art: An Introduction. McGraw-Hill Companies.Google Scholar
- Boreczky, J. S. and Rowe, L. A. 1996. Comparison of video shot boundary detection techniques. In Proceedings of SPIE Storage and Retrieval for Still Image and Video Databases IV (Feb.). San Jose, CA, 170--179.Google Scholar
- Branigan, E. 1992. Narrative Comprehension and Film (Sightlines Series). Routledge.Google Scholar
- Brooks, K. M. 1996. Do story agents use rocking chairs? The theory and implementation of one model for computational narrative. In Proceedings of the ACM International Conference on Multimedia. Boston, MA, 317--328. Google Scholar
Digital Library
- Chatman, S. 1980. Story and Discourse---Narrative Structure in Fiction and Film. Cornell University Press.Google Scholar
- Hanjalic, A., Kakes, G., Lagendijk, R. L., and Biemond, J. 2001. Indexing and retrieval of broadcast news programs using DANCERS. SPIE Journal of Electronic Imaging, Special Issue on Storage, Processing and Retrieval of Digital Media, 10, 4 (Oct.), 871--882.Google Scholar
- Hanjalic, A., Lagendijk, R. L., and Biemond, J. 1999. Automated high-level movie segmentation for advanced video-retrieval systems. IEEE Trans. Circuits Syst. Video Tech. 9, 4 (June), 580--588. Google Scholar
Digital Library
- Hanjalic, A. and Zhang, H. J. 1999. An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE Trans. Circuits Syst. Video Tech. 9, 8 (Dec.), 1280--1289. Google Scholar
Digital Library
- He, L., Sanocki, E., Gupta, A., and Grudin, J. 1999. Auto-summarization of audio-video presentations. In Proceedings of ACM Multimedia. Orlando, Florida, 489--498. Google Scholar
Digital Library
- Jung, B., Ha, M., Kim, H., Park, K., Lee, H., and Kim, W. 2004a. A component-based DCT/LDA face recognition method for character retrieval in TV programs. In Proceedings of the International Workshop on Image Analysis for Multimedia Interactive Services. WedAmPo1 (April).Google Scholar
- Jung, B., Kwak, T., Song, J., and Lee, Y. 2004b. Narrative abstraction model for story-oriented video. In Proceedings of the ACM International Conference on Multimedia (Oct.). New York, NY, 828--835. Google Scholar
Digital Library
- KBS Access Statistics. http://access.kbs.co.kr/.Google Scholar
- Kender, J. R. and Yeo, B. L. 1998. Video scene segmentation via continuous video coherence. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (June). Santa Barbara, CA, 367--373. Google Scholar
Digital Library
- Kupiec, J., Pedersen, J., and Chen, F. 1995. A trainable document summarizer. In Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (July). Seattle, Washington, 68--73. Google Scholar
Digital Library
- Lienhart, R., Pfeiffer, S., and Effelsberg, W. 1999. Scene determination based on video and audio features. In Proceedings of IEEE International Conference on Multimedia Computing and Systems (June). Florence, Italy, 685--690. Google Scholar
Digital Library
- Meng, J., Juan, Y., and Chang, S. F. 1995. Scene change detection in a MPEG compressed video sequences. In Proceedings of IS&T/SPIE Symposium Electronic Imaging (Feb.). San Jose, CA, 14--25.Google Scholar
- Nang, J., Jeong, J., Ha, M., Jung, B., and Kim, K. 2002. An authoring tool generating various video abstractions semi-automatically. In Proceedings of the Third IEEE Pacific Rim Conference on Multimedia: Advances in Multimedia Information Processing (Dec.). Hsinchu, Taiwan, 311--318. Google Scholar
Digital Library
- Nam, J. and Tewfik, A. H. 1999. Video Abstract of Video. In Proceedings of the Third IEEE Workshop on Multimedia Signal Processing (Sep.). Copenhagen, Denmark, 117--122.Google Scholar
- Omoigui, N., He, L., Gupta, A., Grudin, J. and Sanocki, E. 1999. Time-compression: Systems concerns, usage, and benefits. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (May). Pittsburgh, 136--143. Google Scholar
Digital Library
- Patel, N. V. and Sethi, I. K. 1996. Audio characterization for video indexing. In Proceedings of IS&T SPIE on Storage and Retrieval for Still Image and Video Databases IV (Mar.). San Jose, CA, 373--384.Google Scholar
- Pfeiffer, S., Lienhart, R., Fischer, S., and Effelsberg, W. 1996. Abstracting digital movies automatically. J. Visual Comm. Image Repres. 7, 4 (July), 345--353.Google Scholar
Cross Ref
- Sharff, S. 1982. The Elements of Cinema: Towards a Theory of Cinesthetic Impact. Columbia University Press.Google Scholar
- Smith, M. A. and Kanade, T. 1997. Video skimming and characterization through the combination of image and language understanding techniques. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (June). Puerto Rico, 775--781. Google Scholar
Digital Library
- Song, J. and Yeo, B. L. 1999. Fast extraction of spatially reduced image sequences from MPEG-2 compressed video. IEEE Trans. Circuits Syst. Video Tech. 9, 7 (Oct.). 1100--1114. Google Scholar
Digital Library
- Song, J. and Yeo, B. L. 2000. A fast algorithm for DCT-domain inverse motion compensation based on shared information in a macroblock. IEEE Trans. Circuits Syst. Video Tech. 10, 5 (Aug.), 767--775. Google Scholar
Digital Library
- Sundaram, H. and Chang, S. F. 2002. Computable scenes and structures in films. IEEE Trans. Multimedia 4, 4 (Dec.), 482--491. Google Scholar
Digital Library
- Sundaram, H., Xie, L., and Chang, S. F. 2002. A Utility framework for the automatic generation of audio-visual skims. In Proceedings of the ACM International Conference on Multimedia (Dec.). Juan-les-Pins, France, 189--198. Google Scholar
Digital Library
- Syeda-Mahmood, T. and Ponceleon, D. 2001. Learning video browsing behavior and its application in the generation of video previews. In Proceedings of the ACM International Conference on Multimedia (Sept.). Ottawa, Canada, 119--128. Google Scholar
Digital Library
- Uchihashi, S. and Foote, J. 1999. Summarizing video using a shot importance measure and a frame-packing algorithm. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (Mar.). Phoenix, AZ, 3041--3044. Google Scholar
Digital Library
- Vasconcelos, N. and Lippman, A. 1998. A spatiotemporal motion model for video summarization. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (June). Santa Barbara, CA, 361--366. Google Scholar
Digital Library
- Yeo, B. L. and Liu, B. 1995a. A unified approach to temporal segmentation of motion JPEG and MPEG compressed video. In Proceedings of the International Conference on Multimedia Computing and Systems (May). Washington, DC, 81--88. Google Scholar
Digital Library
- Yeo, B. L. and Liu, B. 1995b. Rapid scene analysis on compressed video. IEEE Trans. Circuits Syst. Video Tech. 5, 6 (Dec.), 533--544. Google Scholar
Digital Library
- Yeo, B. L. and Liu, B. 1995c. On the extraction of DC sequence from MPEG compressed video. In Proceedings of the IEEE International Conference on Image Processing, vol II (Oct.). Washington, DC, 260--263. Google Scholar
Digital Library
- Yeung, M. M. and Yeo, B. L. 1997. Video visualization for compact presentation and fast browsing of pictorial content. IEEE Trans. Circuits Syst. Video Tech. 7, 5 (Oct.), 771--785. Google Scholar
Digital Library
- Yeung, M. M., Yeo, B. L., and Liu, B. 1996. Extracting story units from long programs for video browsing and navigation. In Proceedings of the IEEE International Conference on Multimedia Computing and Systems (June). Hiroshima, Japan, 296--305. Google Scholar
Digital Library
- Yeung, M. M., Yeo, B. L., Wolf, W., and Liu, B. 1995. Video browsing using clustering and scene transitions on compressed sequences. In Proceedings of IS&T SPIE Multimedia Computing and Networking (Feb.). San Jose, CA, 399--413.Google Scholar
- Yoshitaka, A., Ishii, T., Hirakawa, M., and Ichikawa, T. 1997. Content-based retrieval of video data by the grammar of film. In Proceedings of the IEEE Symposium on Visual Languages (Sept.). Capri, Italy, 310--317. Google Scholar
Digital Library
- Zhang, H. J., Low, C. Y., Smoliar, S. W., and Wu, J. H. 1995. Video parsing, retrieval and browsing: An integrated and content-based solution. In Proceedings of the ACM International Conference on Multimedia (Nov.). New York, NY, 15--24. Google Scholar
Digital Library
Index Terms
A narrative-based abstraction framework for story-oriented video
Recommendations
Narrative abstraction model for story-oriented video
MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on MultimediaTV program review services, especially drama review services, are one of the most popular video on demand services on the Web. In this paper, we propose a novel video abstraction model for a review service of story-oriented video such as dramas. In a ...
Story Designer: Towards a Mixed-Initiative Tool to Create Narrative Structures
FDG '22: Proceedings of the 17th International Conference on the Foundations of Digital GamesNarratives are a predominant part of games, and their design poses challenges when identifying, encoding, interpreting, evaluating, and generating them. One way to address this would be to approach narrative design in a more abstract layer, such as ...
A framework for narrative adaptation in interactive story-based learning environments
INT3 '10: Proceedings of the Intelligent Narrative Technologies III WorkshopA key functionality provided by interactive narrative systems is narrative adaptation: augmenting story experiences in response to users' actions, and tailoring story elements to individual users' preferences and needs. However, the task of interactive ...






Comments