skip to main content
research-article

Interactive Film Recombination

Authors Info & Claims
Published:12 August 2017Publication History
Skip Abstract Section

Abstract

In this article, we discuss an innovative media entertainment application called Interactive Movietelling. As an offspring of Interactive Storytelling applied to movies, we propose to integrate narrative generation through artificial intelligence (AI) planning with video processing and modeling to construct filmic variants starting from the baseline content. The integration is possible thanks to content description using semantic attributes pertaining to intermediate-level concepts shared between video processing and planning levels. The output is a recombination of segments taken from the input movie performed so as to convey an alternative plot. User tests on the prototype proved how promising Interactive Movietelling might be, even if it was designed at a proof of concept level. Possible improvements that are suggested here lead to many challenging research issues.

Skip Supplemental Material Section

Supplemental Material

References

  1. K. Bálint, T. Klausch, and T. Pólya. 2016. Watching closely. Journal of Media Psychology 1 (2016), 1--10.Google ScholarGoogle Scholar
  2. S. Benini, L. Canini, and R. Leonardi. 2011. A connotative space for supporting movie affective recommendation. IEEE Trans. on Multimedia 13, 6 (2011), 1356--1370. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. S. Benini, P. Migliorati, and R. Leonardi. 2007. A statistical framework for video skimming based on logical story units and motion activity. In Proc. Int. Workshop on Content-Based Multimedia Indexing. IEEE, Bordeaux, France, 152--156. Google ScholarGoogle ScholarCross RefCross Ref
  4. S. Benini, P. Migliorati, and R. Leonardi. 2010. Hierarchical structuring of video previews by leading-cluster-analysis. Signal, Image and Video Processing 4, 4 (2010), 435--450. Google ScholarGoogle ScholarCross RefCross Ref
  5. S. Benini, P. Migliorati, and R. Leonardi. 2010. Statistical skimming of feature films. Int. Journal of Digital Multimedia Broadcasting 2010 (2010), 1--11. Google ScholarGoogle ScholarCross RefCross Ref
  6. S. Benini, M. Svanera, N. Adami, R. Leonardi, and A. Kovács. 2016. Shot scale distribution in art films. Multimedia Tools and Applications 75, 23 (2016), 16499--16527.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. S. Bocconi, F. Nack, and L. Hardman. 2008. Automatic generation of matter-of-opinion video documentaries. Web Semantics: Science, Services and Agents on the WWW 6, 2 (2008), 139--150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. K. Brooks. 1997. Do story agents use rocking chairs? The theory and implementation of one model for computational narrative. In Proc. ACM Int. Conf. on Multimedia. 317--328.Google ScholarGoogle Scholar
  9. L. Canini, S. Benini, and R. Leonardi. 2013. Affective recommendation of movies based on selected connotative features. IEEE Trans. on Circuits and Systems for Video Technology 23, 4 (2013), 636--647. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. A. Chan, Z.-S. Liang, and N. Vasconcelos. 2008. Privacy preserving crowd monitoring: Counting people without people models or tracking. In Proc. Int. Conf. on Computer Vision and Pattern Recognition. IEEE, 1--7. Google ScholarGoogle ScholarCross RefCross Ref
  11. C. Cotsaces, N. Nikolaidis, and I. Pitas. 2006. Video shot detection and condensed representation. A review. IEEE Signal Processing Magazine 23, 2 (2006), 28--37. Google ScholarGoogle ScholarCross RefCross Ref
  12. Chris Crawford. 2003. Chris Crawford on game design. New Riders.Google ScholarGoogle Scholar
  13. Chris Crawford. 2004. Chris Crawford on Interactive Storytelling. New Riders.Google ScholarGoogle Scholar
  14. M. Davis. 1994. Knowledge representation for video. In Proc. Nat. Conf. on Artificial Intelligence (AAAI). 120--127.Google ScholarGoogle Scholar
  15. P. Garcia, D. Bulterman, and L. Soares. 2008. Human-centered television: Directions in interactive television research. ACM Trans. on Multimedia Computing 4, 4 (2008), 1--7.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Graesser and D. Hemphill. 1991. Question answering in the context of scientific mechanisms. Journal of Memory and Language 30, 2 (1991), 186--209. Google ScholarGoogle ScholarCross RefCross Ref
  17. IRIS. 2011. Integrating Research in Interactive Storytelling (IRIS) Network of Excellence. http://iris.scm.tees.ac.uk. (2011). [Online].Google ScholarGoogle Scholar
  18. B. Jung, J. Song, and Y. Lee. 2007. A narrative-based abstraction framework for story-oriented video. ACM Trans. on Multimedia Computing, Communications, and Applications 3, 2 (2007), 11.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. K. Karsch, C. Liu, and S. Kang. 2014. Depth transfer: Depth extraction from video using non-parametric sampling. IEEE Trans. on Pattern Analysis and Machine Intelligence 36, 11 (2014), 2144--2158. Google ScholarGoogle ScholarCross RefCross Ref
  20. A. Kovács. 2014. Shot scale distribution: An authorial fingerprint or a cognitive pattern? Projections 8, 2 (2014), 50--70.Google ScholarGoogle ScholarCross RefCross Ref
  21. M. Mateas and A. Stern. 2005. Structuring content in the façade interactive drama architecture. In Proc. Conf. on AI and Interactive Digital Entertainment (AIIDE). 93--98.Google ScholarGoogle Scholar
  22. D. Mobbs. 2007. The Kuleshov Effect: The influence of contextual framing on emotional attributions. Social Cognitive and Affective NeuroScience 1, 2 (2007), 95--106. Google ScholarGoogle ScholarCross RefCross Ref
  23. J. Monaco. 2000. How to Read a Film: The World of Movies, Media, and Multimedia: Language, History, Theory. Oxford University Press.Google ScholarGoogle Scholar
  24. F. Nack. 1996. AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing. University of Lancaster.Google ScholarGoogle Scholar
  25. Y. Nakamura and T. Kanade. 1997. Semantic analysis for video contents extraction—spotting by association in news video. In Proc. ACM Int. Conf. on Multimedia. ACM, 393--401.Google ScholarGoogle Scholar
  26. A. Piacenza, F. Guerrini, N. Adami, and R. Leonardi. 2012. Markov chains fusion for video scenes generation. In Proc. Eur. Signal Processing Conf. 405--409.Google ScholarGoogle Scholar
  27. A. Piacenza, F. Guerrini, N. Adami, and R. Leonardi. 2013. Tracking characters in movies within logical story units. In Proc. Int. Workshop on Multimedia Signal Processing. IEEE, 183--188. Google ScholarGoogle ScholarCross RefCross Ref
  28. A. Piacenza, F. Guerrini, N. Adami, R. Leonardi, J. Porteous, J. Teutenberg, and M. Cavazza. 2011. Generating story variants with constrained video recombination. In Proc. ACM Int. Conf. on Multimedia. 223--232. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. J. Porteous, S. Benini, L. Canini, F. Charles, M. Cavazza, and R. Leonardi. 2010. Interactive storytelling via video content recombination. In Proc. ACM Int. Conf. on Multimedia. 1715--1718. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. J. Porteous, M. Cavazza, and F. Charles. 2010. Applying planning to interactive storytelling: Narrative control using state constraints. ACM Trans. on Intelligent Systems and Technology 1, 2 (2010), 10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. M. Radford. 2004. The Merchant of Venice (film adaptation). (2004).Google ScholarGoogle Scholar
  32. W. Sack and M. Davis. 1994. IDIC: Assembling video sequences from story plans and content annotations. In Proc. ICMCS. 30--36. Google ScholarGoogle ScholarCross RefCross Ref
  33. N. Serrano, A. Savakis, and A. Luo. 2002. A computationally efficient approach to indoor/outdoor scene classification. In Proc. Int. Conf. on Pattern Recognition, Vol. 4. IEEE, 146--149. Google ScholarGoogle ScholarCross RefCross Ref
  34. C. Shan, S. Gong, and P. McOwan. 2009. Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing 27, 6 (2009), 803--816. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. E. Yu-Te Shen, H. Lieberman, and G. Davenport. 2009. What’s next?: Emergent storytelling from video collection. In Proc. SIGCHI Conf. on Human Factors in Computing Systems. 809--818.Google ScholarGoogle Scholar
  36. T. Smith and G. Davenport. 1992. The stratification system a design environment for random access video. In Proc. Int. Workshop on Network and Operating System Support for Digital Audio and Video. 250--261.Google ScholarGoogle Scholar
  37. V. Zsombori, M. Ursu, J. Wyver, I. Kegel, and D. Williams. 2008. ShapeShifting documentary: A golden age. In Proc. Eur. Conf. on Interactive Television. 40--50. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Interactive Film Recombination

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!