Abstract
In this article, we discuss an innovative media entertainment application called Interactive Movietelling. As an offspring of Interactive Storytelling applied to movies, we propose to integrate narrative generation through artificial intelligence (AI) planning with video processing and modeling to construct filmic variants starting from the baseline content. The integration is possible thanks to content description using semantic attributes pertaining to intermediate-level concepts shared between video processing and planning levels. The output is a recombination of segments taken from the input movie performed so as to convey an alternative plot. User tests on the prototype proved how promising Interactive Movietelling might be, even if it was designed at a proof of concept level. Possible improvements that are suggested here lead to many challenging research issues.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Interactive Film Recombination
- K. Bálint, T. Klausch, and T. Pólya. 2016. Watching closely. Journal of Media Psychology 1 (2016), 1--10.Google Scholar
- S. Benini, L. Canini, and R. Leonardi. 2011. A connotative space for supporting movie affective recommendation. IEEE Trans. on Multimedia 13, 6 (2011), 1356--1370. Google Scholar
Digital Library
- S. Benini, P. Migliorati, and R. Leonardi. 2007. A statistical framework for video skimming based on logical story units and motion activity. In Proc. Int. Workshop on Content-Based Multimedia Indexing. IEEE, Bordeaux, France, 152--156. Google Scholar
Cross Ref
- S. Benini, P. Migliorati, and R. Leonardi. 2010. Hierarchical structuring of video previews by leading-cluster-analysis. Signal, Image and Video Processing 4, 4 (2010), 435--450. Google Scholar
Cross Ref
- S. Benini, P. Migliorati, and R. Leonardi. 2010. Statistical skimming of feature films. Int. Journal of Digital Multimedia Broadcasting 2010 (2010), 1--11. Google Scholar
Cross Ref
- S. Benini, M. Svanera, N. Adami, R. Leonardi, and A. Kovács. 2016. Shot scale distribution in art films. Multimedia Tools and Applications 75, 23 (2016), 16499--16527.Google Scholar
Digital Library
- S. Bocconi, F. Nack, and L. Hardman. 2008. Automatic generation of matter-of-opinion video documentaries. Web Semantics: Science, Services and Agents on the WWW 6, 2 (2008), 139--150. Google Scholar
Digital Library
- K. Brooks. 1997. Do story agents use rocking chairs? The theory and implementation of one model for computational narrative. In Proc. ACM Int. Conf. on Multimedia. 317--328.Google Scholar
- L. Canini, S. Benini, and R. Leonardi. 2013. Affective recommendation of movies based on selected connotative features. IEEE Trans. on Circuits and Systems for Video Technology 23, 4 (2013), 636--647. Google Scholar
Digital Library
- A. Chan, Z.-S. Liang, and N. Vasconcelos. 2008. Privacy preserving crowd monitoring: Counting people without people models or tracking. In Proc. Int. Conf. on Computer Vision and Pattern Recognition. IEEE, 1--7. Google Scholar
Cross Ref
- C. Cotsaces, N. Nikolaidis, and I. Pitas. 2006. Video shot detection and condensed representation. A review. IEEE Signal Processing Magazine 23, 2 (2006), 28--37. Google Scholar
Cross Ref
- Chris Crawford. 2003. Chris Crawford on game design. New Riders.Google Scholar
- Chris Crawford. 2004. Chris Crawford on Interactive Storytelling. New Riders.Google Scholar
- M. Davis. 1994. Knowledge representation for video. In Proc. Nat. Conf. on Artificial Intelligence (AAAI). 120--127.Google Scholar
- P. Garcia, D. Bulterman, and L. Soares. 2008. Human-centered television: Directions in interactive television research. ACM Trans. on Multimedia Computing 4, 4 (2008), 1--7.Google Scholar
Digital Library
- A. Graesser and D. Hemphill. 1991. Question answering in the context of scientific mechanisms. Journal of Memory and Language 30, 2 (1991), 186--209. Google Scholar
Cross Ref
- IRIS. 2011. Integrating Research in Interactive Storytelling (IRIS) Network of Excellence. http://iris.scm.tees.ac.uk. (2011). [Online].Google Scholar
- B. Jung, J. Song, and Y. Lee. 2007. A narrative-based abstraction framework for story-oriented video. ACM Trans. on Multimedia Computing, Communications, and Applications 3, 2 (2007), 11.Google Scholar
Digital Library
- K. Karsch, C. Liu, and S. Kang. 2014. Depth transfer: Depth extraction from video using non-parametric sampling. IEEE Trans. on Pattern Analysis and Machine Intelligence 36, 11 (2014), 2144--2158. Google Scholar
Cross Ref
- A. Kovács. 2014. Shot scale distribution: An authorial fingerprint or a cognitive pattern? Projections 8, 2 (2014), 50--70.Google Scholar
Cross Ref
- M. Mateas and A. Stern. 2005. Structuring content in the façade interactive drama architecture. In Proc. Conf. on AI and Interactive Digital Entertainment (AIIDE). 93--98.Google Scholar
- D. Mobbs. 2007. The Kuleshov Effect: The influence of contextual framing on emotional attributions. Social Cognitive and Affective NeuroScience 1, 2 (2007), 95--106. Google Scholar
Cross Ref
- J. Monaco. 2000. How to Read a Film: The World of Movies, Media, and Multimedia: Language, History, Theory. Oxford University Press.Google Scholar
- F. Nack. 1996. AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing. University of Lancaster.Google Scholar
- Y. Nakamura and T. Kanade. 1997. Semantic analysis for video contents extraction—spotting by association in news video. In Proc. ACM Int. Conf. on Multimedia. ACM, 393--401.Google Scholar
- A. Piacenza, F. Guerrini, N. Adami, and R. Leonardi. 2012. Markov chains fusion for video scenes generation. In Proc. Eur. Signal Processing Conf. 405--409.Google Scholar
- A. Piacenza, F. Guerrini, N. Adami, and R. Leonardi. 2013. Tracking characters in movies within logical story units. In Proc. Int. Workshop on Multimedia Signal Processing. IEEE, 183--188. Google Scholar
Cross Ref
- A. Piacenza, F. Guerrini, N. Adami, R. Leonardi, J. Porteous, J. Teutenberg, and M. Cavazza. 2011. Generating story variants with constrained video recombination. In Proc. ACM Int. Conf. on Multimedia. 223--232. Google Scholar
Digital Library
- J. Porteous, S. Benini, L. Canini, F. Charles, M. Cavazza, and R. Leonardi. 2010. Interactive storytelling via video content recombination. In Proc. ACM Int. Conf. on Multimedia. 1715--1718. Google Scholar
Digital Library
- J. Porteous, M. Cavazza, and F. Charles. 2010. Applying planning to interactive storytelling: Narrative control using state constraints. ACM Trans. on Intelligent Systems and Technology 1, 2 (2010), 10.Google Scholar
Digital Library
- M. Radford. 2004. The Merchant of Venice (film adaptation). (2004).Google Scholar
- W. Sack and M. Davis. 1994. IDIC: Assembling video sequences from story plans and content annotations. In Proc. ICMCS. 30--36. Google Scholar
Cross Ref
- N. Serrano, A. Savakis, and A. Luo. 2002. A computationally efficient approach to indoor/outdoor scene classification. In Proc. Int. Conf. on Pattern Recognition, Vol. 4. IEEE, 146--149. Google Scholar
Cross Ref
- C. Shan, S. Gong, and P. McOwan. 2009. Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing 27, 6 (2009), 803--816. Google Scholar
Digital Library
- E. Yu-Te Shen, H. Lieberman, and G. Davenport. 2009. What’s next?: Emergent storytelling from video collection. In Proc. SIGCHI Conf. on Human Factors in Computing Systems. 809--818.Google Scholar
- T. Smith and G. Davenport. 1992. The stratification system a design environment for random access video. In Proc. Int. Workshop on Network and Operating System Support for Digital Audio and Video. 250--261.Google Scholar
- V. Zsombori, M. Ursu, J. Wyver, I. Kegel, and D. Williams. 2008. ShapeShifting documentary: A golden age. In Proc. Eur. Conf. on Interactive Television. 40--50. Google Scholar
Digital Library
Index Terms
Interactive Film Recombination
Recommendations
Generating story variants with constrained video recombination
MM '11: Proceedings of the 19th ACM international conference on MultimediaWe present a novel approach to the automatic generation of filmic variants within an implemented Video-Based Storytelling (VBS) system that successfully integrates video segmentation with stochastically controlled re-ordering techniques and narrative ...
An Interactive Narrative Platform for Story Understanding Experiments
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent SystemsInteractive Narratives are systems that use automated narrative generation techniques to create multiple story variants which can be shown to an audience, as virtual narratives, using cinematic staging techniques. The focus of previous research has ...
The experience of interactive storytelling: comparing “fahrenheit” with “façade”
ICEC'11: Proceedings of the 10th international conference on Entertainment ComputingAt the intersection of multimedia, artificial intelligence, and gaming technology, new visions of future entertainment media arise that approximate the “Holodeck” ® idea of interactive storytelling. We report exploratory experiments on the user ...






Comments