Abstract
This article presents an approach to postprocessing casually captured videos to improve apparent camera movement. Re-cinematography transforms each frame of a video such that the video better follows cinematic conventions. The approach breaks a video into shorter segments. Segments of the source video where there is no intentional camera movement are made to appear as if the camera is completely static. For segments with camera motions, camera paths are keyframed automatically and interpolated with matrix logarithms to give velocity-profiled movements that appear intentional and directed. Closeups are inserted to provide compositional variety in otherwise uniform segments. The approach automatically balances the tradeoff between motion smoothness and distortion to the original imagery. Results from our prototype show improvements to poor quality home videos.
- Achanta, R., Yan, W.-Q., and Kankanhalli, M. 2006. Modeling intent for home video repurposing. IEEE Multimedia 13, 46--55. Google Scholar
Digital Library
- Adams, B., Venkatesh, S., and Jain, R. 2005. IMCE: Integrated media creation environment. ACM Trans. Multimed. Comput. Comm. Appl 1, 211--247. Google Scholar
Digital Library
- Alexa, M. 2002. Linear combinations of transformations. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 380--387. Google Scholar
Digital Library
- Ang, T. 2005. Digital Video Handbook. Dorling Kindersley. Google Scholar
Digital Library
- Arijon, D. 1991. Grammar of the Film Language. Silman-James Press.Google Scholar
- Bennett, E. P. and McMillan, L. 2003. Proscenium: A framework for spatio-temporal video editing. In Proceedings of the 11th ACM International Conference on Multimedia (MULTIMEDIA'03). ACM, New York, NY, 177--184. Google Scholar
Digital Library
- Bevilacqua, A. and Azzari, P. 2006. High-quality real time motion detection using ptz cameras. In Proceedings of the IEEE International Conference on Video and Signal Based Surveillance (AVSS'06). 23. Google Scholar
Digital Library
- Block, B. A. 2001. The Visual Story: Seeing the Structure of Film, TV, and New Media. Focal Press.Google Scholar
- Bordwell, D. and Thompson, K. 1997. Film Art: An Introduction. McGraw-Hill.Google Scholar
- Brandon, B. 2005. The Complete Digital Video Guide. Reader's Digest. Google Scholar
Digital Library
- Brown, B. 2002. Cinematography: Theory and Practice: Imagemaking for Cinematographers, Directors & Videographers. Butterworth-Heinemann.Google Scholar
- Buehler, C., Bosse, M., and McMillan, L. 2001. Non-metric image-based rendering for video stabilization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Vol. 2. 609--614.Google Scholar
Cross Ref
- Casares, J., Long, A. C., Myers, B. A., Bhatnagar, R., Stevens, S. M., Dabbish, L., Yocum, D., and Corbett, A. 2002. Simplifying video editing using metadata. In Proceedings of the 4th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques. ACM, London, England, 157--166. Google Scholar
Digital Library
- Chalfen, R. 1987. Snapshot Versions of Life. Bowling Green State University Press.Google Scholar
- Chen, L.-Q., Xie, X., Fan, X., Ma, W.-Y., Zhang, H.-J., and Zhou, H.-Q. 2003. A visual attention model for adapting images on small displays. Multimed. Syst. 9, 4, 353--364.Google Scholar
Digital Library
- Christie, M., Machap, R., Normand, J.-M., Olivier, P., and Pickering, J. 2005. Virtual camera planning: A survey. In Proceedings of Smart Graphics. 40--52. Google Scholar
Digital Library
- Crow, F. C. 1984. Summed-area tables for texture mapping. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 207--212. Google Scholar
Digital Library
- Dony, R., Mateer, J., and Robinson, J. 2005. Techniques for automated reverse storyboarding. IEE J. Vision, Image Signal Process. 152, 4, 425--436.Google Scholar
Cross Ref
- Fischler, M. A. and Bolles, R. C. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 6, 381--395. Google Scholar
Digital Library
- Girgensohn, A., Boreczky, J., Chiu, P., Doherty, J., Foote, J., Golovchinsky, G., Uchihashi, S., and Wilcox, L. 2000. A semi-automatic approach to home video editing. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology. 81--89. Google Scholar
Digital Library
- Gleicher, M. 1997. Projective registration with difference decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 331--337. Google Scholar
Digital Library
- Gleicher, M. and Liu, F. 2007. Re-cinematography: Improving the camera dynamics of casual video. In Proceedings of the 15th International Conference on Multimedia. Google Scholar
Digital Library
- Govindu, V. M. 2004. Lie-algebraic averaging for globally consistent motion estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
Cross Ref
- Hansen, M. and Mcdowel, L. 2001. Apparatus and method for removing blank areas from real-time stabilized images by inserting background information. U.S. Patent 6211913.Google Scholar
- Heck, R., Wallick, M., and Gleicher, M. 2007. Virtual videography. ACM Trans. Multimed. Comput. Comm. Appl. 3, 1, 4. Google Scholar
Digital Library
- Hua, X.-S., Lu, L., and Zhang, H.-J. 2004. Optimization-based automated home video editing system. IEEE Trans. Circ. Syst. Video Tech. 14, 5 (May), 572--583. Google Scholar
Digital Library
- Irani, M. and Anandan, P. 1998. Video indexing based on mosaic representations. Proc. IEEE 86, 5, 905--921.Google Scholar
Cross Ref
- Irani, M., Anandan, P., and Hsu, S. 1995. Mosaic based representations of video sequences and their applications. In Proceedings of the International Conference on Computer Vision. 605--611. Google Scholar
Digital Library
- Irani, M., Rousso, B., and Peleg, S. 1994. Recovery of ego-motion using image stabilization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 454--460.Google Scholar
- Johnston, O. and Thomas, F. 1981. The Illusion of Life: Disney Animation. Abbeville Press.Google Scholar
- Katz, S. D. 1991. Film Directing Shot by Shot: Visualizing from Concept to Screen. Michael Wiese Productions.Google Scholar
- Kavan, L., Collins, S., O'Sullivan, C., and Zara, J. 2006. Dual quaternions for rigid transformation blending. Tech. Rep. TCD-CS-2006-46, Trinity College Dublin.Google Scholar
- Kender, J. and Yeo, B.-L. 2000. On the structure and analysis of home video. In Proceedings of ACCV.Google Scholar
- Kirk, D., Sellen, A., Harper, R., and Wood, K. 2007. Understanding videowork. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 61--70. Google Scholar
Digital Library
- Litvin, A., Konrad, J., and Karl, W. 2003. Probabilistic video stabilization using kalman filtering and mosaicking. In Proceedings of the IS&T/SPIE Symposium on Electronic Imaging, Image, and Video Comm. 663--674.Google Scholar
- Liu, F. and Gleicher, M. 2006. Video retargeting: Automating pan and scan. In ACM Multimed. 241--250. Google Scholar
Digital Library
- Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91--110. Google Scholar
Digital Library
- Matsushita, Y., Ofek, E., Ge, W., Tang, X., and Shum, H.-Y. 2006. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mech. Intel. 28, 7, 1150--1163. Google Scholar
Digital Library
- Mei, T., Hua, X.-S., and Zhou, H.-Q. 2005. Tracking users' capture intention: A novel complementary view for home video content analysis. In Proceedings of ACM Multimedia. 531--534. Google Scholar
Digital Library
- Mikolajczyk, K. and Schmid, C. 2005. A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mech. Intel. 27, 10, 1615--1630. Google Scholar
Digital Library
- Nistér, D. 2005. Preemptive RANSAC for live structure and motion estimation. Machine Vision Appl. 16, 5, 321--329. Google Scholar
Digital Library
- Nocedal, J. and Wright, S. J. 2006. Numerical Optimization, 2nd ed. Springer.Google Scholar
- Osian, M. and Van Gool, L. 2004. Video shot characterization. Machine Vision Appl. 15, 172--177. Google Scholar
Digital Library
- Pan, Z. and Ngo, C.-W. 2004. Structuring home video by snippet detection and pattern parsing. In Proceedings of the ACM SIGMM Workshop on Multimedia Information Retrieval. 69--76. Google Scholar
Digital Library
- Rosenholtz, R. 1999. A simple saliency model predicts a number of motion popout phenomena. Vision Research 39, 19, 3157--3163.Google Scholar
Cross Ref
- Suh, B., Ling, H., Bederson, B. B., and Jacobs, D. W. 2003. Automatic thumbnail cropping and its effectiveness. In Proceedings of the Annual ACM Symposium on User Interface Software and Technology. 95--104. Google Scholar
Digital Library
- Szeliski, R. 1996. Video mosaics for virtual environments. IEEE Comput. Graph. Appl. 16, 2, 22--30. Google Scholar
Digital Library
- Szeliski, R. 2006. Image alignment and stitching: A tutorial. Tech. rep. MSR-TR-2004-92, Microsoft Research.Google Scholar
- Teodosio, L. and Bender, W. 2005. Salient stills. ACM Trans. Multimed. Comput. Commun. Appl. 1, 1, 16--36. Google Scholar
Digital Library
- Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 511--518.Google Scholar
- Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time completion of video. IEEE Trans. Pattern Anal. Mech. Intel. 29, 3, 463--476. Google Scholar
Digital Library
- Wood, D. N., Finkelstein, A., Hughes, J. F., Thayer, C. E., and Salesin, D. H. 1997. Multiperspective panoramas for cel animation. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 243--250. Google Scholar
Digital Library
- Yan, W.-Q. and Kankanhalli, M. S. 2002. Detection and removal of lighting and shaking artifacts in home videos. In Proceedings of ACM Multimedia. 107--116. Google Scholar
Digital Library
Index Terms
Re-cinematography: Improving the camerawork of casual video
Recommendations
Re-cinematography: improving the camera dynamics of casual video
MM '07: Proceedings of the 15th ACM international conference on MultimediaThis paper presents an approach to post-processing casually captured videos to improve apparent camera movement. Re-cinematography transforms each frame of a video such that the video better follows cinematic conventions. The approach breaks videos into ...
Planning animation cinematography and shot structure to communicate theme and mood
SMARTGRAPH '02: Proceedings of the 2nd international symposium on Smart graphicsStandard techniques, such as soundtrack recording, storyboarding and key-framing, are used to create animation adaptations of narratives. Many aspects of the narrative, such as moods, themes, character motivations and plot, must he captured in the audio-...
Intelligent camera control using behavior trees
MIG'11: Proceedings of the 4th international conference on Motion in GamesAutomatic camera systems produce very basic animations for virtual worlds. Users often view environments through two types of cameras: a camera that they control manually, or a very basic automatic camera that follows their character, minimizing ...






Comments