Automated video looping with progressive dynamism

Abstract

Given a short video we create a representation that captures a spectrum of looping videos with varying levels of dynamism, ranging from a static image to a highly animated loop. In such a progressively dynamic video, scene liveliness can be adjusted interactively using a slider control. Applications include background images and slideshows, where the desired level of activity may depend on personal taste or mood. The representation also provides a segmentation of the scene into independently looping regions, enabling interactive local adjustment over dynamism. For a landscape scene, this control might correspond to selective animation and deanimation of grass motion, water ripples, and swaying trees. Converting arbitrary video to looping content is a challenging research problem. Unlike prior work, we explore an optimization in which each pixel automatically determines its own looping period. The resulting nested segmentation of static and dynamic scene regions forms an extremely compact representation.

References

1. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Trans. Graph., 23 (3):294--302.
2. Agarwala, A., Zheng, K. C., Pal, C., Agrawala, M., Cohen, M., Curless, B., Salesin, D., and Szeliski, R. 2005. Panoramic video textures. ACM Trans. Graph., 24(3).
3. Bai, J., Agarwala, A., Agrawala, M., and Ramamoorthi, R. 2012. Selectively de-animating video. ACM Trans. Graph., 31(4).
4. Beck, J. and Burg, K. 2012. Cinemagraphs. http://cinemagraphs.com/.
5. Bennett, E. P. and McMillan, L. 2007. Computational time-lapse video. ACM Trans. Graph., 26(3).
6. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. on Pattern Anal. Mach. Intell., 23(11).
7. Burt, P. J. and Adelson, E. H. 1983. A multiresolution spline with application to image mosaics. ACM Trans. Graph., 2(4).
8. Cho, T. S., Joshi, N., Zitnick, C. L., Kang, S. B., Szeliski, R., and Freeman, W. T. 2010. A content-aware image prior. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR).
9. Chuang, Y.-Y., Goldman, D. B., Zheng, K. C., Curless, B., Salesin, D. H., and Szeliski, R. 2005. Animating pictures with stochastic motion textures. ACM Trans. Graph., 24(3).
10. Cohen, M. and Szeliski, R. 2006. The moment camera. IEEE Computer, 39(8).
11. Couture, V., Langer, M., and Roy, S. 2011. Panoramic stereo video textures. ICCV, pages 1251--1258.
12. Freeman, W. T., Adelson, E. H., and Heeger, D. J. 1991. Motion without movement. ACM SIGGRAPH Proceedings.
13. Horn, B. K. P. and Schunk, B. G. 1981. Determining optical flow. Artificial Intelligence, 17:185--203.
14. Joshi, N., Mehta, S., Drucker, S., Stollnitz, E., Hoppe, H., Uyttendaele, M., and Cohen, M. 2012. Cliplets: Juxtaposing still and dynamic imagery. Proceedings of UIST.
15. Kolmogorov, V. and Zabih, R. 2004. What energy functions can be minimized via graph cuts? IEEE Trans. on Pattern Anal. Mach. Intell., 26(2).
16. Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph., 22(3):277--286.
17. Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Trans. Graph., 24(3):519--526.
18. Mahajan, D., Huang, F.-C., Matusik, W., Ramamoorthi, R., and Belhumeur, P. 2009. Moving gradients: A path-based method for plausible image interpolation. ACM Trans. Graph., 28(3):42.
19. Marks, J., Andalman, B., Beardsley, P. A., Freeman, W., Gibson, S., Hodgins, J., Kang, T., Mirtich, B., Pfister, H., Ruml, W., Ryall, K., Seims, J., and Shieber, S. 1997. Design galleries: A general approach to setting parameters for computer graphics and animation. ACM SIGGRAPH Proceedings.
20. Pritch, Y., Rav-Acha, A., and Peleg, S. 2008. Nonchronological video synopsis and indexing. IEEE Trans. on Pattern Anal. Mach. Intell., 30(11).
21. Rav-Acha, A., Pritch, Y., Lischinski, D., and Peleg, S. 2007. Dynamosaicing: Mosaicing of dynamic scenes. IEEE Trans. on Pattern Anal. Mach. Intell., 29(10).
22. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In SIGGRAPH Proceedings, pages 489--498.
23. Sunkavalli, K., Matusik, W., Pfister, H., and Rusinkiewicz, S. 2007. Factored time-lapse video. ACM Trans. Graph., 26(3).
24. Tompkin, J., Pece, F., Subr, K., and Kautz, J. 2011. Towards moment images: Automatic cinemagraphs. In Proc. of the 8th European Conference on Visual Media Production (CVMP 2011).
25. Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. 2005. Interactive video cutout. ACM Trans. Graph., 24(3).
26. Wu, H.-Y., Rubinstein, M., Shih, E., Guttag, J., Durand, F., and Freeman, W. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM Trans. Graph., 31(4).

Index Terms

• Computing methodologies

  • Computer graphics