Fast computation of seamless video loops

Abstract

Short looping videos concisely capture the dynamism of natural scenes. Creating seamless loops usually involves maximizing spatiotemporal consistency and applying Poisson blending. We take an end-to-end view of the problem and present new techniques that jointly improve loop quality while also significantly reducing processing time. A key idea is to relax the consistency constraints to anticipate the subsequent blending, thereby enabling looping of low-frequency content like moving clouds and changing illumination. We also analyze the input video to remove an undesired bias toward short loops. The quality gains are demonstrated visually and confirmed quantitatively using a new gradient-domain consistency metric. We improve system performance by classifying potentially loopable pixels, masking the 2D graph cut, pruning graph-cut labels based on dominant periods, and optimizing on a coarse grid while retaining finer detail. Together these techniques reduce computation times from tens of minutes to nearly real-time.

References

1. Agarwala, A. 2007. Efficient gradient-domain compositing using quadtrees. ACM Trans. Graph., 26(3):94.
2. Agarwala, A., Zheng, K. C., Pal, C., Agrawala, M., Cohen, M., Curless, B., Salesin, D., and Szeliski, R. 2005. Panoramic video textures. ACM Trans. Graph., 24(3).
3. Bai, J., Agarwala, A., Agrawala, M., and Ramamoorthi, R. 2012. Selectively de-animating video. ACM Trans. Graph., 31(4).
4. Bai, J., Agarwala, A., Agrawala, M., and Ramamoorthi, R. 2013. Automatic cinemagraph portraits. Computer Graphics Forum, 32(4):17--25.
5. Beck, J. and Burg, K. 2012. Cinemagraphs. http://cinemagraphs.com/.
6. Bhat, P., Curless, B., Cohen, M., and Zitnick, L. 2008. Fourier analysis of the 2D screened Poisson equation for gradient domain problems. European Conference on Computer Vision, pages 114--128.
7. Couture, V., Langer, M., and Roy, S. 2011. Panoramic stereo video textures. ICCV, pages 1251--1258.
8. Joshi, N., Mehta, S., Drucker, S., Stollnitz, E., Hoppe, H., Uyttendaele, M., and Cohen, M. 2012. Cliplets: Juxtaposing still and dynamic imagery. Proceedings of UIST.
9. Kolmogorov, V. and Zabih, R. 2004. What energy functions can be minimized via graph cuts? IEEE Trans. on Pattern Anal. Mach. Intell., 26(2).
10. Kwatra, V., Schödl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph., 22(3):277--286.
11. Liao, J., Joshi, N., and Hoppe, H. 2013. Automated video looping with progressive dynamism. ACM Trans. Graph., 32(4).
12. Liu, J. and Sun, J. 2010. Parallel graph-cuts by adaptive bottom-up merging. In Proc. CVPR.
13. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. Graph., 22(3).
14. Schödl, A., Szeliski, R., Salesin, D. H., and Essa, I. 2000. Video textures. In SIGGRAPH Proceedings, pages 489--498.
15. Sevilla-Lara, L., Wulff, J., Sunkavalli, K., and Shechtman, E. 2015. Smooth loops from unconstrained video. Computer Graphics Forum, 34(4):99--107.
16. Tompkin, J., Pece, F., Subr, K., and Kautz, J. 2011. Towards moment images: Automatic cinemagraphs. In Proc. of the 8th European Conference on Visual Media Production (CVMP 2011).

Index Terms

• Computing methodologies

  • Computer graphics