Optimizing color consistency in photo collections

Abstract

With dozens or even hundreds of photos in today's digital photo albums, editing an entire album can be a daunting task. Existing automatic tools operate on individual photos without ensuring consistency of appearance between photographs that share content. In this paper, we present a new method for consistent editing of photo collections. Our method automatically enforces consistent appearance of images that share content without any user input. When the user does make changes to selected images, these changes automatically propagate to other images in the collection, while still maintaining as much consistency as possible. This makes it possible to interactively adjust an entire photo album in a consistent manner by manipulating only a few images.

Our method operates by efficiently constructing a graph with edges linking photo pairs that share content. Consistent appearance of connected photos is achieved by globally optimizing a quadratic cost function over the entire graph, treating user-specified edits as constraints in the optimization. The optimization is fast enough to provide interactive visual feedback to the user. We demonstrate the usefulness of our approach using a number of personal and professional photo collections, as well as internet collections.

References

1. Agarwal, S., Snavely, N., Simon, I., Seitz, S. M., and Szeliski, R. 2009. Building Rome in a day. In Proc. IEEE ICCV.
2. An, X., and Pellacini, F. 2010. User-controllable color transfer. Computer Graphics Forum 29, 2, 263--271.
3. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28, 3.
4. Barnes, C. 2011. PatchMatch: A Fast Randomized Matching Algorithm with Application to Image and Video. PhD thesis, Princeton University.
5. Bychkovsky, V., Paris, S., Chan, E., and Durand, F. 2011. Learning photographic global tonal adjustment with a database of input/output image pairs. In Proc. IEEE CVPR.
6. Caicedo, J. C., Kapoor, A., and Kang, S. B. 2011. Collaborative personalization of image enhancement. In Proc. IEEE CVPR.
7. Dale, K., Johnson, M. K., Sunkavalli, K., Matusik, W., and Pfister, H. 2009. Image restoration using online photo collections. In Proc. IEEE ICCV.
8. Faktor, A., and Irani, M. 2012. "Clustering by Composition" - unsupervised discovery of image categories. In Proc. ECCV (7), 474--487.
9. Farbman, Z., and Lischinski, D. 2011. Tonal stabilization of video. ACM Trans. Graph. 30, 4, 89:1--89:9.
10. Frahm, J.-M., Georgel, P. F., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y.-H., Dunn, E., Clipp, B., and Lazebnik, S. 2010. Building Rome on a cloudless day. In Proc. ECCV (4), vol. 6314, 368--381.
11. Gould, S., and Zhang, Y. 2012. PATCHMATCHGRAPH: building a graph of dense patch correspondences for label transfer. In Proc. ECCV, vol. Part V, 439--452.
12. HaCohen, Y., Shechtman, E., Goldman, D. B., and Lischinski, D. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM Trans. Graph. 30, 4, 70:1--70:9.
13. Hasinoff, S. W., Jóźwiak, M., Durand, F., and Freeman, W. T. 2010. Search-and-replace editing for personal photo collections. In Proc. ICCP.
14. Joshi, N., Matusik, W., Adelson, E. H., and Kriegman, D. J. 2010. Personal photo enhancement using example images. ACM Trans. Graph. 29, 2 (April), 12:1--12:15.
15. Kagarlitsky, S., Moses, Y., and Hel Or, Y. 2009. Piecewise-consistent color mappings of images acquired under various conditions. In Proc. ICCV, 2311--2318.
16. Kang, S. B., Kapoor, A., and Lischinski, D. 2010. Personalization of image enhancement. In Proc. IEEE CVPR.
17. Kim, K. I., Tompkin, J., Theobald, M., Kautz, J., and Theobalt, C. 2012. Match graph construction for large image databases. In Proc. ECCV.
18. Laffont, P.-Y., Bousseau, A., Paris, S., Durand, F., and Drettakis, G. 2012. Coherent intrinsic images from photo collections. ACM Trans. Graph. 31, 6, 202:1--11.
19. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3, 689--694.
20. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60, 2, 91--110.
21. Oskam, T., Hornung, A., Sumner, R. W., and Gross, M. H. 2012. Fast and stable color balancing for images and augmented reality. In 3DIMPVT, IEEE, 49--56.
22. Pitié, F., Kokaram, A. C., and Dahyot, R. 2007. Automated colour grading using colour distribution transfer. Comput. Vis. Image Underst. 107 (July), 123--137.
23. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. (September).
24. Sivic, J., and Zisserman, A. 2003. Video Google: A text retrieval approach to object matching in videos. In Proc. IEEE ICCV, 1470.
25. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. 25 (July), 835--846.
26. Snavely, N., Garg, R., Seitz, S. M., and Szeliski, R. 2008. Finding paths through the world's photos. ACM Trans. Graph. 27, 3, 11--21.
27. van de Weijer, J., Gevers, T., and Gijsenij, A. 2007. Edge-based color constancy. IEEE Trans. Im. Proc. 16, 9, 2207--2214.
28. Yücer, K., Jacobson, A., Hornung, A., and Sorkine, O. 2012. Transfusive image manipulation. ACM Trans. Graph. 31, 6, 176:1--176:9.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

  • Computer graphics

    • Image manipulation