Understanding and improving the realism of image composites

Abstract

Compositing is one of the most commonly performed operations in computer graphics. A realistic composite requires adjusting the appearance of the foreground and background so that they appear compatible; unfortunately, this task is challenging and poorly understood. We use statistical and visual perception experiments to study the realism of image composites. First, we evaluate a number of standard 2D image statistical measures, and identify those that are most significant in determining the realism of a composite. Then, we perform a human subjects experiment to determine how the changes in these key statistics influence human judgements of composite realism. Finally, we describe a data-driven algorithm that automatically adjusts these statistical measures in a foreground to make it more compatible with its background in a composite. We show a number of compositing results, and evaluate the performance of both our algorithm and previous work with a human subjects study.

References

1. Alexander, T., 2011. Visual effects supervisor at Industry Light & Magic. Rules of thumb in image compositing. Personal communication, Oct.
2. Berens, P. 2009. Circstat: a matlab toolbox for circular statistics. Journal of Statistical Software 31, 10, 1--21.
3. Bychkovsky, V., Paris, S., Chan, E., and Durand, F. 2011. Learning photographic global tonal adjustment with a database of input/output image pairs. In Proceedings of CVPR, 97--104.
4. Chang, C.-C., and Lin, C.-J. 2011. LIBSVM: A library for support vector machines. ACM Trans. on Intelligent Systems and Technology 2, 27:1--27:27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
5. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. ACM Trans. Graph. 25 (July), 624--630.
6. David, H. A. 1988. The Method of Paired Comparisons. Oxford University Press, 2nd edition.
7. Gijsenij, A., Gevers, T., and van de Weijer, J. 2011. Computational color constancy: Survey and experiments. IEEE Trans. on Image Processing 20, 9 (Sep), 2475--2489.
8. Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2006. Drag-and-drop pasting. ACM Trans. on Graphics 25, 3 (July), 631--637.
9. Johnson, M. K., and Farid, H. 2005. Exposing digital forgeries by detecting inconsistencies in lighting. In Proceedings of the 7th Workshop on Multimedia and Security, 1--10.
10. Lalonde, J.-F., and Efros, A. 2007. Using color compatibility for assessing image realism. In IEEE 11th International Conference on Computer Vision, 1--8.
11. Liaw, A., and Wiener, M. 2002. Classification and regression by randomforest. R News 2, 3, 18--22.
12. Lopez-Moreno, J., Sundstedt, V., Sangorrin, F., and Gutierrez, D. 2010. Measuring the perception of light in-consistencies. In Proceedings of APGV, ACM, 25--32.
13. Lotto, R., and Purves, D. 2002. The empirical basis of color perception. Consciousness and Cognition 11, 4 (Dec.), 609--629.
14. Ogden, J. M., Adelson, E. H., Bergen, J., and Burt, P. 1985. Pyramid-based computer graphics. RCA Engineer 30, 5, 4--15.
15. Ohta, N., and Robertson, A. R. 2005. Colorimetry: Fundamentals and Applications. Wiley, Chichester.
16. Ostrovsky, Y., Cavanagh, P., and Sinha, P. 2005. Perceiving illumination inconsistencies in scenes. Perception 34, 11, 1301--1314.
17. Paris, S., Hasinoff, S. W., and Kautz, J. 2011. Local Laplacian filters: edge-aware image processing with a Laplacian pyramid. ACM Trans. on Graphics 30 (Aug), 68:1--68:12.
18. Peli, E. 1990. Contrast in complex images. Journal of Optical Society of America 7, 10 (Oct), 2032--2040.
19. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. on Graphics 22 (Jul), 313--318.
20. Pouli, T., and Reinhard, E. 2010. Progressive histogram reshaping for creative color transfer and tone reproduction. In Proceedings of NPAR, 81--90.
21. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. S. 2001. Color transfer between images. IEEE Computer Graphics & Applications 21, 5 (Sept./Oct.), 34--41.
22. Reinhard, E., Aküyz, A., Colbert, M., Hughes, C. E., and OConnor, M. 2004. Real-time color blending of rendered and captured video. In Interservice/Industry Training, Simulation and Education Conference, 1--9.
23. Rhemann, C., Rother, C., Wang, J., Gelautz, M., Kohli, P., and Rott, P. 2009. A perceptually motivated online benchmark for image matting. In Proceedings of CVPR, 1826--1833.
24. Russell, B., Torralba, A., Murphy, K., and Freeman, W. 2008. LabelMe: A Database and Web-Based Tool for Image Annotation. International Journal of Computer Vision 77, 1 (May), 157--173.
25. Smith, A. R., and Blinn, J. F. 1996. Blue screen matting. In Proceedings of SIGGRAPH 96, 259--268.
26. Stokes, M., Anderson, M., Chandrasekar, S., and Motta, R. 1996. A standard default color space for the internet-srgb. Microsoft and Hewlett-Packard Joint Report.
27. Sunkavalli, K., Johnson, M. K., Matusik, W., and Pfister, H. 2010. Multi-scale image harmonization. ACM Trans. on Graphics 29, 4 (July), 125:1--125:10.
28. Tao, M. W., Johnson, M. K., and Paris, S. 2010. Error-tolerant image compositing. In European Conference on Computer Vision, 31--44.
29. Tsoumakas, G., and Katakis, I. 2007. Multi-label classification: An overview. International Journal of Data Warehousing and Mining 3 (July/Sept.), 1--13.
30. Wagberg, J., 2007. Optpop - a color properties toolbox, Mar. Software available at http://www.mathworks.com/matlabcentral/fileexchange/13788.
31. Wang, J., and Cohen, M. F. 2007. Image and video matting: a survey. Found. Trends. Comput. Graph. Vis. 3 (January), 97--175.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision representations

        • Image representations