Optimizing content-preserving projections for wide-angle images

Abstract

Any projection of a 3D scene into a wide-angle image unavoidably results in distortion. Current projection methods either bend straight lines in the scene, or locally distort the shapes of scene objects. We present a method that minimizes this distortion by adapting the projection to content in the scene, such as salient scene regions and lines, in order to preserve their shape. Our optimization technique computes a spatially-varying projection that respects user-specified constraints while minimizing a set of energy terms that measure wide-angle image distortion. We demonstrate the effectiveness of our approach by showing results on a variety of wide-angle photographs, as well as comparisons to standard projections.

References

1. Agarwala, A., Agrawala, M., Cohen, M., Salesin, D., and Szeliski, R. 2006. Photographing long scenes with multi-viewpoint panoramas. ACM Trans. on Graph. 25, 3 (July), 853--861.
2. Avidan, S., and Shamir, A. 2007. Seam carving for content-aware image resizing. ACM Trans. on Graph. 26, 3 (July), 10:1--10:9.
3. Bradski, G., and Kaehler, A. 2008. Learning OpenCV: Computer Vision with the OpenCV Library. O'Reilly, Cambridge, MA.
4. Brown, M., and Lowe, D. G. 2003. Recognising panoramas. In International Conference on Computer Vision (ICCV), 1218--1227.
5. Durand, F. 2002. An invitation to discuss computer depiction. In NPAR 2002: Second International Symposium on Non Photorealistic Rendering, 111--124.
6. Flocon, A., and Barre, A. 1988. Curvilinear Perspective: From Visual Space to the Constructed Image. UC Press.
7. Forsyth, D. A., and Ponce, J. 2002. Computer Vision: A Modern Approach. Prentice Hall, August.
8. Gal, R., Sorkine, O., and Cohen-Or, D. 2006. Feature-aware texturing. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 297--304.
9. Heckbert, P. S. 1989. Fundamentals of texture mapping and image warping. Tech. Rep. UCB/CSD-89-516, EECS Department, University of California, Berkeley, Jun.
10. Hilbert, D., and Cohn-Vossen, S. 1952. Geometry and the Imagination. Chelsea, New York.
11. Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. ACM Trans. on Graph. 24, 3 (Aug.), 1134--1141.
12. Itti, L., Koch, C., and Niebur, E. 1998. A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20, 11, 1254--1259.
13. Kopf, J., Uyttendaele, M., Deussen, O., and Cohen, M. F. 2007. Capturing and viewing gigapixel images. ACM Trans. on Graph. 26, 3 (July), 93:1--93:10.
14. Kubovy, M. 1986. The psychology of perspective and renaissance art. Cambridge University Press.
15. Lévy, B., Petitjean, S., Ray, N., andMaillot, J. 2002. Least squares conformal maps for automatic texture atlas generation. ACM Trans. Graph. 21, 3, 362--371.
16. Longfellow, W. 1901. Applied Perspective for Architects and Painters. The Riverside Press.
17. Palmer, S. E. 1999. Vision Science: Photons to Phenomenology. The MIT Press.
18. Sadourny, R., Arakawa, A., and Mintz, Y. 1968. Integration of the non-divergent barotropic vorticity equation with an icosahedral-hexagonal grid for the sphere. Monthly Weather Review 96 (June), 351--356.
19. Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM Trans. on Graph. 25, 3 (July), 533--540.
20. Schenk, O., and Gärtner, K. 2004. Solving unsymmetric sparse systems of linear equations with pardiso. Journal of Future Generation Computer Systems 20, 3, 475--487.
21. Sheffer, A., Praun, E., and Rose, K. 2006. Mesh parameterization methods and their applications. Found. Trends. Comput. Graph. Vis. 2, 2, 105--171.
22. Snyder, J. P. 1985. Computer-assisted map projection research: U.S. Geological Survey Bulletin 1629. U.S. Gov. Printing Office.
23. Snyder, J. P. 1987. Map Projections -- A Working Manual: U.S. Geological Survey Prof. Paper 1395. U.S. Gov. Printing Office.
24. Snyder, J. P. 1993. Flattening the Earth, two thousand years of map projections. University of Chicago Press.
25. Suh, B., Ling, H., Bederson, B. B., and Jacobs, D. W. 2003. Automatic thumbnail cropping and its effectiveness. In Proc. UIST, 95--104.
26. Szeliski, R., and Shum, H.-Y. 1997. Creating full view panoramic mosaics and environment maps. In Proc. SIGGRAPH, 251--258.
27. Viola, P., and Jones, M. J. 2004. Robust real-time face detection. International Journal of Computer Vision 57, 2, 137--154.
28. Vishwanath, D., Girshick, A., and Banks, M. 2005. Why pictures look right when viewed from the wrong place. Naure Neuroscience 8, 10, 1401.
29. Wang, Y.-S., Tai, C.-L., Sorkine, O., and Lee, T.-Y. 2008. Optimized scale-and-stretch for image resizing. ACM Trans. on Graph. 27, 5 (Dec.), 118:1--118:8.
30. Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Non-homogeneous content-driven video-retargeting. In IEEE International Conference on Computer Vision.
31. Zeki, S. 2001. Artistic creativity and the brain. Science 293, 5527, 51--52.
32. Zelnik-Manor, L., Peters, G., and Perona, P. 2005. Squaring the circles in panoramas. In IEEE International Conference on Computer Vision (ICCV), vol. 2, 1292--1299.
33. Zorin, D., and Barr, A. H. 1995. Correction of geometric perceptual distortion in pictures. In Proc. SIGGRAPH, 257--264.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

      • Computer vision representations

        • Shape representations

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Image manipulation

  • Modeling and simulation

    • Model development and analysis

      • Model verification and validation

      • Modeling methodologies

• Theory of computation

  • Randomness, geometry and discrete structures

    • Computational geometry

    • Random projections and metric embeddings