Rectangling panoramic images via warping

Abstract

Stitched panoramic images mostly have irregular boundaries. Artists and common users generally prefer rectangular boundaries, which can be obtained through cropping or image completion techniques. In this paper, we present a content-aware warping algorithm that generates rectangular images from stitched panoramic images. Our algorithm consists of two steps. The first local step is mesh-free and preliminarily warps the image into a rectangle. With a grid mesh placed on this rectangle, the second global step optimizes the mesh to preserve shapes and straight lines. In various experiments we demonstrate that the results of our approach are often visually plausible, and the introduced distortion is often unnoticeable.

References

1. Adobe, 2009. www.adobe.com/technology/projects/content-aware-fill.html.
2. Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. In SIGGRAPH 2004, 294--302.
3. Agarwala, A. 2007. Efficient gradient-domain compositing using quadtrees. In SIGGRAPH 2007.
4. Avidan, S., and Shamir, A. 2007. Seam carving for content-aware image resizing. In SIGGRAPH 2007.
5. Barnes, C., Shechtman, E., Finkelstein, A., and Goldman, D. B. 2009. Patchmatch: a randomized correspondence algorithm for structural image editing. In SIGGRAPH 2009.
6. Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 1222--1239.
7. Brown, M., and Lowe, D. 2003. Recognising panoramas. In International Conference on Computer Vision (ICCV), 1218--1225 vol.2.
8. Carroll, R., Agrawal, M., and Agarwala, A. 2009. Optimizing content-preserving projections for wide-angle images. In SIGGRAPH 2009, 43:1--43:9.
9. Carroll, R., Agarwala, A., and Agrawala, M. 2010. Image warps for artistic perspective manipulation. In SIGGRAPH 2010, 127:1--127:9.
10. Chang, C.-H., and Chuang, Y.-Y. 2012. A line-structure-preserving approach to image resizing. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1075--1082.
11. Criminisi, A., Pérez, P., and Toyama, K. 2004. Region filling and object removal by exemplar-based image inpainting. IEEE Transactions on Image Processing (TIP), 1200--1212.
12. Hays, J., and Efros, A. A. 2008. Scene completion using millions of photographs. In SIGGRAPH 2008, 87--94.
13. He, K., and Sun, J. 2012. Statistics of patch offsets for image completion. In European Conference on Computer Vision (ECCV), Springer-Verlag, 16--29.
14. Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. In SIGGRAPH 2005.
15. Joshi, P., Meyer, M., DeRose, T., Green, B., and Sanocki, T. 2007. Harmonic coordinates for character articulation. In SIGGRAPH 2007.
16. Ju, T., Schaefer, S., and Warren, J. 2005. Mean value coordinates for closed triangular meshes. In SIGGRAPH 2005.
17. Komodakis, N., and Tziritas, G. 2007. Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Transactions on Image Processing (TIP).
18. Kopf, J., Lischinski, D., Deussen, O., Cohen-Or, D., and Cohen, M. 2009. Locally adapted projections to reduce panorama distortions. In Computer Graphics Forum, Wiley Online Library, 1083--1089.
19. Kopf, J., Kienzle, W., Drucker, S., and Kang, S. B. 2012. Quality prediction for image completion. In SIGGRAPH Asia 2012.
20. Lipman, Y., Levin, D., and Cohen-Or, D. 2008. Green coordinates. In SIGGRAPH 2008.
21. Liu, F., Gleicher, M., Jin, H., and Agarwala, A. 2009. Content-preserving warps for 3d video stabilization. In SIGGRAPH 2009.
22. Peleg, S., Rousso, B., Rav-Acha, A., and Zomet, A. 2000. Mosaicing on adaptive manifolds. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 1144--1154.
23. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. In SIGGRAPH 2003, 313--318.
24. Pritch, Y., Kav-Venaki, E., and Peleg, S. 2009. Shift-map image editing. In International Conference on Computer Vision (ICCV), IEEE, 151--158.
25. Qi, S., and Ho, J. 2012. Seam segment carving: retargeting images to irregularly-shaped image domains. In European Conference on Computer Vision (ECCV), Springer-Verlag, 314--326.
26. Rubinstein, M., Shamir, A., and Avidan, S. 2008. Improved seam carving for video retargeting. In SIGGRAPH 2008.
27. Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. In SIGGRAPH 2006.
28. Summa, B., Tierny, J., and Pascucci, V. 2012. Panorama weaving: fast and flexible seam processing. In SIGGRAPH 2012.
29. Szeliski, R., and Shum, H.-Y. 1997. Creating full view panoramic image mosaics and environment maps. In SIGGRAPH 97, 251--258.
30. Szeliski, R. 2006. Image alignment and stitching: A tutorial. Foundations and Trends® in Computer Graphics and Vision.
31. Von Gioi, R., Jakubowicz, J., Morel, J., and Randall, G. 2010. Lsd: A fast line segment detector with a false detection control. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 722--732.
32. Wang, Y.-S., Tai, C.-L., Sorkine, O., and Lee, T.-Y 2008. Optimized scale-and-stretch for image resizing. In SIGGRAPH Asia 2008.
33. Wexler, Y., Shechtman, E., and Irani, M. 2007. Space-time completion of video. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 463--476.
34. Wolf, L., Guttmann, M., and Cohen-Or, D. 2007. Non-homogeneous content-driven video-retargeting. In International Conference on Computer Vision (ICCV), 1--6.
35. Xiao, J., Ehinger, K., Oliva, A., and Torralba, A. 2012. Recognizing scene viewpoint using panoramic place representation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2695--2702.
36. Zelnik-Manor, L., Peters, G., and Perona, P. 2005. Squaring the circle in panoramas. In International Conference on Computer Vision (ICCV), IEEE, 1292--1299.
37. Zhang, G., Cheng, M., Hu, S., and Martin, R. 2009. A shape-preserving approach to image resizing. In Computer Graphics Forum, Wiley Online Library, 1897--1906.
38. Zorin, D., and Barr, A. H. 1995. Correction of geometric perceptual distortions in pictures. In SIGGRAPH 95.

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

    • Rendering