Image alignment and stitching: a tutorial

Abstract

This tutorial reviews image alignment and image stitching algorithms. Image alignment algorithms can discover the correspondence relationships among images with varying degrees of overlap. They are ideally suited for applications such as video stabilization, summarization, and the creation of panoramic mosaics. Image stitching algorithms take the alignment estimates produced by such registration algorithms and blend the images in a seamless manner, taking care to deal with potential problems such as blurring or ghosting caused by parallax and scene movement as well as varying image exposures. This tutorial reviews the basic motion models underlying alignment and stitching algorithms, describes effective direct (pixel-based) and feature-based alignment algorithms, and describes blending algorithms used to produce seamless mosaics. It ends with a discussion of open research problems in the area.

References

1. {1} A. Agarwala et al., "Interactive digigtal photomontage," ACM Transactions on Graphics, vol. 23, no. 3, pp. 292-300, August 2004.
2. {2} A. Agarwala et al., "Panoramic video textures," ACM Transactions on Graphics , vol. 24, no. 3, pp. 821-827, August 2005.
3. {3} P. Anandan, "A computational framework and an algorithm for the measurement of visual motion," International Journal of Computer Vision, vol. 2, no. 3, pp. 283-310, January 1989.
4. {4} V. Argyriou and T. Vlachos, "Estimation of sub-pixel motion using gradient cross-correlation," Electronic Letters, vol. 39, no. 13, pp. 980-982, June 2003.
5. {5} N. Ayache, Vision Stéréoscopique et Perception Multisensorielle. Paris: InterEditions, 1989.
6. {6} A. Bab Hadiashar and D. Suter, "Robust total least squares based optic flow computation," in Asian Conference on Computer Vision (ACCV'98), (Hong Kong), pp. 566-573, ACM, January 1998.
7. {7} F. Badra, A. Qumsieh, and G. Dudek, "Rotation and zooming in image mosaicing," in IEEE Workshop on Applications of Computer Vision (WACV'98), (Princeton), pp. 50-55, October 1998. IEEE Computer Society.
8. {8} S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1167-1183, September 2002.
9. {9} S. Baker and I. Matthews, "Lucas-Kanade 20 years on: A unifying framework: Part 1: The quantity approximated, the warp update rule, and the gradient descent approximation," International Journal of Computer Vision, vol. 56, no. 3, pp. 221-255, March 2004.
10. {10} S. Baker et al., "Lucas-kanade 20 years on: A unifying framework: Part 4," The Robotics Institute, Carnegie Mellon University, February 2004. Technical Report CMU-RI-TR-04-14.
11. {11} S. Baker et al., "Lucas-kanade 20 years on: A unifying framework: Part 2," The Robotics Institute, Carnegie Mellon University, February 2003. Technical Report CMU-RI-TR-03-01.
12. {12} S. Baker et al., "Lucas-kanade 20 years on: A unifying framework: Part 3," The Robotics Institute, Carnegie Mellon University, November 2003. Technical Report CMU-RI-TR-03-35.
13. {13} J. P. Barreto and K. Daniilidis, "Fundamental matrix for cameras with radial distortion," in Tenth International Conference on Computer Vision (ICCV 2005), (Beijing, China), vol. 1, pp. 625-632, October 2005.
14. {14} A. Bartoli, M. Coquerelle, and P. Sturm, "A framework for pencil-of-points structure-from-motion," in Eighth European Conference on Computer Vision (ECCV 2004), (Prague), vol. II, pp. 28-40, Springer-Verlag, May 2004.
15. {15} P. Baudisch et al., "Panoramic viewfinder: Providing a real-time preview to help users avoid flaws in panoramic pictures," in OZCHI 2005, (Canberra, Australia), November 2005.
16. {16} A. Baumberg, "Reliable feature matching across widely separated views," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2000), (Hilton Head Island), vol. 1, pp. 774-781, June 2000.
17. {17} H. Bay, T. Tuytelaars, and L. V. Gool, "Surf: Speeded up robust features," in Computer Vision - ECCV 2006, (A. Leonardis, H. Bischof, and A. Pinz, eds.), vol. 3951, pp. 404-417, of Lecture Notes in Computer Science, Springer, 2006.
18. {18} J. S. Beis and D. G. Lowe, "Shape indexing using approximate nearest-neighbour search in high-dimensional spaces," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'97), (San Juan, Puerto Rico), pp. 1000-1006, June 1997.
19. {19} R. Benosman and S. B. Kang, eds., Panoramic Vision: Sensors, Theory, and Applications. New York: Springer, 2001.
20. {20} J. R. Bergen, P. Anandan, K. J. Hanna, and R. Hingorani, "Hierarchical model-based motion estimation," in Second European Conference on Computer Vision (ECCV'92), (Santa Margherita Liguere, Italy), pp. 237-252, Springer-Verlag, May 1992.
21. {21} J. R. Bergen, P. J. Burt, R. Hingorani, and S. Peleg, "A three-frame algorithm for estimating two-component image motion," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 9, pp. 886-896, September 1992.
22. {22} M. J. Black and P. Anandan, "The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields," Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
23. {23} M. J. Black and A. D. Jepson, "EigenTracking: Robust matching and tracking of articulated objects using a view-based representation," International Journal of Computer Vision, vol. 26, no. 1, pp. 63-84, January 1998.
24. {24} M. J. Black and A. Rangarajan, "On the unification of line processes, outlier rejection, and robust statistics with applications in early vision," International Journal of Computer Vision, vol. 19, no. 1, pp. 57-91, 1996.
25. {25} J. F. Blinn, "Jim Blinn's corner: Compositing, part 1: Theory," IEEE Computer Graphics and Applications, vol. 14, no. 5, pp. 83-87, September 1994.
26. {26} G. Borgefors, "Distance transformations in digital images," Computer Vision, Graphics and Image Processing, vol. 34, no. 3, pp. 227-248, June 1986.
27. {27} Y. Boykov, O. Veksler, and R. Zabih, "Fast approximate energy minimization via graph cuts," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, November 2001.
28. {28} D. C. Brown, "Close-range camera calibration," Photogrammetric Engineering , vol. 37, no. 8, pp. 855-866, 1971.
29. {29} L. G. Brown, "A survey of image registration techniques," Computing Surveys, vol. 24, no. 4, pp. 325-376, December 1992.
30. {30} M. Brown and D. Lowe, "Recognizing panoramas," in Ninth International Conference on Computer Vision (ICCV'03), (Nice, France), pp. 1218-1225, October 2003.
31. {31} M. Brown, R. Szeliski, and S. Winder, "Multi-image matching using multi-scale oriented patches," Technical Report MSR-TR-2004-133, Microsoft Research, December 2004.
32. {32} M. Brown, R. Szeliski, and S. Winder, "Multi-image matching using multi-scale oriented patches," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 510-517, June 2005.
33. {33} L. M. Bugayevskiy and J. P. Snyder, Map Projections: A Reference Manual. CRC Press, 1995.
34. {34} P. J. Burt and E. H. Adelson, "A multiresolution spline with applications to image mosaics," ACM Transactions on Graphics, vol. 2, no. 4, pp. 217-236, October 1983.
35. {35} D. Capel and A. Zisserman, "Automated mosaicing with super-resolution zoom," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'98), (Santa Barbara), pp. 885-891, June 1998.
36. {36} D. Capel and A. Zisserman, "Super-resolution enhancement of text image sequences," in Fifteenth International Conference on Pattern Recognition (ICPR'2000), (Barcelona, Spain), vol. I, pp. 600-605, IEEE Computer Society Press, September 2000.
37. {37} G. Carneiro and A. Jepson, "The distinctiveness, detectability, and robustness of local image features," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. II, pp. 296-301, June 2005.
38. {38} T. J. Cham and R. Cipolla, "A statistical framework for long-range feature matching in uncalibrated image mosaicing," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'98), (Santa Barbara), pp. 442-447, June 1998.
39. {39} G. Champleboux et al., "Accurate calibration of cameras and range imaging sensors, the NPBS method," in IEEE International Conference on Robotics and Automation, (Nice, France), pp. 1552-1558, IEEE Computer Society Press, May 1992.
40. {40} S. Chaudhuri, Super-Resolution Imaging. Springer, 2001.
41. {41} P. Cheeseman, B. Kanefsky, R. Hanson, and J. Stutz, "Super-resolved surface reconstruction from multiple images," Technical Report FIA-93-02, NASA Ames Research Center, Artificial Intelligence Branch, January 1993.
42. {42} C.-Y. Chen and R. Klette, "Image stitching - comparisons and new techniques," in Computer Analysis of Images and Patterns (CAIP'99), (Ljubljana), pp. 615-622, Springer-Verlag, September 1999.
43. {43} S. E. Chen, "QuickTime VR - an image-based approach to virtual environment navigation," Computer Graphics (SIGGRAPH'95), pp. 29-38, August 1995.
44. {44} O. Chum and J. Matas, "Matching with prosac - progressive sample consensus," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 220-226, June 2005.
45. {45} D. Claus and A. Fitzgibbon, "A rational function lens distortion model for general cameras," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 213-219, June 2005.
46. {46} S. Coorg and S. Teller, "Spherical mosaics with quaternions and dense correlation," International Journal of Computer Vision, vol. 37, no. 3, pp. 259-273, June 2000.
47. {47} J. Corso and G. Hager, "Coherent regions for concise and stable image description," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. II, pp. 184-190, June 2005.
48. {48} I. J. Cox, S. Roy, and S. L. Hingorani, "Dynamic histogram warping of image pairs for constant image brightness," in IEEE International Conference on Image Processing (ICIP'95), vol. 2, pp. 366-369, IEEE Computer Society, 1995.
49. {49} P. E. Danielsson, "Euclidean distance mapping," Computer Graphics and Image Processing, vol. 14, no. 3, pp. 227-248, November 1980.
50. {50} J. Davis, "Mosaics of scenes with moving objects," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'98), (Santa Barbara), pp. 354-360, June 1998.
51. {51} E. De Castro and C. Morandi, "Registration of translated and rotated iimages using finite fourier transforms," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-9, no. 5, pp. 700-703, September 1987.
52. {52} P. E. Debevec and J. Malik, "Recovering high dynamic range radiance maps from photographs," in Proceedings of SIGGRAPH 97, Los Angeles, California, pp. 369-378, August 1997. ISBN 0-89791-896-7.
53. {53} F. Dellaert and R. Collins, "Fast image-based tracking by selective pixel integration," in ICCV Workshop on Frame-Rate Vision, pp. 1-22, 1999.
54. {54} F. Durand and J. Dorsey, "Fast bilateral filtering for the display of high-dynamic-range images," ACM Transactions on Graphics (TOG), vol. 21, no. 3, pp. 257-266, 2002.
55. {55} A. Eden, M. Uyttendaele, and R. Szeliski, "Seamless image stitching of scenes with large motions and exposure differences," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2006), (New York, NY), vol. 3, pp. 2498-2505, June 2006.
56. {56} A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," in SIGGRAPH 2001, Computer Graphics Proceedings, (E. Fiume, ed.), pp. 341-346, ACM Press/ACM SIGGRAPH, 2001.
57. {57} M. El Melegy and A. Farag, "Nonmetric lens distortion calibration: Closed-form solutions, robust estimation and model selection," in Ninth International Conference on Computer Vision (ICCV 2003), (Nice, France), pp. 554-559, October 2003.
58. {58} J. H. Elder and R. M. Golderg, "Image editing in the contour domain," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 291-296, March 2001.
59. {59} R. Fattal, D. Lischinski, and M. Werman, "Gradient domain high dynamic range compression," ACM Transactions on Graphics (TOG), vol. 21, no. 3, pp. 249-256, 2002.
60. {60} M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Communications of the ACM, vol. 24, no. 6, pp. 381-395, June 1981.
61. {61} D. Fleet and A. Jepson, "Computation of component image velocity from local phase information," International Journal of Computer Vision, vol. 5, no. 1, pp. 77-104, January 1990.
62. {62} W. Förstner, "A feature-based correspondence algorithm for image matching," International Archves on Photogrammetry and Remote Sensing, vol. 26, no. 3, pp. 150-166, 1986.
63. {63} W. Förstner, "A framework for low level feature extraction," in Third European Conference on Computer Vision (ECCV'94), (Stockholm, Sweden), pp. 383-394, Springer-Verlag, May 1994.
64. {64} W. T. Freeman and E. H. Adelson, "The design and use of steerable filters," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 13, no. 9, pp. 891-906, September 1991.
65. {65} W. T. Freeman, T. R. Jones, and E. C. Pasztor, "Example-based super-resolution," IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 56-65, March-April 2002.
66. {66} C.-S. Fuh and P. Maragos, "Motion displacement estimation using an affine model for image matching," Optical Engineering, vol. 30, no. 7, pp. 881-887, July 1991.
67. {67} S. Geman and D. Geman, "Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-6, no. 6, pp. 721-741, November 1984.
68. {68} M. A. Gennert, "Brightness-based stereo matching," in Second International Conference on Computer Vision (ICCV'88), (Tampa), pp. 139-143, IEEE Computer Society Press, December 1988.
69. {69} G. Golub and C. F. Van Loan, Matrix Computation. Baltimore and London: The John Hopkins University Press, Third Edition, 1996.
70. {70} A. Goshtasby, "Correction of image deformation from lens distortion using bezier patches," Computer Vision, Graphics, and Image Processing, vol. 47, no. 4, pp. 385-394, November 1989.
71. {71} A. Goshtasby, 2-D and 3-D Image Registration. New York: Wiley, 2005.
72. {72} V. M. Govindu, "Revisiting the brightness constraint: Probabilistic formulation and algorithms," in Computer Vision - ECCV 2006, (A. Leonardis, H. Bischof, and A. Pinz, eds.), vol. 3953, pp. 177-188, of Lecture Notes in Computer Science, Springer, 2006.
73. {73} N. Greene, "Environment mapping and other applications of world projections," IEEE Computer Graphics and Applications, vol. 6, no. 11, pp. 21-29, November 1986.
74. {74} N. Greene and P. Heckbert, "Creating raster Omnimax images from multiple perspective views using the elliptical weighted average filter," IEEE Computer Graphics and Applications, vol. 6, no. 6, pp. 21-27, June 1986.
75. {75} K. D. Gremban, C. E. Thorpe, and T. Kanade, "Geometric camera calibration using systems of linear equations," in IEEE International Conference on Robotics and Automation, (Philadelphia), pp. 562-567, IEEE Computer Society Press, April 1988.
76. {76} M. D. Grossberg and S. K. Nayar, "A general imaging model and a method for finding its parameters," in Eighth International Conference on Computer Vision (ICCV 2001), (Vancouver, Canada), vol. II, pp. 108-115, July 2001.
77. {77} G. D. Hager and P. N. Belhumeur, "Efficient region tracking with parametric models of geometry and illumination," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 10, pp. 1025-1039, October 1998.
78. {78} F. R. Hampel et al., Robust Statistics: The Approach Based on Influence Functions . New York: Wiley, 1986.
79. {79} M. J. Hannah, Computer Matching of Areas in Stereo Images. PhD thesis, PhD thesis, Stanford University, 1974.
80. {80} M. J. Hannah, "Test results from SRI's stereo system," in Image Understanding Workshop, (Cambridge, Massachusetts), pp. 740-744, Morgan Kaufmann Publishers, April 1988.
81. {81} M. Hansen, P. Anandan, K. Dana, G. van der Wal, and P. Burt, "Real-time scene stabilization and mosaic construction," in IEEE Workshop on Applications of Computer Vision (WACV'94), (Sarasota), pp. 54-62, December 1994. IEEE Computer Society.
82. {82} C. Harris and M. J. Stephens, "A combined corner and edge detector," in Alvey Vision Conference, pp. 147-152, 1988.
83. {83} R. I. Hartley, "Self-calibration from multiple views of a rotating camera," in Third European Conference on Computer Vision (ECCV'94), (Stockholm, Sweden), vol. 1, pp. 471-478, Springer-Verlag, May 1994.
84. {84} R. I. Hartley and S. B. Kang, "Parameter-free radial distortion correction with centre of distortion estimation," in Tenth International Conference on Computer Vision (ICCV 2005), (Beijing, China), vol. 2, pp. 1834-1841, October 2005.
85. {85} R. I. Hartley and A. Zisserman, Multiple View Geometry. Cambridge, UK: Cambridge University Press, September 2000.
86. {86} R. I. Hartley and A. Zisserman, Multiple View Geometry. Cambridge, UK: Cambridge University Press, March 2004.
87. {87} C. Herley, "Automatic occlusion removal from minimum number of images," in International Conference on Image Processing (ICIP 2005), (Genova), vol. II, pp. 1046-1049-16, September 2005.
88. {88} B. K. P. Horn, "Determining lightness from an image," Computer Graphics and Image Processing, vol. 3, no. 1, pp. 277-299, December 1974.
89. {89} B. K. P. Horn, Robot Vision. Cambridge, Massachusetts: MIT Press, 1986.
90. {90} B. K. P. Horn and M. J. Brooks, Shape from Shading. Cambridge, Massachusetts: MIT Press, 1989.
91. {91} B. K. P. Horn and B. G. Schunck, "Determining optical flow," Artificial Intelligence , vol. 17, pp. 185-203, 1981.
92. {92} P. J. Huber, Robust Statistics. New York: John Wiley & Sons, 1981.
93. {93} M. Irani and P. Anandan, "Video indexing based on mosaic representations," Proceedings of the IEEE, vol. 86, no. 5, pp. 905-921, May 1998.
94. {94} M. Irani and P. Anandan, "About direct methods," in International Workshop on Vision Algorithms, (Kerkyra, Greece), pp. 267-277, Springer, September 1999.
95. {95} M. Irani, S. Hsu, and P. Anandan, "Video compression using mosaic representations," Signal Processing: Image Communication, vol. 7, pp. 529-552, 1995.
96. {96} M. Irani and S. Peleg, "Improving resolution by image registration," Graphical Models and Image Processing, vol. 53, no. 3, pp. 231-239, May 1991.
97. {97} J. Jia and C.-K. Tang, "Image registration with global and local luminance alignment," in Ninth International Conference on Computer Vision (ICCV 2003), (Nice, France), pp. 156-163, October 2003.
98. {98} F. Jurie and M. Dhome, "Hyperplane approximation for template matching," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 996-1000, July 2002.
99. {99} T. Kadir and M. Brady, "Saliency, scale and image description," International Journal of Computer Vision, vol. 45, no. 2, pp. 83-105, November 2001.
100. {100} T. Kadir, A. Zisserman, and M. Brady, "An affine invariant salient region detector," in Eighth European Conference on Computer Vision (ECCV 2004), (Prague), vol. I, pp. 228-241, Springer-Verlag, May 2004.
101. {101} S. B. Kang, "Radial distortion snakes," IEICE Transactions on Information and Systems, vol. E84-D, no. 12, pp. 1603-1611, December 2001.
102. {102} S. B. Kang, R. Szeliski, and M. Uyttendaele, "Seamless stitching using multiperspective plane sweep," Technical Report MSR-TR-2004-48, Microsoft Research, June 2004.
103. {103} S. B. Kang and R. Weiss, "Characterization of errors in compositing panoramic images," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'97), (San Juan, Puerto Rico), pp. 103-109, June 1997.
104. {104} S. B. Kang et al., "High dynamic range video," ACM Transactions on Graphics , vol. 22, no. 3, pp. 319-325, July 2003.
105. {105} Y. Ke and R. Sukthankar, "PCA-SIFT: A more distinctive representation for local image descriptors," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2004), (Washington, DC), vol. II, pp. 506-513, June 2004.
106. {106} C. Kenney, M. Zuliani, and B. Manjunath, "An axiomatic approach to corner detection," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 191-197, June 2005.
107. {107} D. Keren, S. Peleg, and R. Brada, "Image sequence enhancement using subpixel displacements," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'88), (Ann Arbor, Michigan), pp. 742-746, IEEE Computer Society Press, June 1988.
108. {108} J. Kim, V. Kolmogorov, and R. Zabih, "Visual correspondence using energy minimization and mutual information," in Ninth International Conference on Computer Vision (ICCV 2003), (Nice, France), pp. 1033-1040, October 2003.
109. {109} C. D. Kuglin and D. C. Hines, "The phase correlation image alignment method," in IEEE 1975 Conference on Cybernetics and Society, (New York), pp. 163-165, September 1975.
110. {110} R. Kumar, P. Anandan, and K. Hanna, "Direct recovery of shape from multiple views: A parallax based approach," in Twelfth International Conference on Pattern Recognition (ICPR'94), (Jerusalem, Israel), vol. A, pp. 685-688, IEEE Computer Society Press, October 1994.
111. {111} R. Kumar, P. Anandan, M. Irani, J. Bergen, and K. Hanna, "Representation of scenes from collections of images," in IEEE Workshop on Representations of Visual Scenes, (Cambridge, Massachusetts), pp. 10-17, June 1995.
112. {112} V. Kwatra et al., "Graphcut textures: Image and video synthesis using graph cuts," ACM Transactions on Graphics, vol. 22, no. 3, pp. 277-286, July 2003.
113. {113} D. Le Gall, "MPEG: A video compression standard for multimedia applications," Communications of the ACM, vol. 34, no. 4, pp. 44-58, April 1991.
114. {114} M.-C. Lee et al., "A layered video object coding system using sprite and affine motion model," IEEE Transactions on Circuits and Systems for Video Technology , vol. 7, no. 1, pp. 130-145, February 1997.
115. {115} A. Levin, A. Zomet, and Y. Weiss, "Separating reflections from a single image using local features," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2004), (Washington, DC), vol. I, pp. 306-313, June 2004.
116. {116} A. Levin, A. Zomet, S. Peleg, and Y. Weiss, "Seamless image stitching in the gradient domain," in Eighth European Conference on Computer Vision (ECCV 2004), (Prague), vol. IV, pp. 377-389, Springer-Verlag, May 2004.
117. {117} Y. Li et al., "Lazy snapping," ACM Transactions on Graphics, vol. 23, no. 3, pp. 303-308, August 2004.
118. {118} Y. Li et al., "Stereo reconstruction from multiperspective panoramas," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 44-62, January 2004.
119. {119} T. Lindeberg, "Scale-space for discrete signals," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 3, pp. 234-254, March 1990.
120. {120} D. Lischinski, Z. Farbman, M. Uytendaelle, and R. Szeliski, "Interactive local adjustment of tonal values," ACM Transactions on Graphics, vol. 25, no. 3, pp. 646-653, August 2006.
121. {121} C. Loop and Z. Zhang, "Computing rectifying homographies for stereo vision," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99), (Fort Collins), vol. I, pp. 125-131, June 1999.
122. {122} D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, November 2004.
123. {123} B. D. Lucas and T. Kanade, "An iterative image registration technique with an application in stereo vision," in Seventh International Joint Conference on Artificial Intelligence (IJCAI-81), (Vancouver), pp. 674-679, 1981.
124. {124} S. Mann and R. W. Picard, "Virtual bellows: Constructing high-quality images from video," in First IEEE International Conference on Image Processing (ICIP-94), (Austin), vol. I, pp. 363-367, November 1994.
125. {125} S. Mann and R. W. Picard, "On being 'undigital' with digital cameras: Extending dynamic range by combining differently exposed pictures," in IS&T's 48th Annual Conference, (Washington, D. C.), pp. 422-428, Society for Imaging Science and Technology, May 1995.
126. {126} M. Massey and W. Bender, "Salient stills: Process and practice," IBM Systems Journal, vol. 35, nos. 3&4, pp. 557-573, 1996.
127. {127} J. Matas et al., "Robust wide baseline stereo from maximally stable extremal regions," Image and Vision Computing, vol. 22, no. 10, pp. 761-767, September 2004.
128. {128} L. H. Matthies, R. Szeliski, and T. Kanade, "Kalman filter-based algorithms for estimating depth from image sequences," International Journal of Computer Vision, vol. 3, pp. 209-236, 1989.
129. {129} P. F. McLauchlan and A. Jaenicke, "Image mosaicing using sequential bundle adjustment," Image and Vision Computing, vol. 20, nos. 9-10, pp. 751-759, August 2002.
130. {130} L. McMillan and G. Bishop, "Plenoptic modeling: An image-based rendering system," Computer Graphics (SIGGRAPH'95), pp. 39-46, August 1995.
131. {131} J. Meehan, Panoramic Photography. Watson-Guptill, 1990.
132. {132} K. Mikolajczyk and C. Schmid, "Scale & affine invariant interest point detectors," International Journal of Computer Vision, vol. 60, no. 1, pp. 63-86, October 2004.
133. {133} K. Mikolajczyk and C. Schmid, "A performance evaluation of local descriptors," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, October 2005.
134. {134} K. Mikolajczyk et al., "A comparison of affine region detectors," International Journal of Computer Vision, vol. 65, nos. 1-2, pp. 43-72, November 2005.
135. {135} D. L. Milgram, "Computer methods for creating photomosaics," IEEE Transactions on Computers, vol. C-24, no. 11, pp. 1113-1119, November 1975.
136. {136} D. L. Milgram, "Adaptive techniques for photomosaicking," IEEE Transactions on Computers, vol. C-26, no. 11, pp. 1175-1180, November 1977.
137. {137} T. Mitsunaga and S. K. Nayar, "Radiometric self calibration," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'99), (Fort Collins), vol. 1, pp. 374-380, June 1999.
138. {138} H. Moravec, "The stanford cart and the cmu rover," in Proceedings of the IEEE, vol. 71, no. 7, pp. 872-884, July 1983.
139. {139} M. Mühlich and R. Mester, "The role of total least squares in motion analysis," in Fifth European Conference on Computer Vision (ECCV'98), (Freiburg, Germany), vol. II, pp. 305-321, Springer-Verlag, June 1998.
140. {140} S. K. Nayar and T. Mitsunaga, "High dynamic range imaging: Spatially varying pixel exposures," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2000), (Hilton Head, Island), vol. 1, pp. 472-479, June 2000.
141. {141} S. Nene and S. K. Nayar, "A simple algorithm for nearest neighbor search in high dimensions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 19, no. 9, pp. 989-1003, September 1997.
142. {142} G. M. Nielson, "Scattered data modeling," IEEE Computer Graphics and Applications, vol. 13, no. 1, pp. 60-70, January 1993.
143. {143} D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2006), (New York City, NY), vol. 3, pp. 2161-2168, June 2006.
144. {144} M. Okutomi and T. Kanade, "A multiple baseline stereo," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 4, pp. 353-363, April 1993.
145. {145} OpenGL-ARB, OpenGL Reference Manual: The Official Reference Document to OpenGL, Version 1.1., Second Edition, Addison-Wesley, Reading, MA, 1997.
146. {146} A. V. Oppenheim, R. W. Schafer, and J. R. Buck, Discrete-Time Signal Processing , Second Edition, Pearson Education, 1999.
147. {147} R. Peleg, M. Ben-Ezra, and Y. Pritch, "Omnistereo: Panoramic stereo imaging," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 279-290, March 2001.
148. {148} S. Peleg, "Elimination of seams from photomosaics," Computer Vision, Graphics, and Image Processing, vol. 16, pp. 1206-1210, 1981.
149. {149} S. Peleg and A. Rav Acha, "Lucas-Kanade without iterative warping," in International Conference on Image Processing (ICIP-2006), (Atlanta), pp. 1097-1100, October 2006.
150. {150} S. Peleg et al., "Mosaicing on adaptive manifolds," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1144-1154, October 2000.
151. {151} P. Pérez, M. Gangnet, and A. Blake, "Poisson image editing," ACM Transactions on Graphics, vol. 22, no. 3, pp. 313-318, July 2003.
152. {152} B. Platel, E. Balmachnova, L. M. J. Florack, and B. M. ter Haar Romeny, "Top-points as interest points for image matching," in Computer Vision - ECCV 2006, (A. Leonardis, H. Bischof, and A. Pinz, eds.), vol. 3951, pp. 418-429, of Lecture Notes in Computer Science, Springer, 2006.
153. {153} T. Porter and T. Duff, "Compositing digital images," Computer Graphics (SIGGRAPH'84), vol. 18, no. 3, pp. 253-259, July 1984.
154. {154} L. H. Quam, "Hierarchical warp stereo," in Image Understanding Workshop, (New Orleans), pp. 149-155, Science Applications International Corporation, December 1984.
155. {155} P. Rademacher and G. Bishop, "Multiple-center-of-projection images," in Computer Graphics Proceedings, Annual Conference Series, pp. 199-206, Proc. SIGGRAPH'98 (Orlando), ACM SIGGRAPH, July 1998.
156. {156} A. Rav Acha, Y. Pritch, D. Lischinski, and S. Peleg, "Dynamosaics: Video mosaics with non-chronological time," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 58-65, June 2005.
157. {157} J. Rehg and A. Witkin, "Visual tracking with deformation models," in IEEE International Conference on Robotics and Automation, (Sacramento), pp. 844-850, IEEE Computer Society Press, April 1991.
158. {158} E. Reinhard, G. Ward, S. Pattanaik, and P. Debevec, High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting. Morgan Kaufmann, 2005.
159. {159} E. Reinhard et al., "Photographic tone reproduction for digital images," ACM Transactions on Graphics (TOG), vol. 21, no. 3, pp. 267-276, 2002.
160. {160} E. Rosten and T. Drummond, "Machine learning for high-speed corner detection," in Computer Vision - ECCV 2006, (A. Leonardis, H. Bischof, and A. Pinz, eds.), vol. 3951, pp. 430-443, of Lecture Notes in Computer Science, Springer, 2006.
161. {161} C. Rother, V. Kolmogorov, and A. Blake, ""GrabCut" - interactive foreground extraction using iterated graph cuts," ACM Transactions on Graphics, vol. 23, no. 3, pp. 309-314, August 2004.
162. {162} P. J. Rousseeuw, "Least median of squares regresssion," Journal of the American Statistical Association, vol. 79, pp. 871-880, 1984.
163. {163} P. J. Rousseeuw and A. M. Leroy, Robust Regression and Outlier Detection. New York: Wiley, 1987.
164. {164} H. Samet, The Design and Analysis of Spatial Data Structures. Reading, Massachusetts: Addison-Wesley, 1989.
165. {165} H. S. Sawhney, "3D geometry from planar parallax," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), (Seattle), pp. 929-934, IEEE Computer Society, June 1994.
166. {166} H. S. Sawhney and S. Ayer, "Compact representation of videos through dominant multiple motion estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 18, no. 8, pp. 814-830, August 1996.
167. {167} H. S. Sawhney and R. Kumar, "True multi-image alignment and its application to mosaicing and lens distortion correction," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 3, pp. 235-243, March 1999.
168. {168} H. S. Sawhney et al., "Videobrush: Experiences with consumer video mosaicing," in IEEE Workshop on Applications of Computer Vision (WACV'98), (Princeton), pp. 56-62, IEEE Computer Society, October 1998.
169. {169} F. Schaffalitzky and A. Zisserman, "Multi-view matching for unordered image sets, or "How do I organize my holiday snaps?", in Seventh European Conference on Computer Vision (ECCV 2002), (Copenhagen), vol. I, pp. 414-431, Springer-Verlag, May 2002.
170. {170} D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense two-frame stereo correspondence algorithms," International Journal of Computer Vision, vol. 47, no. 1, pp. 7-42, May 2002.
171. {171} C. Schmid, R. Mohr, and C. Bauckhage, "Evaluation of interest point detectors," International Journal of Computer Vision, vol. 37, no. 2, pp. 151-172, June 2000.
172. {172} G. Shakhnarovich, P. Viola, and T. Darrell, "Fast pose estimation with parameter sensitive hashing," in Ninth International Conference on Computer Vision (ICCV 2003), (Nice, France), vol. 2, pp. 750-757, October 2003.
173. {173} J. Shi and C. Tomasi, "Good features to track," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), (Seattle), pp. 593-600, IEEE Computer Society, June 1994.
174. {174} K. Shoemake, "Animating rotation with quaternion curves," Computer Graphics (SIGGRAPH'85), vol. 19, no. 3, pp. 245-254, July 1985.
175. {175} H.-Y. Shum and L.-W. He, "Rendering with concentric mosaics," in SIGGRAPH' 99, (Los Angeles), pp. 299-306, ACM SIGGRAPH, August 1999.
176. {176} H.-Y. Shum and R. Szeliski, "Panoramic image mosaicing," Technical Report MSR-TR-97-23, Microsoft Research, September 1997.
177. {177} H.-Y. Shum and R. Szeliski, "Stereo reconstruction from multiperspective panoramas," in Seventh International Conference on Computer Vision (ICCV'99), (Kerkyra, Greece), pp. 14-21, September 1999.
178. {178} H.-Y. Shum and R. Szeliski, "Construction of panoramic mosaics with global and local alignment," International Journal of Computer Vision, vol. 36, no. 2, pp. 101-130, February 2000, Erratum published July 2002, vol. 48, no. 2, pp. 151-152, 2000.
179. {179} H.-Y. Shum et al., "Omnivergenet stereo," in Seventh International Conference on Computer Vision (ICCV'99), (Greece), pp. 22-29, September 1999.
180. {180} E. P. Simoncelli, E. H. Adelson, and D. J. Heeger, "Probability distributions of optic flow," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'91), (Maui, Hawaii), pp. 310-315, IEEE Computer Society Press, June 1991.
181. {181} C. C. Slama, ed., Manual of Photogrammetry. Fourth Edition, Falls Church, Virginia, 1980. American Society of Photogrammetry.
182. {182} P. Soille, "Morphological image compositing," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 5, pp. 673-683, May 2006.
183. {183} D. Steedly et al., "Efficiently registering video into panoramic mosaics," in Tenth International Conference on Computer Vision (ICCV 2005), (Beijing, China), pp. 1300-1307, October 2005.
184. {184} R. Steele and C. Jaynes, "Feature uncertainty arising from covariant image noise," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 1063-1070, June 2005.
185. {185} R. M. Steele and C. Jaynes, "Overconstrained linear estimation of radial distortion and multi-view geometry," in Computer Vision - ECCV 2006, (A. Leonardis, H. Bischof, and A. Pinz, eds.), vol. 3951, pp. 253-264, of Lecture Notes in Computer Science, Springer, 2006.
186. {186} G. Stein, "Accurate internal camera calibration using rotation, with analysis of sources of error," in Fifth International Conference on Computer Vision (ICCV'95), (Cambridge, Massachusetts), pp. 230-236, June 1995.
187. {187} G. Stein, "Lens distortion calibration using point correspondences," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'97), (San Juan, Puerto Rico), pp. 602-608, June 1997.
188. {188} C. V. Stewart, "Robust parameter estimation in computer vision," SIAM Reviews, vol. 41, no. 3, pp. 513-537, September 1999.
189. {189} P. Sturm, "Multi-view geometry for general camera models," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 206-212, June 2005.
190. {190} J. Sun, N. Zheng, and H. Shum, "Stereo matching using belief propagation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 787-800, July 2003.
191. {191} R. Szeliski, Bayesian Modeling of Uncertainty in Low-Level Vision. Boston: Kluwer Academic Publishers, 1989.
192. {192} R. Szeliski, "Fast surface interpolation using hierarchical basis functions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 6, pp. 513-528, June 1990.
193. {193} R. Szeliski, "Image mosaicing for tele-reality applications," in IEEE Workshop on Applications of Computer Vision (WACV'94), (Sarasota), pp. 44-53, IEEE Computer Society, December 1994.
194. {194} R. Szeliski, "Video mosaics for virtual environments," IEEE Computer Graphics and Applications, vol. 16, no. 2, pp. 22-30, March 1996.
195. {195} R. Szeliski, "Locally adapted hierarchical basis preconditioning," ACM Transactions on Graphics, vol. 25, no. 3, pp. 1135-1143, August 2006.
196. {196} R. Szeliski and J. Coughlan, "Hierarchical spline-based image registration," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), (Seattle), pp. 194-201, IEEE Computer Society, June 1994.
197. {197} R. Szeliski and S. B. Kang, "Recovering 3D shape and motion from image streams using nonlinear least squares," Journal of Visual Communication and Image Representation, vol. 5, no. 1, pp. 10-28, March 1994.
198. {198} R. Szeliski and S. B. Kang, "Direct methods for visual scene reconstruction," in IEEE Workshop on Representations of Visual Scenes, (Cambridge, Massachusetts), pp. 26-33, June 1995.
199. {199} R. Szeliski and H.-Y. Shum, "Creating full view panoramic image mosaics and texture-mapped models," in Computer Graphics (SIGGRAPH'97 Proceedings) , pp. 251-258, August 1997.
200. {200} M. F. Tappen and W. T. Freeman, "Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters," in Ninth International Conference on Computer Vision (ICCV 2003), (Nice, France), pp. 900-907, October 2003.
201. {201} J.-P. Tardif, P. Sturm, and S. Roy, "Self-calibration of a general radially symmetric distortion model," in Computer Vision - ECCV 2006, (A. Leonardis, H. Bischof, and A. Pinz, eds.), vol. 3954, pp. 186-199, of Lecture Notes in Computer Science, Springer, 2006.
202. {202} J.-P. Tardif et al., "Self-calibration of a general radially symmetric distortion model," in Seventh European Conference on Computer Vision (ECCV 2002), (Graz), vol. 4, pp. 186-199, Springer-Verlag, May 2006.
203. {203} L. Teodosio and W. Bender, "Salient video stills: Content and context preserved," in ACM Multimedia 93, (Anaheim, California), pp. 39-46, August 1993.
204. {204} S. Thirthala and M. Pollefeys, "The radial trifocal tensor: A tool for calibrating the radial distortion of wide-angle cameras," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 321-328, June 2005.
205. {205} Q. Tian and M. N. Huhns, "Algorithms for subpixel registration," Computer Vision, Graphics, and Image Processing, vol. 35, pp. 220-233, 1986.
206. {206} B. Triggs, "Detecting keypoints with stable position, orientation, and scale under illumination changes," in Eighth European Conference on Computer Vision (ECCV 2004), (Prague), vol. IV, pp. 100-113, Springer-Verlag, May 2004.
207. {207} B. Triggs et al., "International Workshop on Vision Algorithms," in Bundle adjustment - a modern synthesis, (Kerkyra, Greece), pp. 298-372, Springer, September 1999.
208. {208} J. Tumblin, A. Agrawal, and R. Raskar, "Why i want a gradient camera," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), (San Diego, CA), vol. I, pp. 103-110, June 2005.
209. {209} T. Tuytelaars and L. Van Gool, "Matching widely separated views based on affine invariant regions," International Journal of Computer Vision, vol. 59, no. 1, pp. 61-85, August 2004.
210. {210} M. Uyttendaele, A. Eden, and R. Szeliski, "Eliminating ghosting and exposure artifacts in image mosaics," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2001), (Kauai, Hawaii), vol. II, pp. 509-516, December 2001.
211. {211} M. Uyttendaele et al., "Image-based interactive exploration of real-world environments," IEEE Computer Graphics and Applications, vol. 24, no. 3, May/June 2004.
212. {212} S. v. Huffel and J. Vandewalle, "The Total Least Squares Problem: Computational Aspects and Analysis," in Society for Industrial and Applied Mathematics , (Philadephia), 1991.
213. {213} J. van de Weijer and C. Schmid, "Coloring local feature extraction," in Computer Vision - ECCV 2006, (A. Leonardis, H. Bischof, and A. Pinz, eds.), vol. 3952, pp. 334-348, of Lecture Notes in Computer Science, Springer, 2006.
214. {214} P. Viola and W. Wells III, "Alignment by maximization of mutual information," in Fifth International Conference on Computer Vision (ICCV'95), (Cambridge, Massachusetts), pp. 16-23, June 1995.
215. {215} L. Wang, S. B. Kang, R. Szeliski, and H.-Y. Shum, "Optimal texture map reconstruction from multiple views," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2001), (Kauai, Hawaii), vol. I, pp. 347-354, December 2001.
216. {216} A. Watt, 3D Computer Graphics. Addison-Wesley, Third Edition, 1995.
217. {217} J. Weber and J. Malik, "Robust computation of optical flow in a multi-scale differential framework," International Journal of Computer Vision, vol. 14, no. 1, pp. 67-81, January 1995.
218. {218} Y. Weiss, "Deriving intrinsic images from image sequences," in Eighth International Conference on Computer Vision (ICCV 2001), (Vancouver, Canada), vol. II, pp. 7-14, July 2001.
219. {219} L. Williams, "Pyramidal parametrics," Computer Graphics, vol. 17, no. 3, pp. 1-11, July 1983.
220. {220} D. N. Wood et al., "Multiperspective panoramas for cell animation," in Computer Graphics Proceedings, Annual Conference Series, (Los Angeles), pp. 243-250, Proc. SIGGRAPH'97, ACM SIGGRAPH, August 1997.
221. {221} R. J. Woodham, "Analysing images of curved surfaces," Artificial Intelligence, vol. 17, pp. 117-140, 1981.
222. {222} Y. Xiong and K. Turkowski, "Creating image-based VR using a self-calibrating fisheye lens," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'97), (San Juan, Puerto Rico), pp. 237-243, June 1997.
223. {223} Y. Xiong and K. Turkowski, "Registration, calibration and blending in creating high quality panoramas," in IEEE Workshop on Applications of Computer Vision (WACV'98), (Princeton), pp. 69-74, IEEE Computer Society, October 1998.
224. {224} D. Yang et al., "A 640×512 CMOS image sensor with ultra-wide dynamic range floating-point pixel level ADC," IEEE Journal of Solid State Circuits, vol. 34, no. 12, pp. 1821-1834, December 1999.
225. {225} R. Zabih and J. Woodfill, "Non-parametric local transforms for computing visual correspondence," in Third European Conference on Computer Vision (ECCV'94), (Stockholm, Sweden), vol. II, pp. 151-158, Springer-Verlag, May 1994.
226. {226} B. Zitov'aa and J. Flusser, "Image registration methods: A survey," Image and Vision Computing, vol. 21, pp. 997-1000, October 2003.
227. {227} I. Zoghlami, O. Faugeras, and R. Deriche, "Using geometric corners to build a 2D mosaic from a set of images," in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'97), (San Juan, Puerto Rico), pp. 420-425, June 1997.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Reconstruction

  • Computer graphics

    • Image manipulation

  • Machine learning