skip to main content
research-article

Image registration for foveated panoramic sensing

Authors Info & Claims
Published:22 May 2012Publication History
Skip Abstract Section

Abstract

This article addresses the problem of registering high-resolution, small field-of-view images with low-resolution panoramic images provided by a panoramic catadioptric video sensor. Such systems may find application in surveillance and telepresence systems that require a large field of view and high resolution at selected locations. Although image registration has been studied in more conventional applications, the problem of registering panoramic and conventional video has not previously been addressed, and this problem presents unique challenges due to (i) the extreme differences in resolution between the sensors (more than a 16:1 linear resolution ratio in our application), and (ii) the resolution inhomogeneity of panoramic images. The main contributions of this article are as follows. First, we introduce our foveated panoramic sensor design. Second, we show how a coarse registration can be computed from the raw images using parametric template matching techniques. Third, we propose two refinement methods allowing automatic and near real-time registration between the two image streams. The first registration method is based on matching extracted interest points using a closed form method. The second registration method is featureless and based on minimizing the intensity discrepancy allowing the direct recovery of both the geometric and the photometric transforms. Fourth, a comparison between the two registration methods is carried out, which shows that the featureless method is superior in accuracy. Registration examples using the developed methods are presented.

References

  1. Amintabar, A. and Boufama, B. 2008. Homography-based plane identification and matching. In Proceedings of the IEEE International Conference on Image Processing.Google ScholarGoogle Scholar
  2. Bay, H., Ess, A., Tuytelaars, T., and Gool, L. V. 2008. SURF: Speeded Up Robust Features. Comput Vision Image Understand. 110, 3, 346--359. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Boult, T. E., Gao, X., Micheals, R., and Eckmann, M. 2004. Omni-directional visual surveillance. Image Vision Comput. 22, 7, 515--534.Google ScholarGoogle ScholarCross RefCross Ref
  4. Brown, M. and Lowe, D. G. 2007. Automatic panoramic image stitching using invariant features. Int. J. Comput. Vision 74, 1, 59--73. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Chen, J., Chen, C., and Chen, Y. 2003. Fast algorithm for robust template matching with M-estimators. IEEE Trans. Signal Process 51, 1, 230--243. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Conroy, T. L. and Moore, J. B. September 1999. Resolution invariant surfaces for panoramic vision systems. InProceedings of the IEEE Conference on Computer Vision.Google ScholarGoogle Scholar
  7. Danilidis, K. and Geyer, C. 2000. Omnidirectional vision: Theory and algorithms. In Proceedings of the IEEE International Conference on Patter Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Dornaika, F. and Elder, J. 2002. Image registration for foveated omnidirectional sensing. In Proceedings of the European Conference on Computer Vision. Lecture Notes in Computer Science, vol. 2353. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Dufourneau, Y., Schmid, C., and Horaud, R. 2000. Matching images with different resolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle Scholar
  10. Fischler, M. A. and Bolles, R. C. 1981. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. ACM 24, 6, 381--395. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Fletcher, R. 1990. Practical Methods of Optimization. Wiley, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Harris, C. and Stephens, M. 1988. A combined corner and edge detector. In Proceedings of the Alvey Vision Conference.Google ScholarGoogle Scholar
  13. He Q. and Chu, C. H. 2006. Planar surface detection in image pairs using homographic constraints. In Proceedings of the 2nd International Symposium on Advances in Visual Computing. Lecture Notes in Computer Science, Vol. 4291. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Kanatani, K. and Ohta, N. 1999. Accuracy bounds and optimal computation of homography for image mosaicing applications. In Proceedings of the IEEE Conference on Computer Vision.Google ScholarGoogle Scholar
  15. Kim, D. H., Yoon, Y. I., and Choi, J. S. 2003. An efficient method to build panoramic image mosaics. Patt. Recog. Lett 24, 14, 2421--2429. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Lin, S. S. and Bajcsy, R. 2006. Single-view-point omnidirectional catadioptric cone mirror imager. IEEE Trans. Patt. Anal. Machine Intell. 28, 5, 840--845. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Lowe, D. 2004. Distinctive image features from scale invariant keypoints. Int. J. Comput. Vision 60, 2, 91--100. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Ohta, J. 2007. Smart CMOS Image Sensors and Applications. CRC Press, Taylor & Francis Group.Google ScholarGoogle Scholar
  19. Peng, G., Xie, S., and Cheng, L. 2005. An HVSM for improving the homing ability of visual robots. Int. J. Intell. Syst. Techn. Appl. 1, 2, 18--31. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Pilu, M. 1997. A direct method for stereo correspondence based on singular value decomposition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Prime, S. L., Tsotsos, L., Keith, G. P., and Crawford, J. D. 2007. Visual memory capacity in transsaccadic integration. Exp. Brain Resear. 180, 4, 609--628.Google ScholarGoogle ScholarCross RefCross Ref
  22. Scaramuzza, D. and Siegwart, R. 2007. A practical toolbox for calibrating omnidirectional cameras. In Vision Systems: Applications, Intech.Google ScholarGoogle Scholar
  23. Schwartz, W. B., Kembhavi, A., Harwood, D., and Davis, L. S. 2009. Human detection using partial least squares analysis. In Proceedings of the IEEE International Conference on Computer Vision.Google ScholarGoogle Scholar
  24. Spacek, L. and Burbridge, C. 2007. Instantaneous robot self-localisation and motion estimation with omnidirectional vision. Rob. Auton. Syst. 55, 667--674. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Swaminathan, R., Grossberg, M. D., and Nayar, S. K. 2006. Non-single viewpoint catadioptric cameras: Geometry and analysis. Int. J. Comput. Vision 66, 3, 211--229. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Tanaka, K., Sano, M., Ohara, S., and Okudaira, M. 2000. A parametric template method and its application to robust matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarGoogle Scholar
  27. Torre, F., Vallespi, C., Rybski, P. E., Veloso, M., and Kanade, T. 2005. Learning to track multiple people in omnidirectional video. In Proceedings of the IEEE International Conference on Robotics and Automation.Google ScholarGoogle Scholar
  28. Traver, V. J. and Bernardino, A. 2010. A review of log-polar imaging for visual perception in robotics. Rob. Auton. Syst. 58, 4, 378--398. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Wu, Y., Kanade, T., Li, C., and Cohn, J. 2000. Image registration using wavelet-based motion model. Int. J. Comput. Vision 38, 2, 129--152. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Zitová, B. and Flusser, J. 2003. Image registration methods: a survey. Image Vision Comput. 21, 977--1000.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Image registration for foveated panoramic sensing

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 8, Issue 2
          May 2012
          144 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/2168996
          Issue’s Table of Contents

          Copyright © 2012 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 22 May 2012
          • Accepted: 1 January 2011
          • Revised: 1 April 2010
          • Received: 1 January 2010
          Published in tomm Volume 8, Issue 2

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!