Omnistereo: Panoramic Stereo Imaging

Abstract

An Omnistereo panorama consists of a pair of panoramic images, where one panorama is for the left eye and another panorama is for the right eye. The panoramic stereo pair provides a stereo sensation up to a full 360 degrees. Omnistereo panoramas cannot be photographed by two omnidirectional cameras from two viewpoints, but can be constructed by mosaicing together images from a rotating stereo pair. A more convenient approach to generate omnistereo panoramas is by mosaicing images from a single rotating camera. This approach also enables the control of stereo disparity, giving larger baselines for faraway scenes, and a smaller baseline for closer scenes. Capturing panoramic omnistereo images with a rotating camera makes it impossible to capture dynamic scenes at video rates and limits omnistereo imaging to stationary scenes. We, therefore, present two possibilities for capturing omnistereo panoramas using optics without any moving parts. A special mirror is introduced such that viewing the scene through this mirror creates the same rays as those used with the rotating cameras. A lens for omnistereo panorama is also introduced. The designs of the mirror and of the lens are based on curves whose caustic is a circle. Omnistereo panoramas can also be rendered by computer graphics methods to represent virtual environments.

References

1. S. Chen, “Quicktime VR—An Image-Based Approach to Virtual Environment Navigation,” Proc. SIGGRAPH '95, pp. 29-38, Aug. 1995.
2. J. Gluckman S. Nayar and K. Thoresz, “Real-Time Omnidirectional and Panoramic Stereo,” Proc. DARPA Image Understanding Workshop '98, pp. 299-303, Nov. 1998.
3. S. Gortler R. Grzeszczuk R. Szeliski and M. Cohen, “The Lumigraph,” Proc. SIGGRAPH '96, pp. 43-54, Aug. 1996.
4. H.-C. Huang and Y.-P. Hung, “Panoramic Stereo Imaging System with Automatic Disparity Warping and Seaming,” Graphical Models and Image Processing, vol. 60, no. 3, pp. 196-208, May 1998.
5. H. Ishiguro M. Yamamoto and S. Tsuji, “Omni-Directional Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14,no. 2, pp. 257-262, Feb. 1992.
6. S. Kang and R. Szelisky, “3D Scene Data Recovery Using Omnidirectional Multibaseline Stereo,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 364-370, June 1996.
7. T. Kawanishi K. Yamazawa H. Iwasa H. Takemura and N. Yokoya, “Generation of High-Resolution Stereo Panoramic Images by Omnidirectional Sensor Using Hexagonal Pyramidal Mirrors,” Proc. 14th Int'l Conf. Pattern Recognition, pp. 485-489, Aug. 1998.
8. S. Mann and R. Picard, “Virtual Bellows: Constructing High Quality Stills from Video,” Proc. First IEEE Int'l Conf. Image Processing, vol. I, pp. 363-367, Nov. 1994.
9. L. McMillan and G. Bishop, “Plenoptic Modeling: An Image-Based Rendering System,” Proc. SIGGRAPH '95, pp. 39-46, Aug. 1995.
10. S. Nayar, “Catadioptric Omnidirectional Cameras,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 482-488, June 1997.
11. S. Nayar and A. Karmarkar, “360 x 360 Mosaics,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 388-395, June 2000.
12. S. Peleg and M. Ben-Ezra, “Stereo Panorama with a Single Camera,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 395-401, June 1999.
13. S. Peleg and J. Herman, “Panoramic Mosaics by Manifold Projection,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 338-343, June 1997.
14. Y. Pritch M. Ben-Ezra and S. Peleg, “Optical Projections for Panoramic Stereo,” Foundations of Image Processing and Computer Vision: A Festricft in Honor of Prof. Azriel Rosenfeld, L. Davis, ed., Kluwer, 2001.
15. P. Rademacher and G. Bishop, “Multiple-Center-of-Projection Images,” Proc. SIGGRAPH '98, pp. 199-206, July 1998.
16. J. Shimamura N. Yokoya H. Takemura and K. Yamazawa, “Construction of an Immersive Mixed Environment Using an Omnidirectional Stereo Image Sensor,” Proc. IEEE Workshop Omnidirectional Vision, pp. 62-69, June 2000.
17. H. Shum A. Kalai and S. Seitz, “Omnivergent Stereo,” Proc. Seventh Int'l Conf. Computer Vision, pp. 22-29, Sept. 1999.
18. H. Shum and R. Szeliski, “Stereo Reconstruction from Multiperspective Panoramas,” Proc. Seventh Int'l Conf. Computer Vision, pp. 14-21, Sept. 1999.
19. R. Szeliski, “Video Mosaics for Virtual Environments,” IEEE Computer Graphics and Applications, vol. 16, no. 2, pp. 22-30, 1996.
20. D. Wood A. Finkelstein J. Hughes C. Thayer and D. Salesin, “Multiperspective Panoramas for Cel Animation,” Proc. SIGGRAPH '97, pp. 243-250, Aug. 1997.
21. R.C. Yates, A Handbook on Curves and Their Properties, rev. ed., Nat'l Council of Teachers of Math., 1952, reprinted, 1974.
22. T. Naemura M. Kaneko and H. Harashima, “Multi-User Immersive Stereo,” Proc. IEEE 1998 Int'l Conf. Image Processing, vol. 1, pp. 903-907, Oct. 1998.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

        • Scene understanding

  • Machine learning

  • Modeling and simulation

    • Model development and analysis

      • Model verification and validation