Harpreet S. Sawhney
ABSTRACT
This paper introduces a novel application of IBR technology for efficient rendering of high quality CG and live action stereoscopic sequences. Traditionally, IBR has been applied to render novel views using image and depth based representations of the plenoptic functions. In this work, we present a restricted form of IBR in which lower resolution images for the views to be generated at a very high resolution are assumed to be available. Specifically, the paper addresses the problem of synthesizing stereo IMAX(R)1 3D motion picture images at a standard resolution of 4-6K. At such high resolutions, producing CG content is extremely time consuming and capturing live action requires bulky cameras. We propose a Hybrid Stereo Camera concept in which one view is rendered at the target high resolution but the other is rendered at a much lower resolution. Methods for synthesizing the second view sequence at the target resolution using image analysis and IBR techniques are the focus of this work. The high quality results from the techniques presented in this paper have been visually evaluated in the IMAX 3D large screen projection environment. The paper also highlights generalizations and extensions of the hybrid stereo camera concept.
- 1.G. Adiv. Determining 3D motion and structure from optical flows generated by several moving objects. IEEE TPAMI, 7(4):384-401, 1985.Google Scholar
- 2.J. R. Bergen et al. Hierarchical model-based motion estimation. In Euro. Conf. on Comp. Vision, pages 237-252, 1992. Google ScholarDigital Library
- 3.P. J. Burt and E. H. Adelson. The laplacian pyramid as a compact image code. IEEE TCOMM, 31(4):532-540, 1983.Google ScholarCross Ref
- 4.K. J. Hanna and Neil E. Okamoto. Combining stereo and motion analysis for direct estimation of scene structure. In Proc. Intl. Conf. on Computer Vision, pages 357-365, 1993.Google ScholarCross Ref
- 5.Phil Harman. Home based 3d entertainment - an overview. In IEEE Intl. Conf. on Image Processing, pages 1-4, 2000.Google ScholarCross Ref
- 6.B.K.P. Horn. Robot Vision. MIT Press, 1986. Google ScholarDigital Library
- 7.M. Irani, B. Rousso, and S. Peleg. Computing occluding and transparent motions. IJCV, 12:5-16, 1994. Google ScholarDigital Library
- 8.B. Julesz. Foundations of Cyclopean Perception. The University of Chicago Press, 1971.Google Scholar
- 9.R. Kumar, P. Anandan, and K. Hanna. Direct recovery of shape from multiple views: A parallax based approach. In ICPR, pages 685-688, 1994.Google ScholarCross Ref
- 10.B. D. Lucas and T. Kanade. An iterative image registration technique with an application in stereo vision. In IJCAI, 1981.Google Scholar
- 11.W. R. Mark, L. McMillan, and G. Bishop. Post-rendering 3d warping. In Symp. on Interactive 3D Graphics (Providence, RI), pages 7-16, 1997. Google ScholarDigital Library
- 12.Y. Matsumoto et al. Conversion system of monocular image sequence to stereo using motion parallax. In SPIE Stereo. Disp. and VR Sys.(Vol 3012), 1997.Google Scholar
- 13.L. McMillan and G. Bishop. Plenoptic modeling: An imagebased rendering system. In SIGGRAPH, pages 39-46, 1995. Google ScholarDigital Library
- 14.P.J. Narayanan, P.W. Rander, and T. Kanade. Constructing virtual worlds using dense stereo. In ICCV, pages 3-10, 1998. Google ScholarDigital Library
- 15.Michael G. Perkins. Data compression of stereo pairs. IEEE Trans. on Comm., 40(4):684-696, 1992.Google ScholarCross Ref
- 16.W. K. Pratt. Digital Image Processing (Second Edition).Wiley, 1991. Google ScholarDigital Library
- 17.H. S. Sawhney. 3D geometry from planar parallax. In CVPR, pages 929-934, 1994.Google ScholarCross Ref
- 18.A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In CVPR, pages 483-489, 1994.Google ScholarCross Ref
- 19.L. Stelmach et al. Human perception of mismatched stereoscopic 3d stereo inputs. In ICIP, 2000.Google Scholar
- 20.R. Szeliski. Prediction error as a quality metric for motion and stereo. In Proc. Intl. Conf. on Computer Vision, 1999. Google ScholarDigital Library
- 21.R. Szeliski and J. Coughlan. Spline-based image registration. IJCV, 22(3):199-218, 1997. Google ScholarDigital Library
- 22.W.H.Press, B.P.Flannery, S.A.Teukolsky, and W.T.Vetterling. Numerical Recipes in C. Cambridge University Press, 1986.Google Scholar
- 23.G. Wolberg. Digital Image Warping. IEEE Computer Society Press, CA, 1990. Google ScholarDigital Library
Index Terms
Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences
Recommendations
Free-viewpoint Indoor Neural Relighting from Multi-view Stereo
We introduce a neural relighting algorithm for captured indoors scenes, that allows interactive free-viewpoint navigation. Our method allows illumination to be changed synthetically, while coherently rendering cast shadows and complex glossy materials. We ...
A fast relighting engine for interactive cinematic lighting design
SIGGRAPH '00: Proceedings of the 27th annual conference on Computer graphics and interactive techniquesWe present new techniques for interactive cinematic lighting design of complex scenes that use procedural shaders. Deep-framebuffers are used to store the geometric and optical information of the visible surfaces of an image. The geometric information ...
The multiple-camera 3-D production studio
Multiple-camera systems are currently widely used in research and development as a means of capturing and synthesizing realistic 3-D video content. Studio systems for 3-D production of human performance are reviewed from the literature, and the ...
Comments