3DTV view generation with virtual pan/tilt/zoom functionality

ABSTRACT

This paper presents a novel rendering algorithm based on depth image warping to support virtual pan-tilt-zoom (PTZ) functionalities during 3D view generation. A method based on "3D-ness" knob is proposed for automatically specifying the virtual camera positions along a path, to model the PTZ mechanism in projective framework. Two novel quality enhancing techniques based on segmentation cues are proposed to add pan, tilt and zoom capabilities during arbitrary view synthesis. In addition to reduce the computational load that results in providing such functionalities, a novel selective warping scheme is presented to reduce the computational cost by as much as 40% while maintaining an acceptable quality of rendering results. Experiments are performed using standard "Breakdancers" and "Ballet" video sequences to demonstrate the effectiveness of the proposed methods as compared to currently published results.

References

1. S. Zinger, L. Do, and P. H. N. de With. Free-viewpoint depth image based rendering. JVCI. 21(5--6), 2010.
2. Y. Mori, N. Fukushima, T. Yendo, T. Fujii, and M. Tanimoto. View generation with 3d warping using depth information for ftv. Image Communication'09.
3. Wan-Yu Chen, Yu-Lin Chang, Shyh-Feng Lin, Li-Fu Ding, and Liang-Gee Chen. Efficient depth image based rendering with edge dependent depth filter and interpolation. ICME, pp. 1314--1317, 2005.
4. Y. Horng, Y. Tseng, T. Chang. Stereoscopic images generation with directional gaussian filter. ISCAS'10.
5. A. Criminisi, P. Perez, and K. Toyama. Region filling and object removal by exemplar-based image inpainting. IEEE TIP. 13(9): 2004.
6. I. Daribo and B. Pesquet Popescu. Depth aided image inpainting for novel view synthesis. MMSP. 2010.
7. K. Oh, S. Yea, and Y. Ho. Hole filling method using depth based in-painting for view synthesis in free-viewpoint television and 3-d video. PCS. 2009.
8. K. Muller, A. Smolic, K. Dix, P. Merkle, P. Kauff, and T. Wiegand. View synthesis for advanced 3d video systems. In EURASIP JIVP. 2008.
9. Andreas Ruf and Radu Horaud. Projective Rotations applied to a Pan-Tilt Stereo Head. CVPR, 1999.
10. A. Colombari, A. Fusiello, and V. Murino. Continuous parallax adjustment for 3D-TV. CVMP, 2005.
11. Andreas Ruf, Gabriela Csurka, and Radu Horaud. Projective Translations and Affine Stereo Calibration. CVPR, pp. 475--481, 1998.
12. Janusz Konrad. Enhancement of Viewer Comfort in Stereoscopic Viewing: Parallax Adjustment, 1999.
13. M. Bertalmio, A. L. Bertozzi, and G. Sapiro. Navier-stokes, fluid dynamics, and image and video inpainting. CVPR, pp. 355--362, 2001.
14. A. V. Bhavsar and A. N. Rajagopalan. Inpainting large missing regions in range images. ICPR, 2010.
15. P. F. Felzenszwalb and D. P. Huttenlocher. Efficient graph-based image segmentation. IJCV, 2004.
16. S. Choi, Taemin Kim, and Wonpil Yu. Performance Evaluation of RANSAC Family. BMVC, 2009.
17. C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski. High-quality video view interpolation using a layered representation. SIGGRAPH, 2004.

Index Terms

• Applied computing

  • Document management and text processing

    • Document capture

      • Graphics recognition and interpretation

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation