Abstract
With the growing demand for 3D video, efforts are underway to incorporate it in the next generation of broadcast and streaming applications and standards. 3D video is currently available in games, entertainment, education, security, and surveillance applications. A typical scenario for multiview 3D consists of several 3D video sequences captured simultaneously from the same scene with the help of multiple cameras from different positions and through different angles. Multiview video coding provides a compact representation of these multiple views by exploiting the large amount of inter-view statistical dependencies. One of the major challenges in this field is how to transmit the large amount of data of a multiview sequence over error prone channels to heterogeneous mobile devices with different bandwidth, resolution, and processing/battery power, while maintaining a high visual quality. Scalable Multiview 3D Video Coding (SMVC) is one of the methods to address this challenge; however, the evaluation of the overall visual quality of the resulting scaled-down video requires a new objective perceptual quality measure specifically designed for scalable multiview 3D video. Although several subjective and objective quality assessment methods have been proposed for multiview 3D sequences, no comparable attempt has been made for quality assessment of scalable multiview 3D video. In this article, we propose a new methodology to build suitable objective quality assessment metrics for different scalable modalities in multiview 3D video. Our proposed methodology considers the importance of each layer and its content as a quality of experience factor in the overall quality. Furthermore, in addition to the quality of each layer, the concept of disparity between layers (inter-layer disparity) and disparity between the units of each layer (intra-layer disparity) is considered as an effective feature to evaluate overall perceived quality more accurately. Simulation results indicate that by using this methodology, more efficient objective quality assessment metrics can be introduced for each multiview 3D video scalable modalities.
- Barkowsky, M., Wang, K., Cousseau, R., Brunnstrom, K., Olsson, R., and Le Callet, P. 2010. Subjective quality assessment of error concealment strategies for 3DTV in the presence of asymmetric transmission errors. In Proceedings of the 18th International Packet Video Workshop (PV). 193--200.Google Scholar
Cross Ref
- Do, L., Zinger, S., and de With, P. H. N. 2010. Conversion of free-viewpoint 3D multiview video for stereoscopic displays. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). 1730--1734.Google Scholar
- Dodgson, N. A. 2005. Autostereoscopic 3D displays. In Computer Journal Published by the IEEE Computer Society 38, 9, 31--36. Google Scholar
Digital Library
- Feghali, R., Speranza, F., Wang, D., and Vincent, A. 2007. Video quality metric for bit rate control via joint adjustment of quantization and frame rate. IEEE Transactions on Broadcasting. 53, 1, 441--446.Google Scholar
Cross Ref
- Grois, D., Kaminsky, E., and Hadar, O. 2010. Dynamically adjustable and scalable ROI video coding. In Proceedings of the IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB). 1--5.Google Scholar
- Gutierrez, J., Perez, P., Jaureguizar, F., Cabrera, J., and Garcia, N. 2011. Subjective assessment of the impact of transmission errors in 3DTV compared to HDTV. In Proceedings of the 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON). Antalya, Turkey. 1--4.Google Scholar
- Ha, K. and Kim, M. 2011. A perceptual quality assessment metric using temporal complexity and disparity information for stereoscopic video. In Proceedings of the 18th IEEE International Conference on Image Processing (ICIP), Brussels, Belgium. 2525--2528.Google Scholar
- Hewage, C. T. E. R., Worrall, S. T., Dogan, S., and Kondoz, A. M. 2008. Prediction of stereoscopic video quality using objective quality models of 2-D video. Electron. Lett. 44, 16, 963--965.Google Scholar
Cross Ref
- Hewage, C. T. E. R. and Martini, M. G. 2010a. Reduced-reference quality evaluation for compressed depth maps associated with colour plus depth 3D video. In Proceedings of the 17th IEEE International Conference on Image Processing (ICIP). 4017--4020.Google Scholar
- Hewage, C. T. E. R. and Martini, M. G. 2010b. Reduced-reference quality metric for 3D depth map transmission. In Proceedings of the 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON). 1--4.Google Scholar
- Hewage, C. T. E. R. and Martini, M. G. 2011. Reduced-Reference quality assessment for 3D video compression and transmission. IEEE Trans. Consumer Electron. 57, 3, 1185--1193.Google Scholar
Cross Ref
- Ho, Y. S. and Oh, K. J. 2007. Overview of multi-view video coding. In Proceedings of the 14th EURASIP Conference Focused on Speech and Image Processing Metromedia Communications and Service. 5--12Google Scholar
- ISO/IES JTC 1/SC 29/WG11.2005. Introduction to multi-view video coding. http://www.itscj.ipsj or jp./sc29/.Google Scholar
- ITU-R Recommendation BT.500-10. 2000. Methodology for the subjective assessment of the quality of television picture. http://www.itu.int/rec/R-REC-BT.500/en.Google Scholar
- ITU-R BT.812. 1994. Subjective assessment of the quality of alphanumeric and graphic pictures in teletext and similar services.Google Scholar
- ITU-T Rec H.264/ISO/IEC, Annex H. 2010. Multiview video coding. http://www.itu.int/rec/T-REC-H264.Google Scholar
- Jia, H., Gao, W., and Lu, Y. 2003. Stereoscopic video coding based on global displacement compensated prediction. In Proceedings of the Information, Communications and Signal Processing and the Pacific Rim Conference on Multimedia. 61--65.Google Scholar
- Jin, L., Boev, A., Gotchev, A., Egiazarian, K. 2011. 3D-DCT based perceptual quality assessment of stereo video. In Proceedings of the 18th IEEE International Conference on Image Processing (ICIP). 2521--2524.Google Scholar
Cross Ref
- Joveluro, P., Malekmohamadi, H., Fernando., W. A. C., and Kondoz, A. M. 2010. Perceptual video quality metric for 3D video quality assessment. In Proceedings of The True Vision - Capture, Transmission and Display of 3D Video,Tampere, Finland. 1--4.Google Scholar
- Kim, D., Min, D., Oh, J., Jeon, S., and Sohn, K. 2009. Depth map quality metric for three-dimensional video. In Proceedings of SPIE 7237, 723719.Google Scholar
- Koumaras, H., Kourtis, A., Lin, C.-H., and Shieh, C.-K. 2007. A theoretical framework for end-to-end video Quality Prediction of MPEG-based Sequences. In 3rd International Conference onNetworking and Services, ICNS. Athens, Greece. 62. Google Scholar
Digital Library
- Leon, G., Kalva, H., and Furht, B. 2008. 3D video quality evaluation with depth quality variations. In Proceedings of the 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video. 301--304.Google Scholar
- Li, C., Yang, X., Chu, B., Lu, W., and Pang, L. 2010. A new image fusion quality assessment method based on contourlet and SSIM. In Proceedings of the 3rd IEEE International Conference on Computer Science and Information Technology. 246--249.Google Scholar
- Li, Q. and Wang Z. 2009. Reduced-Reference image quality assessment using divisive normalization-based image representation. IEEE J. Select. Topics Sig. Proce. 3, 2, 202--211.Google Scholar
Cross Ref
- Lin, W. and Kuo, C.-C. J. 2011. Perceptual visual quality metrics: a survey. J. Visual Commun. Image Represent. 22, 4, 297--312. Google Scholar
Digital Library
- Liyuan, X., Junyong, Y., Ebrahimi, T., and Perkis, A. 2011. Objective metrics for quality of experience in stereoscopic images. In Proceedings of the 18th IEEE International Conference on Image Processing (ICIP). 3105--3108.Google Scholar
- Martini, M. G. and Hewage, C. T. E. R. 2011. Objective quality assessment for 3-D video delivery. In Proceedings of the IEEE ComSoc MMTC E-letter 6, 8, 8--22.Google Scholar
- Maalouf, A. and Larabi, M.-C. 2010. A no-reference color video quality metric based on a 3D multispectral wavelet transform. In Proceedings of the International Workshop on Quality of Multimedia Experience (QoMEX). 11--16.Google Scholar
- Merl, ftp://ftp.merl.com/pub/avetro/mvc-testseq/stereo-interlaced/420/. (Last accessed 1/12).Google Scholar
- Micallef, B. W., Debono, C. J., and Farrugia, R. A. 2010. Exploiting depth information for fast multiview video coding. In Proceedings of the Picture Coding Symposium (PCS). 38--41.Google Scholar
- Mittal, A., Moorthy, A. K., Ghosh, J., and Bovik, A.C. 2011. Algorithmic assessment of 3D quality of experience for images and videos. In Proceedings of the Digital Signal Processing and IEEE Signal Processing Education Workshop. 338--343.Google Scholar
- Muller, K., Merkle, P., and Wiegand, T. 2011. 3-D video representation using depth maps. In Proc. IEEE. 99, 4, 643--656.Google Scholar
Cross Ref
- Nur, G., Arachchi, H. K., Dogan, S., and Kondoz, A.M. 2011. Extended VQM model for predicting 3D video quality considering ambient illumination context . In Proceedings of the 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON). 1--4.Google Scholar
- Ozbek, N., Ertan, G., and Karakus, O. 2011. Interactive quality assessment for asymmetric coding of 3D video. In Proceedings of the 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON). 1--4.Google Scholar
- Pandit, S. P., Chen, Y., and Ye, S. 2008. Text of ISO/IEC 14496- 5:2001/PDAM 15 Reference software for multiview video coding, ISO/IEC JTC1/SC29/WG11 MPEG2008/W9974.Google Scholar
- Rodriguez, E. J., Nunome, T., and Tasaka, S. 2009. Assessment of user behavior and QoE in multiview video and audio IP transmission. In Proceedings of the 15th Asia-Pacific Conference on Communications, APCC. 790--793. Google Scholar
Digital Library
- Roodaki, H., Hashemi, M. R., and Shirmohammadi, S. 2011. A new scalable multiview video coding configuration for mobile applications. In Proceedings of IEEE International Conference on Multimedia & Expo. 1--4. Google Scholar
Digital Library
- Saygili, G., Gurler, C. G., and Tekalp, A. M. 2010. Quality assessment of asymmetric stereo video coding. In Proceedings of the 17th IEEE International Conference on Image Processing (ICIP). 4009--4012.Google Scholar
- Schwarz, H., Marpe, D., and Wiegand, T. 2007. Overview of the scalable video coding extension of the H.264/AVC standard. IEEE Trans. Circ. Syst. Video Techn. 17, 9, 1103--1120. Google Scholar
Digital Library
- Shao, H., Cao, X., and Er, G. 2009. Objective quality of depth image based rendering in 3DTV system. In Proceedings of the 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video. 1--4.Google Scholar
- Shimizu, Sh., Kitahara, M., Kimata, H., Kamikura, K., and Yashima, Y. 2007. View scalable multiview video coding using 3-D warping with depth map. IEEE Trans. Circ. Syst. Video Techn. 17, 11, 1485--1495. Google Scholar
Digital Library
- Tanimoto, M., Fujii, T., Suzuki, K., Fukushima, N., and Mori, Y. 2008. Reference softwares for depth estimation and view synthesis. ISO/IEC JTC1/SC29/WG11 MPEG2008/M15377.Google Scholar
- Tanimoto, M. 2009. Overview of FTV (free-viewpoint television), In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), 1552--1553. Google Scholar
Digital Library
- Tanimoto Laboratory. 2012. http://www.tanimoto.nuee.nagoya-u.ac.jp/~fukushima/mpegftv/yuv/. (Last accessed 1/12).Google Scholar
- Umar, A. S., Swash, R. M., and Sadka, A. H. 2011. Subjective quality assessment of 3D videos. In Proceedings of the IEEE Africon, Livingstone. 1--6.Google Scholar
- Vetro, A., Tourapis, A. M., Muller, K., and Chen, T. 2011a. 3D-TV content storage and transmission. Trans. Broadcast. 57, 2, 384--394.Google Scholar
Cross Ref
- Vetro, A., Wiegand, T., and Sullivan, G. J. 2011b. Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC standard. Proc. IEEE. 99, 4, 626--642.Google Scholar
Cross Ref
- Wang, Z., Bovik, A. C., Sheikh, H. R., and Simoncelli, E. P. 2004. Image Quality Assessment: from Error Visibility to Structural Similarity. Ieee Trans. Image Process. 13, 4, 600--612. Google Scholar
Digital Library
- Wong, K.-M., PO, L.-M., Cheung, K.-W., Ng, K.-H., and Xu, X. 2011. Stretching, compression and shearing disparity compensated prediction techniques for stereo and multiview video coding. In Proceedings of The IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 841--844.Google Scholar
- Yamagishi, K., Karam, L., Okamoto, J., and Hayashi, T. 2011. Subjective characteristics for stereoscopic high definition video. In Proceedings of the 3rd International Workshop on Quality of Multimedia Experience (QOME). 37--42.Google Scholar
- Ylxla, L., yllln, C., andna, Z. 2009. Method of ensuring eye-to-eye contact in multi-view video conference. In Proceedings of the IEEE International Conference on Broadband Network & Multimedia Technology. 882--886.Google Scholar
- Zhang, Y., An, P., Wu, Y., and Zhang, Z. 2010. A multiview video quality assessment method based on disparity and ssim. In Proceedings of the IEEE 10th International Conference on Signal Processing (ISCP). 1044--1047.Google Scholar
- Zhu, Y. and Zhen, T. 2009. 3D multi-view autostereoscopic display and its key technologies. In Proceedings of the Asia-Pacific Conference on Information Processing (AS1P). 31--35. Google Scholar
Digital Library
- Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. In Proceedings of the ACM SIGGRAPH. 600-608. http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload. (Last accessed 1/12). Google Scholar
Digital Library
Index Terms
A new methodology to derive objective quality assessment metrics for scalable multiview 3D video coding
Recommendations
Binocular vision based objective quality assessment method for stereoscopic images
Human visual system (HVS) can perceive the difference between two retinal images to create a mental image with depth perception, which is the result of two binocular interactions, i.e., binocular fusion and suppression. According to perceptual ...
Objective Quality Assessment of Noised Stereoscopic Images
ICMTMA '11: Proceedings of the 2011 Third International Conference on Measuring Technology and Mechatronics Automation - Volume 02Stereoscopic images have been widely studied in recent years from a technical point of view, but the related quality assessment does not follow this enthusiasm. An objective quality assessment method for serious noised stereoscopic images is proposed. ...
An adaptive framework for scalable multi-view video coding for the H.264/AVC standard
MM '12: Proceedings of the 20th ACM international conference on MultimediaMulti-view 3D video is currently attracting growing attention in several applications such as the 3DTV, free-view point video and entertainment industry where it can be used to provide multi-perspective viewing and 3D scene experiences. In multi-view 3D ...






Comments