skip to main content
research-article

Blind Image Quality Assessment by Natural Scene Statistics and Perceptual Characteristics

Authors Info & Claims
Published:25 August 2020Publication History
Skip Abstract Section

Abstract

Opinion-unaware blind image quality assessment (OU BIQA) refers to establishing a blind quality prediction model without using the expensive subjective quality scores, which is a highly promising direction in the BIQA research. In this article, we focus on OU BIQA and propose a novel OU BIQA method. Specifically, in our proposed method, we deeply investigate the natural scene statistics (NSS) and the perceptual characteristics of the human brain for visual perception. Accordingly, a set of quality-aware NSS and perceptual characteristics-related features are designed to characterize the image quality effectively. For inferring the image quality, we learn a pristine multivariate Gaussian (MVG) model on a collection of pristine images, which serves as the reference information for quality evaluation. At last, the quality of a new given image is defined by measuring the divergence between its MVG model and the learned pristine MVG model. Thorough experiments performed on seven popular image databases demonstrate that the proposed OU BIQA method delivers superior performance to the state-of-the-art OU BIQA methods. The Matlab source code of the proposed method will be made publicly available at https://github.com/YT2015?tab=;repositories.

References

  1. Karl Friston. 2010. The free-energy principle: A unified brain theory? Nature Rev. Neurosci. 11, 2 (2010), 127--138.Google ScholarGoogle ScholarCross RefCross Ref
  2. Karl Friston, James Kilner, and Lee Harrison. 2006. A free energy principle for the brain. J. Physiol. Paris 100, 1 (2006), 70--87.Google ScholarGoogle Scholar
  3. Jan-Mark Geusebroek and Arnold W. M. Smeulders. 2005. A six-stimulus theory for stochastic texture. Int. J. Comput. Vis. 62, 1--;2 (2005), 7--16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Deepti Ghadiyaram and Alan C. Bovik. 2016. Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. Image Process. 25, 1 (2016), 372--387.Google ScholarGoogle ScholarCross RefCross Ref
  5. Ke Gu, Leida Li, Hong Lu, Xiongkuo Min, and Weisi Lin. 2017. A fast reliable image quality predictor by fusing micro- and macro-structures. IEEE Trans. Ind. Electron. 64, 5 (2017), 3903--3912.Google ScholarGoogle ScholarCross RefCross Ref
  6. Ke Gu, Guangtao Zhai, Weisi Lin, and Min Liu. 2016. The analysis of image contrast: From quality assessment to automatic enhancement. IEEE Trans. Cybernet. 46, 1 (2016), 284--297.Google ScholarGoogle ScholarCross RefCross Ref
  7. Ke Gu, Guangtao Zhai, Xiaokang Yang, and Wenjun Zhang. 2014. Hybrid no-reference quality metric for singly and multiply distorted images. IEEE Trans. Broadcast. 60, 3 (2014), 555--567.Google ScholarGoogle ScholarCross RefCross Ref
  8. Ke Gu, Guangtao Zhai, Xiaokang Yang, and Wenjun Zhang. 2015. Using free energy principle for blind image quality assessment. IEEE Trans. Multimedia 17, 1 (2015), 50--63.Google ScholarGoogle ScholarCross RefCross Ref
  9. Stephen R. Gulliver and Gheorghita Ghinea. 2006. Defining user perception of distributed multimedia quality. ACM Trans. Multimedia Comput. Commun. Appl. 2, 4 (Nov. 2006), 241--257.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. R. Hassen, Z. Wang, and M. Salama. 2010. No-reference image sharpness assessment based on local phase coherence measurement. In IEEE Int. Conf. Acoust. Speech Sig. Process. 2434--2437.Google ScholarGoogle Scholar
  11. Yuukou Horita, Keiji Shibata, Yoshikazu Kawayoke, and Z. M. Parvez Sazzad. 2011. MICT image quality evaluation database. Retrieved from http://mict.eng.u-toyama.ac.jp/mictdb.html.Google ScholarGoogle Scholar
  12. L. Kang, P. Ye, Y. Li, and D. Doermann. 2014. Convolutional neural networks for no-reference image quality assessment. In Proceedings of the IEEE Conference Computer Vision Pattern Recognition (CVPR’14). 1733--1740.Google ScholarGoogle Scholar
  13. J. Kim and S. Lee. 2017. Fully deep blind image quality predictor. IEEE J. Sel. Topics Signal Process 11, 1 (2017), 206--220.Google ScholarGoogle ScholarCross RefCross Ref
  14. Eric C. Larson and D. M. Chandler. 2010. Categorical image quality (CSIQ) database. Retrieved from http://vision.okstate.edu/csiq.Google ScholarGoogle Scholar
  15. Q. Li, W. Lin, J. Xu, and Y. Fang. 2016. Blind image quality assessment using statistical structural and luminance features. IEEE Trans. Multimedia. 18, 12 (2016), 2457--2469.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Xianguo Li, Yemei Sun, Yanli Yang, and Changyun Miao. 2019. Symmetrical residual connections for single image super-resolution. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1 (Feb. 2019), 19:1--19:10.Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Yutao Liu, Ke Gu, Shiqi Wang, Debin Zhao, and Wen Gao. 2019. Blind quality assessment of camera images based on low-level and high-level statistical features. IEEE Trans. Multimedia 21, 1 (2019), 135--146.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Yutao Liu, Ke Gu, Guangtao Zhai, Xianming Liu, Debin Zhao, and Wen Gao. 2017. Quality assessment for real out-of-focus blurred images. J. Vis. Commun. Image Represent. 46 (2017), 70--80.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Y. Liu, G. Zhai, K. Gu, X. Liu, D. Zhao, and W. Gao. 2018. Reduced-reference image quality assessment in free-energy principle and sparse representation. IEEE Trans. Multimedia 20, 2 (2018), 379--391.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Yutao Liu, Guangtao Zhai, Xianming Liu, and Debin Zhao. 2016. Perceptual image quality assessment combining free-energy principle and sparse representation. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS’16). IEEE, 1586--1589.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. K. Ma, Z. Duanmu, Q. Wu, Z. Wang, H. Yong, H. Li, and L. Zhang. 2017. Waterloo exploration database: New challenges for image quality assessment models. IEEE Trans. Image Process. 26, 2 (Feb. 2017), 1004--1016.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. K. Ma, W. Liu, T. Liu, Z. Wang, and D. Tao. 2017. dipIQ: Blind image quality assessment by learning-to-rank discriminable image pairs. IEEE Trans. Image Process. 26, 8 (Aug. 2017), 3951--3964.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. K. Ma, W. Liu, K. Zhang, Z. Duanmu, Z. Wang, and W. Zuo. 2018. End-to-end blind image quality assessment using deep neural networks. IEEE Trans. Image Process. 27, 3 (2018), 1202--1213.Google ScholarGoogle ScholarCross RefCross Ref
  24. D. Martin, C. Fowlkes, D. Tal, and J. Malik. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of the IEEE International Conference on Computer Vision, Vol. 2. 416--423.Google ScholarGoogle Scholar
  25. Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 12 (2012), 4695--4708.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Anish Mittal, Ravi Soundararajan, and Alan C. Bovik. 2013. Making a “completely blind” image quality analyzer. IEEE Signal Process. Lett. 20, 3 (2013), 209--212.Google ScholarGoogle ScholarCross RefCross Ref
  27. Anush Krishna Moorthy and Alan Conrad Bovik. 2010. A two-step framework for constructing blind image quality indices. IEEE Signal Process. Lett. 17, 5 (2010), 513--516.Google ScholarGoogle ScholarCross RefCross Ref
  28. Anush Krishna Moorthy and Alan Conrad Bovik. 2011. Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Trans. Image Process. 20, 12 (2011), 3350--3364.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. T. Ojala, M. Pietikainen, and T. Maenpaa. 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 7 (July 2002), 971--987.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Bruno A. Olshausen et al. 1996. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 6583 (1996), 607--609.Google ScholarGoogle Scholar
  31. Bruno A. Olshausen and David J. Field. 1997. Sparse coding with an overcomplete basis set: A strategy employed by V1?Vision Res. 37, 23 (1997), 3311--3325.Google ScholarGoogle Scholar
  32. Nikolay Ponomarenko, Oleg Ieremeiev, Vladimir Lukin, Karen Egiazarian, Lina Jin, Jaakko Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, et al. 2013. Color image database TID2013: Peculiarities and preliminary results. In Proceedings of the 4th European Workshop on Visual Information Processing. 106--111.Google ScholarGoogle Scholar
  33. Ann Marie Rohaly, John Libert, Philip Corriveau, Arthur Webster, et al. 2000. Final report from the video quality experts group on the validation of objective models of video quality assessment. ITU-T Stand. Contrib. COM (2000), 9--80. Retrieved from https://drive.google.com/file/d/1jhHSifr9hTbCpWVxcX_l2NGT6dAN4AnR/view?usp=sharing.Google ScholarGoogle Scholar
  34. Michele A. Saad, Alan C. Bovik, and Christophe Charrier. 2012. Blind image quality assessment: A natural scene statistics approach in the DCT domain. IEEE Trans. Image Process. 21, 8 (2012), 3339--3352.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. K. Sharifi and A. Leon-Garcia. 1995. Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video. IEEE Trans. Circuits Syst. Video Technol. 5, 1 (1995), 52--56.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Hamid R. Sheikh, Muhammad F. Sabir, and Alan C. Bovik. 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15, 11 (2006), 3440--3451.Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Rajiv Soundararajan and Alan C. Bovik. 2012. RRED indices: Reduced reference entropic differencing for image quality assessment. IEEE Trans. Image Process. 21, 2 (2012), 517--526.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Joel A. Tropp and Anna C. Gilbert. 2007. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Info. Theory 53, 12 (2007), 4655--4666.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Toni Virtanen, Mikko Nuutinen, Mikko Vaahteranoksa, Pirkko Oittinen, and Jukka Häkkinen. 2015. CID2013: A database for evaluating no-reference image quality assessment algorithms. IEEE Trans. Image Process. 24, 1 (2015), 390--402.Google ScholarGoogle ScholarCross RefCross Ref
  40. Zhou Wang, Alan Conrad Bovik, Hamid Rahim Sheikh, and Eero P. Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600--612.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Q. Wu, H. Li, F. Meng, and K. N. Ngan. 2018. A perceptually weighted rank correlation indicator for objective image quality assessment. IEEE Trans. Image Process. 27, 5 (2018), 2499--2513.Google ScholarGoogle ScholarCross RefCross Ref
  42. Qingbo Wu, Zhou Wang, and Hongliang Li. 2015. A highly efficient method for blind image quality assessment. In Proceedings of the IEEE International Conference on Image Processing. 339--343.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. X. Wu, G. Zhai, X. Yang, and W. Zhang. 2011. Adaptive sequential prediction of multidimensional signals with applications to lossless image coding. IEEE Trans. Image Process. 20, 1 (Jan. 2011), 36--42.Google ScholarGoogle Scholar
  44. Wufeng Xue, Lei Zhang, and Xuanqin Mou. 2013. Learning without human scores for blind image quality assessment. In Proc. IEEE Int. Conf. Comput. Vis. Pattern Recogn. 995--1002.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Peng Ye, Jayant Kumar, Le Kang, and David Doermann. 2012. Unsupervised feature learning framework for no-reference image quality assessment. In Proc. IEEE Int. Conf. Comput. Vis. Pattern Recogn. IEEE, 1098--1105.Google ScholarGoogle Scholar
  46. Guangtao Zhai, Xiaolin Wu, Xiaokang Yang, Weisi Lin, and Wenjun Zhang. 2012. A psychovisual quality metric in free-energy principle. IEEE Trans. Image Process. 21, 1 (2012), 41--52.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Baochang Zhang, Yongsheng Gao, Sanqiang Zhao, and Jianzhuang Liu. 2010. Local derivative pattern versus local binary pattern: face recognition with high-order local pattern descriptor. IEEE Trans. Image Process. 19, 2 (2010), 533--544.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Jian Zhang, Debin Zhao, and Wen Gao. 2014. Group-based sparse representation for image restoration. IEEE Trans. Image Process. 23, 8 (2014), 3336--3351.Google ScholarGoogle Scholar
  49. Lin Zhang, Ying Shen, and Hongyu Li. 2014. VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Trans. Image Process. 23, 10 (2014), 4270--4281.Google ScholarGoogle ScholarCross RefCross Ref
  50. Lin Zhang, Lei Zhang, and Alan C. Bovik. 2015. A feature-enriched completely blind image quality evaluator. IEEE Trans. Image Process. 24, 8 (2015), 2579--2591.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Guoying Zhao and Matti Pietikainen. 2007. Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 6 (2007), 915--928.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Yi Zhu, Sharath Chandra Guntuku, Weisi Lin, Gheorghita Ghinea, and Judith A. Redi. 2018. Measuring individual video QoE: A survey, and proposal for future directions using social media. ACM Trans. Multimedia Comput. Commun. Appl. 14, 2s (May 2018), 30:1--30:24.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Blind Image Quality Assessment by Natural Scene Statistics and Perceptual Characteristics

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!