skip to main content
research-article

Precise No-Reference Image Quality Evaluation Based on Distortion Identification

Authors Info & Claims
Published:15 November 2021Publication History
Skip Abstract Section

Abstract

The difficulty of no-reference image quality assessment (NR IQA) often lies in the lack of knowledge about the distortion in the image, which makes quality assessment blind and thus inefficient. To tackle such issue, in this article, we propose a novel scheme for precise NR IQA, which includes two successive steps, i.e., distortion identification and targeted quality evaluation. In the first step, we employ the well-known Inception-ResNet-v2 neural network to train a classifier that classifies the possible distortion in the image into the four most common distortion types, i.e., Gaussian white noise (WN), Gaussian blur (GB), jpeg compression (JPEG), and jpeg2000 compression (JP2K). Specifically, the deep neural network is trained on the large-scale Waterloo Exploration database, which ensures the robustness and high performance of distortion classification. In the second step, after determining the distortion type of the image, we then design a specific approach to quantify the image distortion level, which can estimate the image quality specially and more precisely. Extensive experiments performed on LIVE, TID2013, CSIQ, and Waterloo Exploration databases demonstrate that (1) the accuracy of our distortion classification is higher than that of the state-of-the-art distortion classification methods, and (2) the proposed NR IQA method outperforms the state-of-the-art NR IQA methods in quantifying the image quality.

REFERENCES

  1. [1] Yan Chenggang, Xie Hongtao, Chen Jianjun, Zha Zhengjun, Hao Xinhong, Zhang Yongdong, and Dai Qionghai. 2018. A fast Uyghur text detector for complex background images. IEEE Trans. Multimedia 20, 12 (2018), 33893398.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Yan Chenggang, Li Liang, Zhang Chunjie, Liu Bingtao, Zhang Yongdong, and Dai Qionghai. 2019. Cross-modality bridging and knowledge transferring for image understanding. IEEE Trans. Multimedia 21, 10 (2019), 2675–2685.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Yan Chenggang, Tu Yunbin, Wang Xingzheng, Zhang Yongbing, Hao Xinhong, Zhang Yongdong, and Dai Qionghai. 2019. STAT: Spatial-temporal attention mechanism for video captioning. IEEE Trans. Multimedia 22, 1 (2019), 229–241.Google ScholarGoogle Scholar
  4. [4] Zhu Zunjie, Xu Feng, Yan Chenggang, Hao Xinhong, Ji Xiangyang, Zhang Yongdong, and Dai Qionghai. 2019. Real-time indoor scene reconstruction with RGBD and inertial input. In 2019 IEEE International Conference on Multimedia and Expo (ICME’19). IEEE, 712.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Zhu Yi, Guntuku Sharath Chandra, Lin Weisi, Ghinea Gheorghita, and Redi Judith A.. 2018. Measuring individual video QoE: A survey, and proposal for future directions using social media. ACM Trans. Multimedia Comput. Commun. Appl. 14, 2s (May 2018), 30:1–30:24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. [6] Gulliver Stephen R. and Ghinea Gheorghita. 2006. Defining user perception of distributed multimedia quality. ACM Trans. Multimedia Comput. Commun. Appl. 2, 4 (Nov. 2006), 241257. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Wang Zhou, Bovik Alan Conrad, Sheikh Hamid Rahim, and Simoncelli Eero P.. 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600612. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Liu Y., Zhai G., Gu K., Liu X., Zhao D., and Gao W.. 2018. Reduced-reference image quality assessment in free-energy principle and sparse representation. IEEE Trans. Multimedia 20, 2 (2018), 379391. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. [9] Liu Yutao, Gu Ke, Li Xiu, and Zhang Yongbing. 2020. Blind image quality assessment by natural scene statistics and perceptual characteristics. ACM Trans. Multimedia Comput. Comm. Appl. (TOMM) 16, 3 (2020), 191.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Liu Yutao, Gu Ke, Zhai Guangtao, Liu Xianming, Zhao Debin, and Gao Wen. 2017. Quality assessment for real out-of-focus blurred images. J. Vis. Comm. Image Represent. 46 (2017), 7080. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. [11] Vu Phong V. and Chandler Damon M.. 2012. A fast wavelet-based algorithm for global and local image sharpness estimation. IEEE Signal Process. Lett. 19, 7 (2012), 423426.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Hassen Rania, Wang Zhou, and Salama Magdy M. A.. 2013. Image sharpness assessment based on local phase coherence. IEEE Trans. Image Process. 22, 7 (2013), 27982810.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Zoran Daniel and Weiss Yair. 2009. Scale invariance and noise in natural images. In Proceedings of the IEEE International Conference on Computer Vision.22092216.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Liu X., Tanaka M., and Okutomi M.. 2013. Single-image noise level estimation for blind denoising. IEEE Trans. Image Process. 22, 12 (Dec. 2013), 52265237. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Wang Zhou, Bovik A. C., and Evan B. L.. 2000. Blind measurement of blocking artifacts in images. In Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101), Vol. 3. 981984.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Pan F., Lin X., Rahardja S., Lin W., Ong E., Yao S., Lu Z., and Yang X.. 2004. A locally-adaptive algorithm for measuring blocking artifacts in images and videos. In 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512), Vol. 3. III–925.Google ScholarGoogle Scholar
  17. [17] Liu Yutao, Gu Ke, Wang Shiqi, Zhao Debin, and Gao Wen. 2019. Blind quality assessment of camera images based on low-level and high-level statistical features. IEEE Trans. Multimedia 21, 1 (2019), 135146.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Gu Ke, Zhai Guangtao, Yang Xiaokang, and Zhang Wenjun. 2015. Using free energy principle for blind image quality assessment. IEEE Trans. Multimedia 17, 1 (2015), 5063.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Moorthy Anush Krishna and Bovik Alan Conrad. 2010. A two-step framework for constructing blind image quality indices. IEEE Signal Process. Lett. 17, 5 (2010), 513516.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Liu Y., Gu K., Zhang Y., Li X., Zhai G., Zhao D., and Gao W.. 2019. Unsupervised blind image quality evaluation via statistical measurements of structure, naturalness and perception. IEEE Trans. Circuits Syst. Video Technol. 30, 4 (2019), 929–943.Google ScholarGoogle Scholar
  21. [21] Zhang Lin, Shen Ying, and Li Hongyu. 2014. VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Trans. Image Process. 23, 10 (2014), 42704281.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Moorthy Anush Krishna and Bovik Alan Conrad. 2011. Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Trans. Image Process. 20, 12 (2011), 33503364. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Ma K., Liu W., Zhang K., Duanmu Z., Wang Z., and Zuo W.. 2018. End-to-end blind image quality assessment using deep neural networks. IEEE Trans. Image Process. 27, 3 (2018), 12021213.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Szegedy Christian, Ioffe Sergey, Vanhoucke Vincent, and Alemi Alexander A.. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In 31st AAAI Conference on Artificial Intelligence. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Ma Kede, Duanmu Zhengfang, Wu Qingbo, Wang Zhou, Yong Hongwei, Li Hongliang, and Zhang Lei. 2016. Waterloo exploration database: New challenges for image quality assessment models. IEEE Trans. Image Process. 26, 2 (2016), 10041016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Wang Zhou, Sheikh H. R., and Bovik A. C.. 2002. No-reference perceptual quality assessment of JPEG compressed images. In Proceedings International Conference on Image Processing.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Perra C., Massidda F., and Giusto D. D.. 2005. Image blockiness evaluation based on Sobel operator. In IEEE International Conference on Image Processing 2005, Vol. 1. I–389.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Min X., Ma K., Gu K., Zhai G., Wang Z., and Lin W.. 2017. Unified blind quality assessment of compressed natural, graphic, and screen content images. IEEE Trans. Image Process. 26, 11 (Nov. 2017), 54625474.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Marziliano Pina, Dufaux Frederic, Winkler Stefan, and Ebrahimi Touradj. 2002. A no-reference perceptual blur metric. In Proceedings of the IEEE International Conference on Image Processing.5760.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Ferzli Rony and Karam Lina J.. 2009. A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). IEEE Trans. Image Process. 18, 4 (2009), 717728. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Gu K., Lin W., Zhai G., Yang X., Zhang W., and Chen C. W.. 2017. No-reference quality metric of contrast-distorted images based on information maximization. IEEE Trans. Cybernetics 47, 12 (Dec. 2017), 45594565.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Liu Y. and Li X.. 2020. No-reference quality assessment for contrast-distorted images. IEEE Access 8 (2020), 8410584115.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Mittal Anish, Moorthy Anush Krishna, and Bovik Alan Conrad. 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 12 (2012), 46954708. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Saad Michele A., Bovik Alan C., and Charrier Christophe. 2012. Blind image quality assessment: A natural scene statistics approach in the DCT domain. IEEE Trans. Image Process. 21, 8 (2012), 33393352. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Mittal Anish, Soundararajan Ravi, and Bovik Alan C.. 2013. Making a completely blind image quality analyzer. IEEE Signal Process. Lett. 20, 3 (2013), 209212.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Zhang Y. and Chandler D. M.. 2018. Opinion-unaware blind quality assessment of multiply and singly distorted images via distortion parameter estimation. IEEE Trans. Image Process. 27, 11 (Nov. 2018), 54335448. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. [37] Kang L., Ye P., Li Y., and Doermann D.. 2014. Convolutional neural networks for no-reference image quality assessment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). 17331740. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. [38] Bosse S., Maniry D., Müller K., Wiegand T., and Samek W.. 2018. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27, 1 (Jan. 2018), 206219.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Wu J., Ma J., Liang F., Dong W., Shi G., and Lin W.. 2020. End-to-end blind image quality prediction with cascaded deep neural network. IEEE Trans. Image Process. 29 (2020), 74147426.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. [40] Liu X., Weijer J. Van De, and Bagdanov A. D.. 2017. RankIQA: Learning from rankings for no-reference image quality assessment. In 2017 IEEE International Conference on Computer Vision (ICCV’17). 10401049.Google ScholarGoogle ScholarCross RefCross Ref
  41. [41] Kim J. and Lee S.. 2017. Fully deep blind image quality predictor. IEEE J. Sel. Topics Signal Process. 11, 1 (2017), 206220. DOI: DOI: http://dx.doi.org/10.1109/JSTSP.2016.2639328Google ScholarGoogle ScholarCross RefCross Ref
  42. [42] Zhang W., Ma K., Zhai G., and Yang X.. 2021. Uncertainty-aware blind image quality assessment in the laboratory and wild. IEEE Trans. Image Process. 30 (2021), 34743486. DOI: DOI: http://dx.doi.org/10.1109/TIP.2021.3061932Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. [43] Yan Q., Gong D., and Zhang Y.. 2019. Two stream convolutional networks for blind image quality assessment. IEEE Transactions on Image Processing 28, 5 (2019), 22002211. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Haralick Robert M., Shanmugam Karthikeyan, and Dinstein Its’ Hak. 1973. Textural features for image classification. IEEE Trans. Syst. Man Cybernetics 6 (1973), 610621.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Szummer Martin and Picard Rosalind W.. 1998. Indoor-outdoor image classification. In Proceedings of the 1998 IEEE International Workshop on Content-Based Access of Image and Video Database. IEEE, 4251. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Khotanzad Alireza and Hong Yaw Hua. 1990. Invariant image recognition by Zernike moments. IEEE Trans. Pattern Anal. Mach. Intell. 12, 5 (1990), 489497. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. [47] He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770778.Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Szegedy Christian, Vanhoucke Vincent, Ioffe Sergey, Shlens Jon, and Wojna Zbigniew. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 28182826.Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] Zhai G., Kaup A., Wang J., and Yang X.. 2013. A dual-model approach to blind quality assessment of noisy images. In 2013 Picture Coding Symposium (PCS’13). 2932.Google ScholarGoogle Scholar
  50. [50] Simoncelli Eero P. and Olshausen Bruno A.. 2001. Natural image statistics and neural representation. Ann. Rev. Neurosci. 24, 1 (2001), 11931216.Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Morrone M. Concetta and Burr D. C.. 1988. Feature detection in human vision: A phase-dependent energy model. Proc. Roy. Soc. London B Biol. Sci. 235, 1280 (1988), 221245.Google ScholarGoogle ScholarCross RefCross Ref
  52. [52] Morrone M. Concetta, Ross John, Burr David C., and Owens Robyn. 1986. Mach bands are phase dependent. Nature 324, 6094 (1986), 250253.Google ScholarGoogle ScholarCross RefCross Ref
  53. [53] Liu Yutao, Zhai Guangtao, Liu Xianming, and Zhao Debin. 2015. Quality assessment for out-of-focus blurred images. In Visual Communications and Image Processing (VCIP’15). IEEE, 14.Google ScholarGoogle Scholar
  54. [54] Kovesi Peter et al. 1999. Image features from phase congruency. Videre: Journal of Computer Vision Research 1, 3 (1999), 126.Google ScholarGoogle Scholar
  55. [55] Cohen Albert, Daubechies Ingrid, and Feauveau J.-C.. 1992. Biorthogonal bases of compactly supported wavelets. Comm. Pure Appl. Math. 45, 5 (1992), 485560.Google ScholarGoogle ScholarCross RefCross Ref
  56. [56] Min Xiongkuo, Zhai Guangtao, Gu Ke, Fang Yuming, Yang Xiaokang, Wu Xiaolin, Zhou Jiantao, and Liu Xianming. 2016. Blind quality assessment of compressed images via pseudo structural similarity. In 2016 IEEE International Conference on Multimedia and Expo (ICME’16). IEEE, 16.Google ScholarGoogle ScholarCross RefCross Ref
  57. [57] Li Leida, Lin Weisi, and Zhu Hancheng. 2014. Learning structural regularity for evaluating blocking artifacts in JPEG images. IEEE Signal Process. Lett. 21, 8 (2014), 918922.Google ScholarGoogle ScholarCross RefCross Ref
  58. [58] Li Leida, Lin Weisi, and Zhu Hancheng. 2014. Learning structural regularity for evaluating blocking artifacts in JPEG images. IEEE Signal Process. Lett. 21, 8 (2014), 918922.Google ScholarGoogle ScholarCross RefCross Ref
  59. [59] Shi Jianbo et al. 1994. Good features to track. In 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 593600.Google ScholarGoogle Scholar
  60. [60] Vu Phong V. and Chandler Damon M.. 2013. A no-reference quality assessment algorithm for JPEG2000-compressed images based on local sharpness. In Image Quality and System Performance X, Vol. 8653. International Society for Optics and Photonics, 865302.Google ScholarGoogle Scholar
  61. [61] Vu Cuong T., Phan Thien D., and Chandler Damon M.. 2012. S3: A spectral and spatial measure of local perceived sharpness in natural images. IEEE Trans. Image Process. 21, 3 (2012), 934945. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. [62] Sheikh Hamid R., Sabir Muhammad F., and Bovik Alan C.. 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15, 11 (2006), 34403451. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. [63] Larson Eric Cooper and Chandler Damon Michael. 2010. Most apparent distortion: full-reference image quality assessment and the role of strategy. Journal of Electron. Imaging 19, 1 (2010), 011006.Google ScholarGoogle ScholarCross RefCross Ref
  64. [64] Ponomarenko Nikolay, Jin Lina, Ieremeiev Oleg, Lukin Vladimir, Egiazarian Karen, Astola Jaakko, Vozel Benoit, Chehdi Kacem, Carli Marco, Battisti Federica, et al. 2015. Image database TID2013: Peculiarities, results and perspectives. Signal Process. Image Comm. 30 (2015), 5777. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. [65] Rohaly Ann Marie, Libert John, Corriveau Philip, Webster Arthur, et al. 2000. Final report from the video quality experts group on the validation of objective models of video quality assessment. ITU-T Standards Contribution COM (2000), 980.Google ScholarGoogle Scholar
  66. [66] Wu Qingbo, Wang Zhou, and Li Hongliang. 2015. A highly efficient method for blind image quality assessment. In IEEE International Conference on Image Processing.339343.Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. [67] Xue Wufeng, Zhang Lei, and Mou Xuanqin. 2013. Learning without human scores for blind image quality assessment. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition.9951002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. [68] Zhang Lin, Zhang Lei, and Bovik Alan C.. 2015. A feature-enriched completely blind image quality evaluator. IEEE Trans. Image Process. 24, 8 (2015), 25792591.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Precise No-Reference Image Quality Evaluation Based on Distortion Identification

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 3s
      October 2021
      324 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3492435
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 15 November 2021
      • Revised: 1 May 2021
      • Accepted: 1 May 2021
      • Received: 1 November 2020
      Published in tomm Volume 17, Issue 3s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!