Abstract
Low-light image enhancement algorithms (LIEA) can light up images captured in dark or back-lighting conditions. However, LIEA may introduce various distortions such as structure damage, color shift, and noise into the enhanced images. Despite various LIEAs proposed in the literature, few efforts have been made to study the quality evaluation of low-light enhancement. In this article, we make one of the first attempts to investigate the quality assessment problem of low-light image enhancement. To facilitate the study of objective image quality assessment (IQA), we first build a large-scale low-light image enhancement quality (LIEQ) database. The LIEQ database includes 1,000 light-enhanced images, which are generated from 100 low-light images using 10 LIEAs. Rather than evaluating the quality of light-enhanced images directly, which is more difficult, we propose to use the multi-exposure fused (MEF) image and stack-based high dynamic range (HDR) image as a reference and evaluate the quality of low-light enhancement following a full-reference (FR) quality assessment routine. We observe that distortions introduced in low-light enhancement are significantly different from distortions considered in traditional image IQA databases that are well-studied, and the current state-of-the-art FR IQA models are also not suitable for evaluating their quality. Therefore, we propose a new FR low-light image enhancement quality assessment (LIEQA) index by evaluating the image quality from four aspects: luminance enhancement, color rendition, noise evaluation, and structure preserving, which have captured the most key aspects of low-light enhancement. Experimental results on the LIEQ database show that the proposed LIEQA index outperforms the state-of-the-art FR IQA models. LIEQA can act as an evaluator for various low-light enhancement algorithms and systems. To the best of our knowledge, this article is the first of its kind comprehensive low-light image enhancement quality assessment study.
- [1] . 2003. Formula for the contrast sensitivity of the human eye. In Image Quality and System Performance, Vol. 5294. International Society for Optics and Photonics, 231–238.Google Scholar
- [2] . 2014. Special section on graphics interface: ExpoBlend: Information preserving exposure blending based on normalized log-domain entropy. Comput. Graph. 39 (2014), 12–23. Google Scholar
Digital Library
- [3] . 2002. Methodology for the subjective assessment of the quality of television pictures. International Telecommunication Union.Google Scholar
- [4] . 2018. Learning a deep single image contrast enhancer from multi-exposure images. IEEE Trans. Image Process. 27, 4 (2018), 2049–2062.Google Scholar
Cross Ref
- [5] . 1987. A computational approach to edge detection. In Readings in Computer Vision. Elsevier, 184–203. Google Scholar
Digital Library
- [6] . 2011. Contextual and variational contrast enhancement. IEEE Trans. Image Process. 20, 12 (2011), 3431–3441. Google Scholar
Digital Library
- [7] . 1974. Pattern classification and scene analysis. IEEE Trans. Automat. Control 19, 4 (1974), 462–463.Google Scholar
Cross Ref
- [8] . 2007. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16 (2007), 2080–2095. Google Scholar
Digital Library
- [9] . 2011. Fast efficient algorithm for enhancement of low lighting video. In Proceedings of the IEEE International Conference on Multimedia and Expo. IEEE, 1–6. Google Scholar
Digital Library
- [10] . 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Comput. Graph. Forum 22 (2003), 419–426.Google Scholar
Cross Ref
- [11] . 2015. A probabilistic method for image enhancement with simultaneous illumination and reflectance estimation. IEEE Trans. Image Process. 24, 12 (2015), 4965–4977.Google Scholar
Digital Library
- [12] . 2016. A fusion-based enhancing method for weakly illuminated images. Sig. Process. 129 (2016), 82–96. Google Scholar
Digital Library
- [13] . 2016. A weighted variational model for simultaneous reflectance and illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2782–2790.Google Scholar
Cross Ref
- [14] . 2020. Extended geometric models for stereoscopic 3D with vertical screen disparity. Displays 65 (2020), 101972.Google Scholar
Cross Ref
- [15] . 2016. No-reference quality metric of contrast-distorted images based on information maximization. IEEE Trans. Cybern. 47, 12 (2016), 4559–4565.Google Scholar
Cross Ref
- [16] . 2016. Blind quality assessment of tone-mapped images via analysis of information, naturalness, and structure. IEEE Trans. Multimedia 18, 3 (2016), 432–443. Google Scholar
Digital Library
- [17] . 2014. Using free energy principle for blind image quality assessment. IEEE Trans. Multimedia 17, 1 (2014), 50–63.Google Scholar
Cross Ref
- [18] . 2017. LIME: Low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 26, 2 (2017), 982–993. Google Scholar
Digital Library
- [19] . 2013. HDR deghosting: How to deal with saturation? In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1163–1170. Google Scholar
Digital Library
- [20] . 2007. Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Trans. Consum. Electron. 53, 4 (2007), 1752–1758. Google Scholar
Digital Library
- [21] . 1995. Machine Vision. Vol. 5. McGraw-Hill New York. Google Scholar
Digital Library
- [22] . 2019. BLIQUE-TMI: Blind quality evaluator for tone-mapped images based on local and global feature analyses. IEEE Trans. Circ. Syst. Vid. Technol. 29, 2 (2019), 323–335. Google Scholar
Digital Library
- [23] . 1997. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Process. 6, 7 (1997), 965–976. Google Scholar
Digital Library
- [24] . 1997. Properties and performance of a center/surround retinex. IEEE Trans. Image Process. 6, 3 (1997), 451–462. Google Scholar
Digital Library
- [25] . 2017. Multi-scale exposure fusion via gradient domain guided image filtering. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). 1105–1110.Google Scholar
Cross Ref
- [26] . 2017. No-reference quality assessment of tone-mapped HDR pictures. IEEE Trans. Image Process. 26, 6 (2017), 2957–2971. Google Scholar
Digital Library
- [27] . 2010. Most apparent distortion: Full-reference image quality assessment and the role of strategy. J. Electron. Imag. 19, 1 (2010), 011006.Google Scholar
Cross Ref
- [28] . 2013. Contrast enhancement based on layered difference representation of 2D histograms. IEEE Trans. Image Process. 22, 12 (2013), 5372–5384. Google Scholar
Digital Library
- [29] . 2018. Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans. Image Process. 27, 6 (2018), 2828–2841.Google Scholar
Cross Ref
- [30] . 2016. Blind image quality assessment using statistical structural and luminance features. IEEE Trans. Multimedia 18, 12 (2016), 2457–2469. Google Scholar
Digital Library
- [31] . 2013. Image fusion with guided filtering. IEEE Trans. Image Process. 22, 7 (2013), 2864–2875.Google Scholar
Cross Ref
- [32] . 2011. Image quality assessment based on gradient similarity. IEEE Trans. Image Process. 21, 4 (2011), 1500–1512. Google Scholar
Digital Library
- [33] . 2017. LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recog. 61 (2017), 650–662. Google Scholar
Digital Library
- [34] . 2018. MBLLEN: Low-light image/video enhancement using CNNs. In Proceedings of the British Machine Vision Conference.Google Scholar
- [35] . 2008. Nonlinear image representation using divisive normalization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 1–8.Google Scholar
- [36] . 2017. Robust multi-exposure image fusion: A structural patch decomposition approach. IEEE Trans. Image Process. 26, 5 (2017), 2519–2532. Google Scholar
Digital Library
- [37] . 2019. A multi-attribute blind quality evaluator for tone-mapped images. IEEE Trans. Multimedia (2019), 1–1.Google Scholar
- [38] . 2005. A perceptual framework for contrast processing of high dynamic range images. ACM Transactions on Applied Perception (TAP) 3, 3 (2006), 286–308. Google Scholar
Digital Library
- [39] . 2009. Exposure fusion: A simple and practical alternative to high dynamic range photography. Comput. Graph. For. 28, 1 (2009), 161–171.Google Scholar
- [40] . 2017. Blind quality assessment based on pseudo-reference image. IEEE Trans. Multimedia 20, 8 (2017), 2049–2062.Google Scholar
Cross Ref
- [41] . 2017. Unified blind quality assessment of compressed natural, graphic, and screen content images. IEEE Trans. Image Process. 26, 11 (2017), 5462–5474.Google Scholar
Digital Library
- [42] . 2018. Blind image quality estimation via distortion aggravation. IEEE Trans. Broadcast. 64, 2 (2018), 508–517.Google Scholar
Cross Ref
- [43] . 2019. Quality evaluation of image dehazing methods using synthetic hazy images. IEEE Trans. Multimedia (2019).Google Scholar
Cross Ref
- [44] . 2020. Study of subjective and objective quality assessment of audio-visual signals. IEEE Trans. Image Process. 29 (2020), 6054–6068.Google Scholar
Cross Ref
- [45] . 2020. A multimodal saliency model for videos with high audio-visual correspondence. IEEE Trans. Image Process. 29 (2020), 3805–3819.Google Scholar
Digital Library
- [46] . 2020. A metric for light field reconstruction, compression, and display quality evaluation. IEEE Trans. Image Process. 29 (2020), 3790–3804.Google Scholar
Digital Library
- [47] . 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21 (2012), 4695–4708. Google Scholar
Digital Library
- [48] . 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 12 (2012), 4695–4708. Google Scholar
Digital Library
- [49] . 2012. Making a completely blind image quality analyzer. IEEE Sig. Process. Lett. 20, 3 (2012), 209–212.Google Scholar
Cross Ref
- [50] . 2011. Edge detection techniques for image segmentation. Int. J. Comput. Sci. Inf. Technol. 3, 6 (2011), 259.Google Scholar
Cross Ref
- [51] . 2015. FSITM: A feature similarity index for tone-mapped images. IEEE Sig. Process. Lett. 22, 8 (2015), 1026–1029.Google Scholar
Cross Ref
- [52] . 2015. Robust high dynamic range imaging by rank minimization. IEEE Trans. Pattern Anal. Mach. Intell. 37, 6 (2015), 1219–1232.Google Scholar
Digital Library
- [53] . 2002. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell.7 (2002), 971–987. Google Scholar
Digital Library
- [54] . 2020. Commercially available HDR Processing Software. Retrieved from http://www.hdrsoft.com/.Google Scholar
- [55] . 2015. Image database TID2013: Peculiarities, results and perspectives. Sig. Process.: Image Commun. 30 (2015), 57–77. Google Scholar
Digital Library
- [56] . 2009. TID2008—A database for evaluation of full-reference visual quality assessment metrics. Advanc. Mod. Radioelectron. 10, 4 (2009), 30–45.Google Scholar
- [57] . 2009. Bilateral filter based compositing for variable exposure photography. In Proceedings of the Eurographics. 1–4.Google Scholar
- [58] . 2002. Photographic tone reproduction for digital images. In Proceedings of the 29th annual Conference on Computer graphics and Interactive Techniques. 267–276. Google Scholar
Digital Library
- [59] . 1963. Machine Perception of Three-dimensional Solids. Ph.D. Dissertation. Massachusetts Institute of Technology.Google Scholar
- [60] . 2000. The earth mover’s distance as a metric for image retrieval. Int. J. Comput. Vis. 40, 2 (2000), 99–121. Google Scholar
Digital Library
- [61] . 2012. Robust patch-based HDR reconstruction of dynamic scenes. ACM Trans. Graph. 31, 6 (2012). Google Scholar
Digital Library
- [62] . 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15, 11 (2006), 3440–3451. Google Scholar
Digital Library
- [63] . 2014. Exposure fusion using boosting Laplacian pyramid. IEEE Trans. Syst., Man, Cybern. 44, 9 (2014), 1579–1590.Google Scholar
- [64] . 2017. MSR-net: Low-light image enhancement using deep convolutional network. arXiv:1711.02488 (2017).Google Scholar
- [65] . 2011. Generalized random walks for fusion of multi-exposure images. IEEE Trans. Image Process. 20, 12 (2011), 3634–3646. Google Scholar
Digital Library
- [66] . 2018. Luminance enhancement and detail preservation of images and videos adapted to ambient illumination. IEEE Trans. Image Process. 27 (2018), 4901–4915.Google Scholar
Cross Ref
- [67] . 2018. A large-scale compressed 360-degree spherical image database: From subjective quality evaluation to objective model comparison. In Proceedings of the IEEE 20th International Workshop on Multimedia Signal Processing (MMSP). IEEE, 1–6.Google Scholar
Cross Ref
- [68] . 2017. CVIQD: Subjective quality evaluation of compressed virtual reality images. In Proceedings of the IEEE International Conference on Image Processing (ICIP). IEEE, 3450–3454.Google Scholar
Cross Ref
- [69] . 2019. MC360IQA: A multi-channel CNN for blind 360-degree image quality assessment. IEEE J. Select. Topics Sig. Process. 14, 1 (2019), 64–77.Google Scholar
Cross Ref
- [70] . 2020. Dynamic backlight scaling considering ambient luminance for mobile videos on LCD displays. IEEE Trans. Mob. Comput. (2020).Google Scholar
- [71] . 2017. Dynamic backlight scaling considering ambient luminance for mobile energy saving. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). IEEE, 25–30.Google Scholar
Cross Ref
- [72] . 2019. Underexposed photo enhancement using deep illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6849–6857.Google Scholar
Cross Ref
- [73] . 2013. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process. 22, 9 (2013), 3538–3548. Google Scholar
Digital Library
- [74] . 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600–612. Google Scholar
Digital Library
- [75] . 2003. Multiscale structural similarity for image quality assessment. In Proceedings of the 37th Asilomar Conference on Signals, Systems & Computers. IEEE, 1398–1402.Google Scholar
Cross Ref
- [76] . 2015. A highly efficient method for blind image quality assessment. In Proceedings of the IEEE International Conference on Image Processing (ICIP). IEEE, 339–343.Google Scholar
Digital Library
- [77] . 2019. Blind night-time image quality assessment: Subjective and objective approaches. IEEE Trans. Multimedia 22, 5 (2019), 1259–1272.Google Scholar
Cross Ref
- [78] . 2013. Gradient magnitude similarity deviation: A highly efficient perceptual image quality index. IEEE Trans. Image Process. 23, 2 (2013), 684–695. Google Scholar
Digital Library
- [79] . 2019. Deep objective quality assessment driven single image super-resolution. IEEE Trans. Multimedia 21 (2019), 2957–2971.Google Scholar
Cross Ref
- [80] . 2018. CrowdDBS: A crowdsourced brightness scaling optimization for display energy reduction in mobile video. IEEE Trans. Mob. Comput. 17, 11 (2018), 2536–2549.Google Scholar
Digital Library
- [81] . 2013. Objective quality assessment of tone-mapped images. IEEE Trans. Image Process. 22, 2 (2013), 657–667. Google Scholar
Digital Library
- [82] . 2017. A new image contrast enhancement algorithm using exposure fusion framework. In Proceedings of the International Conference on Computer Analysis of Images and Patterns. Springer, 36–46.Google Scholar
Cross Ref
- [83] . 2017. A new low-light image enhancement algorithm using camera response model. In Proceedings of the IEEE International Conference on Computer Vision. 3015–3022.Google Scholar
Cross Ref
- [84] . 2020. Perceptual image quality assessment: A survey. Sci. China Series F: Inf. Sci. 63, 11 (2020).Google Scholar
- [85] . 2019. Free-energy principle inspired visual quality assessment: An overview. Dig. Sig. Process. 91 (2019), 11–20.Google Scholar
Cross Ref
- [86] . 2011. A psychovisual quality metric in free-energy principle. IEEE Trans. Image Process. 21, 1 (2011), 41–52. Google Scholar
Digital Library
- [87] . 2014. VSI: A visual saliency-induced index for perceptual image quality assessment. IEEE Trans. Image Process. 23, 10 (2014), 4270–4281.Google Scholar
Cross Ref
- [88] . 2011. FSIM: A feature similarity index for image quality assessment. IEEE Trans. Image Process. 20, 8 (2011), 2378–2386. Google Scholar
Digital Library
- [89] . 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 586–595.Google Scholar
Cross Ref
- [90] . 2012. Gradient-directed multiexposure composition. IEEE Trans. Image Process. 21, 4 (2012), 2318–2323. Google Scholar
Digital Library
- [91] . 2020. Optical flow in the dark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
Cross Ref
Index Terms
Perceptual Quality Assessment of Low-light Image Enhancement
Recommendations
A Low-light Image Enhancement Method Balancing Brightness Boosting and Noise Smoothing
CSAI '22: Proceedings of the 2022 6th International Conference on Computer Science and Artificial IntelligenceImages captured in low-light or uneven lighting environments need to use low-light image enhancement techniques to boost brightness and contrast for high-quality improved results. Some existing low-light image enhancement methods over-enhance the image ...
Feature spatial pyramid network for low-light image enhancement
AbstractLow-light images usually contain high noise and low contrast. This brings bad visual feelings and hinders subsequent computer vision work. At present, many algorithms have been proposed to enhance low-light images. However, the existing methods ...
Low-light image enhancement using inverted image normalized by atmospheric light
Highlights- This paper presents a novel low-light image enhancement algorithm using the inverted image normalized by atmospheric light.
AbstractImages captured under low-light conditions are characterized by problems, including low brightness, low contrast, and color distortion. Low-light image enhancement, aims to enhance visual effects for the benefit of subsequent image ...






Comments