Abstract
Face images are typically a key component in the fields of security and criminal investigation. However, due to lighting and shooting angles, faces taken under low-light conditions are often difficult to recognize. Face super-resolution (FSR) technology can restore high-resolution faces based on low-resolution inputs. However, existing face super-resolution methods typically rely on prior knowledge of inaccurate faces estimated from low-resolution images. Faces restored by low-light inputs may suffer from problems such as low brightness and many missing details. In this article, we proposed an Illumination-Enhanced Face Super-Resolution (IEFSR) model that can progressively super-resolve low-light faces of 32 × 32 pixels by an upscaling factor of 8. While reconstructing the low-light low-resolution face into a clear and high-quality face, we introduce a coarse low-resolution (LR) restoration network to recover the LR face details hidden in the dark. In the generator, we use a series of style blocks with noise to make the generated faces appear to have a more realistic visual aesthetic. Additionally, we introduce spectrum normalization in the discriminator to improve training stability. Extensive experimental evaluations show that the proposed IEFSR yields visually and metrically more attractive results than existing state-of-the-art FSR methods.
- [1] . 2020. Introduction to the special issue on smart communications and networking for future video surveillance. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 58 (Jul. 2020), 1–2.
DOI: Google ScholarDigital Library
- [2] . 2011. Traffic prediction and QoS transmission of real-time live VBR videos in WLANs. ACM Transactions on Multimedia Computing, Communications, and Applications 7, 36 (Nov. 2011), 1–21.
DOI: Google ScholarDigital Library
- [3] . 2020. A decision support system with intelligent recommendation for multi-disciplinary medical treatment. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 33 (Apr. 2020), 1–23.
DOI: Google ScholarDigital Library
- [4] . 1986. Understanding face recognition. British Journal of Psychology. 77, 3 (Aug. 1986), 305–327.
DOI: Google ScholarCross Ref
- [5] . 1996. Virage image search engine: an open framework for image management. In Proceedings of the SPIE. Storage and Retrieval for Still Image and Video Databases, Vol. IV, International Society for Optics and Photonics, 76–87.
DOI: Google ScholarCross Ref
- [6] . 1991. Improving resolution by image registration. Graphical Models and Image Processing. 53, 3 (May. 1991), 231–239.
DOI: Google ScholarDigital Library
- [7] . 2008. Generalized face super-resolution. IEEE Transactions on Image Processing 17, 6 (Jun. 2008), 873–886.
DOI: Google ScholarDigital Library
- [8] Hua Huang, Huiting He, and Xin Fan. 2010. Super-resolution of human face image using canonical correlation analysis. Pattern Recognition 43, 7 (Feb. 2010), 2532--2543.
DOI: Google ScholarDigital Library
- [9] . 2016. Deep cascaded bi-network for face hallucination. In Proceedings of the European Conference on Computer Vision. 614–630.
DOI: Google ScholarCross Ref
- [10] . 2018. Face super-resolution guided by facial component heatmaps. In Proceedings of the European Conference on Computer Vision. 217–233.
DOI: Google ScholarCross Ref
- [11] . 2018. FSRNet: End-to-end learning face super-resolution with facial priors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2492–2501.
DOI: Google ScholarCross Ref
- [12] . 2018. MBLLEN: Low-light image/video enhancement using CNNs. In Proceedings of the British Machine Vision Conference. 220–233.Google Scholar
- [13] . 2018. Spectral normalization for generative adversarial networks. In Proceedings of the International Conference on Learning Representation. 1–26.Google Scholar
- [14] . 2018. Super-resolving very low-resolution face images with supplementary attributes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 908–917.
DOI: Google ScholarCross Ref
- [15] . 2018. Attribute augmented convolutional neural network for face hallucination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 721–729.
DOI: Google ScholarCross Ref
- [16] . 2014. Generative adversarial networks. Advances in Neural Information Processing Systems 27 (2014), 2672–2680.Google Scholar
Digital Library
- [17] . 2017. Photo-Realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4681–4690.
DOI: Google ScholarCross Ref
- [18] . 2018. ESRGAN: Enhanced super-resolution generative adversarial networks. In Proceedings of the European Conference on Computer Vision. 1–16.
DOI: Google ScholarDigital Library
- [19] . 2018. Super-FAN: Integrated facial landmark localization and super-resolution of real-world low-resolution faces in arbitrary poses with GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 109–117.
DOI: Google ScholarCross Ref
- [20] . 2020. Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5569–5578.
DOI: Google ScholarCross Ref
- [21] . 2017. High-resolution image synthesis and semantic manipulation with conditional GANs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8798–8807.Google Scholar
- [22] . 2018. Progressive growing of GANs for improved quality, stability, and variation. In Proceedings of the International Conference on Learning Representations. 1–26.Google Scholar
- [23] . 2019. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4401–4410.
DOI: Google ScholarCross Ref
- [24] . 2020. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8110–8119.Google Scholar
Cross Ref
- [25] . 2021. Learning spatial attention for face super-resolution. IEEE Transactions on Image Processing 30 (2021), 1219–1231.
DOI: Google ScholarDigital Library
- [26] . 1998. Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms. Journal of Digital Imaging 11, 4 (Nov. 1998), 193–200.Google Scholar
- [27] . 2011. Fast efficient algorithm for enhancement of low lighting video. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1–6.
DOI: Google ScholarDigital Library
- [28] . 1977. The retinex theory of color vision. Scientific American. 237, 6 (Dec. 1977), 108–128.Google Scholar
Cross Ref
- [29] . 1997. Properties and performance of a center/surround retinex. IEEE Transactions on Image Processing 6, 3 (Mar. 1997), 451–462.
DOI: Google ScholarDigital Library
- [30] . 1997. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing. 6, 7 (Jul. 1997), 965–976.
DOI: Google ScholarDigital Library
- [31] . 2016. LIME: Low-light image enhancement via illumination map estimation. IEEE Transactions on Image Processing 26, 2 (Feb. 2016), 982–993.
DOI: Google ScholarDigital Library
- [32] . 2017. LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognition 61 (2017), 650–662.
DOI: Google ScholarDigital Library
- [33] . 2018. Learning to see in the dark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3291–3300.
DOI: Google ScholarCross Ref
- [34] . 2017. LLCNN: A convolutional neural network for low-light image enhancement. In Proceedings of theIEEE Visual Communications and Image Processing. 1–4.
DOI: Google ScholarCross Ref
- [35] . 2018. Deep retinex decomposition for low-light enhancement. In Proceedings of the British Machine Vision Conference. 127–136.Google Scholar
- [36] . 2019. Underexposed photo enhancement using deep illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6849–6857.
DOI: Google ScholarCross Ref
- [37] . 2017. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 6626–6637.Google Scholar
- [38] . 2014. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3rd International Conference on Learning Representations. 1–14.Google Scholar
- [39] Jongyeon Lee. 2020. Face super-resolution with styled feature channel attention. Hongik University Graduation Project. Retrieved Sept. 27, 2020, from https://github.com/tinnunculus/Styled-Attention-Face-Super-Resolution.Google Scholar
- [40] . 2018. Image super-resolution using very deep residual channel attention networks. In Proceedings of the European Conference on Computer Vision. 286–301.
DOI: Google ScholarCross Ref
- [41] . 2019. Self-attention generative adversarial networks. In Proceedings of the International Conference on Machine Learning. 7354–7363.Google Scholar
- [42] . 2017. How far are we from solving the 2D & 3D face alignment problem? (and a dataset of 230,000 3D facial landmarks). In Proceedings of the IEEE International Conference on Computer Vision. 1021–1030.
DOI: Google ScholarCross Ref
- [43] . 2008. Scope of validity of PSNR in image/video quality assessment. Electronics Letters 44, 13 (Feb. 2008), 800–801.
DOI: Google ScholarCross Ref
- [44] . 2004. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (Apr. 2004), 600–612.
DOI: Google ScholarDigital Library
- [45] . 2018. Deep back-projection networks for super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1664–1673.
DOI: Google ScholarCross Ref
- [46] . 2021. Towards real-world blind face restoration with generative facial prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 9164--9174.
DOI: Google ScholarCross Ref
- [47] . 2021. Urban perception: Sensing cities via a deep interactive multi-task learning framework. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1 (Jan. 2021), Article 13, 20 pages.
DOI: Google ScholarDigital Library
- [48] . 2017. VoxCeleb: A large-scale speaker identification dataset. In Proceedings of the Conference of the International Speech Communication Association. 1–6.Google Scholar
Cross Ref
- [49] . 2017. Deep high dynamic range imaging of dynamic scenes. ACM Transactions on Graphics 36, 4 (Jul. 2017), Article 144, 12 pages.
DOI: Google ScholarDigital Library
Index Terms
Deep Illumination-Enhanced Face Super-Resolution Network for Low-Light Images
Recommendations
Eigentransformation-based face super-resolution in the wavelet domain
In this paper, we propose a wavelet-based eigentransformation method for human face hallucination. Our algorithm uses the wavelet transform to decompose interpolated low-resolution (LR) images in the wavelet domain to obtain high-frequency information ...
Learning the Relationship Between High and Low Resolution Images in Kernel Space for Face Super Resolution
ICPR '10: Proceedings of the 2010 20th International Conference on Pattern RecognitionThis paper proposes a new nonlinear face super resolution algorithm to address an important issue in face recognition from surveillance video namely, recognition of low resolution face image with nonlinear variations. The proposed method learns the ...
Feature super-resolution based Facial Expression Recognition for multi-scale low-resolution images
AbstractFacial Expression Recognition (FER) for various low-resolution images is an important task and need in applications of analyzing crowd scenes (station, classroom, etc.). Due to the discriminative feature loss caused by reduced ...






Comments