skip to main content
research-article

Robust Facial Image Super-Resolution by Kernel Locality-Constrained Coupled-Layer Regression

Authors Info & Claims
Published:09 June 2021Publication History
Skip Abstract Section

Abstract

Super-resolution methods for facial image via representation learning scheme have become very effective methods due to their efficiency. The key problem for the super-resolution of facial image is to reveal the latent relationship between the low-resolution (LR) and the corresponding high-resolution (HR) training patch pairs. To simultaneously utilize the contextual information of the target position and the manifold structure of the primitive HR space, in this work, we design a robust context-patch facial image super-resolution scheme via a kernel locality-constrained coupled-layer regression (KLC2LR) scheme to obtain the desired HR version from the acquired LR image. Here, KLC2LR proposes to acquire contextual surrounding patches to represent the target patch and adds an HR layer constraint to compensate the detail information. Additionally, KLC2LR desires to acquire more high-frequency information by searching for nearest neighbors in the HR sample space. We also utilize kernel function to map features in original low-dimensional space into a high-dimensional one to obtain potential nonlinear characteristics. Our compared experiments in the noisy and noiseless cases have verified that our suggested methodology performs better than many existing predominant facial image super-resolution methods.

References

  1. Simon Baker and Takeo Kanade. 2000. Hallucinating faces. In Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition. IEEE, 83–88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Simon Baker and Takeo Kanade. 2002. Limits on super-resolution and how to break them.IEEE Trans. Pattern Anal. Mach. Intell. 24, 9 (September 2002), 1167–1183. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Debarpan Bhattacharya, Sudip Misra, Nidhi Pathak, and Anandarup Mukherjee. 2020. IDeA: IoT-Based autonomous aerial demarcation and path planning for precision agriculture with UAVs.ACM Trans. Internet Things 1, 3 (June 2020), 1–21. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Ayan Chakrabarti, AN Rajagopalan, and Rama Chellappa. 2007. Super-resolution of face images using kernel PCA-based prior.IEEE Trans. Multimedia 9, 4 (June 2007), 888–892. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Berk Dogan, Shuhang Gu, and Radu Timofte. 2019. Exemplar guided face image super-resolution without facial landmarks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1–10.Google ScholarGoogle ScholarCross RefCross Ref
  6. Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2016. Image super-resolution using deep convolutional networks.IEEE Trans. Pattern Anal. Mach. Intell. 38, 2 (February 2016), 295–307. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Guangwei Gao and Jian Yang. 2014. A novel sparse representation based framework for face image super-resolution.Neurocomputing 134 (June 2014), 92–99. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Guangwei Gao, Jian Yang, Xiao-Yuan Jing, Fumin Shen, Wankou Yang, and Dong Yue. 2017. Learning robust and discriminative low-rank representations for face recognition with occlusion.Pattern Recogn. 66 (June 2017), 129–143. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Guangwei Gao, Yi Yu, Jin Xie, Jian Yang, Meng Yang, and Jian Zhang. 2020. Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution.Pattern Recogn. (July 2020), 107539.Google ScholarGoogle Scholar
  10. Guangwei Gao, Yi Yu, Meng Yang, Heyou Chang, Pu Huang, and Dong Yue. 2020. Cross-resolution face recognition with pose variations via multilayer locality-constrained structural orthogonal procrustes regression.Inf. Sci. 506 (January 2020), 19–36.Google ScholarGoogle Scholar
  11. Wen Gao, Bo Cao, Shiguang Shan, Xilin Chen, Delong Zhou, Xiaohua Zhang, and Debin Zhao. 2008. The CAS-PEAL large-scale Chinese face database and baseline evaluations.IEEE Trans. Syst., Man, Cybern. A, Syst., Humans 38, 1 (January 2008), 149–161. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Shiming Ge, Shengwei Zhao, Xindi Gao, and Jia Li. 2019. Fewer-shots and lower-resolutions: Towards ultrafast face recognition in the wild. In Proceedings of the 27th ACM International Conference on Multimedia. ACM, 229–237. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Bilal Ghanem, Paolo Rosso, and Francisco Rangel. 2020. An emotional analysis of false information in social media and news articles.ACM Trans. Internet Technol. 20, 2 (April 2020), 1–18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Klemen Grm, Walter J. Scheirer, and Vitomir Štruc. 2019. Face hallucination using cascaded super-resolution and identity priors.IEEE Trans. Image Proc. 29, 1 (October 2019), 2150–2165.Google ScholarGoogle Scholar
  15. Hu He and Lisimachos P. Kondi. 2006. An image super-resolution algorithm for different error levels per frame.IEEE Trans. Image Proc. 15, 3 (March 2006), 592–603. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Alain Hore and Djemel Ziou. 2010. Image quality metrics: PSNR vs. SSIM. In Proceedings of the IEEE International Conference on Pattern Recognition. IEEE, 2366–2369. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Xiao Hu, Peirong Ma, Zhuohao Mai, Shaohu Peng, Zhao Yang, and Li Wang. 2019. Face hallucination from low quality images using definition-scalable inference.Pattern Recogn. 94 (October 2019), 110–121.Google ScholarGoogle Scholar
  18. Kui Jia and Shaogang Gong. 2008. Generalized face super-resolution.IEEE Trans. Image Proc. 17, 6 (June 2008), 873–886. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Junjun Jiang, Ruimin Hu, Zhongyuan Wang, and Zhen Han. 2014. Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning.IEEE Trans. Image Process. 23, 10 (August 2014), 4220–4231.Google ScholarGoogle ScholarCross RefCross Ref
  20. Junjun Jiang, Ruimin Hu, Zhongyuan Wang, and Zhen Han. 2014. Noise robust face hallucination via locality-constrained representation.IEEE Trans. Multimedia 16, 5 (August 2014), 1268–1281.Google ScholarGoogle Scholar
  21. Junjun Jiang, Jiayi Ma, Chen Chen, Xinwei Jiang, and Zheng Wang. 2017. Noise robust face image super-resolution through smooth sparse representation.IEEE Trans. Cybern. 47, 11 (November 2017), 3991–4002.Google ScholarGoogle Scholar
  22. Junjun Jiang, Yi Yu, Suhua Tang, Jiayi Ma, Akiko Aizawa, and Kiyoharu Aizawa. 2019. Context-patch based face hallucination via thresholding locality-constrained representation and reproducing learning.IEEE Trans. Cybern. 50, 1 (January 2019), 324–337.Google ScholarGoogle Scholar
  23. Juncheng Li, Faming Fang, Kangfu Mei, and Guixu Zhang. 2018. Multi-scale residual network for image super-resolution. In Proceedings of the European Conference on Computer Vision. 517–532. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Mengyan Li, Yuechuan Sun, Zhaoyu Zhang, and Jun Yu. 2018. A coarse-to-fine face hallucination method by exploiting facial prior knowledge. In Proceedings of the 25th IEEE International Conference on Image Processing (ICIP ’18). IEEE, 61–65.Google ScholarGoogle ScholarCross RefCross Ref
  25. Xiaoming Li, Ming Liu, Yuting Ye, Wangmeng Zuo, Liang Lin, and Ruigang Yang. 2018. Learning warped guidance for blind face restoration. In Proceedings of the European Conference on Computer Vision (ECCV ’18). 272–289.Google ScholarGoogle ScholarCross RefCross Ref
  26. Zhouchen Lin and Heung-Yeung Shum. 2004. Fundamental limits of reconstruction-based superresolution algorithms under local translation.IEEE Trans. Pattern Anal. Mach. Intell. 26, 1 (January 2004), 83–97. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Tao Lu, Jiaming Wang, Junjun Jiang, and Yanduo Zhang. 2020. Global-local fusion network for face super-resolution.Neurocomputing 387 (April 2020), 309–320.Google ScholarGoogle Scholar
  28. Xiang Ma, Junping Zhang, and Chun Qi. 2010. Hallucinating face by position-patch.Pattern Recogn. 43, 6 (June 2010), 2224–2236. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Mohamed Naleer Haju Mohamed, Yao Lu, and Feng Lv. 2013. Effective two-step method for face hallucination based on sparse compensation on over-complete patches.IET Image Process. 7, 6 (August 2013), 624–632.Google ScholarGoogle Scholar
  30. Sivaram Prasad Mudunuri, Shashanka Venkataramanan, and Soma Biswas. 2019. Dictionary alignment with re-ranking for low-resolution NIR-VIS face recognition.IEEE Trans. Inf. Forensics Secur. 14, 4 (April 2019), 886–896.Google ScholarGoogle Scholar
  31. Sung Cheol Park, Min Kyu Park, and Moon Gi Kang. 2003. Super-resolution image reconstruction: A technical overview.IEEE Signal Process. Mag. 20, 3 (May 2003), 21–36.Google ScholarGoogle Scholar
  32. Shyam Singh Rajput, K. V. Arya, and Vinay Singh. 2018. Robust face super-resolution via iterative sparsity and locality-constrained representation.Inf. Sci. 463 (October 2018), 227–244.Google ScholarGoogle Scholar
  33. Ching-Ting Tu and Jang-Ren Luo. 2015. Robust face hallucination using ensemble of feature-based regression functions and classifiers.Image Vis. Comput. 44 (December 2015), 59–72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Chih-Yuan Yang, Sifei Liu, and Ming-Hsuan Yang. 2013. Structured face hallucination. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1099–1106. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Yi Yu, Suhua Tang, Francisco Raposo, and Lei Chen. 2019. Deep cross-modal correlation learning for audio and lyrics in music retrieval.ACM Trans. Multim. Comput. 15, 1 (February 2019), 1–16. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Robust Facial Image Super-Resolution by Kernel Locality-Constrained Coupled-Layer Regression

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!