Abstract
Since the neural network was introduced into the super-resolution (SR) field, many SR deep models have been proposed and have achieved excellent results. However, there are two main drawbacks: one is that the methods based on the best peak-signal-to-noise ratio (PSNR) do not have enough comfortable visual quality; the other is that although the SR models based on generative adversarial network (GAN) have satisfactory visual quality, the structure of the reconstructed image has apparent defects. Therefore, according to the characteristics that human eyes are sensitive to high-frequency components in images, this article proposes an improved image SRGAN model based on high-frequency information fusion (HIFGAN). It builds a feature extraction network for high-frequency information fusion by designing a lightweight spatial attention module and improving the network architecture of enhanced super-resolution GAN (ESRGAN). It makes the generator in the GAN network have better feature recovery ability, reduces the dependence of the later training on the decider and loss function, and makes the generated image structure more consistent with the real situation. In addition, we build a high-frequency loss function to optimize the training of the generator network. Detailed experimental results show that HIFGAN performs excellently in both objective criterion evaluation and subjective visual effect. Compared with the state-of-the-art GAN-based SR networks, the reconstructed image by our model is more precise and complete in texture details.
- [1] . 2014. Learning a deep convolutional network for image super-resolution. In European Conference on Computer Vision. Springer, Cham. 184–199. Retrieved Jan 10, 2020 from Google Scholar
Cross Ref
- [2] .. 2022. Lightweight image super-resolution with feature cheap convolution and attention mechanism. Cluster Computing 25 (2022), 1–16. Retrieved Jan 25, 2022 from Google Scholar
Digital Library
- [3] . 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 136–144. Retrieved Jan 10, 2013 from Google Scholar
Cross Ref
- [4] . 2021. An image super-resolution deep learning network based on multi-level feature extraction module. Multimedia Tools and Applications 80, 5 (2021), 7063–7075. Retrieved Dec 22, 2021 from Google Scholar
Digital Library
- [5] . 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4681–4690. Retrieved Jan 10, 2020 from Google Scholar
Cross Ref
- [6] . 2020. Generative adversarial networks. Communications of the ACM 63, 11 (2020), 139–144. Retrieved Dec 22, 2020 from Google Scholar
Cross Ref
- [7] . 2018. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 606–615. Retrieved Jan 10, 2020 from Google Scholar
Cross Ref
- [8] . 2014. Very deep convolutional networks for large-scale image recognition. arXiv.1409.1556. Retrieved Jan 10, 2020 from Google Scholar
Cross Ref
- [9] . 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778. Retrieved Dec 10, 2020 from Google Scholar
Cross Ref
- [10] . 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1874–1883. Retrieved July 15, 2020 from Google Scholar
Cross Ref
- [11] . 2018. ESRGAN: 2018. Enhanced super-resolution generative adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV’18). Retrieved July 15, 2020 from Google Scholar
Cross Ref
- [12] . 2020. Residual feature aggregation network for image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2359–2368. Retrieved Dec 15, 2020 from Google Scholar
Cross Ref
- [13] . 2015. Spatial transformer networks. Advances in Neural Information Processing Systems. Montreal: [s.n.], 2017–2025. Retrieved July 10, 2020 from Google Scholar
Cross Ref
- [14] . 2004. Adaptive image interpolation based on local gradient features. IEEE Signal Processing Letters 11, 3 (2004), 359–362. Retrieved Jan 18, 2020 from Google Scholar
Cross Ref
- [15] . 2017. Deep Laplacian pyramid networks for fast and accurate super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 624–632. Retrieved July 10, 2020 from Google Scholar
Cross Ref
- [16] . 2016. Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1637–1645. Retrieved July 12, 2020 from Google Scholar
Cross Ref
- [17] . 2017. Image super-resolution via deep recursive residual net- work. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 3147–3155. Retrieved July 10, 2020 from https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8099781.Google Scholar
Cross Ref
- [18] . 2017. MemNet: A persistent memory network for image restoration. In Proceedings of the IEEE International Conference on Computer Vision. 4539--4547. Retrieved July 12, 2020 from Google Scholar
Cross Ref
- [19] . 2018. Two-stage convolutional network for image super-resolution. In ICPR. 2670–2675. Retrieved July 12, 2020 from Google Scholar
Cross Ref
- [20] . 2019. Deep back-projection networks for single image super-resolution. arXiv:1904.05677. Retrieved July 15, 2020 from Google Scholar
Cross Ref
- [21] . 2018. Residual dense network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2472–2481. Retrieved Dec 20, 2021 from Google Scholar
Cross Ref
- [22] . 2021. Image super-resolution based on deep neural network of multiple attention mechanism. Journal of Visual Communication and Image Representation 75 (2021), 103019. Retrieved July 16, 2020 from Google Scholar
Cross Ref
- [23] . 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. Retrieved July 16, 2020 from http://www.bmva.org/bmvc/2012/BMVC/paper135/abstract135.pdf.Google Scholar
- [24] . 2010. On Single Image Scale-Up Using Sparse Representations. International Conference on Curves and Surfaces. Springer, Berlin, 711–730. Retrieved July 16, 2020 from Google Scholar
Digital Library
- [25] . 2010. Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 5 (2010), 898–916. Retrieved July 16, 2020 from Google Scholar
Digital Library
- [26] . 2015. Single image super-resolution from transformed self-exemplars. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5197–5206. Google Scholar
Cross Ref
- [27] . 2017. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications 76, 20 (2017), 21811–21838. Retrieved July 16, 2020 from Google Scholar
Digital Library
- [28] . 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 586–595. Retrieved July 18, 2020 from Google Scholar
Cross Ref
- [29] . 2018. The perception-distortion tradeoff. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6228–6237. Retrieved July 18, 2020 from Google Scholar
Cross Ref
- [30] . 2020. Multi-attention based ultra lightweight image super-resolution. ECCV 2020 Workshops. Retrieved Dec 15, 2020 from Google Scholar
Digital Library
- [31] . 2021. Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data. Retrieved Jan 10, 2022 from Google Scholar
Cross Ref
- [32] . 2020. Structure-preserving super resolution with gradient guidance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7769–7778. Retrieved Jan 10, 2022 from https://ieeexplore.ieee.org/document/7938379.Google Scholar
Cross Ref
Index Terms
HIFGAN: A High-Frequency Information-Based Generative Adversarial Network for Image Super-Resolution
Recommendations
Spatial-Spectral Deep Residual Network for Hyperspectral Image Super-Resolution
AbstractRecently, single hyperspectral image super-resolution (SR) methods based on deep learning have been extensively studied. However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-...
Dilated projection correction network based on autoencoder for hyperspectral image super-resolution
AbstractThis paper focuses on improving the spatial resolution of the hyperspectral image (HSI) by taking the prior information into consideration. In recent years, single HSI super-resolution methods based on deep learning have achieved good ...
Highlights- We transform the single HSI SR problem into the abundance domain.
- We deploy a ...
Super-Resolution Image Reconstruction Using Wavelet Based Patch and Discrete Wavelet Transform
This paper proposes a novel method that combines the discrete wavelet transform (DWT) and example-based technique to reconstruct a high-resolution from a low-resolution image. Although previous interpolation- and example-based methods consider the ...






Comments