Abstract
In recent years, thanks to the inherent powerful feature representation and learning abilities of the convolutional neural network (CNN), deep CNN-steered single image super-resolution approaches have achieved remarkable performance improvements. However, these methods are often accompanied by large consumption of computing and memory resources, which is difficult to be adopted in real-world application scenes. To handle this issue, we design an efficient Feature De-redundancy and Self-calibration Super-resolution network (FDSCSR). In particular, a Feature De-redundancy and Self-calibration Block (FDSCB) is proposed to reduce the repetitive feature information extracted by the model and further enhance the efficiency of the model. Then, based on FDSCB, a Local Feature Fusion Module is presented to elaborately utilize and fuse the feature information extracted by each FDSCB. Abundant experiments on benchmarks have demonstrated that our FDSCSR achieves superior performance with relatively less computational consumption and storage resource than other state-of-the-art approaches. The code is available at https://github.com/IVIPLab/FDSCSR.
- [1] . 2017. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17). 126–135.Google Scholar
Cross Ref
- [2] . 2018. Fast, accurate, and lightweight super-resolution with cascading residual network. In Proceedings of the European Conference on Computer Vision (ECCV’18). 252–268.Google Scholar
Digital Library
- [3] . 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In Proceedings of the British Machine Vision Conference (BMVC’12). 135.1–135.10.Google Scholar
Cross Ref
- [4] . 2022. Real-world single image super-resolution: A brief review. Inf. Fusion 79 (2022), 124–145.Google Scholar
Digital Library
- [5] . 2022. Activating more pixels in image super-resolution transformer. arXiv:2205.04437. Retrieved from https://arxiv.org/abs/2005.04437.Google Scholar
- [6] . 2019. Second-order attention network for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19). 11065–11074.Google Scholar
Cross Ref
- [7] . 2014. Learning a deep convolutional network for image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV’14). 184–199.Google Scholar
Cross Ref
- [8] . 2022. Feature distillation interaction weighting network for lightweight image super-resolution. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’22). 661–669.Google Scholar
Cross Ref
- [9] . 2022. Lightweight bimodal network for single-image super-resolution via symmetric CNN and recursive transformer. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’22). 913–919.Google Scholar
Cross Ref
- [10] . 2022. Context-patch representation learning with adaptive neighbor embedding for robust face image super-resolution. IEEE Trans Multimedia (2022).Google Scholar
- [11] . 2021. Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution. Pattern Recogn. 110 (2021), 107539.Google Scholar
Cross Ref
- [12] . 2020. Closed-loop matters: Dual regression networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 5407–5416.Google Scholar
Cross Ref
- [13] . 2020. Ghostnet: More features from cheap operations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 1580–1589.Google Scholar
Cross Ref
- [14] . 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 770–778.Google Scholar
Cross Ref
- [15] . 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). 7132–7141.Google Scholar
Cross Ref
- [16] . 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 4700–4708.Google Scholar
Cross Ref
- [17] . 2015. Single image super-resolution from transformed self-exemplars. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 5197–5206.Google Scholar
Cross Ref
- [18] . 2019. Lightweight image super-resolution with information multi-distillation network. In Proceedings of the ACM International Conference on Multimedia (ACMMM’19). 2024–2032.Google Scholar
Digital Library
- [19] . 2018. Fast and accurate single image super-resolution via information distillation network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). 723–731.Google Scholar
Cross Ref
- [20] . 2020. Hierarchical dense recursive network for image super-resolution. Pattern Recogn. 107 (2020), 107475.Google Scholar
Cross Ref
- [21] . 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 1646–1654.Google Scholar
Cross Ref
- [22] . 2016. Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 1637–1645.Google Scholar
Cross Ref
- [23] . 2018. Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans. Pattern Anal. Mach. Intell. 41, 11 (2018), 2599–2613.Google Scholar
Cross Ref
- [24] . 2020. Madnet: A fast and lightweight network for single-image super resolution. IEEE Trans. Cybernet. 51, 3 (2020), 1443–1453.Google Scholar
Cross Ref
- [25] . 2018. Multi-scale residual network for image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV’18). 517–532.Google Scholar
Digital Library
- [26] . 2021. From beginner to master: A survey for deep learning-based single-image super-resolution. arXiv:2109.14335. Retrieved from https://arixv.org/abs/2109.14335.Google Scholar
- [27] . 2022. Cross-receptive focused inference network for lightweight image super-resolution. arXiv:2207.02796. Retrieved from https://arxiv.org/abs/2207.02796.Google Scholar
- [28] . 2021. Lapar: Linearly-assembled pixel-adaptive regression network for single image super-resolution and beyond. arXiv:2105.10422. Retrieved from https://arxiv.org/abs/2105.10422.Google Scholar
- [29] . 2021. Lightweight single image super-resolution with dense connection distillation network. ACM Trans. Multimedia Comput. Commun. Appl. 17, 1s (2021), 1–17.Google Scholar
Digital Library
- [30] . 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17). 136–144.Google Scholar
Cross Ref
- [31] . 2020. Residual feature distillation network for lightweight image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV’20). Springer, 41–55.Google Scholar
Digital Library
- [32] . 2020. Improving convolutional networks with self-calibrated convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 10096–10105.Google Scholar
Cross Ref
- [33] . 2022. Transformer for single image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22) Workshops. 457–466.Google Scholar
Cross Ref
- [34] . 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’01). 416–423.Google Scholar
Cross Ref
- [35] . 2017. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools Appl. 76, 20 (2017), 21811–21838.Google Scholar
Digital Library
- [36] . 2020. Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 5690–5699.Google Scholar
Cross Ref
- [37] . 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 1874–1883.Google Scholar
Cross Ref
- [38] . 2017. Image super-resolution via deep recursive residual network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 3147–3155.Google Scholar
Cross Ref
- [39] . 2017. Memnet: A persistent memory network for image restoration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). 4539–4547.Google Scholar
Cross Ref
- [40] . 2020. Coarse-to-fine CNN for image super-resolution. IEEE Trans Multimedia 23 (2020), 1489–1502.Google Scholar
Digital Library
- [41] . 2019. Lightweight image super-resolution with adaptive weighted learning network. arXiv:1904.02358. Retrieved from https://arxiv.org/abs/1904.02358.Google Scholar
- [42] . 2021. Exploring sparsity in image super-resolution for efficient inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21). 4917–4926.Google Scholar
Cross Ref
- [43] . 2022. Learning a degradation-adaptive network for light field image super-resolution. arXiv:2206.06214. Retrieved from https://arxiv.org/abs/2206.06214.Google Scholar
- [44] . 2022. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22). 17683–17693.Google Scholar
Cross Ref
- [45] . 2021. Lightweight image super-resolution with multi-scale feature interaction network. In Proceedings of the IEEE International Conference on Multimedia & Expo (ICME’21). IEEE, 1–6.Google Scholar
Cross Ref
- [46] . 2022. A progressive fusion generative adversarial network for realistic and consistent video super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 44, 5 (2022), 2264–2280.Google Scholar
- [47] . 2010. On single image scale-up using sparse-representations. In Proceedings of the International Conference on Curves and Surfaces (ICCS’10). 711–730.Google Scholar
Digital Library
- [48] . 2021. Large factor image super-resolution with cascaded convolutional neural networks. IEEE Trans. Multimedia 23 (2021), 2172–2184.Google Scholar
Cross Ref
- [49] . 2022. A two-stage attentive network for single image super-resolution. IEEE Trans. Circ. Syst. Vid. Technol. 32, 3 (2022), 1020–1033.Google Scholar
Cross Ref
- [50] . 2021. Accurate and efficient image super-resolution via global-local adjusting dense network. IEEE Trans. Multimedia 23 (2021), 1924–1937.Google Scholar
Cross Ref
- [51] . 2021. Edge-oriented convolution block for real-time super resolution on mobile devices. In Proceedings of the ACM International Conference on Multimedia (ACMMM’21). 4034–4043.Google Scholar
Digital Library
- [52] . 2018. Image super-resolution using very deep residual channel attention networks. In Proceedings of the European Conference on Computer Vision (ECCV’18). 286–301.Google Scholar
Digital Library
- [53] . 2019. Efficient single image super-resolution via hybrid residual feature learning with compact back-projection network. In Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW’19). 1–8.Google Scholar
Cross Ref
- [54] . 2022. Lightweight image super-resolution with expectation-maximization attention mechanism. IEEE Trans. Circ. Syst. Vid. Technol. 32, 3 (2022), 1273–1284.Google Scholar
Cross Ref
- [55] . 2022. Self-calibrated efficient transformer for lightweight super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22) Workshops. 930–939.Google Scholar
Cross Ref
Index Terms
Lightweight Feature De-redundancy and Self-calibration Network for Efficient Image Super-resolution
Recommendations
Lightweight image super-resolution based on deep learning: State-of-the-art and future directions
AbstractRecently, super-resolution (SR) techniques based on deep learning have taken more and more attention, aiming to improve the images and videos resolutions. Most of the SR methods are related to other fields of computer vision such as ...
Graphical abstractDisplay Omitted
Highlights- Lightweight image super-resolution based on deep learning models are reviewed.
- ...
Lightweight image super-resolution with a feature-refined network
AbstractIn recent years, deep Convolutional Neural Networks (CNNs) have achieved impressive successes on the Single Image Super-Resolution task (SISR). However, it remains difficult to apply these CNN-based SISR methods in embedded devices due ...
Highlights- We propose a shadow module with a lower cost to generate similar feature maps.
- ...
Multi-branch aware module with channel shuffle pixel-wise attention for lightweight image super-resolution
AbstractDeep convolutional neural networks (CNNs) have boosted the performance of image super-resolution (SR) in recent years. However, existing deep CNN-based SR approaches often have massive computation costs to obtain high performance, which are still ...






Comments