Abstract
The advent of convolutional neural networks (CNNs) has brought substantial progress in image super-resolution (SR) reconstruction. However, most SR methods pursue deep architectures to boost performance, and the resulting large model sizes are impractical for real-world applications. Furthermore, they insufficiently explore the internal structural information of image features, disadvantaging the restoration of fine texture details. To solve these challenges, we propose a lightweight architecture based on a CNN named attention-directed feature aggregation network (AFAN), consisting of chained stacking multi-aware attention modules (MAAMs) and a simple channel attention module (SCAM), for image SR. Specifically, in each MAAM, we construct a space-aware attention block (SAAB) and a dimension-aware attention block (DAAB) that individually yield unique three-dimensional modulation coefficients to adaptively recalibrate structural information from an asymmetric convolution residual block (ACRB). The synergistic strategy captures multiple content features that are both space-aware and dimension-aware to preserve more fine-grained details. In addition, to further enhance the accuracy and robustness of the network, SCAM is embedded in the last MAAM to highlight channels with high activated values at low computational load. Comprehensive experiments verify that our proposed network attains high qualitative accuracy while employing fewer parameters and moderate computational requirements, exceeding most state-of-the-art lightweight approaches.
- [1] . 2018. Fast, accurate, and lightweight super-resolution with cascading residual network. In Proceedings of the European Conference on Computer Vision (ECCV’18). 252–268.Google Scholar
Digital Library
- [2] Supratik Banerjee, Cagri Ozcinar, Aakanksha Rana, Aljosa Smolic, and Michael Manzke. 2020. Sub-Pixel Back-Projection Network For Lightweight Single Image Super-Resolution. arXiv:2008.01116 (2020). https://arxiv.org/abs/2008.01116.Google Scholar
- [3] . 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In Proceedings of the 23rd British Machine Vision Conference (BMVC’12). British Machine Vision Association, Surrey, 1–10.Google Scholar
Cross Ref
- [4] . 2016. Accelerating the super-resolution convolutional neural network. In European Conference on Computer Vision (ECCV’16). 391–407.Google Scholar
- [5] . 2020. Progressive attentional learning for underwater image super-resolution. In International Conference on Intelligent Robotics and Applications. 233–243.Google Scholar
Digital Library
- [6] . 2017. A deep convolutional neural network with selection units for super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17). 1150–1156.Google Scholar
Cross Ref
- [7] . 2021. Fast, accurate and lightweight super-resolution with neural architecture search. In International Conference on Pattern Recognition (ICPR’21). 59–64.Google Scholar
Cross Ref
- [8] . 2020. Multi-objective reinforced evolution in mobile neural architecture search. In Computer Vision - ECCV 2020 Workshops, and (Eds.). 99–113.Google Scholar
Digital Library
- [9] . 2014. Learning a deep convolutional network for image super-resolution. In European Conference on Computer Vision. 184–199.Google Scholar
Cross Ref
- [10] . 2021. Feature distillation interaction weighting network for lightweight image super-resolution. CoRRw abs/2112.08655 (2021).Google Scholar
- [11] . 2003. Eigenface-domain super-resolution for face recognition. IEEE Transactions on Image Processing 12, 5 (2003), 597–606.Google Scholar
Digital Library
- [12] Meng-Hao Guo, Tian-Xing Xu, Jiang-Jiang Liu, Zheng-Ning Liu, Peng-Tao Jiang, Tai-Jiang Mu, Song-Hai Zhang, Ralph R. Martin, Ming-Ming Cheng, and Shi-Min Hu. 2022. Attention mechanisms in computer vision: A survey. Computational Visual Media 8, 3 (2022), 331–368.Google Scholar
- [13] . 2019. ODE-inspired network design for single image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19). 1732–1741.Google Scholar
Cross Ref
- [14] . 2017. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 7132–7141.Google Scholar
- [15] . 2020. Channel-wise and spatial feature modulation network for single image super-resolution. IEEE Transactions on Circuits and Systems for Video Technology 30, 11 (2020), 3911–3927.Google Scholar
- [16] . 2015. Single image super-resolution from transformed self-exemplars. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 5179–5206.Google Scholar
Cross Ref
- [17] . 2019. Lightweight image super-resolution with information multi-distillation network. In Proceedings of the 27th ACM International Conference on Multimedia. 2024–2032.Google Scholar
Digital Library
- [18] . 2018. Fast and accurate single image super-resolution via information distillation network. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 723–731.Google Scholar
Cross Ref
- [19] . 2020. Underwater image super-resolution using deep residual multipliers. In IEEE International Conference on Robotics and Automation (ICRA’20). 900–906.Google Scholar
Cross Ref
- [20] . 2020. Lightweight super-resolution using deep neural learning. IEEE Transactions on Broadcasting 66, 4 (2020), 814–823.Google Scholar
- [21] . 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 1646–1654.Google Scholar
Cross Ref
- [22] . 2016. Deeply-recursive convolutional network for image super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 1637–1645.Google Scholar
Cross Ref
- [23] Jun Hyuk Kim, Jun Ho Choi, Manri Cheon, and Jong Seok Lee. 2018. RAM: Residual Attention Module for Single Image Super-Resolution. arXiv: 1811.12043 (2018). https://arxiv.org/abs/1811.12043.Google Scholar
- [24] . 2017. Deep Laplacian pyramid networks for fast and accurate super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 624–632.Google Scholar
Cross Ref
- [25] . 2021. MADNet: A fast and lightweight network for single-image super resolution. IEEE Transactions on Cybernetics 51, 3 (2021), 1443–1453.Google Scholar
Cross Ref
- [26] . 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 4681–4690.Google Scholar
Cross Ref
- [27] . 2020. s-LWSR: Super lightweight super-resolution network. IEEE Transactions on Image Processing 29 (2020), 8368–8380.Google Scholar
Cross Ref
- [28] . 2018. Multi-scale residual network for image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV’18). 517–532.Google Scholar
Digital Library
- [29] Zhuangzi Li. 2019. Image Super-Resolution Using Attention Based DenseNet with Residual Deconvolution. arXiv:1907.05282 (2019). http://arxiv.org/abs/1907.05282.Google Scholar
- [30] . 2019. Feedback network for image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’19).Google Scholar
Digital Library
- [31] . 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17). 1132–1140.Google Scholar
Cross Ref
- [32] . 2020. Lightweight multi-scale residual networks with attention for image super-resolution. Knowledge-Based Systems 203 (2020), 106103.Google Scholar
Cross Ref
- [33] . 2020. Residual feature aggregation network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 2356–2365.Google Scholar
Cross Ref
- [34] . 2021. Cross-SRN: Structure-preserving super-resolution network with cross convolution. IEEE Transactions on Circuits and Systems for Video Technology (2021), 1–1.Google Scholar
- [35] . 2019. Efficient dense modules of asymmetric convolution for real-time semantic segmentation. In Proceedings of the ACM Multimedia Asia. ACM, 1–6.Google Scholar
Digital Library
- [36] . 2021. Single image super-resolution via multi-scale information polymerization network. IEEE Signal Processing Letters 28 (2021), 1305–1309.Google Scholar
Cross Ref
- [37] . 2002. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In IEEE International Conference on Computer Vision (CVPR’02). 416–423.Google Scholar
- [38] . 2017. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools & Applications 76, 20 (2017), 21811–21838.Google Scholar
Digital Library
- [39] . 2021. Rotate to attend: Convolutional triplet attention module. In IEEE Winter Conference on Applications of Computer Vision (WACV’21). 3138–3147.Google Scholar
Cross Ref
- [40] . 2019. Video-based person re-identification via spatio-temporal attentional and two-stream fusion convolutional networks. Pattern Recognition Letters 117 (2019), 153–160.Google Scholar
Cross Ref
- [41] . 2021. Dynamic residual self-attention network for lightweight single image super-resolution. IEEE Transactions on Multimedia (2021), 1–1.Google Scholar
Digital Library
- [42] . 2016. Convolutional neural network super resolution for face recognition in surveillance monitoring. In International Conference on Articulated Motion and Deformable Objects. 175–184.Google Scholar
Cross Ref
- [43] . 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 1874–1883.Google Scholar
Cross Ref
- [44] . 2013. Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In International Conference on Medical Image Computing and Computer-Assisted Intervention. 9–16.Google Scholar
Cross Ref
- [45] . 2021. Lightweight image super-resolution via weighted multi-scale residual network. IEEE/CAA Journal of Automatica Sinica 8, 7 (2021), 1271–1280.Google Scholar
Cross Ref
- [46] . 2016. Rethinking the inception architecture for computer vision. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 2818–2826.Google Scholar
Cross Ref
- [47] . 2017. Image super-resolution via deep recursive residual network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 3147–3155.Google Scholar
Cross Ref
- [48] . 2017. MemNet: A persistent memory network for image restoration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). 4539–4547.Google Scholar
Cross Ref
- [49] Chunwei Tian, Yong Xu, Wangmeng Zuo, Chia-Wen Lin, and David Zhang. 2022. Asymmetric CNN for Image Superresolution. IEEE Transactions on Systems, Man, and Cybernetics: Systems 52, 6 (2022), 3718–3730.Google Scholar
- [50] . 2020. Lightweight image super-resolution with enhanced CNN. Knowledge-Based Systems 205 (2020), 106235.Google Scholar
Cross Ref
- [51] . 2017. NTIRE 2017 challenge on single image super-resolution: Methods and results. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17). 114–125.Google Scholar
Cross Ref
- [52] . 2021. Lightweight image super-resolution by multi-scale aggregation. IEEE Transactions on Broadcasting 67, 2 (2021), 372–382.Google Scholar
- [53] Chaofeng Wang, Zheng Li, and Jun Shi. 2019. Lightweight Image Super-Resolution with Adaptive Weighted Learning Network. arXiv:1904.02358 (2019). https://arxiv.org/abs/1904.02358Google Scholar
- [54] Xuehui Wang, Qing Wang, Yuzhi Zhao, Junchi Yan, Lei Fan, and Long Chen. 2020. Lightweight Single-Image Super-Resolution Network with Attentive Auxiliary Feature Learning. In Proceedings of the Asian Conference on Computer Vision (ACCV). 268–285.Google Scholar
- [55] . 2019. ESRGAN: Enhanced super-resolution generative adversarial networks. In Proceedings of the European Conference on Computer Vision (ECCV’19) Workshops. 63–79.Google Scholar
Cross Ref
- [56] . 2022. Channel rearrangement multi-branch network for image super-resolution. Digital Signal Processing 120 (2022), 103254.Google Scholar
Digital Library
- [57] . 2018. CBAM: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV’18). 3–19.Google Scholar
Digital Library
- [58] Huapeng Wu, Jie Gui, Jun Zhang, James T. Kwok, and Zhihui Wei. 2021. Pyramidal Dense Attention Networks for Lightweight Image Super-Resolution. arXiv:2106.06996 (2021). https://arxiv.org/abs/2106.06996.Google Scholar
- [59] . 2010. On single image scale-up using sparse-representations. In International Conference on Curves and Surfaces. 711–730.Google Scholar
Digital Library
- [60] . 2021. Remote sensing image super-resolution via mixed high-order attention network. IEEE Transactions on Geoscience and Remote Sensing 59, 6 (2021), 5183–5196.Google Scholar
Cross Ref
- [61] . 2021. Multi-scale image super-resolution via a single extendable deep network. IEEE Journal of Selected Topics in Signal Processing 15, 2 (2021), 253–263.Google Scholar
Cross Ref
- [62] . 2017. Learning deep CNN denoiser prior for image restoration. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 3929–3938.Google Scholar
Cross Ref
- [63] . 2017. Learning a single convolutional super-resolution network for multiple degradations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 3262–3271.Google Scholar
- [64] Kai Zhang, Wangmeng Zuo, and Lei Zhang. 2019. Deep Plug-And-Play Super-Resolution for Arbitrary Blur Kernels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1671–1681.Google Scholar
- [65] . 2018. Image super-resolution using very deep residual channel attention networks. In Proceedings of the European Conference on Computer Vision (ECCV’18). 286–301.Google Scholar
Digital Library
- [66] . 2018. Residual dense network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). 2472–2481.Google Scholar
Cross Ref
- [67] . 2021. Attention-guided multi-path cross-CNN for underwater image super-resolution. Signal, Image and Video Processing 16, 1 (2021), 155–163.Google Scholar
Cross Ref
- [68] Hengyuan Zhao, Xiangtao Kong, Jingwen He, Yu Qiao, and Chao Dong. 2020. Efficient Image Super-Resolution Using Pixel Attention. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 56–72.Google Scholar
- [69] . 2019. Efficient single image super-resolution via hybrid residual feature learning with compact back-projection network. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV’19) Workshops.Google Scholar
Cross Ref
Index Terms
Image Super-Resolution via Lightweight Attention-Directed Feature Aggregation Network
Recommendations
Deep Residual Attention Network for Spectral Image Super-Resolution
Computer Vision – ECCV 2018 WorkshopsAbstractSpectral imaging sensors often suffer from low spatial resolution, as there exists an essential tradeoff between the spectral and spatial resolutions that can be simultaneously achieved, especially when the temporal resolution needs to be ...
Edge-directed single image super-resolution via cross-resolution sharpening function learning
Edge-directed single image super-resolution methods have been paid more attentions due to their sharp edge preserving in the recovered high-resolution image. Their core is the high-resolution gradient estimation. In this paper, we propose a novel cross-...
Image super-resolution based on adaptive cascading attention network▪
AbstractSingle image super-resolution (SISR) refers to the task restoring a high-resolution (HR) image from its low-resolution (LR) counterpart. Deep convolutional neural networks (CNNs) have significantly improved SISR performance. However, ...
Highlights- A novel lightweight attention network has been proposed.
- We use adaptive ...






Comments