skip to main content
research-article

Lightweight Feature De-redundancy and Self-calibration Network for Efficient Image Super-resolution

Authors Info & Claims
Published:25 February 2023Publication History
Skip Abstract Section

Abstract

In recent years, thanks to the inherent powerful feature representation and learning abilities of the convolutional neural network (CNN), deep CNN-steered single image super-resolution approaches have achieved remarkable performance improvements. However, these methods are often accompanied by large consumption of computing and memory resources, which is difficult to be adopted in real-world application scenes. To handle this issue, we design an efficient Feature De-redundancy and Self-calibration Super-resolution network (FDSCSR). In particular, a Feature De-redundancy and Self-calibration Block (FDSCB) is proposed to reduce the repetitive feature information extracted by the model and further enhance the efficiency of the model. Then, based on FDSCB, a Local Feature Fusion Module is presented to elaborately utilize and fuse the feature information extracted by each FDSCB. Abundant experiments on benchmarks have demonstrated that our FDSCSR achieves superior performance with relatively less computational consumption and storage resource than other state-of-the-art approaches. The code is available at https://github.com/IVIPLab/FDSCSR.

REFERENCES

  1. [1] Agustsson Eirikur and Timofte Radu. 2017. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17). 126135.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Ahn Namhyuk, Kang Byungkon, and Sohn Kyung-Ah. 2018. Fast, accurate, and lightweight super-resolution with cascading residual network. In Proceedings of the European Conference on Computer Vision (ECCV’18). 252268.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. [3] Bevilacqua Marco, Roumy Aline, Guillemot Christine, and Alberi-Morel Marie Line. 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In Proceedings of the British Machine Vision Conference (BMVC’12). 135.1–135.10.Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Chen Honggang, He Xiaohai, Qing Linbo, Wu Yuanyuan, Ren Chao, Sheriff Ray E., and Zhu Ce. 2022. Real-world single image super-resolution: A brief review. Inf. Fusion 79 (2022), 124145.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Chen Xiangyu, Wang Xintao, Zhou Jiantao, and Dong Chao. 2022. Activating more pixels in image super-resolution transformer. arXiv:2205.04437. Retrieved from https://arxiv.org/abs/2005.04437.Google ScholarGoogle Scholar
  6. [6] Dai Tao, Cai Jianrui, Zhang Yongbing, Xia Shu-Tao, and Zhang Lei. 2019. Second-order attention network for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’19). 1106511074.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] Dong Chao, Loy Chen Change, He Kaiming, and Tang Xiaoou. 2014. Learning a deep convolutional network for image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV’14). 184199.Google ScholarGoogle ScholarCross RefCross Ref
  8. [8] Gao Guangwei, Li Wenjie, Li Juncheng, Wu Fei, Lu Huimin, and Yu Yi. 2022. Feature distillation interaction weighting network for lightweight image super-resolution. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’22). 661669.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Gao Guangwei, Wang Zhengxue, Li Juncheng, Li Wenjie, Yu Yi, and Zeng Tieyong. 2022. Lightweight bimodal network for single-image super-resolution via symmetric CNN and recursive transformer. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’22). 913919.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Gao Guangwei, Yu Yi, Lu Huimin, Yang Jian, and Yue Dong. 2022. Context-patch representation learning with adaptive neighbor embedding for robust face image super-resolution. IEEE Trans Multimedia (2022).Google ScholarGoogle Scholar
  11. [11] Gao Guangwei, Yu Yi, Xie Jin, Yang Jian, Yang Meng, and Zhang Jian. 2021. Constructing multilayer locality-constrained matrix regression framework for noise robust face super-resolution. Pattern Recogn. 110 (2021), 107539.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Guo Yong, Chen Jian, Wang Jingdong, Chen Qi, Cao Jiezhang, Deng Zeshuai, Xu Yanwu, and Tan Mingkui. 2020. Closed-loop matters: Dual regression networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 54075416.Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Han Kai, Wang Yunhe, Tian Qi, Guo Jianyuan, Xu Chunjing, and Xu Chang. 2020. Ghostnet: More features from cheap operations. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 15801589.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 770778.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Hu Jie, Shen Li, and Sun Gang. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). 71327141.Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Huang Gao, Liu Zhuang, Maaten Laurens Van Der, and Weinberger Kilian Q. 2017. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 47004708.Google ScholarGoogle ScholarCross RefCross Ref
  17. [17] Huang Jia-Bin, Singh Abhishek, and Ahuja Narendra. 2015. Single image super-resolution from transformed self-exemplars. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’15). 51975206.Google ScholarGoogle ScholarCross RefCross Ref
  18. [18] Hui Zheng, Gao Xinbo, Yang Yunchu, and Wang Xiumei. 2019. Lightweight image super-resolution with information multi-distillation network. In Proceedings of the ACM International Conference on Multimedia (ACMMM’19). 20242032.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. [19] Hui Zheng, Wang Xiumei, and Gao Xinbo. 2018. Fast and accurate single image super-resolution via information distillation network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’18). 723731.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Jiang Kui, Wang Zhongyuan, Yi Peng, and Jiang Junjun. 2020. Hierarchical dense recursive network for image super-resolution. Pattern Recogn. 107 (2020), 107475.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Kim Jiwon, Lee Jung Kwon, and Lee Kyoung Mu. 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 16461654.Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Kim Jiwon, Lee Jung Kwon, and Lee Kyoung Mu. 2016. Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 16371645.Google ScholarGoogle ScholarCross RefCross Ref
  23. [23] Lai Wei-Sheng, Huang Jia-Bin, Ahuja Narendra, and Yang Ming-Hsuan. 2018. Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans. Pattern Anal. Mach. Intell. 41, 11 (2018), 25992613.Google ScholarGoogle ScholarCross RefCross Ref
  24. [24] Lan Rushi, Sun Long, Liu Zhenbing, Lu Huimin, Pang Cheng, and Luo Xiaonan. 2020. Madnet: A fast and lightweight network for single-image super resolution. IEEE Trans. Cybernet. 51, 3 (2020), 14431453.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Li Juncheng, Fang Faming, Mei Kangfu, and Zhang Guixu. 2018. Multi-scale residual network for image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV’18). 517532.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Li Juncheng, Pei Zehua, and Zeng Tieyong. 2021. From beginner to master: A survey for deep learning-based single-image super-resolution. arXiv:2109.14335. Retrieved from https://arixv.org/abs/2109.14335.Google ScholarGoogle Scholar
  27. [27] Li Wenjie, Li Juncheng, Gao Guangwei, Zhou Jiantao, Yang Jian, and Qi Guo-Jun. 2022. Cross-receptive focused inference network for lightweight image super-resolution. arXiv:2207.02796. Retrieved from https://arxiv.org/abs/2207.02796.Google ScholarGoogle Scholar
  28. [28] Li Wenbo, Zhou Kun, Qi Lu, Jiang Nianjuan, Lu Jiangbo, and Jia Jiaya. 2021. Lapar: Linearly-assembled pixel-adaptive regression network for single image super-resolution and beyond. arXiv:2105.10422. Retrieved from https://arxiv.org/abs/2105.10422.Google ScholarGoogle Scholar
  29. [29] Li Yanchun, Cao Jianglian, Li Zhetao, Oh Sangyoon, and Komuro Nobuyoshi. 2021. Lightweight single image super-resolution with dense connection distillation network. ACM Trans. Multimedia Comput. Commun. Appl. 17, 1s (2021), 117.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Lim Bee, Son Sanghyun, Kim Heewon, Nah Seungjun, and Lee Kyoung Mu. 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’17). 136144.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Liu Jie, Tang Jie, and Wu Gangshan. 2020. Residual feature distillation network for lightweight image super-resolution. In Proceedings of the European Conference on Computer Vision (ECCV’20). Springer, 4155.Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. [32] Liu Jiang-Jiang, Hou Qibin, Cheng Ming-Ming, Wang Changhu, and Feng Jiashi. 2020. Improving convolutional networks with self-calibrated convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 1009610105.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Lu Zhisheng, Liu Hong, Li Juncheng, and Zhang Linlin. 2022. Transformer for single image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22) Workshops. 457466.Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Martin David, Fowlkes Charless, Tal Doron, and Malik Jitendra. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’01). 416423.Google ScholarGoogle ScholarCross RefCross Ref
  35. [35] Matsui Yusuke, Ito Kota, Aramaki Yuji, Fujimoto Azuma, Ogawa Toru, Yamasaki Toshihiko, and Aizawa Kiyoharu. 2017. Sketch-based manga retrieval using manga109 dataset. Multimedia Tools Appl. 76, 20 (2017), 2181121838.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Mei Yiqun, Fan Yuchen, Zhou Yuqian, Huang Lichao, Huang Thomas S., and Shi Honghui. 2020. Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’20). 56905699.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Shi Wenzhe, Caballero Jose, Huszár Ferenc, Totz Johannes, Aitken Andrew P., Bishop Rob, Rueckert Daniel, and Wang Zehan. 2016. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 18741883.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Tai Ying, Yang Jian, and Liu Xiaoming. 2017. Image super-resolution via deep recursive residual network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’17). 31473155.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Tai Ying, Yang Jian, Liu Xiaoming, and Xu Chunyan. 2017. Memnet: A persistent memory network for image restoration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV’17). 45394547.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Tian Chunwei, Xu Yong, Zuo Wangmeng, Zhang Bob, Fei Lunke, and Lin Chia-Wen. 2020. Coarse-to-fine CNN for image super-resolution. IEEE Trans Multimedia 23 (2020), 14891502.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Wang Chaofeng, Li Zheng, and Shi Jun. 2019. Lightweight image super-resolution with adaptive weighted learning network. arXiv:1904.02358. Retrieved from https://arxiv.org/abs/1904.02358.Google ScholarGoogle Scholar
  42. [42] Wang Longguang, Dong Xiaoyu, Wang Yingqian, Ying Xinyi, Lin Zaiping, An Wei, and Guo Yulan. 2021. Exploring sparsity in image super-resolution for efficient inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’21). 49174926.Google ScholarGoogle ScholarCross RefCross Ref
  43. [43] Wang Yingqian, Liang Zhengyu, Wang Longguang, Yang Jungang, An Wei, and Guo Yulan. 2022. Learning a degradation-adaptive network for light field image super-resolution. arXiv:2206.06214. Retrieved from https://arxiv.org/abs/2206.06214.Google ScholarGoogle Scholar
  44. [44] Wang Zhendong, Cun Xiaodong, Bao Jianmin, Zhou Wengang, Liu Jianzhuang, and Li Houqiang. 2022. Uformer: A general u-shaped transformer for image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22). 1768317693.Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Wang Zhengxue, Gao Guangwei, Li Juncheng, Yu Yi, and Lu Huimin. 2021. Lightweight image super-resolution with multi-scale feature interaction network. In Proceedings of the IEEE International Conference on Multimedia & Expo (ICME’21). IEEE, 16.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Yi Peng, Wang Zhongyuan, Jiang Kui, Jiang Junjun, Lu Tao, and Ma Jiayi. 2022. A progressive fusion generative adversarial network for realistic and consistent video super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 44, 5 (2022), 22642280.Google ScholarGoogle Scholar
  47. [47] Zeyde Roman, Elad Michael, and Protter Matan. 2010. On single image scale-up using sparse-representations. In Proceedings of the International Conference on Curves and Surfaces (ICCS’10). 711730.Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. [48] Zhang Dongyang, Shao Jie, Liang Zhenwen, Gao Lianli, and Shen Heng Tao. 2021. Large factor image super-resolution with cascaded convolutional neural networks. IEEE Trans. Multimedia 23 (2021), 21722184.Google ScholarGoogle ScholarCross RefCross Ref
  49. [49] Zhang Jiqing, Long Chengjiang, Wang Yuxin, Piao Haiyin, Mei Haiyang, Yang Xin, and Yin Baocai. 2022. A two-stage attentive network for single image super-resolution. IEEE Trans. Circ. Syst. Vid. Technol. 32, 3 (2022), 10201033.Google ScholarGoogle ScholarCross RefCross Ref
  50. [50] Zhang Xinyan, Gao Peng, Liu Sunxiangyu, Zhao Kongya, Li Guitao, Yin Liuguo, and Chen Chang Wen. 2021. Accurate and efficient image super-resolution via global-local adjusting dense network. IEEE Trans. Multimedia 23 (2021), 19241937.Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Zhang Xindong, Zeng Hui, and Zhang Lei. 2021. Edge-oriented convolution block for real-time super resolution on mobile devices. In Proceedings of the ACM International Conference on Multimedia (ACMMM’21). 40344043.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. [52] Zhang Yulun, Li Kunpeng, Li Kai, Wang Lichen, Zhong Bineng, and Fu Yun. 2018. Image super-resolution using very deep residual channel attention networks. In Proceedings of the European Conference on Computer Vision (ECCV’18). 286301.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Zhu Feiyang and Zhao Qijun. 2019. Efficient single image super-resolution via hybrid residual feature learning with compact back-projection network. In Proceedings of the IEEE International Conference on Computer Vision Workshops (ICCVW’19). 18.Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Zhu Xiangyuan, Guo Kehua, Ren Sheng, Hu Bin, Hu Min, and Fang Hui. 2022. Lightweight image super-resolution with expectation-maximization attention mechanism. IEEE Trans. Circ. Syst. Vid. Technol. 32, 3 (2022), 12731284.Google ScholarGoogle ScholarCross RefCross Ref
  55. [55] Zou Wenbin, Ye Tian, Zheng Weixin, Zhang Yunchen, Chen Liang, and Wu Yi. 2022. Self-calibrated efficient transformer for lightweight super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’22) Workshops. 930939.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Lightweight Feature De-redundancy and Self-calibration Network for Efficient Image Super-resolution

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM Transactions on Multimedia Computing, Communications, and Applications
            ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 3
            May 2023
            514 pages
            ISSN:1551-6857
            EISSN:1551-6865
            DOI:10.1145/3582886
            • Editor:
            • Abdulmotaleb El Saddik
            Issue’s Table of Contents

            Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 25 February 2023
            • Online AM: 29 October 2022
            • Accepted: 20 October 2022
            • Revised: 5 September 2022
            • Received: 7 May 2022
            Published in tomm Volume 19, Issue 3

            Permissions

            Request permissions about this article.

            Request Permissions

            Check for updates

            Qualifiers

            • research-article
          • Article Metrics

            • Downloads (Last 12 months)274
            • Downloads (Last 6 weeks)22

            Other Metrics

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Full Text

          View this article in Full Text.

          View Full Text

          HTML Format

          View this article in HTML Format .

          View HTML Format
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!