skip to main content
research-article

Symmetrical Residual Connections for Single Image Super-Resolution

Authors Info & Claims
Published:25 February 2019Publication History
Skip Abstract Section

Abstract

Single-image super-resolution (SISR) methods based on convolutional neural networks (CNN) have shown great potential in the literature. However, most deep CNN models don’t have direct access to subsequent layers, seriously hindering the information flow. Furthermore, they fail to make full use of the hierarchical features from different low-level layers, thereby resulting in relatively low accuracy. In this article, we present a new SISR CNN, called SymSR, which incorporates symmetrical nested residual connections to improve both the accuracy and the execution speed. SymSR takes a larger image region for contextual spreading. It symmetrically combines multiple short paths for the forward propagation to improve the accuracy and for the backward propagation of gradient flow to accelerate the convergence speed. Extensive experiments based on open challenge datasets show the effectiveness of symmetrical residual connections. Compared with four other state-of-the-art super-resolution CNN methods, SymSR is superior in both accuracy and runtime.

References

  1. Eirikur Agustsson and Radu Timofte. 2017. Ntire 2017 challenge on single image super-resolution: Dataset and study. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Vol. 3. 2.Google ScholarGoogle ScholarCross RefCross Ref
  2. Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks 5, 2 (1994), 157--166. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie Line Alberi-Morel. 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In Proceedings of the 23rd British Machine Vision Conference (BMVC'12). 135.1--135.10.Google ScholarGoogle ScholarCross RefCross Ref
  4. Marco Bevilacqua, Aline Roumy, Christine Guillemot, and Marie-Line Alberi Morel. 2013. Super-resolution using neighbor embedding of back-projection residuals. In 18th International Conference on Digital Signal Processing (DSP’13). IEEE, 1--8.Google ScholarGoogle ScholarCross RefCross Ref
  5. Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2016. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 2 (2016), 295--307. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Gilad Freedman and Raanan Fattal. 2011. Image and video upscaling from local self-examples. ACM Transactions on Graphics (TOG) 30, 2 (2011), 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Daniel Glasner, Shai Bagon, and Michal Irani. 2009. Super-resolution from a single image. In IEEE 12th International Conference on Computer Vision. IEEE, 349--356.Google ScholarGoogle ScholarCross RefCross Ref
  8. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 770--778.Google ScholarGoogle ScholarCross RefCross Ref
  9. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Identity mappings in deep residual networks. In European Conference on Computer Vision. Springer, 630--645.Google ScholarGoogle ScholarCross RefCross Ref
  10. Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja. 2015. Single image super-resolution from transformed self-exemplars. In IEEE Conference on Computer Vision and Pattern Recognition. 5197--5206.Google ScholarGoogle ScholarCross RefCross Ref
  11. Kui Jia, Xiaogang Wang, and Xiaoou Tang. 2013. Image transformation based on learning dictionaries across image spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 2 (2013), 367--380. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. In 22nd ACM International Conference on Multimedia. ACM, 675--678. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Accurate image super-resolution using very deep convolutional networks. In IEEE Conference on Computer Vision and Pattern Recognition. 1646--1654.Google ScholarGoogle ScholarCross RefCross Ref
  14. Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Deeply-recursive convolutional network for image super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition. 1637--1645.Google ScholarGoogle ScholarCross RefCross Ref
  15. Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. 2017. Deep Laplacian pyramid networks for fast and accurate super-resolution. In IEEE Conference on Computer Vision and Pattern Recognition. 624--632.Google ScholarGoogle ScholarCross RefCross Ref
  16. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (1998), 2278--2324.Google ScholarGoogle ScholarCross RefCross Ref
  17. Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. 2016. Photo-realistic single image super-resolution using a generative adversarial network. arXiv preprint (2016).Google ScholarGoogle Scholar
  18. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, 234--241.Google ScholarGoogle ScholarCross RefCross Ref
  19. Samuel Schulter, Christian Leistner, and Horst Bischof. 2015. Fast and accurate image upscaling with super-resolution forests. In IEEE Conference on Computer Vision and Pattern Recognition. 3791--3799.Google ScholarGoogle ScholarCross RefCross Ref
  20. Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google ScholarGoogle Scholar
  21. Abhishek Singh and Narendra Ahuja. 2014. Super-resolution using sub-band self-similarity. In Asian Conference on Computer Vision. Springer, 552--568.Google ScholarGoogle Scholar
  22. Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A. Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In AAAI, Vol. 4. 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Ying Tai, Jian Yang, Xiaoming Liu, and Chunyan Xu. 2017. MemNet: A persistent memory network for image restoration. (2017), 4549--4557.Google ScholarGoogle Scholar
  24. Radu Timofte, Vincent De, and Luc Van Gool. 2013. Anchored neighborhood regression for fast example-based super-resolution. In IEEE International Conference on Computer Vision (ICCV’13). IEEE, 1920--1927. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Radu Timofte, Vincent De Smet, and Luc Van Gool. 2014. A+: Adjusted anchored neighborhood regression for fast super-resolution. In Asian Conference on Computer Vision. Springer, 111--126.Google ScholarGoogle Scholar
  26. Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3D convolutional networks. In IEEE International Conference on Computer Vision (ICCV’15). IEEE, 4489--4497. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Chih-Yuan Yang and Ming-Hsuan Yang. 2013. Fast direct super-resolution by simple functions. In IEEE International Conference on Computer Vision (ICCV’13). IEEE, 561--568. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Jianchao Yang, John Wright, Thomas S. Huang, and Yi Ma. 2010. Image super-resolution via sparse representation. IEEE Transactions on Image Processing 19, 11 (2010), 2861--2873. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Lequan Yu, Xin Yang, Hao Chen, Jing Qin, and Pheng-Ann Heng. 2017. Volumetric ConvNets with mixed residual connections for automated prostate segmentation from 3D MR images. In AAAI. 66--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Sergey Zagoruyko and Nikos Komodakis. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146 (2016).Google ScholarGoogle Scholar
  31. Roman Zeyde, Michael Elad, and Matan Protter. 2010. On single image scale-up using sparse-representations. In International Conference on Curves and Surfaces. Springer, 711--730. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Y. Zhang, L. Sun, C. Yan, X. Ji, and Q. Dai. 2018. Adaptive residual networks for high-quality image restoration. IEEE Transactions on Image Processing PP, 99 (2018), 1--1.Google ScholarGoogle Scholar

Index Terms

  1. Symmetrical Residual Connections for Single Image Super-Resolution

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!