Abstract
Due to the diversity of the degradation process that is difficult to model, the recovery of mixed distorted images is still a challenging problem. The deep learning model trained under certain degradation declines significantly in other degradation situations. In this article, we explore ways to use a combination of tools to deal with the mixed distortion. First, we illustrate the limitations of a single deep network in dealing with multiple distortion types and then introduce a hierarchical toolkit with distinguished powerful tools. Second, we investigate how an efficient representation of images combined with a reinforcement learning (RL) paradigm helps to deal with tool noise in continuous restoration. The proposed method can accurately capture the distortion preferences for selecting the optimal recovery tools by RL agent. Finally, to fully utilize random tools for unknown distortion combinations, we adopt the exploration scheme with various quality evaluation methods to achieve more quality improvements. Experimental results demonstrate that the peak signal-to-noise ratio of the proposed method is 3.30 dB higher than other state-of-the-art RL-based methods on the CSIQ single distortion dataset and 0.95 dB higher on the DIV2K mixed distortion dataset.
- [1] . 2017. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 126–135.Google Scholar
Cross Ref
- [2] . 2018. Abstract Dynamic Programming. Athena Scientific Nashua, NH.Google Scholar
- [3] . 2021. Disentangling image distortions in deep feature space. Pattern Recogn. Lett. (2021).Google Scholar
Cross Ref
- [4] . 2017. Deep neural networks for no-reference and full-reference image quality assessment. IEEE Trans. Image Process. 27, 1 (2017), 206–219.Google Scholar
Cross Ref
- [5] . 2011. Non-local means denoising. Image Process. On Line 1 (2011), 208–212.Google Scholar
Cross Ref
- [6] . 2007. Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16, 8 (2007), 2080–2095.Google Scholar
Digital Library
- [7] . 2015. Compression artifacts reduction by a deep convolutional network. In Proceedings of the IEEE International Conference on Computer Vision. 576–584.Google Scholar
Digital Library
- [8] . 2020. PixelRL: Fully convolutional network with reinforcement learning for image processing. IEEE Trans. Multimedia 22, 7 (2020), 1704–1719.Google Scholar
Cross Ref
- [9] . 2015. Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. Image Process. 25, 1 (2015), 372–387.Google Scholar
Digital Library
- [10] . 2019. Modulating image restoration with continual levels via adaptive feature modification layers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11056–11064.Google Scholar
Cross Ref
- [11] . 2019. Lightweight image super-resolution with information multi-distillation network. In Proceedings of the 27th ACM International Conference on Multimedia. 2024–2032.Google Scholar
Digital Library
- [12] . 1996. Reinforcement learning: A survey. J. Artif. Intell. Res. 4 (1996), 237–285.Google Scholar
Digital Library
- [13] . 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1646–1654.Google Scholar
Cross Ref
- [14] . 2015. Dark image enhancement based onpairwise target contrast and multi-scale detail boosting. In Proceedings of the IEEE International Conference on Image Processing (ICIP’15). IEEE, 1404–1408.Google Scholar
Digital Library
- [15] . 2019. Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 8878–8887.Google Scholar
Cross Ref
- [16] . 2010. Most apparent distortion: Full-reference image quality assessment and the role of strategy. J. Electr. Imag. 19, 1 (2010), 011006.Google Scholar
Cross Ref
- [17] . 2019. Image/video restoration via multiplanar autoregressive model and low-rank optimization. ACM Trans. Multimedia Comput. Commun. Appl. 15, 4 (2019), 1–23.Google Scholar
Digital Library
- [18] . 2017. Enhanced deep residual networks for single image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 136–144.Google Scholar
Cross Ref
- [19] . 2012. Noise level estimation using weak textured patches of a single noisy image. In Proceedings of the 19th IEEE International Conference on Image Processing. IEEE, 665–668.Google Scholar
Cross Ref
- [20] . 2021. Affinity derivation for accurate instance segmentation. ACM Trans. Multimedia Comput. Commun. Appl. 17, 1 (2021), 1–20.Google Scholar
Digital Library
- [21] . 2020. FIN: Feature integrated network for object detection. ACM Trans. Multimedia Comput. Commun. Appl. 16, 2 (2020), 1–18.Google Scholar
Digital Library
- [22] . 2017. The epsilon-greedy Algorithm. [EB/OL]. Retrieved from https://jamesmccaffrey.wordpress.com/2017/11/30/the-epsilon-greedy-algorithm/.Google Scholar
- [23] . 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 12 (2012), 4695–4708.Google Scholar
Digital Library
- [24] . 2019. Trinity of pixel enhancement: A joint solution for demosaicking, denoising and super-resolution. arXiv:1905.02538. Retrieved from https://arxiv.org/abs/1905.02538.Google Scholar
- [25] . 2019. Attention-based adaptive selection of operations for image restoration in the presence of unknown combined distortions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9039–9048.Google Scholar
Cross Ref
- [26] . 2018. Scale-recurrent network for deep image deblurring. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8174–8182.Google Scholar
Cross Ref
- [27] . 1995. Temporal difference learning and TD-Gammon. Commun. ACM 38, 3 (1995), 58–68.Google Scholar
Digital Library
- [28] . 2019. Cfsnet: Toward a controllable feature space for image restoration. (2019), 4140–4149.Google Scholar
- [29] . 2004. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 13, 4 (2004), 600–612.Google Scholar
Digital Library
- [30] . 2002. No-reference perceptual quality assessment of JPEG compressed images. In Proceedings of the International Conference on Image Processing, Vol. 1. IEEE, I–I.Google Scholar
Cross Ref
- [31] . 1989. Learning from delayed rewards.Google Scholar
- [32] . 1964. Extrapolation, interpolation, and smoothing of stationary time series: With engineering applications. 8 (1964).Google Scholar
- [33] . 2020. Depth image denoising using nuclear norm and learning graph model. ACM Trans. Multimedia Comput. Commun. Appl. 16, 4 (2020), 1–17.Google Scholar
Digital Library
- [34] . 2021. Compressed imaging reconstruction with sparse random projection. ACM Trans. Multimedia Comput. Commun. Appl. 17, 1 (2021), 1–25.Google Scholar
Digital Library
- [35] . 2020. From patches to pictures (PaQ-2-PiQ): Mapping the perceptual space of picture quality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3575–3585.Google Scholar
Cross Ref
- [36] . 2018. Crafting a toolchain for image restoration by deep reinforcement learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2443–2452.Google Scholar
Cross Ref
- [37] . 2021. Path-restore: Learning network path selection for image restoration. IEEE Trans. Pattern Anal. Mach. Intell. (2021).Google Scholar
Digital Library
- [38] . 2017. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 26, 7 (2017), 3142–3155.Google Scholar
Digital Library
- [39] . 2018. FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27, 9 (2018), 4608–4622.Google Scholar
- [40] . 2018. DMCNN: Dual-domain multi-scale convolutional neural network for compression artifacts removal. In Proceedings of the 25th IEEE International Conference on Image Processing (ICIP’18). IEEE, 390–394.Google Scholar
Cross Ref
Index Terms
Image Quality Assessment–driven Reinforcement Learning for Mixed Distorted Image Restoration
Recommendations
A survey of deep learning approaches to image restoration
AbstractIn this paper, we present an extensive review on deep learning methods for image restoration tasks. Deep learning techniques, led by convolutional neural networks, have received a great deal of attention in almost all areas of image ...
Blurred image restoration: A fast method of finding the motion length and angle
Motion blur in photographic images is a result of camera movement or shake. Methods such as Blind Deconvolution are used when information about the direction and size of blur is not known. Restoration methods, such as Lucy and Richardson or Wiener ...
Reduced-reference Perceptual Discrepancy Learning for Image Restoration Quality Assessment
Artificial IntelligenceAbstractImage restoration has been receiving extensive attention owing to its widespread applications, and numerous restoration algorithms have been proposed. However, how to accurately evaluate the performances of image restoration algorithms remains ...






Comments