Abstract
Underwater imaging often suffers from color cast and contrast degradation due to range-dependent medium absorption and light scattering. Introducing image statistics as prior has been proved to be an effective solution for underwater image enhancement. However, relative to the modal divergence of light propagation and underwater scenery, the existing methods are limited in representing the inherent statistics of underwater images resulting in color artifacts and haze residuals. To address this problem, this article proposes a convolutional neural network (CNN)-based framework to learn hierarchical statistical features related to color cast and contrast degradation and to leverage them for underwater image enhancement. Specifically, a pixel disruption strategy is first proposed to suppress intrinsic colors’ influence and facilitate modeling a unified statistical representation of underwater image. Then, considering the local variation of depth of field, two parallel sub-networks: Color Correction Network (CC-Net) and Contrast Enhancement Network (CE-Net) are presented. The CC-Net and CE-Net can generate pixel-wise color cast and transmission map and achieve spatial-varied color correction and contrast enhancement. Moreover, to address the issue of insufficient training data, an imaging model-based synthesis method that incorporates pixel disruption strategy is presented to generate underwater patches with global degradation consistency. Quantitative and subjective evaluations demonstrate that our proposed method achieves state-of-the-art performance.
- [1] . 2001. Underwater robotics. Adv. Robot. 15, 5 (2001), 609–639.Google Scholar
Cross Ref
- [2] . 2003. Automatic indexing of underwater survey video: Algorithm and benchmarking method. IEEE J. Ocean. Eng. 28, 4 (2003), 673–686.Google Scholar
Cross Ref
- [3] . 2012. Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans. Image Process. 21, 4 (2012), 1756–1769. Google Scholar
Digital Library
- [4] . 2017. High-resolution underwater robotic vision-based mapping and three-dimensional reconstruction for archaeology. J. Field Robot. 34, 4 (2017), 625–643.Google Scholar
Cross Ref
- [5] . 2002. Detecting man-made objects in unconstrained subsea videos. In Proceedings of the 13th British Machine Vision Conference (BMVC’02). 1–10.Google Scholar
Cross Ref
- [6] . 1980. A spatial processor model for object colour perception. J. Franklin Inst. 310, 1 (1980), 1–26.Google Scholar
Cross Ref
- [7] . 2010. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 12 (2010), 2341–2353. Google Scholar
Digital Library
- [8] . 2013. Single underwater image enhancement with a new optical model. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS’13). IEEE, 753–756.Google Scholar
- [9] . 2013. Transmission estimation in underwater single images. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 825–830. Google Scholar
Digital Library
- [10] . 2019. All-in-One underwater image enhancement using domain-adversarial learning. In Proceedings of the Computer Vision and Pattern Recognition Workshops (CVPR’19). 1–8.Google Scholar
- [11] . 2019. A fusion adversarial underwater image enhancement network with a public test dataset. Retrieved from https://arXiv:1906.06819.Google Scholar
- [12] . 2018. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot. Autom. Lett. 3, 1 (2018), 387–394.Google Scholar
- [13] . 2016. Estimation of ambient light and transmission map with common convolutional architecture. In Proceedings of the MTS/IEEE Oceans Conference (OCEANS’16). IEEE, 1–7.Google Scholar
Cross Ref
- [14] . 2017. A deep CNN method for underwater image enhancement. In Proceedings of the IEEE International Conference on Image Processing (ICIP’17). IEEE, 1382–1386.Google Scholar
Cross Ref
- [15] . 2011. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 12 (2011), 2341–2353. Google Scholar
Digital Library
- [16] . 2010. Removal of water scattering. In Proceedings of the IEEE International Conference on Computer Engineering Technology (ICCET’10). 35–39.Google Scholar
- [17] . 2011. Low complexity underwater image enhancement based on dark channel prior. In Proceedings of the 2nd International Conference on Innovations in Bio-inspired Computing and Applications (IBICA’11). IEEE, 17–20. Google Scholar
Digital Library
- [18] . 2015. Deriving inherent optical properties from background color and underwater image enhancement. Ocean Eng. 94 (2015), 163–172.Google Scholar
Cross Ref
- [19] . 2010. Initial results in underwater single image dehazing. In Proceedings of the MTS/IEEE Oceans Conference (OCEANS’10). IEEE, 1–8.Google Scholar
Cross Ref
- [20] . 2017. Locally adaptive color correction for underwater image dehazing and matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 1–9.Google Scholar
Cross Ref
- [21] . 2019. Progressive retinex: Mutually reinforced illumination-noise perception network for low-light image enhancement. In Proceedings of the 27th ACM International Conference on Multimedia. 2015–2023. Google Scholar
Digital Library
- [22] . 2016. Dehazenet: An end-to-end system for single image haze removal. IEEE Trans. Image Process. 25, 11 (2016), 5187–5198. Google Scholar
Digital Library
- [23] . 2017. An all-in-one network for dehazing and beyond. Retrieved from https://arXiv:1707.06543.Google Scholar
- [24] . 2021. A shape transformation-based dataset augmentation framework for pedestrian detection. Int. J. Comput. Vision 129, 4 (2021), 1121–1138.Google Scholar
Cross Ref
- [25] . 2018. Aesthetic-driven image enhancement by adversarial learning. In Proceedings of the 26th ACM International Conference on Multimedia. 870–878. Google Scholar
Digital Library
- [26] . 2020. Nighttime dehazing with a synthetic benchmark. In Proceedings of the 28th ACM International Conference on Multimedia. 2355–2363. Google Scholar
Digital Library
- [27] . 2017. Fast haze removal for nighttime image using maximum reflectance prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7418–7426.Google Scholar
Cross Ref
- [28] . 2019. Dense 3D-convolutional neural network for person re-identification in videos. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1s (2019), 1–19. Google Scholar
Digital Library
- [29] . 2020. Object relational graph with teacher-recommended learning for video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 13278–13288.Google Scholar
Cross Ref
- [30] . 2019. Convolutional attention networks for scene text recognition. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1s (2019), 1–17. Google Scholar
Digital Library
- [31] . 2020. Deep degradation prior for low-quality image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 11049–11058.Google Scholar
Cross Ref
- [32] . 2018. Emerging from water: Underwater image color correction based on weakly supervised color transfer. IEEE Signal Process. Lett. 25, 3 (2018), 323–327.Google Scholar
Cross Ref
- [33] . 2018. Fully point-wise convolutional neural network for modeling statistical regularities in natural images. In Proceedings of the 26th ACM International Conference on Multimedia. 984–992. Google Scholar
Digital Library
- [34] . 2013. Maxout networks. Retrieved from https://arXiv:1302.4389. Google Scholar
Digital Library
- [35] . 2019. An underwater image enhancement benchmark dataset and beyond. IEEE Trans. Image Process. 29 (2019), 4376–4389.Google Scholar
Cross Ref
- [36] . 2014. Underwater stereo image enhancement using a new physical model. In Proceedings of the IEEE International Conference on Image Processing (ICIP’14). IEEE, 5422–5426.Google Scholar
Cross Ref
- [37] . 2014. Illuminant estimation for color constancy: Why spatial-domain methods work and the role of the color distribution. J. Optic. Soc. Amer. A 31, 5 (2014), 1049–1058.Google Scholar
Cross Ref
- [38] . 2010. Guided image filtering. In Proceedings of the European Conference on Computer Vision. Springer, 1–14.Google Scholar
Digital Library
- [39] . 1977. The retinex theory of color vision. Sci. Amer. 237, 6 (1977), 108–129.Google Scholar
Cross Ref
- [40] . 2007. Edge-based color constancy. IEEE Trans. Image Process. 16, 9 (2007), 2207–2214. Google Scholar
Digital Library
- [41] . 2004. Shades of gray and colour constancy. In Proceedings of the Color and Imaging Conference. Society for Imaging Science and Technology, 37–41.Google Scholar
- [42] . 2015. Color constancy using CNNs. Retrieved from https://arXiv:1504.04548.Google Scholar
- [43] . 2016. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans. Image Process. 25, 12 (2016), 5664–5677. Google Scholar
Digital Library
- [44] . 2017. Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 26, 4 (2017), 1579–1594. Google Scholar
Digital Library
- [45] . 2018. Color balance and fusion for underwater image enhancement. IEEE Trans. Image Process. 27, 1 (2018), 379–393.Google Scholar
Cross Ref
- [46] . 2017. Aod-net: All-in-one dehazing network. In Proceedings of the IEEE International Conference on Computer Vision. 4770–4778.Google Scholar
Cross Ref
- [47] . 2020. FAMED-Net: A fast and accurate multi-scale end-to-end dehazing network. IEEE Trans. Image Process.: Pub. IEEE Signal Process. Soc. 29 (2020), 72.Google Scholar
Cross Ref
- [48] . 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47, 1–3 (2002), 7–42. Google Scholar
Digital Library
- [49] . 2003. High-accuracy stereo depth maps using structured light. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1. IEEE, I–I. Google Scholar
Digital Library
- [50] . 2007. Learning conditional random fields for stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07). IEEE, 1–8.Google Scholar
Cross Ref
- [51] . 2017. Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 26, 4 (2017), 1579–1594. Google Scholar
Digital Library
- [52] . 2012. Enhancing underwater images and videos by fusion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 81–88. Google Scholar
Digital Library
- [53] . 2015. Human-visual-system-inspired underwater image quality measures. IEEE J. Ocean. Eng. 41, 3 (2015), 541–551.Google Scholar
Cross Ref
- [54] . 2019. Sea-thru: A method for removing water from underwater images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1682–1691.Google Scholar
Cross Ref
- [55] . 2020. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn. 98 (2020), 107038.Google Scholar
Digital Library
- [56] . 2016. Salient object detection via structured matrix decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 4 (2016), 818–832. Google Scholar
Digital Library
- [57] . 2006. SURF: Speeded up robust features. In Proceedings of the European Conference on Computer Vision. Springer, 404–417. Google Scholar
Digital Library
- [58] . 2020. Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light. IEEE Trans. Circ. Syst. Video Technol. 30, 12 (2020), 4861–4875.Google Scholar
Digital Library
Index Terms
Leveraging Deep Statistics for Underwater Image Enhancement
Recommendations
Underwater image enhancement by color correction and color constancy via Retinex for detail preserving
AbstractIn underwater, light attenuation causes non-uniform illumination that degrades underwater image. To enhance the degraded image, we propose an underwater image enhancement method that includes color correction, color constancy, multi-...
Graphical abstractDisplay Omitted
Highlights- The formulation of color correction compensates the red and blue channels by masking.
Underwater image enhancement method with light scattering characteristics
Highlights- Underwater image is divided into five categories.
- Calculate the color loss rate ...
AbstractLight is absorbed and scattered when propagating in water, which results in low quality and poor visibility of underwater optical images. Furthermore, the absorption of light by water causes color distortion, whereas the scattering of ...
Graphical abstractDisplay Omitted
Underwater image enhancement by combining color constancy and dehazing based on depth estimation
Highlights- Proposed an underwater image enhancement method using color constancy and dehazing.
AbstractThe physical properties which are present in the underwater environment affects the images captured by the visual sensors. As a consequence of these properties, the captured image includes non-uniform illumination. This non-uniform ...






Comments