skip to main content
research-article

Leveraging Deep Statistics for Underwater Image Enhancement

Authors Info & Claims
Published:26 October 2021Publication History
Skip Abstract Section

Abstract

Underwater imaging often suffers from color cast and contrast degradation due to range-dependent medium absorption and light scattering. Introducing image statistics as prior has been proved to be an effective solution for underwater image enhancement. However, relative to the modal divergence of light propagation and underwater scenery, the existing methods are limited in representing the inherent statistics of underwater images resulting in color artifacts and haze residuals. To address this problem, this article proposes a convolutional neural network (CNN)-based framework to learn hierarchical statistical features related to color cast and contrast degradation and to leverage them for underwater image enhancement. Specifically, a pixel disruption strategy is first proposed to suppress intrinsic colors’ influence and facilitate modeling a unified statistical representation of underwater image. Then, considering the local variation of depth of field, two parallel sub-networks: Color Correction Network (CC-Net) and Contrast Enhancement Network (CE-Net) are presented. The CC-Net and CE-Net can generate pixel-wise color cast and transmission map and achieve spatial-varied color correction and contrast enhancement. Moreover, to address the issue of insufficient training data, an imaging model-based synthesis method that incorporates pixel disruption strategy is presented to generate underwater patches with global degradation consistency. Quantitative and subjective evaluations demonstrate that our proposed method achieves state-of-the-art performance.

REFERENCES

  1. [1] Yuh Junku and West Michael. 2001. Underwater robotics. Adv. Robot. 15, 5 (2001), 609639.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Lebart Katia, Smith Chris, Trucco Emanuele, and Lane David M.. 2003. Automatic indexing of underwater survey video: Algorithm and benchmarking method. IEEE J. Ocean. Eng. 28, 4 (2003), 673686.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Chiang John Y. and Chen Ying-Ching. 2012. Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans. Image Process. 21, 4 (2012), 17561769. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Johnson-Roberson Matthew, Bryson Mitch, Friedman Ariell, Pizarro Oscar, Troni Giancarlo, Ozog Paul, and Henderson Jon C.. 2017. High-resolution underwater robotic vision-based mapping and three-dimensional reconstruction for archaeology. J. Field Robot. 34, 4 (2017), 625643.Google ScholarGoogle ScholarCross RefCross Ref
  5. [5] Olmos Adriana and Trucco Emanuele. 2002. Detecting man-made objects in unconstrained subsea videos. In Proceedings of the 13th British Machine Vision Conference (BMVC’02). 110.Google ScholarGoogle ScholarCross RefCross Ref
  6. [6] Buchsbaum Gershon. 1980. A spatial processor model for object colour perception. J. Franklin Inst. 310, 1 (1980), 126.Google ScholarGoogle ScholarCross RefCross Ref
  7. [7] He Kaiming, Sun Jian, and Tang Xiaoou. 2010. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 12 (2010), 23412353. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Wen Haocheng, Tian Yonghong, Huang Tiejun, and Gao Wen. 2013. Single underwater image enhancement with a new optical model. In Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS’13). IEEE, 753756.Google ScholarGoogle Scholar
  9. [9] Drews Paul, Nascimento Erickson, Moraes Filipe, Botelho Silvia, and Campos Mario. 2013. Transmission estimation in underwater single images. In Proceedings of the IEEE International Conference on Computer Vision Workshops. 825830. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. [10] Uplavikar Pritish M., Wu Zhenyu, and Wang Zhangyang. 2019. All-in-One underwater image enhancement using domain-adversarial learning. In Proceedings of the Computer Vision and Pattern Recognition Workshops (CVPR’19). 18.Google ScholarGoogle Scholar
  11. [11] Li Hanyu, Li Jingjing, and Wang Wei. 2019. A fusion adversarial underwater image enhancement network with a public test dataset. Retrieved from https://arXiv:1906.06819.Google ScholarGoogle Scholar
  12. [12] Li Jie, Skinner Katherine A, Eustice Ryan M., and Johnson-Roberson Matthew. 2018. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot. Autom. Lett. 3, 1 (2018), 387394.Google ScholarGoogle Scholar
  13. [13] Shin Young-Sik, Cho Younggun, Pandey Gaurav, and Kim Ayoung. 2016. Estimation of ambient light and transmission map with common convolutional architecture. In Proceedings of the MTS/IEEE Oceans Conference (OCEANS’16). IEEE, 17.Google ScholarGoogle ScholarCross RefCross Ref
  14. [14] Wang Yang, Zhang Jing, Cao Yang, and Wang Zengfu. 2017. A deep CNN method for underwater image enhancement. In Proceedings of the IEEE International Conference on Image Processing (ICIP’17). IEEE, 13821386.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] He Kaiming, Sun Jian, and Tang Xiaoou. 2011. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 12 (2011), 23412353. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. [16] Chao L. and Wang M.. 2010. Removal of water scattering. In Proceedings of the IEEE International Conference on Computer Engineering Technology (ICCET’10). 3539.Google ScholarGoogle Scholar
  17. [17] Yang Hung-Yu, Chen Pei-Yin, Huang Chien-Chuan, Zhuang Ya-Zhu, and Shiau Yeu-Horng. 2011. Low complexity underwater image enhancement based on dark channel prior. In Proceedings of the 2nd International Conference on Innovations in Bio-inspired Computing and Applications (IBICA’11). IEEE, 1720. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. [18] Zhao Xinwei, Jin Tao, and Qu Song. 2015. Deriving inherent optical properties from background color and underwater image enhancement. Ocean Eng. 94 (2015), 163172.Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Carlevaris-Bianco Nicholas, Mohan Anush, and Eustice Ryan M.. 2010. Initial results in underwater single image dehazing. In Proceedings of the MTS/IEEE Oceans Conference (OCEANS’10). IEEE, 18.Google ScholarGoogle ScholarCross RefCross Ref
  20. [20] Ancuti Codruta O., Ancuti Cosmin, Vleeschouwer Christophe De, and Garcia Rafael. 2017. Locally adaptive color correction for underwater image dehazing and matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 19.Google ScholarGoogle ScholarCross RefCross Ref
  21. [21] Wang Yang, Cao Yang, Zha Zheng-Jun, Zhang Jing, Xiong Zhiwei, Zhang Wei, and Wu Feng. 2019. Progressive retinex: Mutually reinforced illumination-noise perception network for low-light image enhancement. In Proceedings of the 27th ACM International Conference on Multimedia. 20152023. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Cai Bolun, Xu Xiangmin, Jia Kui, Qing Chunmei, and Tao Dacheng. 2016. Dehazenet: An end-to-end system for single image haze removal. IEEE Trans. Image Process. 25, 11 (2016), 51875198. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Li Boyi, Peng Xiulian, Wang Zhangyang, Xu Jizheng, and Feng Dan. 2017. An all-in-one network for dehazing and beyond. Retrieved from https://arXiv:1707.06543.Google ScholarGoogle Scholar
  24. [24] Chen Zhe, Ouyang Wanli, Liu Tongliang, and Tao Dacheng. 2021. A shape transformation-based dataset augmentation framework for pedestrian detection. Int. J. Comput. Vision 129, 4 (2021), 11211138.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Deng Yubin, Loy Chen Change, and Tang Xiaoou. 2018. Aesthetic-driven image enhancement by adversarial learning. In Proceedings of the 26th ACM International Conference on Multimedia. 870878. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Zhang Jing, Cao Yang, Zha Zheng-Jun, and Tao Dacheng. 2020. Nighttime dehazing with a synthetic benchmark. In Proceedings of the 28th ACM International Conference on Multimedia. 23552363. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. [27] Zhang Jing, Cao Yang, Fang Shuai, Kang Yu, and Chen Chang Wen. 2017. Fast haze removal for nighttime image using maximum reflectance prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 74187426.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Liu Jiawei, Zha Zheng-Jun, Chen Xuejin, Wang Zilei, and Zhang Yongdong. 2019. Dense 3D-convolutional neural network for person re-identification in videos. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1s (2019), 119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. [29] Zhang Ziqi, Shi Yaya, Yuan Chunfeng, Li Bing, Wang Peijin, Hu Weiming, and Zha Zheng-Jun. 2020. Object relational graph with teacher-recommended learning for video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1327813288.Google ScholarGoogle ScholarCross RefCross Ref
  30. [30] Xie Hongtao, Fang Shancheng, Zha Zheng-Jun, Yang Yating, Li Yan, and Zhang Yongdong. 2019. Convolutional attention networks for scene text recognition. ACM Trans. Multimedia Comput. Commun. Appl. 15, 1s (2019), 117. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. [31] Wang Yang, Cao Yang, Zha Zheng-Jun, Zhang Jing, and Xiong Zhiwei. 2020. Deep degradation prior for low-quality image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1104911058.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Li Chongyi, Guo Jichang, and Guo Chunle. 2018. Emerging from water: Underwater image color correction based on weakly supervised color transfer. IEEE Signal Process. Lett. 25, 3 (2018), 323327.Google ScholarGoogle ScholarCross RefCross Ref
  33. [33] Zhang Jing, Cao Yang, Wang Yang, Wen Chenglin, and Chen Chang Wen. 2018. Fully point-wise convolutional neural network for modeling statistical regularities in natural images. In Proceedings of the 26th ACM International Conference on Multimedia. 984992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Goodfellow Ian J., Warde-Farley David, Mirza Mehdi, Courville Aaron, and Bengio Yoshua. 2013. Maxout networks. Retrieved from https://arXiv:1302.4389. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Li Chongyi, Guo Chunle, Ren Wenqi, Cong Runmin, Hou Junhui, Kwong Sam, and Tao Dacheng. 2019. An underwater image enhancement benchmark dataset and beyond. IEEE Trans. Image Process. 29 (2019), 43764389.Google ScholarGoogle ScholarCross RefCross Ref
  36. [36] Zhang Shijie, Zhang Jing, Fang Shuai, and Cao Yang. 2014. Underwater stereo image enhancement using a new physical model. In Proceedings of the IEEE International Conference on Image Processing (ICIP’14). IEEE, 54225426.Google ScholarGoogle ScholarCross RefCross Ref
  37. [37] Cheng Dongliang, Prasad Dilip K., and Brown Michael S.. 2014. Illuminant estimation for color constancy: Why spatial-domain methods work and the role of the color distribution. J. Optic. Soc. Amer. A 31, 5 (2014), 10491058.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] He Kaiming, Sun Jian, and Tang Xiaoou. 2010. Guided image filtering. In Proceedings of the European Conference on Computer Vision. Springer, 114.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. [39] Land Edwin H.. 1977. The retinex theory of color vision. Sci. Amer. 237, 6 (1977), 108129.Google ScholarGoogle ScholarCross RefCross Ref
  40. [40] Weijer Joost Van De, Gevers Theo, and Gijsenij Arjan. 2007. Edge-based color constancy. IEEE Trans. Image Process. 16, 9 (2007), 22072214. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Finlayson Graham D. and Trezzi Elisabetta. 2004. Shades of gray and colour constancy. In Proceedings of the Color and Imaging Conference. Society for Imaging Science and Technology, 3741.Google ScholarGoogle Scholar
  42. [42] Bianco Simone, Cusano Claudio, and Schettini Raimondo. 2015. Color constancy using CNNs. Retrieved from https://arXiv:1504.04548.Google ScholarGoogle Scholar
  43. [43] Li Chong-Yi, Guo Ji-Chang, Cong Run-Min, Pang Yan-Wei, and Wang Bo. 2016. Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans. Image Process. 25, 12 (2016), 56645677. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. [44] Peng Yan-Tsung and Cosman Pamela C.. 2017. Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 26, 4 (2017), 15791594. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. [45] Ancuti Codruta O., Ancuti Cosmin, Vleeschouwer Christophe De, and Bekaert Philippe. 2018. Color balance and fusion for underwater image enhancement. IEEE Trans. Image Process. 27, 1 (2018), 379393.Google ScholarGoogle ScholarCross RefCross Ref
  46. [46] Li Boyi, Peng Xiulian, Wang Zhangyang, Xu Jizheng, and Feng Dan. 2017. Aod-net: All-in-one dehazing network. In Proceedings of the IEEE International Conference on Computer Vision. 47704778.Google ScholarGoogle ScholarCross RefCross Ref
  47. [47] Zhang J. and Tao D.. 2020. FAMED-Net: A fast and accurate multi-scale end-to-end dehazing network. IEEE Trans. Image Process.: Pub. IEEE Signal Process. Soc. 29 (2020), 72.Google ScholarGoogle ScholarCross RefCross Ref
  48. [48] Scharstein Daniel and Szeliski Richard. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47, 1–3 (2002), 742. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. [49] Scharstein Daniel and Szeliski Richard. 2003. High-accuracy stereo depth maps using structured light. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1. IEEE, I–I. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. [50] Scharstein Daniel and Pal Chris. 2007. Learning conditional random fields for stereo. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07). IEEE, 18.Google ScholarGoogle ScholarCross RefCross Ref
  51. [51] Peng Yan-Tsung and Cosman Pamela C.. 2017. Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 26, 4 (2017), 15791594. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. [52] Ancuti Cosmin, Ancuti Codruta Orniana, Haber Tom, and Bekaert Philippe. 2012. Enhancing underwater images and videos by fusion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, 8188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. [53] Panetta Karen, Gao Chen, and Agaian Sos. 2015. Human-visual-system-inspired underwater image quality measures. IEEE J. Ocean. Eng. 41, 3 (2015), 541551.Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Akkaynak Derya and Treibitz Tali. 2019. Sea-thru: A method for removing water from underwater images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 16821691.Google ScholarGoogle ScholarCross RefCross Ref
  55. [55] Li Chongyi, Anwar Saeed, and Porikli Fatih. 2020. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn. 98 (2020), 107038.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. [56] Peng Houwen, Li Bing, Ling Haibin, Hu Weiming, Xiong Weihua, and Maybank Stephen J.. 2016. Salient object detection via structured matrix decomposition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 4 (2016), 818832. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. [57] Bay Herbert, Tuytelaars Tinne, and Gool Luc Van. 2006. SURF: Speeded up robust features. In Proceedings of the European Conference on Computer Vision. Springer, 404417. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. [58] Liu Risheng, Fan Xin, Zhu Ming, Hou Minjun, and Luo Zhongxuan. 2020. Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light. IEEE Trans. Circ. Syst. Video Technol. 30, 12 (2020), 48614875.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Leveraging Deep Statistics for Underwater Image Enhancement

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Multimedia Computing, Communications, and Applications
      ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 3s
      October 2021
      324 pages
      ISSN:1551-6857
      EISSN:1551-6865
      DOI:10.1145/3492435
      Issue’s Table of Contents

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 26 October 2021
      • Accepted: 1 August 2021
      • Revised: 1 July 2021
      • Received: 1 December 2020
      Published in tomm Volume 17, Issue 3s

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    View Full Text

    HTML Format

    View this article in HTML Format .

    View HTML Format
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!