skip to main content
research-article

A Real-Time Effective Fusion-Based Image Defogging Architecture on FPGA

Published:22 July 2021Publication History
Skip Abstract Section

Abstract

Foggy weather reduces the visibility of photographed objects, causing image distortion and decreasing overall image quality. Many approaches (e.g., image restoration, image enhancement, and fusion-based methods) have been proposed to work out the problem. However, most of these defogging algorithms are facing challenges such as algorithm complexity or real-time processing requirements. To simplify the defogging process, we propose a fusional defogging algorithm on the linear transmission of gray single-channel. This method combines gray single-channel linear transform with high-boost filtering according to different proportions. To enhance the visibility of the defogging image more effectively, we convert the RGB channel into a gray-scale single channel without decreasing the defogging results. After gray-scale fusion, the data in the gray-scale domain should be linearly transmitted. With the increasing real-time requirements for clear images, we also propose an efficient real-time FPGA defogging architecture. The architecture optimizes the data path of the guided filtering to speed up the defogging speed and save area and resources. Because the pixel reading order of mean and square value calculations are identical, the shift register in the box filter after the average and the computation of the square values is separated from the box filter and put on the input terminal for sharing, saving the storage area. What’s more, using LUTs instead of the multiplier can decrease the time delays of the square value calculation module and increase efficiency. Experimental results show that the linear transmission can save 66.7% of the total time. The architecture we proposed can defog efficiently and accurately, meeting the real-time defogging requirements on 1920 × 1080 image size.

References

  1. C. Ancuti, C. O. Ancuti, T. Haber, and P. Bekaert. 2012. Enhancing underwater images and videos by fusion. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 81–88. DOI:https://doi.org/10.1109/CVPR.2012.6247661 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. C. O. Ancuti and C. Ancuti. 2013. Single image dehazing by multi-scale fusion. IEEE Transactions on Image Processing 22, 8 (August 2013), 3271–3282. DOI:https://doi.org/10.1109/TIP.2013.2262284Google ScholarGoogle ScholarCross RefCross Ref
  3. Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and Dacheng Tao. 2016. DehazeNet: An end-to-end system for single image haze removal. IEEE Transactions on Image Processing 25, 11 (2016), 5187–5198. DOI:http://dx.doi.org/10.1109/TIP.2016.2598681 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Akshay Dudhane and Subrahmanyam Murala. 2020. RYF-Net: Deep fusion network for single image haze removal. IEEE Transactions on Image Processing 29 (2020), 628–640. DOI:https://doi.org/10.1109/TIP.2019.2934360Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Dragomir El-Mezeni and Lazar Saranovac. 2018. Fast guided filter for power-efficient real-time 1080p streaming video processing. Journal of Real-Time Image Processing (04 July 2018). DOI:https://doi.org/10.1007/s11554-018-0802-zGoogle ScholarGoogle Scholar
  6. Raanan Fattal. 2014. Dehazing using color-lines. ACM Trans. Graph. 34, 1, Article 13 (December 2014), 14 pages. DOI:https://doi.org/10.1145/2651362 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. X. Fu, Y. Huang, D. Zeng, X. Zhang, and X. Ding. 2014. A fusion-based enhancing approach for single sandstorm image. In 2014 IEEE 16th International Workshop on Multimedia Signal Processing (MMSP). 1–5. DOI:https://doi.org/10.1109/MMSP.2014.6958791Google ScholarGoogle ScholarCross RefCross Ref
  8. Yin Gao, Yijing Su, Qiming Li, and Jun Li. 2018. Single image dehazing via relativity-of-Gaussian. In 2018 IEEE 4th International Conference on Computer and Communications (ICCC). 1665–1669. DOI:https://doi.org/10.1109/CompComm.2018.8780924Google ScholarGoogle ScholarCross RefCross Ref
  9. Yin Gao, Lijun Yun, Junsheng Shi, Feiyan Chen, and Liansha Lei. 2014. Enhancement MSRCR algorithm of color fog image based on the adaptive scale. In 6th International Conference on Digital Image Processing (ICDIP 2014), Charles M. Falco, Chin-Chen Chang, and Xudong Jiang (Eds.), Vol. 9159. International Society for Optics and Photonics, 253–59. https://doi.org/10.1117/12.2064391Google ScholarGoogle Scholar
  10. Li Guo, Long Chen, and C. L. Philip Chen. 2018. Shadowed non-local image guided filter. In 9th International Conference on Graphic and Image Processing (ICGIP 2017), Vol. 10615. SPIE, Qingdao, China, 1424–1430. DOI:https://doi.org/10.1117/12.2302633Google ScholarGoogle Scholar
  11. Kaiming He, Jian Sun, and Xiaoou Tang. 2011. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence 33, 12 (2011), 2341–2353. DOI:http://dx.doi.org/10.1109/TPAMI.2010.168 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided image filtering. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 35 (2013), 1397–1409. DOI:http://dx.doi.org/10.1109/TPAMI.2012.213 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. D. J. Jobson, Z. Rahman, and G. A. Woodell. 1997. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Transactions on Image Processing 6, 7 (July 1997), 965–976. DOI:https://doi.org/10.1109/83.597272 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. D. J. Jobson, Z. Rahman, and G. A. Woodell. 1997. Properties and performance of a center/surround retinex. IEEE Transactions on Image Processing 6, 3 (March 1997), 451–462. DOI:https://doi.org/10.1109/83.557356 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J.-Y. Kim, L.-S. Kim, and S.-H. Hwang. 2001. An advanced contrast enhancement using partially overlapped sub-block histogram equalization. IEEE Transactions on Circuits and Systems for Video Technology 11, 4 (2001), 475–484. DOI:http://dx.doi.org/10.1109/76.915354 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Rahul Kumar, Brajesh Kumar Kaushik, and R. Balasubramanian. 2017. FPGA implementation of image dehazing algorithm for real time applications. In Applications of Digital Image Processing XL 2017. SPIE, San Diego, CA, The Society of Photo–Optical Instrumentation Engineers (SPIE) –. DOI:https://doi.org/10.1117/12.2274682Google ScholarGoogle Scholar
  17. Wahengbam Kanan Kumar, Kishorjit Nongmeikapam, and Aheibam Dinamani Singh. 2019. Enhancing scene perception using a multispectral fusion of visible/near-infrared image pair. IET Image Processing 13 (14 November 2019), 2467–2479(12). DOI:https://doi.org/10.1049/iet-ipr.2018.5812Google ScholarGoogle Scholar
  18. Edwin H. Land and John J. McCann. 1971. Lightness and retinex theory. J. Opt. Soc. Am. 61, 1 (January 1971), 1–11. DOI:https://doi.org/10.1364/JOSA.61.000001Google ScholarGoogle ScholarCross RefCross Ref
  19. Y. Lee and B. Wu. 2019. Algorithm and architecture design of a hardware-efficient image dehazing engine. IEEE Transactions on Circuits and Systems for Video Technology 29, 7 (2019), 2146–2161. DOI:https://doi.org/10.1109/TCSVT.2018.2862906Google ScholarGoogle ScholarCross RefCross Ref
  20. Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, and Dan Feng. 2017. AOD-Net: All-in-one dehazing network. In IEEE International Conference on Computer Vision. Venice, Italy, 4780–4788. DOI:http://dx.doi.org/10.1109/ICCV.2017.511Google ScholarGoogle ScholarCross RefCross Ref
  21. Zhengfa Lianga, Hengzhu Liu, Botao Zhang, and Wang Benzhang. 2014. Real-time hardware accelerator for single image haze removal using dark channel prior and guided filter. IEICE Electronics Express 11, 24 (2014). DOI:http://dx.doi.org/10.1587/elex.11.20141002Google ScholarGoogle ScholarCross RefCross Ref
  22. Heng Liu, Dongdong Huang, Shudong Hou, and Ruan Yue. 2017. Large-size single-image fast-defogging and the real -time video defogging FPGA architecture. Neurocomputing 269 (2017), 97–107. DOI:http://dx.doi.org/10.1016/j.neucom.2016.09.139 Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Zhongli Ma, Jie Wen, and Xiumei Liang. 2013. Video image clarity algorithm research of USV visual system under the sea fog. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 7929 LNCS, PART 2, 436–444. DOI:https://doi.org/10.1007/978-3-642-38715-9-52Google ScholarGoogle Scholar
  24. Zhongli Ma, Jie Wen, Cheng Zhang, Quanyong Liu, and Danniang Yan. 2016. An effective fusion defogging approach for single sea fog image. Neurocomputing 173 (2016), 1257–1267. DOI:http://dx.doi.org/10.1016/j.neucom.2015.08.084 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Gaofeng Meng, Ying Wang, Jiangyong Duan, Shiming Xiang, and Chunhong Pan. 2013. Efficient image dehazing with boundary constraint and contextual regularization. In IEEE International Conference on Computer Vision, 617–624. DOI:https://doi.org/10.1109/ICCV.2013.82 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Yongmin Park and Tae-Hwan Kim. 2017. A video dehazing system based on fast airlight estimation. In 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP 2017). Institute of Electrical and Electronics Engineers Inc., United States, 779–783. DOI:https://doi.org/10.1109/GlobalSIP.2017.8309066Google ScholarGoogle ScholarCross RefCross Ref
  27. Z. Rahman, D. J. Jobson, and G. A. Woodell. 1996. Multi-scale retinex for color image enhancement. In 3rd IEEE International Conference on Image Processing, Vol. 3. Lausanne, Switzerland, 1003–1006. DOI:https://doi.org/10.1109/ICIP.1996.560995Google ScholarGoogle ScholarCross RefCross Ref
  28. Wenqi Ren, Lin Ma, Jiawei Zhang, Jinshan Pan, Xiaochun Cao, Wei Liu, and Ming-Hsuan Yang. 2018. Gated fusion network for single image dehazing. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, 3253–3261. DOI:http://dx.doi.org/10.1109/CVPR.2018.00343Google ScholarGoogle ScholarCross RefCross Ref
  29. L. Schaul, C. Fredembach, and S. Sijsstrunk. 2009. Color image dehazing using the near-infrared. In 2009 16th IEEE International Conference on Image Processing (ICIP). 1629–1632. DOI:https://doi.org/10.1109/ICIP.2009.5413700 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Yu Shen, Jian-Wu Dang, Ji-Xiang Gou, Rui Guo, Cheng Liu, Xiao-Peng Wang, and Lei Li. 2019. A dehaze algorithm based on near-infrared and visible dual channel sensor information fusion. Guang Pu Xue Yu Guang Pu Fen Xi/Spectroscopy and Spectral Analysis 39, 5 (2019), 1420–1427. DOI:http://dx.doi.org/10.3964/j.issn.1000-0593(2019)05-1420-08Google ScholarGoogle Scholar
  31. Y. Shiau, H. Yang, P. Chen, and Y. Chuang. 2013. Hardware implementation of a fast and efficient haze removal method. IEEE Transactions on Circuits and Systems for Video Technology 23, 8 (2013), 1369–1374. DOI:https://doi.org/10.1109/TCSVT.2013.2243650 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Dilbag Singh and Vijay Kumar. 2018. Comprehensive survey on haze removal techniques. Multimedia Tools and Applications 77, 8 (2018), 9595–9620. DOI:http://dx.doi.org/10.1007/s11042-017-5321-6 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Jean-Philippe Tarel and Nicolas Hautiere. 2009. Fast visibility restoration from a single color or gray level image. In 2009 IEEE 12th International Conference on Computer Vision. Kyoto, Japan, 2201–2208. DOI:https://doi.org/10.1109/ICCV.2009.5459251Google ScholarGoogle ScholarCross RefCross Ref
  34. J. Varalakshmi, Deepa Jose, and P. Nirmal Kumar. 2020. FPGA implementation of haze removal technique based on dark channel prior. In 3rd International Conference on Computational Vision and Bio Inspired Computing (ICCVBIC 2019), Vol. 1108 AISC. Springer Science and Business Media Deutschland GmbH, Coimbatore, India, 624–630. DOI:https://doi.org/10.1007/978-3-030-37218-7_71Google ScholarGoogle Scholar
  35. Chunxia Xiao and Jiajia Gan. 2012. Fast image dehazing using guided joint bilateral filter. Visual Computer 28, 6–8 (2012), 713–721. DOI:http://dx.doi.org/10.1007/s00371-012-0679-y Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Long Xu, Dong Zhao, Yihua Yan, Sam Kwong, Jie Chen, and Ling-Yu Duan. 2019. IDeRs: Iterative dehazing method for single remote sensing image. Information Sciences (2019), 50–62. DOI:http://dx.doi.org/10.1016/j.ins.2019.02.058Google ScholarGoogle Scholar
  37. Z. Xu, X. Liu, and X. Chen. 2009. Fog removal from video sequences using contrast limited adaptive histogram equalization. In 2009 International Conference on Computational Intelligence and Software Engineering. Wuhan, China, 1–4. DOI:https://doi.org/10.1109/CISE.2009.5366207Google ScholarGoogle Scholar
  38. Dong Zhao, Long Xu, Yihua Yan, Jie Chen, and Ling-Yu Duan. 2019. Multi-scale optimal fusion model for single image dehazing. Signal Processing: Image Communication 74 (2019), 253–265. DOI:http://dx.doi.org/10.1016/j.image.2019.02.004Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Karel Zuiderveld. 1994. Contrast Limited Adaptive Histogram Equalization. Academic Press Professional, Inc., San Diego, CA, 474–485. http://dlacm.xilesou.top/citation.cfm?id=180895.180940. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Shih-Chia Huang, Bo-Hao Chen, and Wei-Jheng Wang. 2014. Visibility restoration of single hazy images captured in real-world weather conditions. IEEE Transactions on Circuits and Systems for Video Technology 24, 10 (2014), 1814–1824. http://dx.doi.org/10.1109/TCSVT.2014.2317854Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. A Real-Time Effective Fusion-Based Image Defogging Architecture on FPGA

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 17, Issue 3
          August 2021
          443 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/3476118
          Issue’s Table of Contents

          Copyright © 2021 Association for Computing Machinery.

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 22 July 2021
          • Revised: 1 December 2020
          • Accepted: 1 December 2020
          • Received: 1 September 2020
          Published in tomm Volume 17, Issue 3

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed
        • Article Metrics

          • Downloads (Last 12 months)73
          • Downloads (Last 6 weeks)4

          Other Metrics

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format .

        View HTML Format
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!