Abstract
Recent works on single image high dynamic range (HDR) reconstruction fail to hallucinate plausible textures, resulting in information missing and artifacts in large-scale under/over-exposed regions. In this article, a decoupled kernel prediction network is proposed to infer an HDR image from a low dynamic range (LDR) image. Specifically, we first adopt a simple module to generate a preliminary result, which can precisely estimate well-exposed HDR regions. Meanwhile, an encoder-decoder backbone network with a soft mask guidance module is presented to predict pixel-wise kernels, which is further convolved with the preliminary result to obtain the final HDR output. Instead of traditional kernels, our predicted kernels are decoupled along the spatial and channel dimensions. The advantages of our method are threefold at least. First, our model is guided by the soft mask so that it can focus on the most relevant information for under/over-exposed regions. Second, pixel-wise kernels are able to adaptively solve the different degradations for differently exposed regions. Third, decoupled kernels can avoid information redundancy across channels and reduce the solution space of our model. Thus, our method is able to hallucinate fine details in the under/over-exposed regions and renders visually pleasing results. Extensive experiments demonstrate that our model outperforms state-of-the-art ones.
Supplemental Material
Available for Download
Supplementary material
- [1] . 2021. Single image HDR synthesis using a densely connected dilated ConvNet. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2021, virtual, June 19–25, 2021. Computer Vision Foundation/IEEE, 526–531.Google Scholar
- [2] . 2008. Extending quality metrics to full dynamic range images. In Proceedings of the Human Vision and Electronic Imaging XIII (Proceedings of SPIE). San Jose, 6806–10.Google Scholar
Cross Ref
- [3] . 2017. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Transactions on Graphics 36, 4 (2017), 97:1–97:14.Google Scholar
Digital Library
- [4] . 2017. Advanced High Dynamic Range Imaging (2nd Edition). AK Peters (CRC Press), Natick, MA. Google Scholar
Digital Library
- [5] . 2009. High dynamic range imaging and low dynamic range expansion for generating HDR content. Computer Graphics Forum 28, 8 (2009), 2343–2367.Google Scholar
Cross Ref
- [6] . 2007. A framework for inverse tone mapping. Visual Computer 23, 7 (2007), 467–478.Google Scholar
Digital Library
- [7] . 2011. Learning photographic global tonal adjustment with a database of input/output image pairs. In Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011. IEEE Computer Society, 97–104.Google Scholar
- [8] . 2021. A brightness-adaptive kernel prediction network for inverse tone mapping. Neurocomputing 464 (2021), 1–14.Google Scholar
Digital Library
- [9] . 2021. Single-image HDR reconstruction with task-specific network based on channel adaptive RDN. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2021, virtual, June 19–25, 2021. Computer Vision Foundation/IEEE, 398–403.Google Scholar
Cross Ref
- [10] . 2021. HDRUNet: Single image HDR reconstruction with denoising and dequantization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2021, virtual, June 19-25, 2021. Computer Vision Foundation/IEEE, 354–363.Google Scholar
Cross Ref
- [11] . 2015. RAISE: A raw images dataset for digital image forensics. In Proceedings of the 6th ACM Multimedia Systems Conference, MMSys 2015. ACM, 219–224.Google Scholar
Digital Library
- [12] . 1997. Recovering high dynamic range radiance maps from photographs. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1997. ACM, 369–378.Google Scholar
Digital Library
- [13] . 2021. How to cheat with metrics in single-image HDR reconstruction. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops, ICCVW 2021. IEEE, 3981–3990.Google Scholar
Cross Ref
- [14] . 2017. HDR image reconstruction from a single exposure using deep CNNs. ACM Transactions on Graphics 36, 6 (2017), 178:1–178:15.Google Scholar
Digital Library
- [15] . 2017. Deep reverse tone mapping. ACM Transactions on Graphics 36, 6 (2017), 177:1–177:10.Google Scholar
Digital Library
- [16] . 2003. What is the space of camera response functions?. In Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2003). IEEE Computer Society, 602–612.Google Scholar
Cross Ref
- [17] . 2018. Squeeze-and-excitation networks. In Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018. Computer Vision Foundation/IEEE Computer Society, 7132–7141.Google Scholar
Cross Ref
- [18] . 2018. Over- and under-exposure reconstruction of a single plenoptic capture. ACM Transactions on Multimedia Computing, Communications, and Applications 14, 2 (2018), 52:1–52:21.Google Scholar
Digital Library
- [19] . 2014. Physiological inverse tone mapping based on retina response. Visual Computer 30, 5 (2014), 507–517.Google Scholar
Cross Ref
- [20] . 2017. Image-to-image translation with conditional adversarial networks. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. IEEE Computer Society, 5967–5976.Google Scholar
Cross Ref
- [21] . 2019. FHDR: HDR image reconstruction from a single LDR image using feedback network. In Proceedings of the 2019 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2019. IEEE, 1–5.Google Scholar
Cross Ref
- [22] . 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, Conference Track Proceedings.Google Scholar
- [23] . 2017. Reinhard’s global operator based inverse tone mapping with one parameter. In Proceedings of the 8th International Workshop on Signal Design and Its Applications in Communications, IWSDA 2017. IEEE, 49–53.Google Scholar
Cross Ref
- [24] . 2014. High-quality reverse tone mapping for a wide range of exposures. In Proceedings of the 27th SIBGRAPI Conference on Graphics, Patterns and Images, SIBGRAPI 2014. IEEE Computer Society, 49–56.Google Scholar
Digital Library
- [25] . 2002. Production-ready global illumination. Siggraph Course Notes 16, 2002 (2002), 11.Google Scholar
- [26] . 2017. Photo-realistic single image super-resolution using a generative adversarial network. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. IEEE Computer Society, 105–114.Google Scholar
Cross Ref
- [27] . 2020. Single-image HDR reconstruction by learning to reverse the camera pipeline. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle. Computer Vision Foundation/IEEE, 1648–1657.Google Scholar
Cross Ref
- [28] . 2011. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Transactions on Graphics 30, 4 (2011), 40.Google Scholar
Digital Library
- [29] . 2018. ExpandNet: A deep convolutional neural network for high dynamic range expansion from low dynamic range content. Computer Graphics Forum 37, 2 (2018), 37–49.Google Scholar
Cross Ref
- [30] . 2017. Video frame interpolation via adaptive separable convolution. In Proceedings of the IEEE International Conference on Computer Vision, ICCV 2017. IEEE Computer Society, 261–270.Google Scholar
Cross Ref
- [31] . 2005. Dynamic range reduction inspired by photoreceptor physiology. IEEE Transactions on Visualization and Computer Graphics 11, 1 (2005), 13–24.Google Scholar
Digital Library
- [32] . 2007. Ldr2Hdr: On-the-fly reverse tone mapping of legacy video and photographs. ACM Transactions on Graphics 26, 3 (2007), 39.Google Scholar
Digital Library
- [33] . 2015. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015-18th International Conference Munich, Proceedings, Part III (Lecture Notes in Computer Science), Vol. 9351. Springer, 234–241.Google Scholar
Cross Ref
- [34] . 2020. Single image HDR reconstruction using a CNN with masked features and perceptual loss. ACM Transactions on Graphics 39, 4 (2020), 1–10.Google Scholar
Digital Library
- [35] . 2018. Recovering realistic texture in image super-resolution by deep spatial feature transform. In Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018. Computer Vision Foundation/IEEE Computer Society, 606–615.Google Scholar
Cross Ref
- [36] . 2004. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612.Google Scholar
Digital Library
- [37] . 2003. Multiscale structural similarity for image quality assessment. In Proceedings of the 37th Asilomar Conference on Signals, Systems & Computers, 2003, Vol. 2. IEEE, 1398–1402.Google Scholar
Cross Ref
- [38] . 2018. CBAM: Convolutional block attention module. In Proceedings of the Computer Vision - ECCV 2018-15th European Conference, Proceedings, Part VII (Lecture Notes in Computer Science), Vol. 11211. Springer, 3–19.Google Scholar
Digital Library
- [39] . 2020. Basis prediction networks for effective burst denoising with large kernels. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020. Computer Vision Foundation/IEEE, 11841–11850.Google Scholar
Cross Ref
- [40] . 2021. Exploring image enhancement for salient object detection in low light images. ACM Transactions on Multimedia Computing, Communications, and Applications 17, 1s (2021), 1–19.Google Scholar
Digital Library
- [41] . 2019. Attention-guided network for ghost-free high dynamic range imaging. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019. Computer Vision Foundation/IEEE, 1751–1760.Google Scholar
Cross Ref
- [42] . 2022. Dual-attention-guided network for ghost-free high dynamic range imaging. International Journal of Computer Vision 130, 1 (2022), 76–94.Google Scholar
Digital Library
- [43] . 2022. High dynamic range imaging via gradient-aware context aggregation network. Pattern Recognition 122 (2022), 108342.Google Scholar
Digital Library
- [44] . 2020. Deep HDR imaging via A non-local network. IEEE Transactions on Image Processing 29 (2020), 4308–4322.Google Scholar
Cross Ref
- [45] . 2020. Kernel attention network for single image super-resolution. ACM Transactions on Multimedia Computing, Communications, and Applications 16, 3 (2020), 90:1–90:15.Google Scholar
Digital Library
- [46] . 2019. Self-attention generative adversarial networks. In Proceedings of the 36th International Conference on Machine Learning, ICML (Proceedings of Machine Learning Research), Vol. 97. PMLR, 7354–7363.Google Scholar
- [47] . 2021. Ultra-high-definition image HDR reconstruction via collaborative bilateral learning. In Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021. IEEE, 4429–4438.Google Scholar
Cross Ref
Index Terms
A Decoupled Kernel Prediction Network Guided by Soft Mask for Single Image HDR Reconstruction
Recommendations
HDR image reconstruction from a single exposure using deep CNNs
Camera sensors can only capture a limited range of luminance simultaneously, and in order to create high dynamic range (HDR) images a set of different exposures are typically combined. In this paper we address the problem of predicting information that ...
A brightness-adaptive kernel prediction network for inverse tone mapping
Highlights- A brightness-adaptive kernel prediction network is proposed for iTM task.
- ...
AbstractThe inverse tone mapping (iTM) technique that produces a high dynamic range (HDR) image from one single standard dynamic range (SDR) image has received much attention in industry and academia recently. However, existing methods to ...
Binocular tone reproduction display for an HDR panorama image
SAP '16: Proceedings of the ACM Symposium on Applied PerceptionIn virtual reality (VR) applications, it is often required to display a surrounding scenery with a high dynamic range (HDR) of luminance to a binocular display device such as an Oculus Rift. A surrounding scenery of an outdoor environment usually ...






Comments