research-article

Edge-preserving decompositions for multi-scale tone and detail manipulation

Authors:

Dani Lischinski,

Richard SzeliskiAuthors Info & Claims

SIGGRAPH '08: ACM SIGGRAPH 2008 papers

Article No.: 67, Pages 1 - 10

https://doi.org/10.1145/1399504.1360666

Published: 01 August 2008 Publication History

Abstract

Many recent computational photography techniques decompose an image into a piecewise smooth base layer, containing large scale variations in intensity, and a residual detail layer capturing the smaller scale details in the image. In many of these applications, it is important to control the spatial scale of the extracted details, and it is often desirable to manipulate details at multiple scales, while avoiding visual artifacts.

In this paper we introduce a new way to construct edge-preserving multi-scale image decompositions. We show that current basedetail decomposition techniques, based on the bilateral filter, are limited in their ability to extract detail at arbitrary scales. Instead, we advocate the use of an alternative edge-preserving smoothing operator, based on the weighted least squares optimization framework, which is particularly well suited for progressive coarsening of images and for multi-scale detail extraction. After describing this operator, we show how to use it to construct edge-preserving multi-scale decompositions, and compare it to the bilateral filter, as well as to other schemes. Finally, we demonstrate the effectiveness of our edge-preserving decompositions in the context of LDR and HDR tone mapping, detail enhancement, and other applications.

Supplementary Material

MOV File (a67-farbman.mov)

Download
21.75 MB

References

[1]

Alvarez, L., Lions, P.-L., and Morel, J.-M. 1992. Image selective smoothing and edge detection by nonlinear diffusion. ii. SIAM Journal on Numerical Analysis 29, 3 (June), 845--866.

Digital Library

[2]

Aubert, G., and Kornprobst, P. 2006. Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, vol. 147 of Applied Mathematical Sciences. Springer.

Digital Library

[3]

Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. ACM Trans. Graph. 25, 3 (July), 654--662.

Digital Library

[4]

Barash, D. 2002. A fundamental relationship between bilateral filtering, adaptive smoothing, and the nonlinear diffusion equation. IEEE Trans. Pattern Anal. Mach. Intell. 24, 6, 844--847.

Digital Library

[5]

Black, M. J., Sapiro, G., Marimont, D. H., and Heeger, D. 1998. Robust anisotropic diffusion. IEEE Trans. Image Proc. 7, 3 (Mar.), 421--432.

Digital Library

[6]

Buades, A., Coll, B., and Morel, J. M. 2006. The staircasing effect in neighborhood filters and its solution. IEEE Transactions on Image Processing 15, 6, 1499--1505.

Digital Library

[7]

Buatois, L., Caumon, G., and Levy, B. 2007. Concurrent number cruncher: An efficient sparse linear solver on the GPU. In High Performance Computation Conference (HPCC), Springer.

Digital Library

[8]

Burt, P., and Adelson, E. H. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. Comm. 31, 532--540.

[9]

Chen, J., Paris, S., and Durand, F. 2007. Real-time edge-aware image processing with the bilateral grid. ACM Trans. Graph. 26, 3 (July), Article 103.

Digital Library

[10]

Choudhury, P., and Tumblin, J. 2003. The trilateral filter for high contrast images and meshes. In Proc. EGSR 2003, Eurographics, 186--196.

Digital Library

[11]

Comaniciu, D., and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 5, 603--619.

Digital Library

[12]

DeCarlo, D., and Santella, A. 2002. Stylization and abstraction of photographs. ACM Trans. Graph. 21, 3 (July), 769--776.

Digital Library

[13]

Durand, F., and Dorsey, J. 2002. Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. 21, 3 (July), 257--266.

Digital Library

[14]

Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Trans. Graph. 23, 3 (August), 673--678.

Digital Library

[15]

Elad, M. 2002. On the bilateral filter and ways to improve it. IEEE Trans. Image Proc. 11, 10, 1141--1151.

Digital Library

[16]

Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Trans. Graph. 21, 3 (July), 249--256.

Digital Library

[17]

Fattal, R., Agrawala, M., and Rusinkiewicz, S. 2007. Multiscale shape and detail enhancement from multi-light image collections. ACM Trans. Graph. 26, 3 (July), Article 51.

Digital Library

[18]

Jobson, D. J., Rahman, Z., and Woodell, G. A. 1997. A multi-scale Retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Proc. 6, 7 (July), 965--976.

Digital Library

[19]

Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Trans. Graph. 25, 3 (July), 654--663.

Digital Library

[20]

Lagendijk, R. L., Biemond, J., and Boekee, D. E. 1988. Regularized iterative image restoration with ringing reduction. IEEE Trans. Acoustics, Speech, and Signal Proc., Speech, Signal Proc. 36, 12 (December), 1874--1888.

[21]

Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3 (August), 689--694.

Digital Library

[22]

Levin, A., Fergus, R., Durand, F., and Freeman, W. T. 2007. Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph. 26, 3 (July), Article 70.

Digital Library

[23]

Li, Y., Sharan, L., and Adelson, E. H. 2005. Compressing and companding high dynamic range images with subband architectures. ACM Trans. Graph. 24, 3 (July), 836--844.

Digital Library

[24]

Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM Trans. Graph. 25, 3 (July), 646--653.

Digital Library

[25]

Mantiuk, R., Myszkowski, K., and Seidel, H.-P. 2006. A perceptual framework for contrast processing of high dynamic range images. ACM Trans. Appl. Percept. 3, 3, 286--308.

Digital Library

[26]

Mrázek, P., Weickert, J., and Bruhn, A. 2006. On robust estimation and smoothing with spatial and tonal kernels. In Geometric Properties from Incomplete Data, R. Klette, R. Kozera, L. Noakes, and J. Weickert, Eds. Springer, Dordrecht, 335--352.

[27]

Nordstrom, K. N. 1989. Biased anisotropic diffusion --- a unified regularization and diffusion approach to edge detection. Tech. Rep. UCB/CSD-89-514, EECS Department, University of California, Berkeley, May.

Digital Library

[28]

Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-based modeling and photo editing. In Proc. ACM SIGGRAPH 2001, ACM, E. Fiume, Ed., 433--442.

Digital Library

[29]

Oppenheim, A. V., and Schafer, R. W. 1989. Discrete-Time Signal Processing. Prentice Hall.

Digital Library

[30]

Paris, S., and Durand, F. 2006. A fast approximation of the bilateral filter using a signal processing approach. In Proc. ECCV '06, IV: 568--580.

Digital Library

[31]

Paris, S. 2007. A gentle introduction to bilateral filtering and its applications. In ACM SIGGRAPH 2007 courses, Course 13.

Digital Library

[32]

Pattanaik, S. N., Ferwerda, J. A., Fairchild, M. D., and Greenberg, D. P. 1998. A multiscale model of adaptation and spatial vision for realistic image display. In Proc. ACM SIGGRAPH 98, M. Cohen, Ed., 287--298.

Digital Library

[33]

Perona, P., and Malik, J. 1990. Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Machine Intell. 12, 7 (July), 629--639.

Digital Library

[34]

Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23, 3 (August), 664--672.

Digital Library

[35]

Reinhard, E., Stark, M., Shirley, P., and Ferwerda, J. 2002. Photographic tone reproduction for digital images. ACM Trans. Graph. 21, 3 (July), 267--276.

Digital Library

[36]

Saad, Y. 2003. Iterative Methods for Sparse Linear Systems, second ed. SIAM.

Digital Library

[37]

Scales, J. A., and Gersztenkorn, A. 1988. Robust methods in inverse theory. Inverse Problems 4, 1071--1091.

[38]

Scherzer, O., and Weickert, J. 2000. Relations between regularization and diffusion filtering. Journal of Mathematical Imaging and Vision 12, 1 (February), 43--63.

Digital Library

[39]

Schlick, C. 1994. Quantization techniques for visualization of high dynamic range pictures. In Photorealistic Rendering Techniques, Springer-Verlag, P. Shirley, G. Sakas, and S. Müller, Eds., 7--20.

[40]

Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV '98, IEEE Computer Society, 839--846.

Digital Library

[41]

Tumblin, J., and Turk, G. 1999. LCIS: A boundary hierarchy for detail-preserving contrast reduction. In Proc. ACM SIGGRAPH 99, A. Rockwood, Ed., ACM, 83--90.

Digital Library

[42]

Weiss, B. 2006. Fast median and bilateral filtering. ACM Trans. Graph. 25, 3 (July), 519--526.

Digital Library

[43]

Winnemöller, H., Olsen, S. C., and Gooch, B. 2006. Realtime video abstraction. ACM Trans. Graph. 25, 3 (July), 1221--1226.

Digital Library

[44]

Zervakis, M. E. 1990. Nonlinear image restoration techniques. PhD thesis, Univ. Toronto, Toronto, ON, Canada.

Cited By

Yan RSun JDavis A(2024)Chromaticity Gradient Mapping for Interactive Control of Color Contrast in Images and VideoProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676340(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676340
Zhou MWu XWei XXiang TFang BKwong S(2024)Low-Light Enhancement Method Based on a Retinex Model for Structure PreservationIEEE Transactions on Multimedia10.1109/TMM.2023.326886726(650-662)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3268867
Yang YMa WGao WZeng LPan Z(2024)Hyperbolic tangent penalty function for edge-preserving image filteringDigital Signal Processing10.1016/j.dsp.2024.104591(104591)Online publication date: May-2024
https://doi.org/10.1016/j.dsp.2024.104591
Show More Cited By

Index Terms

Edge-preserving decompositions for multi-scale tone and detail manipulation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Photographic tone reproduction for digital images

A classic photographic task is the mapping of the potentially high dynamic range of real world luminances to the low dynamic range of the photographic print. This tone reproduction problem is also faced by computer graphics practitioners who map digital ...
Edge-preserving decompositions for multi-scale tone and detail manipulation

Many recent computational photography techniques decompose an image into a piecewise smooth base layer, containing large scale variations in intensity, and a residual detail layer capturing the smaller scale details in the image. In many of these ...
Fast bilateral filtering for the display of high-dynamic-range images

We present a new technique for the display of high-dynamic-range images, which reduces the contrast while preserving detail. It is based on a two-scale decomposition of the image into a base layer, encoding large-scale variations, and a detail layer. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '08: ACM SIGGRAPH 2008 papers

August 2008

887 pages

ISBN:9781450301121

DOI:10.1145/1399504

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGGRAPH '08

Sponsor:

SIGGRAPH

SIGGRAPH '08: Special Interest Group on Computer Graphics and Interactive Techniques Conference

August 11 - 15, 2008

California, Los Angeles

Acceptance Rates

SIGGRAPH '08 Paper Acceptance Rate 90 of 518 submissions, 17%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

183
Total Citations
View Citations
4,178
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)2

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yan RSun JDavis A(2024)Chromaticity Gradient Mapping for Interactive Control of Color Contrast in Images and VideoProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676340(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676340
Zhou MWu XWei XXiang TFang BKwong S(2024)Low-Light Enhancement Method Based on a Retinex Model for Structure PreservationIEEE Transactions on Multimedia10.1109/TMM.2023.326886726(650-662)Online publication date: 2024
https://doi.org/10.1109/TMM.2023.3268867
Yang YMa WGao WZeng LPan Z(2024)Hyperbolic tangent penalty function for edge-preserving image filteringDigital Signal Processing10.1016/j.dsp.2024.104591(104591)Online publication date: May-2024
https://doi.org/10.1016/j.dsp.2024.104591
Sicat RIbrahim MAgeeli AMannuss FRautek PHadwiger M(2023)Real-Time Visualization of Large-Scale Geological Models With Nonlinear Feature-Preserving Levels of DetailIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.312037229:2(1491-1505)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TVCG.2021.3120372
Zheng CYing WWu SLi Z(2023)Neural Augmentation-Based Saturation Restoration for LDR Images of HDR ScenesIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2023.330467572(1-11)Online publication date: 2023
https://doi.org/10.1109/TIM.2023.3304675
Zhang RWu J(2023)A Bidirectional Guided Filter Used for RGB-D MapsIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2023.325646772(1-14)Online publication date: 2023
https://doi.org/10.1109/TIM.2023.3256467
Zhang YLee H(2023)Multisensor Infrared and Visible Image Fusion via Double Joint Edge Preservation Filter and Nonglobally Saliency Gradient OperatorIEEE Sensors Journal10.1109/JSEN.2023.326277523:9(10252-10267)Online publication date: 1-May-2023
https://doi.org/10.1109/JSEN.2023.3262775
Jiang HWang ZZheng PZhang HWang YSong BCheng D(2023)PoissonNet: Single Image Detail Enhancement Based on Poisson Regression2023 4th International Conference on Computer, Big Data and Artificial Intelligence (ICCBD+AI)10.1109/ICCBD-AI62252.2023.00100(549-553)Online publication date: 15-Dec-2023
https://doi.org/10.1109/ICCBD-AI62252.2023.00100
Zhang RHe YDai LZhu H(2023)A Medical Image Fusion Method Based on Detail Enhancement Decomposition2023 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech)10.1109/DASC/PiCom/CBDCom/Cy59711.2023.10361318(0934-0941)Online publication date: 14-Nov-2023
https://doi.org/10.1109/DASC/PiCom/CBDCom/Cy59711.2023.10361318
Yang BPan DJiang ZHuang JGui W(2023)CSDMSignal Processing10.1016/j.sigpro.2022.108752202:COnline publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1016/j.sigpro.2022.108752
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents