research-article

Intrinsic colorization

Authors:

Tien-Tsin Wong,

Chi-Sing Leung,

Pheng-Ann HengAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 27, Issue 5

Article No.: 152, Pages 1 - 9

https://doi.org/10.1145/1409060.1409105

Published: 01 December 2008 Publication History

Abstract

In this paper, we present an example-based colorization technique robust to illumination differences between grayscale target and color reference images. To achieve this goal, our method performs color transfer in an illumination-independent domain that is relatively free of shadows and highlights. It first recovers an illumination-independent intrinsic reflectance image of the target scene from multiple color references obtained by web search. The reference images from the web search may be taken from different vantage points, under different illumination conditions, and with different cameras. Grayscale versions of these reference images are then used in decomposing the grayscale target image into its intrinsic reflectance and illumination components. We transfer color from the color reflectance image to the grayscale reflectance image, and obtain the final result by relighting with the illumination component of the target image. We demonstrate via several examples that our method generates results with excellent color consistency.

Supplementary Material

JPG File (a152-liu-mp4_hi.jpg)

Download
12.64 KB

MOV File (a152-liu-mp4_hi.mov)

Download
283.88 MB

References

[1]

Barrow, H., and Tenenbaum., J. 1978. Recovering intrinsic scene characteristics from images. Computer Vision Systems, 3--26.

[2]

Comaniciu, D., and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 5, 603--619.

Digital Library

[3]

Fischler, M. A., and Bolles, R. C. 1987. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Readings in computer vision: issues, problems, principles, and paradigms, 726--740.

[4]

Freeman, W. T., and Viola, P. A. 1998. Bayesian model of surface perception. In NIPS '97: Proceedings of the 1997 conference on Advances in neural information processing systems 10, MIT Press, Cambridge, MA, USA, 787--793.

Digital Library

[5]

Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Trans. Graph. 26, 3, 4.

Digital Library

[6]

Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. ACM Trans. Graph., 327--340.

[7]

Huang, Y.-C., Tung, Y.-S., Chen, J.-C., Wang, S.-W., and Wu, J.-L. 2005. An adaptive edge detection based colorization algorithm and its applications. In MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia, ACM, New York, NY, USA, 351--354.

[8]

Irony, R., Cohen-Or, D., and Lischinski, D. 2005. Colorization by example. In Rendering Techniques 2005, IEEE Computer Society Press, 201--210.

[9]

Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. ACM Trans. Graph. 26, 3, 3.

Digital Library

[10]

Land, E. H., and McCann, J. J. 1971. Lightness and retinex theory. Journal of the Optical Society of America (1917--1983) 61 (Jan.), 1.

[11]

Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3, 689--694.

Digital Library

[12]

Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2, 91--110.

Digital Library

[13]

Luan, Q., Wen, F., Cohen-Or, D., Liang, L., Xu, Y.-Q., and Shum, H.-Y. 2007. Natural image colorization. In Rendering Techniques 2007 (Proceedings Eurographics Symposium on Rendering), 309--320.

[14]

Portilla, J., Strela, V., Wainwright, M. J., and Simon-celli, E. P. 2002. Image denoising using gaussian scale mixtures in the wavelet domain. IEEE Transactions on Image Processing 12, 1338--1351.

Digital Library

[15]

Qu, Y., Wong, T.-T., and Heng, P.-A. 2006. Manga colorization. ACM Trans. Graph. 25, 3, 1214--1220.

Digital Library

[16]

Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Computer Graphics and Applications 21, 5, 34--41.

Digital Library

[17]

Schnitman, Y., Caspi, Y., Cohen-Or, D., and Lischinski, D. 2006. Inducing semantic segmentation from an example. In Asian Conference on Computer Vision, 373--384.

[18]

Shewchuk, J. R. 1996. Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator. Applied Computational Geometry: Towards Geometric Engineering 1148 (May), 203--222. From the First ACM Workshop on Applied Computational Geometry.

[19]

Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Trans. Graph. 25, 3, 835--846.

Digital Library

[20]

Tappen, M. F., Freeman, W. T., and Adelson, E. H. 2005. Recovering intrinsic images from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 9, 1459--1472.

Digital Library

[21]

Weiss, Y. 2001. Deriving intrinsic images from image sequences. In Eighth IEEE International Conference on Computer Vision, vol. 2, 68--75.

[22]

Welsh, T., Ashikhmin, M., and Mueller, K. 2002. Transferring color to greyscale images. ACM Trans. Graph. 21, 3, 277--280.

Digital Library

[23]

Yatziv, L., and Sapiro, G. 2006. Fast image and video colorization using chrominance blending. IEEE Transactions on Image Processing 15, 5, 1120--1129.

Digital Library

Cited By

Li SMa DDing YXian YZhang T(2024)DBSF-Net: Infrared Image Colorization Based on the Generative Adversarial Model with Dual-Branch Feature Extraction and Spatial-Frequency-Domain DiscriminationRemote Sensing10.3390/rs1620376616:20(3766)Online publication date: 10-Oct-2024
https://doi.org/10.3390/rs16203766
Sun QNie YZhang QLi G(2024)Building Coarse to Fine Convex Hulls With Auxiliary Vertices for Palette-Based Image RecoloringIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329638630:8(5581-5595)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3296386
Cong XWu YChen QLei C(2024)Automatic Controllable Colorization via Imagination2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00252(2609-2619)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00252
Show More Cited By

Index Terms

Intrinsic colorization
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Image manipulation

Recommendations

Intrinsic colorization
SIGGRAPH Asia '08: ACM SIGGRAPH Asia 2008 papers

In this paper, we present an example-based colorization technique robust to illumination differences between grayscale target and color reference images. To achieve this goal, our method performs color transfer in an illumination-independent domain that ...
Using assorted color spaces and pixel window sizes for colorization of grayscale images
ICWET '10: Proceedings of the International Conference and Workshop on Emerging Trends in Technology

There is no exact solution for colorization of grayscale images. In the initial work done [23], color traits transfer techniques to color grayscale images are proposed. The main focus of the techniques [23] is to minimise the human efforts needed in ...
Colorization in YCbCr color space and its application to JPEG images

This paper presents a colorization method in YCbCr color space, which is based on the maximum a posteriori estimation of a color image given a monochrome image as is our previous method in RGB color space. The presented method in YCbCr space is much ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 27, Issue 5

December 2008

552 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1409060

Issue’s Table of Contents

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2008

Published in TOG Volume 27, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

124
Total Citations
View Citations
1,634
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li SMa DDing YXian YZhang T(2024)DBSF-Net: Infrared Image Colorization Based on the Generative Adversarial Model with Dual-Branch Feature Extraction and Spatial-Frequency-Domain DiscriminationRemote Sensing10.3390/rs1620376616:20(3766)Online publication date: 10-Oct-2024
https://doi.org/10.3390/rs16203766
Sun QNie YZhang QLi G(2024)Building Coarse to Fine Convex Hulls With Auxiliary Vertices for Palette-Based Image RecoloringIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.329638630:8(5581-5595)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3296386
Cong XWu YChen QLei C(2024)Automatic Controllable Colorization via Imagination2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00252(2609-2619)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00252
Ward RBigioi DBasak SBreslin JCorcoran P(2024)LatentColorization: Latent Diffusion-Based Speaker Video ColorizationIEEE Access10.1109/ACCESS.2024.340624912(81105-81121)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3406249
Jiang QYao GFeng MJin XMiao SGao YCheng X(2024)MCU-GAN: Colorization method for infrared images based on multi-convolution fusion and generative adversarial networkInfrared Physics & Technology10.1016/j.infrared.2024.105673(105673)Online publication date: Dec-2024
https://doi.org/10.1016/j.infrared.2024.105673
Liu YZhao HChan KWang XLoy CQiao YDong C(2024)Temporally consistent video colorization with deep feature propagation and self-regularization learningComputational Visual Media10.1007/s41095-023-0342-810:2(375-395)Online publication date: 3-Jan-2024
https://doi.org/10.1007/s41095-023-0342-8
Duan XCao YZhang RWang XLi P(2024)Shadow-aware image colorizationThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-024-03500-540:7(4969-4979)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00371-024-03500-5
Smirnov DLeGendre CYu XDebevec P(2023)Magenta Green Screen: Spectrally Multiplexed Alpha Matting with Deep ColorizationProceedings of the 2023 Digital Production Symposium10.1145/3603521.3604293(1-13)Online publication date: 5-Aug-2023
https://dl.acm.org/doi/10.1145/3603521.3604293
Zhao YPo LYu WRehman YLiu MZhang YOu W(2023)VCGAN: Video Colorization With Hybrid Generative Adversarial NetworkIEEE Transactions on Multimedia10.1109/TMM.2022.315460025(3017-3032)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3154600
Liu SLiu S(2023)ColorizationImage and Video Color Editing10.1007/978-3-031-26030-8_4(31-41)Online publication date: 21-Mar-2023
https://doi.org/10.1007/978-3-031-26030-8_4
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents