research-article

Automatic Cartoon Colorization Based on Convolutional Neural Network

Authors:

Domonkos Varga,

Csaba Attila Szabó,

Tamás SzirányiAuthors Info & Claims

CBMI '17: Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing

Article No.: 28, Pages 1 - 6

https://doi.org/10.1145/3095713.3095742

Published: 19 June 2017 Publication History

Abstract

This paper deals with automatic cartoon colorization. This is a hard issue, since it is an ill-posed problem that usually requires user intervention to achieve high quality. Motivated by the recent successes in natural image colorization based on deep learning techniques, we investigate the colorization problem at the cartoon domain using Convolutional Neural Network. To our best knowledge, no existing papers or research studies address this problem using deep learning techniques. Here we investigate a deep Convolutional Neural Network based automatic color filling method for cartoons.

References

[1]

Aurélie Bugeau and Vinh-Thong Ta. 2012. Patch-based image colorization. In Pattern Recognition (ICPR), 2012 21st International Conference on. IEEE, 3058--3061.

[2]

Guillaume Charpiat, Matthias Hofmann, and Bernhard Schölkopf. 2008. Automatic image colorization via multimodal predictions. Computer Vision--ECCV 2008 (2008), 126--139.

[3]

Zezhou Cheng, Qingxiong Yang, and Bin Sheng. 2015. Deep colorization. In Proceedings of the IEEE International Conference on Computer Vision. 415--423.

Digital Library

[4]

François Chollet. 2015. Keras. (2015).

[5]

Federico Girosi, Michael Jones, and Tomaso Poggio. 1995. Regularization theory and neural networks architectures. Neural computation 7, 2 (1995), 219--269.

Digital Library

[6]

Karunesh Kumar Gupta and RP Pareek. 2014. A Survey of Image Quality Assessment Techniques for Medical Imaging. New Delhi Nov 1st and 2nd (2014), 114.

[7]

Bharath Hariharan, Pablo Arbeláez, Ross Girshick, and Jitendra Malik. 2015. Hypercolumns for object segmentation and fine-grained localization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 447--456.

[8]

Kaiming He, Jian Sun, and Xiaoou Tang. 2013. Guided image filtering. IEEE transactions on pattern analysis and machine intelligence 35, 6 (2013), 1397--1409.

Digital Library

[9]

Yi-Chin Huang, Yi-Shin Tung, Jun-Cheng Chen, Sung-Wen Wang, and Ja-Ling Wu. 2005. An adaptive edge detection based colorization algorithm and its applications. In Proceedings of the 13th annual ACM international conference on Multimedia. ACM, 351--354.

Digital Library

[10]

Satoshi Iizuka, Edgar Simo-Serra, and Hiroshi Ishikawa. 2016. Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification. ACM Transactions on Graphics (TOG) 35, 4 (2016), 110.

Digital Library

[11]

Revital Ironi, Daniel Cohen-Or, and Dani Lischinski. 2005. Colorization by Example. In Rendering Techniques. Citeseer, 201--210.

[12]

Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[13]

Amir Kolaman and Orly Yadid-Pecht. 2012. Quaternion structural similarity: a new quality index for color images. IEEE Transactions on Image Processing 21, 4 (2012), 1526--1536.

Digital Library

[14]

Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. 2016. Learning representations for automatic colorization. In European Conference on Computer Vision. Springer, 577--593.

[15]

Anat Levin, Dani Lischinski, and Yair Weiss. 2004. Colorization using optimization. In ACM Transactions on Graphics (ToG), Vol. 23. ACM, 689--694.

Digital Library

[16]

Xiangguo Liang, Zhuo Su, Yiqi Xiao, Jiaming Guo, and Xiaonnan Luo. 2016. Deep patch-wise colorization model for grayscale images. In SIGGRAPH ASIA 2016 Technical Briefs. ACM, 13.

Digital Library

[17]

Yingge Qu, Tien-Tsin Wong, and Pheng-Ann Heng. 2006. Manga colorization. In ACM Transactions on Graphics (TOG), Vol. 25. ACM, 1214--1220.

Digital Library

[18]

Erik Reinhard, Michael Adhikhmin, Bruce Gooch, and Peter Shirley. 2001. Color transfer between images. IEEE Computer graphics and applications 21, 5 (2001), 34--41.

Digital Library

[19]

Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

[20]

Daniel Sykora, Jan Buriánek, and Jiří Žára. 2004. Unsupervised colorization of black-and-white cartoons. In Proceedings of the 3rd international symposium on Non-photorealistic animation and rendering. ACM, 121--127.

Digital Library

[21]

Dániel Szolgay and Tamás Szirányi. 2012. Adaptive image decomposition into cartoon and texture parts optimized by the orthogonality criterion. IEEE Transactions on Image Processing 21, 8 (2012), 3405--3415.

Digital Library

[22]

Engin Tola, Vincent Lepetit, and Pascal Fua. 2010. Daisy: An efficient dense descriptor applied to wide-baseline stereo. IEEE transactions on pattern analysis and machine intelligence 32, 5 (2010), 815--830.

Digital Library

[23]

Domonkos Varga and Tamás Szirányi. 2016. Fully automatic image colorization based on Convolutional Neural Network. In Pattern Recognition (ICPR), 2016 23rd International Conference on. IEEE, 3691--3696.

[24]

Tomihisa Welsh, Michael Ashikhmin, and Klaus Mueller. 2002. Transferring color to greyscale images. In ACM Transactions on Graphics (TOG), Vol. 21. ACM, 277--280.

Digital Library

[25]

Liron Yatziv and Guillermo Sapiro. 2006. Fast image and video colorization using chrominance blending. IEEE Transactions on Image Processing 15, 5 (2006), 1120--1129.

Digital Library

[26]

Richard Zhang, Phillip Isola, and Alexei A Efros. 2016. Colorful image colorization. In European Conference on Computer Vision. Springer, 649--666.

Cited By

Cao YMeng XMok PLee TLiu XLi P(2024)AnimeDiffusion: Anime Diffusion ColorizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.335756830:10(6956-6969)Online publication date: Oct-2024
https://doi.org/10.1109/TVCG.2024.3357568
Luo RPeng YZhou NLiang J(2024)Periodic Iterative Segmentation-Colorization Training: Line Drawing Colorization Using Text Tag with CBAMCatPattern Recognition and Computer Vision10.1007/978-981-97-8502-5_16(215-229)Online publication date: 1-Nov-2024
https://doi.org/10.1007/978-981-97-8502-5_16
Shi MZhang JChen SGao LLai YZhang F(2023)Reference-Based Deep Line Art Video ColorizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.314600029:6(2965-2979)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TVCG.2022.3146000
Show More Cited By

Index Terms

Automatic Cartoon Colorization Based on Convolutional Neural Network
1. Computing methodologies
  1. Computer graphics
    1. Image manipulation
      1. Image processing

Recommendations

Colorization of grayscale images and videos using a semi-automatic approach
ICIP'09: Proceedings of the 16th IEEE international conference on Image processing

Colorization is a computer-aided process of adding color to a grayscale image or video. The task of colorizing a grayscale image involves assigning three dimensional (RGB) pixel values to an image which varies along only one dimension (luminance or ...
Automatic grayscale image colorization using histogram regression

Colorization aims to adding colors to a grayscale image. This task is ill-posed in the sense that assigning the colors to a grayscale image without any prior knowledge is ambiguous. Most of the previous methods require some amount of user interventions, ...
Deep patch-wise colorization model for grayscale images
SA '16: SIGGRAPH ASIA 2016 Technical Briefs

To handle the colorization problem, we propose a deep patch-wise colorization model for grayscale images. Distinguished with some constructive color mapping models with complicated mathematical priors, we alternately apply two loss metric functions in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

CBMI '17: Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing

June 2017

237 pages

ISBN:9781450353335

DOI:10.1145/3095713

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

CBMI '17

CBMI '17: International Workshop on Content-Based Multimedia Indexing

June 19 - 21, 2017

Florence, Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
374
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)1

Reflects downloads up to 21 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cao YMeng XMok PLee TLiu XLi P(2024)AnimeDiffusion: Anime Diffusion ColorizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.335756830:10(6956-6969)Online publication date: Oct-2024
https://doi.org/10.1109/TVCG.2024.3357568
Luo RPeng YZhou NLiang J(2024)Periodic Iterative Segmentation-Colorization Training: Line Drawing Colorization Using Text Tag with CBAMCatPattern Recognition and Computer Vision10.1007/978-981-97-8502-5_16(215-229)Online publication date: 1-Nov-2024
https://doi.org/10.1007/978-981-97-8502-5_16
Shi MZhang JChen SGao LLai YZhang F(2023)Reference-Based Deep Line Art Video ColorizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.314600029:6(2965-2979)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TVCG.2022.3146000
Cao YTian HMok P(2023)Attention-Aware Anime Line Drawing Colorization2023 IEEE International Conference on Multimedia and Expo (ICME)10.1109/ICME55011.2023.00282(1637-1642)Online publication date: Jul-2023
https://doi.org/10.1109/ICME55011.2023.00282
Chen SZhang JGao LHe YXia SShi MZhang F(2022)Active Colorization for Cartoon Line DrawingsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.300994928:2(1198-1208)Online publication date: 1-Feb-2022
https://doi.org/10.1109/TVCG.2020.3009949
Treneska SJovanov M(2021)Colorizing images with Conditional Adversarial Networks and Transfer Learning2021 IEEE 19th International Symposium on Intelligent Systems and Informatics (SISY)10.1109/SISY52375.2021.9582537(33-38)Online publication date: 16-Sep-2021
https://doi.org/10.1109/SISY52375.2021.9582537
Tsukasa KKazunori U(2021)Image Colorization Algorithm Based on Graph Signal Processing Using Two-Steps Image Segmentation2021 IEEE International Midwest Symposium on Circuits and Systems (MWSCAS)10.1109/MWSCAS47672.2021.9531797(520-523)Online publication date: 9-Aug-2021
https://doi.org/10.1109/MWSCAS47672.2021.9531797
Aldahoul NKarim HAbdullah MWazir AFauzi MTan MMansor SLyn H(2021)An Evaluation of Traditional and CNN-Based Feature Descriptors for Cartoon Pornography DetectionIEEE Access10.1109/ACCESS.2021.30643929(39910-39925)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3064392
Joshi MNkenyereye LJoshi GIslam SAbdullah-Al-Wadud MShrestha S(2020)Auto-Colorization of Historical Images Using Deep Convolutional Neural NetworksMathematics10.3390/math81222588:12(2258)Online publication date: 21-Dec-2020
https://doi.org/10.3390/math8122258
Chybicki MKozakiewicz WSielski DFabijańska A(2019)Deep cartoon colorizerEngineering Applications of Artificial Intelligence10.1016/j.engappai.2019.02.00681:C(37-46)Online publication date: 1-May-2019
https://dl.acm.org/doi/10.1016/j.engappai.2019.02.006
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents