skip to main content
research-article

Scalable Color Quantization for Task-centric Image Compression

Authors Info & Claims
Published:17 February 2023Publication History
Skip Abstract Section

Abstract

Conventional image compression techniques targeted for the perceptual quality are not generally optimized for classification tasks using deep neural networks (DNNs). To compress images for DNN inference tasks, recent studies have proposed task-centric image compression methods with quantization techniques optimized for DNN inference. Among them, color quantization was proposed to reduce the amount of data per pixel by limiting the number of distinct colors (color space) in an image. However, quantizing images into various color space sizes requires training and inference of multiple DNNs, each of which is dedicated to each color space. To overcome this limitation, we propose a scalable color quantization method, where images with variable color space sizes can be extracted from a master image generated by a single DNN model. This scalability is enabled by weighted color grouping that constructs a color palette using critical color components for the classification task. We also propose an adaptive training method that can jointly optimize images with various color-space sizes. The results show that the proposed method supports dynamic changes of the color space size between 1–6 bit color space per pixel, while even increasing the inference accuracy at a low bit precision up to 20.2% and 46.6% compared to other task- and human-centric color quantizations, respectively.

REFERENCES

  1. [1] Burger Wilhelm and Burge Mark J.. 2016. Color quantization. In Digital Image Processing. Springer, 329339.Google ScholarGoogle ScholarCross RefCross Ref
  2. [2] Chamain Lahiru D., Racapé Fabien, Bégaint Jean, Pushparaja Akshay, and Feltman Simon. 2021. End-to-end optimized image compression for machines, a study. In Data Compression Conference (DCC). IEEE, 163172.Google ScholarGoogle ScholarCross RefCross Ref
  3. [3] Choi Jinyoung and Han Bohyung. 2020. Task-aware quantization network for JPEG image compression. In European Conference on Computer Vision. Springer, 309324.Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. [4] Coates Adam, Ng Andrew, and Lee Honglak. 2011. An analysis of single-layer networks in unsupervised feature learning. In 14th International Conference on Artificial Intelligence and Statistics. 215223.Google ScholarGoogle Scholar
  5. [5] Deng Yining, Kenney Charles, Moore Michael S., and Manjunath B. S.. 1999. Peer group filtering and perceptual color image quantization. In IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 2124.Google ScholarGoogle Scholar
  6. [6] Deng Yining and Manjunath Bangalore S.. 2001. Unsupervised segmentation of color-texture regions in images and video. IEEE Trans. Pattern Anal. Mach. Intell. 23, 8 (2001), 800810.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Gao Changsheng, Liu Dong, Li Li, and Wu Feng. 2021. Towards task-generic image compression: A study of semantics-oriented metrics. IEEE Trans. Multim. (2021).Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. [8] Gervautz Michael and Purgathofer Werner. 1988. A simple method for color quantization: Octree quantization. In New Trends in Computer Graphics. Springer, 219231.Google ScholarGoogle ScholarCross RefCross Ref
  9. [9] Goldberg Naftaly. 1991. Colour image quantization for high resolution graphics display. Image. Vis. Comput. 9, 5 (1991), 303312.Google ScholarGoogle ScholarCross RefCross Ref
  10. [10] Gonzalez Teofilo F.. 1985. Clustering to minimize the maximum intercluster distance. Theoret. Comput. Sci. 38 (1985), 293306.Google ScholarGoogle ScholarCross RefCross Ref
  11. [11] He Kaiming, Zhang Xiangyu, Ren Shaoqing, and Sun Jian. 2016. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition. 770778.Google ScholarGoogle ScholarCross RefCross Ref
  12. [12] Heckbert Paul. 1982. Color image quantization for frame buffer display. ACM Siggraph Comput. Graph. 16, 3 (1982), 297307.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. [13] Hinton Geoffrey, Vinyals Oriol, and Dean Jeff. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).Google ScholarGoogle Scholar
  14. [14] Hou Yunzhong, Zheng Liang, and Gould Stephen. 2020. Learning to structure an image with few colors. In IEEE/CVF Conference on Computer Vision and Pattern Recognition. 1011610125.Google ScholarGoogle ScholarCross RefCross Ref
  15. [15] Houle G. and Dubois E.. 1986. Quantization of color images for display on graphics terminals. In IEEE Global Telecommunications Conference. 11381142.Google ScholarGoogle Scholar
  16. [16] Ioffe Sergey and Szegedy Christian. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015).Google ScholarGoogle Scholar
  17. [17] Kiefer Jack, Wolfowitz Jacob, et al. 1952. Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23, 3 (1952), 462466.Google ScholarGoogle Scholar
  18. [18] Kim Sang Hoon, Park Jae Hyun, and Ko Jong Hwan. 2021. Target-dependent scalable image compression using a reconfigurable recurrent neural network. IEEE Access 9 (2021), 119418119429.Google ScholarGoogle Scholar
  19. [19] Krizhevsky Alex, Hinton Geoffrey, et al. 2009. Learning multiple layers of features from tiny images. https://www.cs.toronto.edu/kriz/learning-features-2009-TR.pdf.Google ScholarGoogle Scholar
  20. [20] Le Ya and Yang Xuan. 2015. Tiny ImageNet visual recognition challenge. CS 231N 7 (2015).Google ScholarGoogle Scholar
  21. [21] Liu Zihao, Liu Tao, Wen Wujie, Jiang Lei, Xu Jie, Wang Yanzhi, and Quan Gang. 2018. DeepN-JPEG: A deep neural network favorable JPEG-based image compression framework. In 55th Annual Design Automation Conference. 16.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. [22] Loshchilov Ilya and Hutter Frank. 2016. SGDR: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016).Google ScholarGoogle Scholar
  23. [23] N100094 ISO/IEC JTC 1/SC29/WG1. 2022. Coding of Still Pictures - ds.jpeg.org. Retrieved from https://ds.jpeg.org/documents/jpegai/wg1n90021-REQ-JPEG_AI_Use_Cases_and_Requirements.pdf.Google ScholarGoogle Scholar
  24. [24] Nguyen Anh, Yosinski Jason, and Clune Jeff. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE Conference on Computer Vision and Pattern Recognition. 427436.Google ScholarGoogle ScholarCross RefCross Ref
  25. [25] Orchard Michael T., Bouman Charles A., et al. 1991. Color quantization of images. IEEE Trans. Sig. Process. 39, 12 (1991), 26772690.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. [26] Patwa Neel, Ahuja Nilesh, Somayazulu Srinivasa, Tickoo Omesh, Varadarajan Srenivas, and Koolagudi Shashidhar. 2020. Semantic-preserving image compression. In IEEE International Conference on Image Processing (ICIP). IEEE, 12811285.Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Ronneberger Olaf, Fischer Philipp, and Brox Thomas. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-assisted Intervention. Springer, 234241.Google ScholarGoogle ScholarCross RefCross Ref
  28. [28] Selvaraju Ramprasaath R., Cogswell Michael, Das Abhishek, Vedantam Ramakrishna, Parikh Devi, and Batra Dhruv. 2017. Grad-Cam: Visual explanations from deep networks via gradient-based localization. In IEEE International Conference on Computer Vision. 618626.Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Simonyan Karen and Zisserman Andrew. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).Google ScholarGoogle Scholar
  30. [30] Smith Leslie N. and Topin Nicholay. 2019. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, Vol. 11006. International Society for Optics and Photonics, 1100612.Google ScholarGoogle Scholar
  31. [31] Wang Qiang, Shen Liquan, and Shi Yuan. 2020. Recognition-driven compressed image generation using semantic-prior information. IEEE Sig. Process. Lett. 27 (2020), 11501154.Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Weber Maurice, Renggli Cedric, Grabner Helmut, and Zhang Ce. 2019. Lossy image compression with recurrent neural networks: From human perceived visual quality to classification accuracy. arXiv preprint arXiv:1910.03472 (2019).Google ScholarGoogle Scholar
  33. [33] Weber Maurice, Renggli Cedric, Grabner Helmut, and Zhang Ce. 2021. Observer dependent lossy image compression. In Pattern Recognition: 42nd DAGM German Conference, DAGM GCPR’20, Tübingen, Germany, September 28–October 1, 2020, Proceedings 42. Springer, 130144.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. [34] Wu Xiaolin. 1992. Color quantization by dynamic programming and principal analysis. ACM Trans. Graph. 11, 4 (1992), 348372.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. [35] Xiang Zhigang. 1997. Color image quantization by minimizing the maximum intercluster distance. ACM Trans. Graph. 16, 3 (1997), 260276.Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. [36] Yu Haichao, Li Haoxiang, Shi Honghui, Huang Thomas S., and Hua Gang. 2019. Any-precision deep neural networks. arXiv preprint arXiv:1911.07346 (2019).Google ScholarGoogle Scholar
  37. [37] Yu Jiahui and Huang Thomas S.. 2019. Universally slimmable networks and improved training techniques. In IEEE/CVF International Conference on Computer Vision. 18031811.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Zhang Yichi, Zhao Ritchie, Hua Weizhe, Xu Nayun, Suh G. Edward, and Zhang Zhiru. 2020. Precision gating: Improving neural network efficiency with dynamic dual-precision activations. arXiv preprint arXiv:2002.07136 (2020).Google ScholarGoogle Scholar

Index Terms

  1. Scalable Color Quantization for Task-centric Image Compression

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 19, Issue 2s
        April 2023
        545 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/3572861
        • Editor:
        • Abdulmotaleb El Saddik
        Issue’s Table of Contents

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 17 February 2023
        • Online AM: 1 August 2022
        • Accepted: 20 July 2022
        • Revised: 19 May 2022
        • Received: 21 May 2021
        Published in tomm Volume 19, Issue 2s

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
      • Article Metrics

        • Downloads (Last 12 months)224
        • Downloads (Last 6 weeks)20

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Full Text

      View this article in Full Text.

      View Full Text

      HTML Format

      View this article in HTML Format .

      View HTML Format
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!