Recoloring Algorithms for Colorblind People: A Survey

Abstract

Color is a powerful communication component, not only as part of the message meaning but also as a way of discriminating contents therein. However, 5% of the world’s population suffers from color vision deficiency (CVD), commonly known as colorblindness. This handicap adulterates the way the color is perceived, compromising the reading and understanding of the message contents. This issue becomes even more pertinent due to the increasing availability of multimedia contents in computational environments (e.g., web browsers). Aware of this problem, a significant number of CVD research works came up in the literature in the past two decades to improve color perception in text documents, still images, video, and so forth. This survey mainly addresses recoloring algorithms toward still images for colorblind people, including the current trends in the field of color adaptation.

References

1. C.-N. Anagnostopoulos, G. Tsekouras, I. Anagnostopoulos, and C. Kalloniatis. 2007. Intelligent modification for the daltonization process of digitized paintings. In Proceedings of the 5th International Conference on Computer Vision Systems (ICVS’07).
2. J. Bao, Y. Wang, Y. Ma, and X. Gu. 2008. Re-coloring images for dichromats based on an improved adaptive mapping algorithm. In Proceedings of the International Conference on Audio, Language and Image Processing (ICALIP’08). IEEE Press, 152--156.
3. S. Bao, G. Tanaka, H. Tamukoh, and N. Suetake. 2016. Lightness modification method considering Craik-O’Brien effect for protanopia and deuteranopia. IEICE Trans. Fundam. Electr. Commun. Comput. Sci. E99. A, 11 (2016), 2008--2011.
4. J. Birch. 2001. Diagnosis of Defective Colour Vision (2nd ed.). Elsevier Science, Edinburgh.
5. J. Birch, I. Chisholm, P. Kinnear, M. Marré, A. Pinckers, J. Pokorny, and G. Verriest. 1979. Acquired color vision defects. In Congetinal and Acquired Color Vision Defects. Grune 8 Stratton.
6. C. Birtolo, P. Pagano, and L. Troiano. 2009. Evolving colors in user interfaces by interactive genetic algorithm. In Proceedings of the World Congress on Nature and Biologically Inspired Computing (NaBIC’09). IEEE Computer Society, 349--355.
7. H. Brettel, F. Viénot, and J. Mollon. 1997. Computerized simulation of color appearance for dichromats. J. Opt. Soc. Am. 14, 10 (1997), 2647--2655.
8. A. Byrne and D. Hilbert. 2010. How do things look to the color-blind? In Color Ontology and Color Science, J. Cohen and M. Metthen (Eds.). MIT Press.
9. B. Caldwell, M. Cooper, L. Reid, and G. Vanderheiden. 2008. Web content accessibility guidelines (WCAG) 2.0. The World Wide Web Consortium. Retrieved from https://www.w3.org/TR/WCAG20.
10. W. Chen, W. Chen, and H. Bao. 2011. An efficient direct volume rendering approach for dichromats. IEEE Trans. Vis. Comput. Graph. 17, 12 (2011), 2144--2152.
11. Y.-C. Chen and T.-S. Liao. 2011. Hardware digital color enhancement for color vision deficiencies. ETRI J. 33, 1 (2011), 71--77.
12. S.-L. Ching and M. Sabudin. 2010. Website image colour transformation for the colour blind. In Proceedings of the 2nd International Conference on Computer Technology and Development (ICCTD’10). IEEE Computer Society, 255--259.
13. S. H. Chua, H. Zhang, M. Hammad, S. Zhao, S. Goyal, and K. Singh. 2015. ColorBless: Augmenting visual information for colorblind people with binocular luster effect. ACM Trans. Comput.-Hum. Interact. 21, 6 (2015), A32:1--A32:20.
14. T. Cox and M. Cox. 2001. Multidimensional Scaling (2nd ed.). Chapman 8 Hall/CRC, Boca Raton, FL, USA.
15. A. Damasio, T. Yamada, H. Damasio, J. Corbett, and J. McKee. 1980. Central achromatopsia: behavioral, anatomic, and physiologic aspects. Neurology 30, 10 (1980), 1064--1071.
16. Y. Deng, Y. Wang, Y. Ma, J. Bao, and X. Gu. 2007. A fixed transformation of color images for dichromats based on similarity matrices. In Proceedings of the 3rd International Conference on Intelligent Computing (ICIC’07), Lecture Notes in Computer Science, Vol. 4681. Springer, Berlin, 1018--1028.
17. P. Doliotis, G. Tsekouras, C.-N. Anagnostopoulos, and V. Athitsos. 2009. Intelligent modification of colors in digitized paintings for enhancing the visual perception of color-blind viewers. In Proceedings of the IFIP International Conference on Artificial Intelligence Applications and Innovations (AIAI’09), Vol. 296. Springer, Berlin, 293--301.
18. M. Fairchild. 2013. Color Appearance Models (3rd ed.). John Wiley 8 Sons, Ltd.
19. D. Flatla, A. Andrade, R. Teviotdale, D. Knowles, and C. Stewart. 2015. ColourID: Improving colour identification for people with impaired colour vision. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI’15). ACM Press, 3543--3552.
20. D. Flatla and C. Gutwin. 2012. SSMRecolor: Improving recoloring tools with situation-specific models of color differentiation. In Proceedings of the 30th International Conference on Human factors in Computing Systems (CHI’12). ACM Press, 2297--2306.
21. D. Flatla, K. Reinecke, C. Gutwin, and K. Gajos. 2013. SPRWeb: Preserving subjective responses to website colour schemes through automatic recolouring. In Proceedings of the Conference on Human Factors in Computing Systems (CHI’2013). ACM Press, 2069--2078.
22. B. Fraser, C. Murphy, and F. Bunting. 2005. Real World Color Management (2nd ed.). Peachpit Press, Berkeley, CA.
23. J. Gardner, M. Michaelides, G. Holder, N. Kanuga, T. Webb, J. Mollon, A. Moore, and A. Hardcastle. 2009. Blue cone monochromacy: Causative mutations and associated phenotypes. Molec. Vis. 15 (2009), 876--884.
24. R. Gonzalez and R. Woods. 1992. Digital Image Processing. Pearson, Prentice Hall.
25. R. Gray, J. Kieffer, and Y. Linde. 1980. Locally optimal block quantizer design. Inf. Contr. 45, 2 (1980), 178--198.
26. M. Hassan and R. Paramesran. 2017. Naturalness preserving image recoloring method for people with red-green deficiency. Sign. Process.: Image Commun. 57 (2017), 126--133.
27. C.-R. Huang, K.-C. Chiu, and C.-S. Chen. 2010. Key color priority based image recoloring for dichromats. In Proceedings of the 11th Pacific-Rim Conference on Multimedia (PCM’10), Lecture Notes in Computer Science, Vol. 6298. Springer-Verlag, Berlin, 637--647.
28. J.-B. Huang, C.-S. Chen, T.-C. Jen, and S.-J. Wang. 2009. Image recolorization for the colorblind. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP’09). IEEE Press, 1161--1164.
29. J.-B. Huang, Y.-C. Tseng, S.-I. Wu, and S.-J. Wang. 2007. Information preserving color transformation for protanopia and deuteranopia. Sign. Process. Lett. 14, 10 (2007), 711--714.
30. J.-B. Huang, S.-Y. Wu, and C.-S. Chen. 2008. Enhancing color representation for the color vision impaired. In Proceedings of the Workshop on Enhancing Color Representation for the Color Vision Impaired, Held in Conjunction with the 10th European Conference on Computer Vision (ECCV’08).
31. M. Huiskes, B. Thomee, and M. Lew. 2010. New trends and ideas in visual concept detection: The MIR flickr retrieval evaluation initiative. In Proceedings of the 11th ACM International Conference on Multimedia Information Retrieval (MIR’10). ACM Press, 527--536.
32. G. Iaccarino, D. Malandrino, M. Del Percio, and V. Scarano. 2006. Efficient edge-services for colorblind users. In Proceedings of the 15th International Conference on World Wide Web (WWW’06). ACM Press, 919--920.
33. M. Ichikawa, K. Tanaka, S. Kondo, K. Hiroshima, K. Ichikawa, S. Tanabe, and K. Fukami. 2003. Web-page color modification for barrier-free color vision with genetic algorithm. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO’03), Lecture Notes in Computer Science, Vol. 2724. Springer, Berlin, 2134--2146.
34. M. Ichikawa, K. Tanaka, S. Kondo, K. Hiroshima, K. Ichikawa, S. Tanabe, and K. Fukami. 2004. Preliminary study on color modification for still images to realize barrier-free color vision. In Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Vol. 1. IEEE Press, 36--41.
35. L. Jefferson and R. Harvey. 2006. Accommodating color blind computer users. In Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’06). ACM Press, 40--47.
36. L. Jefferson and R. Harvey. 2007. An interface to support color blind computer users. In Proceedings of the Conference on Human Factors in Computing Systems (CHI’2007). ACM Press, 1535--1538.
37. J.-Y. Jeong, H.-J. Kim, T.-S. Wang, Y.-J. Yoon, and S.-J. Ko. 2011. An efficient re-coloring method with information preserving for the color-blind. IEEE Trans. Cons. Electr. 57, 4 (2011), 1953--1960.
38. T. Kojima, R. Mochizuki, R. Lenz, and J. Chao. 2014. Riemann geometric color-weak compensation for individual observers. In Proceedings of the Universal Access in Human-Computer Interaction (UAHCI’14), Lecture Notes in Computer Science, Vol. 8514. Springer, Cham, Switzerland, 121--131.
39. H. Kolb, E. Fernandez, and R. Nelson (Eds.). 2011. WebVision: The Organization of the Retina and Visual System. John Moran Eye Center, University of Utah, Salt Lake City, UT. Retrieved from http://webvision.med.utah.edu/.
40. V. Kovalev. 2004. Towards image retrieval for eight percent of color-blind men. In Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04), Vol. 2. IEEE Press, 943--946.
41. V. Kovalev and M. Petrou. 2005. Optimising the choice of colours of an image database for dichromats. In Proceedings of the 4th International Conference on Machine Learning and Data Mining in Pattern Recognition (MLDM’05), Lecture Notes in Computer Science, Vol. 3587. Springer, Berlin, 456--465.
42. G. Kuhn, M. Oliveira, and L. Fernandes. 2008. An efficient naturalness-preserving image-recoloring method for dichromats. IEEE Trans. Vis. Comput. Graph. 14, 6 (2008), 1747--1754.
43. C.-L. Lai and S.-W. Chang. 2009. An image processing based visual compensation system for vision defects. In Proceedings of the 5th International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP’09). IEEE Computer Society, 559--562.
44. J. Lee and W. Santos. 2010. An adaptative fuzzy-based system to evaluate color blindness. In Proceedings of the 17th International Conference on Systems, Signals and Image Processing (IWSSIP’10). 211--214.
45. J. Lee and W. Santos. 2011. An adaptive fuzzy-based system to simulate, quantify and compensate color blindness. Integr. Comput.-Aid. Eng. 18, 1 (2011), 29--40.
46. M. Luo, G. Cui, and C. Li. 2006. Uniform colour spaces based on CIECAM02 colour appearance model. Color Res. Appl. 31, 4 (2006), 320--330.
47. M. R. Luo and R. W. G. Hunt. 1998. The structure of the CIE 1997 colour appearance model (CIECAM97s). Color Res. Appl. 23, 3 (1998), 138--146.
48. Y. Ma, X. Gu, and Y. Wang. 2006. A new color blindness cure model based on bp neural network. In Proceedings of the 3rd International Symposium on Neural Networks (ISNN’06), Lecture Notes in Computer Science, Vol. 3973. Springer-Verlag, Berlin, 740--745.
49. Y. Ma, X. Gu, and Y. Wang. 2008. Color discrimination enhancement for dichromats using self-organizing color transformation. Inf. Sci. 179, 6 (2008), 830--843.
50. G. Machado. 2010. A Model for Simulation of Color Vision Deficiency and a Color Contrast Enhancement for Dichromats. Ph.D. Dissertation. Universidade Federal do Rio Grande do Sul, Porto Alegre, Brazil.
51. G. M. Machado, M. M. Oliveira, and L. Fernandes. 2009. A physiologically-based model for simulation of color vision deficiency. IEEE Trans. Vis. Comput. Graph. 15, 6 (2009), 1291--1298.
52. E. Marieb and K. Hoehn. 2010. Human Anatomy 8 Physiology (8th ed.). Benjamin-Cummings Publishing Company, San Francisco, CA.
53. N. Milic, M. Hoffmann, T. Tomacs, D. Novakovic, and B. Milosavljevic. 2015. A content-dependent naturalness-preserving daltonization method for dichromatic and anomalous trichromatic color vision deficiencies. J. Imag. Sci. Technol. 59, 1 (2015), 010504.
54. Y. Miyake. 2006. Electrodiagnosis of Retinal Diseases. Springer-Verlag, Tokyo, Japan.
55. R. Mochizuki, T. Nakamura, J. Chao, and R. Lenz. 2008. Color-weak correction by discrimination threshold matching. In Proceedings of the 5th Conference on Colour in Graphics, Imaging and Vision (CGIV’08). Society for Imaging Science and Technology, 208--213.
56. R. Mochizuki, S. Oshima, and J. Chao. 2011a. Fast color-weakness compensation with discrimination threshold matching. In Proceedings of the 3rd International Workshop on Computational Color Imaging (CCIW’11), Lecture Notes in Computer Science, Vol. 6626. Springer, Berlin, 176--187.
57. R. Mochizuki, S. Oshima, R. Lenz, and J. Chao. 2011b. Exact compensation of color-weakness with discrimination threshold matching. In Proceedings of the 6th International Conference on Universal Access in Human-Computer Interaction (UAHCI’11), Lecture Notes in Computer Science, Vol. 6768. Springer, Berlin, 155--164.
58. J. Mollon, F. Newcombe, P. Polden, and G. Ratcliff. 1980. On the presence of three cone mechanisms in a case of total achromatopsia. In Proceedings of the 5th Symposium of the International Research Group on Colour Vision Deficiencies, G. Verriest (Ed.). Adam Hilger, Bristol, United Kingdom, 130--135.
59. J. Morovic and M. Luo. 1999. Developing algorithms for universal colour gamut mapping. In Colour Imaging: Vision and Technology, L. MacDonald and M. Luo (Eds.). John Wiley 8 Sons Ltd., 253--283.
60. S. Nakauchi and T. Onouchi. 2008. Detection and modification of confusing color combinations for red-green dichromats to achieve a color universal design. Color Res. Appl. 33, 3 (2008), 203--211.
61. Jamie R. Nuñez, Christopher R. Anderton, and Ryan S. Renslow. 2018. Optimizing colormaps with consideration for color vision deficiency to enable accurate interpretation of scientific data. PLoS One 13, 7 (2018), e0199239:1--14.
62. S. Oshima, R. Mochizuki, J. Chao, and R. Lenz. 2009. Color reproduction using riemann normal coordinates. In Proceedings of the 2nd International Computational Color Imaging Workshop (CCIW’09), Lecture Notes in Computer Science, Vol. 5646. Springer, Berlin, 140--149.
63. S. Oshima, R. Mochizuki, R. Lenz, and J. Chao. 2012. Color-weakness compensation using riemann normal coordinates. In Proceedings of the 2012 IEEE International Symposium on Multimedia (ISM’12). IEEE Computer Society, 175--178.
64. S. Oshima, R. Mochizuki, R. Lenz, and J. Chao. 2016. Modeling, measuring, and compensating color weak vision. IEEE Trans. Image Process. 25, 6 (2016), 2587--2600.
65. L.-C. Ou, M. Luo, A. Woodcock, and A. Wright. 2004. A study of colour emotion and colour preference. Part I: Colour emotions for single colours. Color Res. Appl. 29, 3 (2004), 232--240.
66. L. Petrich. 2012. Color-Blindness Simulators. Retrieved October 28, 2017 from http://lpetrich.org/Science/ColorBlindnessSim/ColorBlindnessSim.html.
67. S. Poret, R. Dony, and S. Gregori. 2009. Image processing for colour blindness correction. In Proceedings of the International Conference on Science and Technology for Humanity (TIC-STH’09). IEEE Computer Society, 539--544.
68. K. Rasche, R. Geist, and J. Westall. 2005a. Detail preserving reproduction of color images for monochromats and dichromats. IEEE Comput. Graph. Appl. 25, 3 (2005), 22--30.
69. K. Rasche, R. Geist, and J. Westall. 2005b. Re-coloring images for gamuts of lower dimension. Comput. Graph. Forum 24, 3 (2005), 423--432.
70. A. Reitner, L. Sharpe, and E. Zrenner. 1991. Is colour vision possible with only rods and blue-sensitive cones? Nature 352, 6338 (1991), 798--800.
71. M. Ribeiro and A. Gomes. 2013. A skillet-based recoloring algorithm for dichromats. In Proceedings of the 15th International Conference on e-Health Networking, Applications and Services (Healthcom’2013). IEEE Computer Society, 654--658.
72. J. Rissanen. 1997. Stochastic complexity in learning. J. Comput. Syst. Sci. 55, 1 (1997), 89--95.
73. R.-L. Rousseau. 1980. Le Language Des Couleurs. Éditions Dangles, St. Jean de Braye, France.
74. J. Ruminski, J. Wtorek, J. Ruminska, M. Kaczmarek, A. Bujnowski, T. Kocejko, and A. Polinski. 2010. Color transformation methods for dichromats. In Proceedings of the 3rd Conference on Human System Interactions (HSI’10). IEEE Computer Society, 634--641.
75. L. Ruttinger, D. Braun, K. Gegenfurtner, D. Petersen, P. Schonle, and L. Sharpe. 1999. Selective color constancy deficits after circumscribed unilateral brain lesions. J. Neurosci. 9, 8 (1999), 3094--3106.
76. B. Sajadi, A. Majumder, M. Oliveira, R. Schneider, and R. Raskar. 2013. Using patterns to encode color information for dichromats. IEEE Trans. Vis. Comput. Graph. 19, 1 (2013), 118--129.
77. J. Sammon. 1969. A nonlinear mapping for data structure analysis. IEEE Trans. Comput. 18, 5 (1969), 401--409.
78. L. Sharpe, A. Stockman, H. Jäagle, and J. Nathans. 1999. Opsin genes, cone photopigments, color vision and color blindness. In Color Vision: From Genes to Perception, K. Gegenfurtner and L. Sharpe (Eds.). Cambridge University Press.
79. Prarthana Shrestha and Bas Hulsken. 2014. Color accuracy and reproducibility in whole slide imaging scanners. J. Med. Imag. 1, 2 (2014), 027501:1--8.
80. N. Smith. 2017. Colorspacious documentation 2017. Retrieved from https://colorspacious.readthedocs.org/en/latest.
81. N. Suetake, G. Tanaka, H. Hashii, and E. Uchino. 2012. Simple lightness modification for color vision impaired based on Craik-O’Brien effect. J. Frank. Inst. 349, 6 (2012), 2093--2107.
82. M. Tkalcic and J. Tasic. 2003. Colour spaces: Perceptual, historical and applicational background. In Proceedings of the IEEE Region 8 International Conference on Computer as a Tool (EUROCON’03). IEEE Computer Society, 304--308.
83. L. Troiano, C. Birtolo, and G. Cirillo. 2009. Interactive genetic algorithm for choosing suitable colors in user interface. In Proceedings of 3rd International Conference on Learning and Intelligent Optimization (LION’09).
84. L. Troiano, C. Birtolo, and M. Miranda. 2008. Adapting palettes to color vision deficiencies by genetic algorithm. In Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation (GECCO’08). ACM Press, 1065--1072.
85. L. Velho, A. Frery, and J. Gomes. 2008. Image Processing for Computer Graphics and Vision. Springer-Verlag, London.
86. F. Vienot, H. Brettel, and J. Mollon. 1999. Digital video colourmaps for checking the legibility of displays by dichromats. Color Res. Appl. 24, 4 (1999), 243--252.
87. K. Wakita and K. Shimamura. 2005. SmartColor: Disambiguation framework for the colorblind. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS’05). ACM Press, 158--165.
88. M. Wang, B. Liu, and X. Hua. 2009. Accessible image search. In Proceedings of the 17th ACM International Conference on Multimedia (MM’09). ACM Press, 291--300.
89. M. Wang, B. Liu, and X. Hua. 2010. Accessible image search for colorblindness. ACM Trans. Intell. Syst. Technol. 1, 1, Article 8 (2010), 1--26.
90. A. Wong and W. Bishop. 2008. Perceptually-adaptive color enhancement of still images for individuals with dichromacy. In Proceedings of the Canadian Conference on Electrical and Computer Engineering (CCECE’08). IEEE Computer Society, 2027--2032.
91. S. Woo, C. Park, Y. S. Baek, and Y. Kwak. 2018. Flexible technique to enhance color-image quality for color-deficient observers. Curr. Opt. Photon. 2, 1 (2018), 101--106.
92. S. Yang and Y. Ro. 2003. Visual contents adaptation for color vision deficiency. In Proceedings of the 2003 International Conference on Image Processing (ICIP’03), Vol. 1. IEEE Computer Society, 453--456.
93. S. Yang, Y. Ro, J. Nam, J. Hong, S. Choi, and J. Lee. 2004. Improving visual accessibility for color vision deficiency based on MPEG-21. ETRI J. 26, 3 (2004), 195--202.
94. S. Yang, Y. Ro, E. Wong, and J. Lee. 2008. Quantification and standardized description of color vision deficiency caused by anomalous trichromats -- part I: Simulation and measurement. EURASIP J. Image Vid. Process. 2008 (2008), Article 487618, 9.
95. J. You and K.-C. Park. 2016. Image processing with color compensation using LCD display for color vision deficiency. J. Display Technol. 12, 6 (2016), 562--566.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Perception

    • Image manipulation

      • Image processing

• Human-centered computing

  • Accessibility

    • Accessibility systems and tools