stelaCSF: a unified model of contrast sensitivity as the function of spatio-temporal frequency, eccentricity, luminance and area

Abstract

A contrast sensitivity function, or CSF, is a cornerstone of many visual models. It explains whether a contrast pattern is visible to the human eye. The existing CSFs typically account for a subset of relevant dimensions describing a stimulus, limiting the use of such functions to either static or foveal content but not both. In this paper, we propose a unified CSF, stelaCSF, which accounts for all major dimensions of the stimulus: spatial and temporal frequency, eccentricity, luminance, and area. To model the 5-dimensional space of contrast sensitivity, we combined data from 11 papers, each of which studied a subset of this space. While previously proposed CSFs were fitted to a single dataset, stelaCSF can predict the data from all these studies using the same set of parameters. The predictions are accurate in the entire domain, including low frequencies. In addition, stelaCSF relies on psychophysical models and experimental evidence to explain the major interactions between the 5 dimensions of the CSF. We demonstrate the utility of our new CSF in a flicker detection metric and in foveated rendering.

References

1. Albert Ahumada and Heidi A. Peterson. 1992. Luminance-model-based DCT quantization for color image compression. In SPIE 1666, Human vision, visual processing, and digital display III.
2. Albert Ahumada, Jihyun Yeonan-Kim, and Andrew B. Watson. 2018. A Dual Channel Spatial-Temporal Detection Model. Electronic Imaging 2018, 14 (2018), 1--4.
3. Hirotugu Akaike. 1974. A new look at the statistical model identification. IEEE transactions on automatic control 19, 6 (1974), 716--723.
4. S. J. Anderson, K. T. Mullen, and R. F. Hess. 1991. Human peripheral spatial resolution for achromatic and chromatic stimuli: limits imposed by optical and retinal factors. The Journal of Physiology 442 (1991), 47--64.
5. Pontus Andersson, Jim Nilsson, Tomas Akenine-Möller, Magnus Oskarsson, Kalle Åström, and Mark D Fairchild. 2020. FLIP: A Difference Evaluator for Alternating Images. Proc. ACM Comput. Graph. Interact. Tech. 3, 2 (2020), 15--1.
6. Peter G. J. Barten. 1999. Contrast sensitivity of the human eye and its effects on image quality. SPIE Press.
7. Peter G. J. Barten. 2003. Formula for the contrast sensitivity of the human eye. In Proc. SPIE 5294, Image Quality and System Performance. 231--238.
8. Christina A. Burbeck and D. H. Kelly. 1980. Spatiotemporal characteristics of visual mechanisms: excitatory-inhibitory model. Journal of the Optical Society of America 70, 9 (September 1980), 1121--1126.
9. Fergus W Campbell and John G Robson. 1968. Application of Fourier analysis to the visibility of gratings. The Journal of physiology 197, 3 (1968), 551--566.
10. Scott J Daly. 1992. Visible differences predictor: an algorithm for the assessment of image fidelity. In SPIE 1666, Human Vision, Visual Processing, and Digital Display III. International Society for Optics and Photonics, 2--15.
11. Scott J. Daly. 1998. Engineering observations from spatiovelocity and spatiotemporal visual models. In SPIE 3299, Human Vision and Electronic Imaging. 180--191.
12. H De Lange. 1952. Experiments on flicker and some calculations on an electrical analogue of the foveal systems. Physica 18, 11 (1952), 935--950.
13. Gyorgy Denes and Rafał K. Mantiuk. 2020. Predicting visible flicker in temporally changing images. Electronic Imaging 2020, 11 (2020), 233--1.
14. M. A. Georgeson and G. D. Sullivan. 1975. Contrast constancy: deblurring in human vision by spatial frequency channels. The J. of Physiology 252, 3 (1975), 627--656.
15. Brian Guenter, Mark Finch, Steven Drucker, Desney Tan, and John Snyder. 2012. Foveated 3D graphics. ACM Transactions on Graphics (TOG) 31, 6 (2012), 1--10.
16. S.T. Hammett and A.T. Smith. 1992. Two temporal channels or three? A re-evaluation. Vision Research 32, 2 (February 1992), 285--291.
17. E. Hartmann, B. Lachenmayr, and H. Brettel. 1979. The peripheral critical flicker frequency. Vision Research 19, 9 (jan 1979), 1019--1023.
18. D. H. Kelly. 1979a. Motion and vision. I. Stabilized images of stationary gratings. Journal of the Optical Society of America 69, 9 (1979), 1266--1274.
19. D. H. Kelly. 1979b. Motion and vision. II. Stabilized spatio-temporal threshold surface. Journal of Optical Society of America 69, 10 (1979), 1340--1349.
20. Jan J. Koenderink, Maarten A. Bouman, Albert E. Bueno de Mesquita, and Sybe Slappendel. 1978. Perimetry of contrast detection thresholds of moving spatial sine wave patterns IV The influence of the mean retinal illuminance. Journal of the Optical Society of America 68, 6 (June 1978), 860 -- 865.
21. Brooke Krajancich, Petr Kellnhofer, and Gordon Wetzstein. 2021. A Perceptual Model for Eccentricity-Dependent Spatio-Temporal Flicker Fusion and Its Applications to Foveated Graphics. ACM Trans. Graph. 40, 4, Article 47 (July 2021), 11 pages.
22. Justin Laird, Mitchell Rosen, Jeff Pelz, Ethan Montag, and Scott Daly. 2006. Spatiovelocity CSF as a function of retinal velocity using unstabilized stimuli. In SPIE 6057, Human Vision and Electronic Imaging XI.
23. G. E. Legge and J. M. Foley. 1980. Contrast masking in human vision. Journal of the Optical Society of America 70, 12 (December 1980), 1458--1471.
24. Rafal Mantiuk, Scott J Daly, Karol Myszkowski, and Hans-Peter Seidel. 2005. Predicting visible differences in high dynamic range images: model and its calibration. In Human Vision and Electronic Imaging X, Vol. 5666. International Society for Optics and Photonics, 204--214.
25. Rafał K. Mantiuk, S. Daly, and L. Kerofsky. 2008. Display adaptive tone mapping. ACM Transactions on Graphics 27, 3 (2008), 1--10.
26. Rafał K. Mantiuk, Gyorgy Denes, Alexandre Chapiro, Anton Kaplanyan, Gizem Rufo, Romain Bachy, Trisha Lian, and Anjul Patney. 2021. FovVideoVDP: A visible difference predictor for wide field-of-view video. ACM Transaction on Graphics 40, 4 (2021), 1--19.
27. Rafał K. Mantiuk, Kil Joong Kim, Allan G. Rempel, and Wolfgang Heidrich. 2011. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions. ACM Transactions on Graphics 30, 4 (July 2011), 1--14.
28. Rafał K. Mantiuk, Minjung Kim, Maliha Ashraf, Qiang Xu, M. Ronnier Luo, Jasna Martinovic, and Sophie Wuerger. 2020. Practical Color Contrast Sensitivity Functions for Luminance Levels up to 10000 cd/m². In Color and Imaging Conference, Vol. 2020. Society for Imaging Science and Technology, 1--6.
29. W. H. Merigan, L. M. Katz, and J. H. Maunsell. 1991. The effects of parvocellular lateral geniculate lesions on the acuity and contrast sensitivity of macaque monkeys. The Journal of Neuroscience 11, 4 (April 1991), 994--1001.
30. Juvi Mustonen, Jyrki Rovamo, and Risto Näsänen. 1993. The effects of grating area and spatial frequency on contrast sensitivity as a function of light level. Vision Research 33, 15 (October 1993), 2065--2072.
31. Anjul Patney, Marco Salvi, Joohwan Kim, Anton Kaplanyan, Chris Wyman, Nir Benty, David Luebke, and Aaron Lefohn. 2016. Towards foveated rendering for gaze-tracked virtual reality. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1--12.
32. D. G. Pelli, Melanie Palomares, and Najib J. Majaj. 2004. Crowding is unlike ordinary masking: Distinguishing feature integration from detection. Journal of Vision 4, 12 (dec 2004), 12.
33. John G. Robson. 1966. Spatial and temporal contrast-sensitivity functions of the visual system. Journal of Optical Society of America 56, 8 (1966), 1141--1142.
34. Ankit Rohatgi. 2021. Webplotdigitizer: Version 4.5. https://automeris.io/WebPlotDigitizer
35. Jyrki Rovamo, Olavi Luntinen, and Risto Näsänen. 1993. Modelling the dependence of contrast sensitivity on grating area and spatial frequency. Vision Research 33, 18 (December 1993), 2773--2788.
36. Jyrki Rovamo, Juvi Mustonen, and Risto Näsänen. 1995. Neural modulation transfer function of the human visual system at various eccentricities. Vision Research 35, 6 (March 1995), 767--774.
37. Robert L. Savoy and John J. McCann. 1975. Visibility of low-spatial-frequency sine-wave targets: Dependence on number of cycles. Journal of the Optical Society of America 65, 3 (March 1975), 343--350.
38. Otto H. Schade. 1956. Optical and photoelectric analog of the eye. Journal of Optical Society of America 46, 9 (1956), 721--739.
39. Vincent Sitzmann, Ana Serrano, Amy Pavel, Maneesh Agrawala, Diego Gutierrez, Belen Masia, and Gordon Wetzstein. 2018. Saliency in VR: How do people explore virtual environments? IEEE transactions on Visualization and Computer Graphics (2018).
40. Robert J. Snowden, Robert F. Hess, and Sarah J. Waugh. 1995. The Processing of Temporal Modulation at Different Levels of Retinal Illuminance. Vision Research 35, 6 (1995), 775--789.
41. Hans Strasburger, Ingo Rentschler, and Martin Jüttner. 2011. Peripheral vision and pattern recognition: A review. Journal of Vision 11, 5 (2011), 1--82.
42. Okan Tarhan Tursun, Elena Arabadzhiyska-Koleva, Marek Wernikowski, Radosław Mantiuk, Hans-Peter Seidel, Karol Myszkowski, and Piotr Didyk. 2019. Luminance-contrast-aware foveated rendering. ACM Transactions on Graphics 38 (2019).
43. Christopher W. Tyler and Russell D. Hamer. 1990. Analysis of visual modulation sensitivity. IV. Validity of the Ferry-Porter law. Journal of Optical Society of America A 7, 4 (1990), 743--758.
44. Christopher W Tyler and Russell D Hamer. 1993. Eccentricity and the Ferry-Porter law. Journal of Optical Society of America A 10, 9 (1993), 2084--2087.
45. Peter Vangorp, Karol Myszkowski, Erich W. Graf, and Rafał K. Mantiuk. 2015. A model of local adaptation. ACM Transactions on Graphics 34, 6 (oct 2015), 1--13.
46. V. Virsu and J. Rovamo. 1979. Visual resolution, contrast sensitivity, and the cortical magnification factor. Experimental Brain Research 37, 3 (1979), 475--494.
47. Veijo Virsu, Jyrki Rovamo, and Pentti Laurinen. 1982. Temporal Contrast Sensitivity and Cortical Magnification. Vision Research 22 (1982), 1211--1217.
48. Andrew B. Watson. 2000. Visual detection of spatial contrast patterns: Evaluation of five simple models. Optics Express 6, 1 (2000), 12--33.
49. Andrew B. Watson. 2018. The field of view, the field of resolution, and the field of contrast sensitivity. Journal of Perceptual Imaging 1, 1 (January 2018), 1--11.
50. Andrew B. Watson and Albert J. Ahumada. 2005. A standard model for foveal detection of spatial contrast. Journal of Vision 5, 9 (2005), 717--740.
51. Andrew B. Watson and Albert J. Ahumada. 2016. The pyramid of visibility. Electronic Imaging 16 (2016), 37--42.
52. M. J. Wright and A. Johnston. 1983. Spatiotemporal contrast sensitivity and visual field locus. Vision Research 23, 10 (1983), 983--989.
53. Sophie Wuerger, Maliha Ashraf, Minjung Kim, Jasna Martinovic, María Pérez-Ortiz, and Rafał K. Mantiuk. 2020. Spatio-chromatic contrast sensitivity under mesopic and photopic light levels. Journal of Vision 20, 4 (April 2020), 23.
54. Shinyoung Yi, Daniel S. Jeon, Ana Serrano, Se-Yoon Jeong, Hui-Yong Kim, Diego Gutierrez, and Min H. Kim. 2022. Modelling Surround-aware Contrast Sensitivity for HDR Displays. Computer Graphics Forum 41, 1 (feb 2022), 350--363.
55. Wenjun Zeng, Scott Daly, and Shawmin Lei. 2002. An overview of the visual optimization tools in JPEG 2000. Signal Processing: Image Communication 17 (2002).

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Perception