Dark stereo: improving depth perception under low luminance

Abstract

It is often desirable or unavoidable to display Virtual Reality (VR) or stereoscopic content at low brightness. For example, a dimmer display reduces the flicker artefacts that are introduced by low-persistence VR headsets. It also saves power, prolongs battery life, and reduces the cost of a display or projection system. Additionally, stereo movies are usually displayed at relatively low luminance due to polarization filters or other optical elements necessary to separate two views. However, the binocular depth cues become less reliable at low luminance. In this paper, we propose a model of stereo constancy that predicts the precision of binocular depth cues for a given contrast and luminance. We use the model to design a novel contrast enhancement algorithm that compensates for the deteriorated depth perception to deliver good-quality stereoscopic images even for displays of very low brightness.

References

1. Reynold Bailey, Cindy Grimm, Christopher Davoli, and Richard Abrams. 2007. The effect of object color on depth ordering. (2007).
2. J. L. Barbur and A. Stockman. 2010. Photopic, mesopic and scotopic vision and changes in visual performance. Encyclopedia of the Eye 3 (2010), 323--331.
3. P. Burt and E. Adelson. 1983. The Laplacian Pyramid as a Compact Image Code. IEEE Transactions on Communications 31, 4 (1983), 532--540.
4. Dingcai Cao, Joel Pokorny, Vivianne C Smith, and Andrew J Zele. 2008. Rod contributions to color perception: linear with rod contrast. Vision Research 48, 26 (nov 2008), 2586--92.
5. Alexandre Chapiro, Robin Atkins, and Scott Daly. 2019. A Luminance-Aware Model of Judder Perception. ACM Transactions on Graphics 38, 5, Article 142 (2019), 10 pages.
6. J. Cutting and P. Vishton. 1995. Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth. In Perception of Space and Motion (Handbook Of Perception And Cognition), W. Epstein and S. Rogers (Eds.). Academic Press, 69--117.
7. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2011. A Perceptual Model for Disparity. ACM Transactions on Graphics, Article 96 (2011), 10 pages.
8. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, and Hans-Peter Seidel. 2012a. Apparent stereo: the Cornsweet illusion can enhance perceived depth. In Human Vision and Electronic Imaging XVII, Bernice E. Rogowitz, Thrasyvoulos N. Pappas, and Huib de Ridder (Eds.), Vol. 8291. International Society for Optics and Photonics, SPIE, 180 -- 191.
9. Piotr Didyk, Tobias Ritschel, Elmar Eisemann, Karol Myszkowski, Hans-Peter Seidel, and Wojciech Matusik. 2012b. A Luminance-Contrast-Aware Disparity Model and Applications. ACM Transactions on Graphics 31, 6 (2012).
10. A. Erickson, K. Kim, G. Bruder, and G. F. Welch. 2020. Effects of Dark Mode Graphics on Visual Acuity and Fatigue with Virtual Reality Head-Mounted Displays. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 434--442.
11. John P Frisby and John E W Mayhew. 1978. Contrast Sensitivity Function for Stereopsis. Perception 7, 4 (1978), 423--429. arXiv:https://doi.org/10.1068/p070423 PMID: 704272.
12. Chenyang Fu, Changjun Li, Guihua Cui, M. Ronnier Luo, Robert W. G. Hunt, and Michael R. Pointer. 2012. An investigation of colour appearance for unrelated colours under photopic and mesopic vision. Color Research & Application 37, 4 (2012), 238--254.
13. B Y M A Georgeson and G D Sullivan. 1975. Contrast constancy: deblurring in human vision by spatial frequency channels. The Journal of Physiology 252, 3 (1975), 627--656.
14. David M. Hoffman and Grace Lee. 2019. Temporal Requirements for VR Displays to Create a More Comfortable and Immersive Visual Experience. Information Display 35, 2 (mar 2019), 9--39.
15. Jochen Jacobs, Xi Wang, and Marc Alexa. 2019. Keep It Simple: Depth-Based Dynamic Adjustment of Rendering for Head-Mounted Displays Decreases Visual Comfort. ACM Trans. Appl. Percept. 16, 3, Article 16 (2019), 16 pages.
16. Petr Kellnhofer, Piotr Didyk, Karol Myszkowski, Mohamed M. Hefeeda, Hans-Peter Seidel, and Wojciech Matusik. 2016. GazeStereo3D: Seamless Disparity Manipulations. ACM Transactions on Graphics 35, 4 (2016).
17. Petr Kellnhofer, Tobias Ritschel, Peter Vangorp, Karol Myszkowski, and Hans-Peter Seidel. 2014. Stereo Day-for-Night: Retargeting Disparity for Scotopic Vision. ACM Trans. Appl. Percept. 11, 3 (2014).
18. Min H Kim, Tim Weyrich, and Jan Kautz. 2009. Modeling human color perception under extended luminance levels. In ACM SIGGRAPH 2009 papers. 1--9.
19. John Krauskopf and Jason D Forte. 2002. Influence of chromaticity on vernier and stereo acuity. Journal of Vision 2, 9 (2002), 6--6.
20. J.J. Kulikowski. 1976. Effective contrast constancy and linearity of contrast sensation. Vision Research 16, 12 (jan 1976), 1419--1431.
21. Youngshin Kwak, Lindsay William MacDonald, and M. Ronnier Luo. 2003. Mesopic color appearance. In Human Vision and Electronic Imaging VIII, Bernice E. Rogowitz and Thrasyvoulos N. Pappas (Eds.), Vol. 5007. International Society for Optics and Photonics, SPIE, 161 -- 169.
22. Manuel Lang, Alexander Hornung, Oliver Wang, Steven Poulakos, Aljoscha Smolic, and Markus Gross. 2010. Nonlinear Disparity Mapping for Stereoscopic 3D. ACM Transactions on Graphics 29, 4, Article 75 (2010), 10 pages.
23. Margaret S. Livingstone and David H. Hubel. 1994. Stereopsis and positional acuity under dark adaptation. Vision Research 34, 6 (1994), 799--802.
24. Thomas Luft, Carsten Colditz, and Oliver Deussen. 2006. Image Enhancement by Unsharp Masking the Depth Buffer. ACM Transactions on Graphics 25, 3 (2006), 1206--1213.
25. Ming Ronnier Luo and Changjun Li. 2013. CIECAM02 and Its Recent Developments. Springer New York, New York, NY, 19--58.
26. Rafał Mantiuk, Allan G. Rempel, and Wolfgang Heidrich. 2009. Display considerations for night and low-illumination viewing. In Proc. of Symposium on Applied Perception in Graphics and Visualization - APGV '09. 53--58.
27. Thomas Oskam, Alexander Hornung, Huw Bowles, Kenny Mitchell, and Markus Gross. 2011. OSCAM - Optimized Stereoscopic Camera Control for Interactive 3D. ACM Transactions on Graphics 30, 6 (2011), 1--8.
28. Eli Peli, Jian Yang, Robert Goldstein, and Adam Reeves. 1991. Effect of luminance on suprathreshold contrast perception. Journal of the Optical Society of America A 8, 8 (aug 1991), 1352.
29. M. Schuchhardt, S. Jha, R. Ayoub, M. Kishinevsky, and G. Memik. 2015. Optimizing mobile display brightness by leveraging human visual perception. In 2015 International Conference on Compilers, Architecture and Synthesis for Embedded Systems (CASES). 11--20.
30. Jae Chul Shin, Hirohisa Yaguchi, and Satoshi Shioiri. 2004. Change of Color Appearance in Photopic, Mesopic and Scotopic Vision. Optical Review 11, 4 (2004), 265--271.
31. David R Simmons and Frederick AA Kingdom. 1994. Contrast thresholds for stereoscopic depth identification with isoluminant and isochromatic stimuli. Vision Research 34, 22 (1994), 2971--2982.
32. Gurjot Singh, Stephen R. Ellis, and J. Edward Swan. 2018. The Effect of Focal Distance, Age, and Brightness on Near-Field Augmented Reality Depth Matching. IEEE Transactions on Visualization and Computer Graphics 26, 2 (2018), 1385--1398. arXiv:1712.00088
33. Bjørn Stabell and Ulf Stabell. 1998. Chromatic rod-cone interaction during dark adaptation. J. Opt. Soc. Am. A 15, 11 (Nov 1998), 2809--2815.
34. Tom Troscianko, Rachel Montagnon, Jacques Le Clerc, Emmanuelle Malbert, and Pierre-Louis Chanteau. 1991. The role of colour as a monocular depth cue. Vision research 31, 11 (1991), 1923--1929.
35. Khrystyna Vasylevska, Hyunjin Yoo, Tara Akhavan, and Hannes Kaufmann. 2019. Towards Eye-Friendly VR: How Bright Should It Be?. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 566--574.
36. Robert Wanat and Rafał K. Mantiuk. 2014. Simulating and compensating changes in appearance between day and night vision. ACM Trans. Graph. 33 (2014), 147:1--147:12.
37. John P Wann, Simon Rushton, and Mark Mon-Williams. 1995. Natural problems for stereoscopic depth perception in virtual environments. Vision research 35, 19 (1995), 2731--2736.
38. Simon J. Watt, Kurt Akeley, Marc O. Ernst, and Martin S. Banks. 2005. Focus cues affect perceived depth. Journal of Vision 5, 10 (12 2005), 7--7. arXiv:https://arvojournals.org/arvo/content_public/journal/jov/932834/jov-5-10-7.pdf
39. Felix A Wichmann and N Jeremy Hill. 2001. The psychometric function: I. Fitting, sampling, and goodness of fit. Perception & Psychophysics 63, 8 (2001), 1293--1313.
40. Sophie Wuerger, Maliha Ashraf, Minjung Kim, Jasna Martinovic, María Pérez-Ortiz, and Rafal K. Mantiuk. 2020. Spatio-chromatic contrast sensitivity under mesopic and photopic light levels. Journal of Vision 20, 4 (apr 2020), 23.
41. Fangcheng Zhong, Akshay Jindal, Ali Özgür Yöntem, Param Hanji, Simon J. Watt, and Rafał K. Mantiuk. 2021. Reproducing reality with a high-dynamic-range multi-focal stereo display. ACM Transactions on Graphics 40, 6 (dec 2021), 1--14.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Perception

      • Virtual reality