skip to main content
research-article

Foveated gaze-contingent displays for peripheral LOD management, 3D visualization, and stereo imaging

Published:12 December 2007Publication History
Skip Abstract Section

Abstract

Advancements in graphics hardware have allowed development of hardware-accelerated imaging displays. This article reviews techniques for real-time simulation of arbitrary visual fields over still images and video. The goal is to provide the vision sciences and perceptual graphics communities techniques for the investigation of fundamental processes of visual perception. Classic gaze-contingent displays used for these purposes are reviewed and for the first time a pixel shader is introduced for display of a high-resolution window over peripherally degraded stimulus. The pixel shader advances current state-of-the-art by allowing real-time processing of still or streamed images, obviating the need for preprocessing or storage.

References

  1. Baudisch, P., DeCarlo, D., Duchowski, A. T., and Geisler, W. S. 2003. Focusing on the essential: Considering attention in display design. Commun. ACM 46, 3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bergström, P. 2003. Eye-Movement controlled image coding. Ph.D. thesis, Linköping University, Linköping, Sweden.Google ScholarGoogle Scholar
  3. Bertera, J. H. and Rayner, K. 2000. Eye movements and the span of the effective stimulus in visual search. Percept. Psychophys. 62, 3, 576--585.Google ScholarGoogle ScholarCross RefCross Ref
  4. Böhme, M., Dorr, M., Martinetz, T., and Barth, E. 2006. Gaze-Contingent temporal filtering of video. In Proceedings of the Eye Tracking Research Applications (ETRA) Symposium. ACM, San Diego, CA, 109--116. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Burt, P. J. and Adelson, E. H. 1983. The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31, 4 (Apr.), 532--540.Google ScholarGoogle Scholar
  6. Campbell, F. and Green, D. 1965. Monocular versus binocular visual acuity. Nature 208, 191--192.Google ScholarGoogle ScholarCross RefCross Ref
  7. Çöltekin, A. 2006. Foveation for 3D visualization and stereo imaging. Ph.D. thesis, Helsinki University of Technology, Helsinki, Finland. ISBN 951-22-8017-5, also <http://lib.tkk.fi/Diss/2006/isbn9512280175/>.Google ScholarGoogle Scholar
  8. Chang, E.-C., Mallat, S., and Yap, C. 2000. Wavelet foveation. J. Appl. Comput. Harmonic Anal. 9, 3, 312--335.Google ScholarGoogle ScholarCross RefCross Ref
  9. Daly, S., Matthews, K., and Ribas-Corbera, J. 2001. As plain as the noise on your face: Adaptive video compression using face detection and visual eccentricity models. J. Electron. Imag. 10, 1, 30--46.Google ScholarGoogle ScholarCross RefCross Ref
  10. Dorr, M., Böhme, M., Martinetz, T., and Barth, E. 2005. Visibility of temporal blur on a gaze-contingent display. In Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization (APGV). ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Drascic, D. and Milgam, P. 1991. Positioning accuracy of a virtual stereographic pointer in a real stereoscopic video world. In the Conference on Stereoscopic Displays and Applications II, Bellingham, WA. SPIE, vol. 1457, 58--69.Google ScholarGoogle Scholar
  12. Duchowski, A. T. 1998. 3D wavelet analysis of eye movements. In Proceedings of the Wavelet Applications IV Conference, Bellingham, WA. SPIE, vol. 3391.Google ScholarGoogle ScholarCross RefCross Ref
  13. Duchowski, A. T. 2000. Acuity-Matching resolution degradation through wavelet coefficient scaling. IEEE Trans. Image Process. 9, 8 (Aug.), 1437--1440. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Duchowski, A. T. 2003. Eye Tracking Methodology: Theory and Practice. Springer, London. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Duchowski, A. T. 2004. Hardware-Accelerated real-time simulation of arbitrary visual fields. In Proceedings of the Eye Tracking Research and Applications (ETRA) Symposium, San Antonio, TX, 59. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Duchowski, A. T. and McCormick, B. H. 1995. Pre-Attentive considerations for gaze-contingent image processing. In Proceedings of the Conference on Human Vision, Visual Processing, and Digital Display VI, Bellingham, WA. SPIE, vol. 2411, 128--139.Google ScholarGoogle ScholarCross RefCross Ref
  17. Ferwerda, J. A., Pattanaik, S. N., Shirley, P., and Greenberg, D. P. 1996. A model of visual adaptation for realistic image synthesis. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. ACM, New York, 249--258. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Foley, J. D., van Dam, A., Feiner, S. K., and Hughes, J. F. 1990. Computer Graphics: Principles and Practice, 2nd ed. Addison-Wesley, Reading, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Foster, D. H., Gravano, S., and Tomoszek, A. 1989. Acuity for fine-grain motion and for two-dot spacing as a function of retinal eccentricity: Differences in specialization of the central and peripheral retina. Vision Res. 29, 8, 1017--1031.Google ScholarGoogle ScholarCross RefCross Ref
  20. Funkhouser, T. A. and Séquin, C. H. 1993. Adaptive display algorithm for interactive frame rates during visualization of complex virtual environments. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Geisler, W. S. and Perry, J. S. 1998. Real-Time foveated multiresolution system for low-bandwidth video communication. In Conference on Human Vision and Electronic Imaging, Bellingham, WA.Google ScholarGoogle Scholar
  22. Geisler, W. S. and Perry, J. S. 2002. Real-Time simulation of arbitrary visual fields. In Proceedings of the Eye Tracking Research and Applications (ETRA) Symposium, New Orleans, LA. ACM, 83--153. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Hahn, P. J. and Mathews, V. J. 1997. Perceptually lossless image compression. In Proceedings of the Data Compression Industry Workshop, Snowbird, UT. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Holliman, N. 2005. 3D Display Systems. IOP Press. ISBN: 0 7503 0646 7. also <http://www.dur.ac.uk/n.s.holliman/Presentations/3dv3-0.pdf>, last accessed January 2007.Google ScholarGoogle Scholar
  25. Levoy, M. and Whitaker, R. 1990. Gaze-Directed volume rendering. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. ACM, New York, 217--223. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Linde, I. V. D. 2003. Space-Variant perceptual image compression for gaze-contingent stereoscopic displays. Ph.D. thesis, Anglia Polytechnic University, UK.Google ScholarGoogle Scholar
  27. Lipton, L. 1982. Foundations of the Stereoscopic Cinema, A Study in Depth. Van Nostrand Reinhold. ISBN O-442-24724-9. also <http://www.stereoscopic.org>.Google ScholarGoogle Scholar
  28. Loschky, L. C. and McConkie, G. W. 2000. User performance with gaze contingent multiresolutional displays. In Proceedings of the Eye Tracking Research and Applications Symposium. Palm Beach Gardens, FL, 97--103. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Loschky, L. C. and Wolverton, G. S. 2007. How late can you update gaze-contingent multiresolutional displays without detection? ACM Trans. Multimedia Comput. Commun. Appl. 3, 4 (Nov.). this issue. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Luebke, D. and Erikson, C. 1997. View-Dependent simplification of arbitrary polygonal environments. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Luebke, D., Reddy, M., Cohen, J. D., Varshney, A., Watson, B., and Huebner, R. 2003. Level of Detail for 3D Graphics. Morgan-Kauffman/Elsevier, San Francisco, CA. ISBN 1-55860-838-9. also <http://www.lodbook.com/>. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Luebke, D., Varshney, A., Cohen, J., Watson, B., and Reddy, M. 2000. Course 41: Advanced issues in level of detail. In Proceedings of the ACM SIGGRAPH International Conference on Interactive Techniques. New York, NY.Google ScholarGoogle Scholar
  33. McConkie, G. W. and Rayner, K. 1975. The span of the effective stimulus during a fixation in reading. Percept. Psychophys. 17, 578--586.Google ScholarGoogle ScholarCross RefCross Ref
  34. Mon-Williams, M. and Wann, J. 1998. Binocular virtual reality displays: When problems do and don't occur. Hum. Factors 40, 18--24. also <http://www.abdn.ac.uk/~psy359/dept/Papers/Human Facts.pdf>.Google ScholarGoogle ScholarCross RefCross Ref
  35. Murphy, H. and Duchowski, A. T. 2001. Gaze-Contingent level of detail. In EuroGraphics Conference, Manchester, UK.Google ScholarGoogle Scholar
  36. National Eye Institute. 2004. Office of Communication and Health Education. Personal communique.Google ScholarGoogle Scholar
  37. National Institutes of Health. 2003. Age-Related macular degneration: What you should know. Rep. 03-2294, National Eye Institute, National Institutes of Health. also <http://www.nei.nih.gov/health/maculardegen/webAMD.pdf>, last accessed January 2007.Google ScholarGoogle Scholar
  38. Nikolov, S. G., Newman, T. D., Bull, D. R., Canagarajah, N. C., Jones, M. G., and Gilchrist, I. D. 2004. Gaze-Contingent display using texture mapping and OpenGL: System and applications. In Proceedings of the Eye Tracking Research and Applications (ETRA) Symposium, San Antonio, TX, 11--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Ohshima, T., Yamamoto, H., and Tamura, H. 1996. Gaze-Directed adaptive rendering for interacting with virtual space. In Proceedings of the Virtual Reality Annual International Symposium (VRAIS). IEEE, 103--110. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. O'Sullivan, C., Dingliana, J., Giang, T., and Kaiser, M. K. 2003. Evaluating the visual fidelity of physically based animations. ACM Trans. Graphics 22, 3, 527--536. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Parkhurst, D., Culurciello, E., and Niebur, E. 2000. Evaluating variable resolution displays with visual search: Task performance and eye movements. In Proceedings of the Eye Tracking Research and Applications Symposium, Palm Beach Gardens, FL, 105--109. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Parkhurst, D. J. and Niebur, E. 2002. Variable resolution displays: A theoretical, practical, and behavioral evaluation. Hum. Factors 44, 4, 611--629.Google ScholarGoogle ScholarCross RefCross Ref
  43. Parkhurst, D. J. and Niebur, E. 2004. A feasibility test for perceptually adaptive level of detail rendering on desktop systems. In Proceedings of the Applied Perception, Graphics and Visualization (APGV) Symposium. ACM, New York, 49--56. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Ramasubramanian, M., Pattanaik, S. N., and Greenberg, D. P. 1999. A perceptually based physical error metric for realistic image synthesis. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. ACM, New York, NY, 73--82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Rayner, K. 1998. Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124, 3, 372--422.Google ScholarGoogle ScholarCross RefCross Ref
  46. Rayner, K. and Bertera, J. H. 1979. Reading without a fovea. Sci. 206, 468--469.Google ScholarGoogle ScholarCross RefCross Ref
  47. Reddy, M. 2001. Perceptually optimized 3D graphics. Comput. Graphics Appl. 21, 5, 68--75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Reingold, E. M., Loschky, L. C., McConkie, G. W., and Stampe, D. M. 2003. Gaze-Contingent multi-resolutional displays: An integrative review. Hum. Factors 45, 2, 307--328.Google ScholarGoogle ScholarCross RefCross Ref
  49. Shreiner, D., Woo, M., Neider, J., and Davis, T. 2006. OpenGL Programming Guide: The Official Guide to Learning OpenGL, Version 2, 5th ed. Addison-Wesley. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Tobii Technology AB. 2003. Tobii ET-17 eye-tracker product description (v1.1). <http://www.tobii.se/> (last accessed January 2007).Google ScholarGoogle Scholar
  51. Ware, C. 2004. Information Visualization---Perception for Design. Morgan-Kauffman, Elsevier, San Francisco, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Watson, B., Walker, N., and Hodges, L. F. 2004. Supra-Threshold control of peripheral LOD. In Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques, 750--759. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Watson, B., Walker, N., Hodges, L. F., and Worden, A. 1997. Managing level of detail through peripheral degradation: Effects on search performance with a head-mounted display. ACM Trans. Comput.-Hum. Interact. 4, 4 (Dec.), 323--346. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Williams, L. 1983. Pyramidal parametrics. Comput. Graphics 17, 3 (Jul.), 1--11. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Foveated gaze-contingent displays for peripheral LOD management, 3D visualization, and stereo imaging

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 3, Issue 4
        December 2007
        147 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/1314303
        Issue’s Table of Contents

        Copyright © 2007 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 12 December 2007
        • Received: 1 August 2007
        • Accepted: 1 August 2007
        Published in tomm Volume 3, Issue 4

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article
        • Research
        • Pre-selected

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!