skip to main content
research-article

Effect of compressed offline foveated video on viewing behavior and subjective quality

Published:22 February 2010Publication History
Skip Abstract Section

Abstract

Offline foveation is a technique to improve the compression efficiency of digitized video. The general idea behind offline foveation is to blur video regions where no or a small number of previewers look without decreasing the subjective quality for later viewers. It relies on the fact that peripheral vision is reduced compared to central vision, and the observation that during free-viewing humans' gaze positions generally coincide when watching video. In this article, we conduct two experiments to assess how offline foveation affects viewing behavior and subjective quality. In the first experiment, 15 subjects free-viewed six video clips before and after offline foveation whereas in the second experiment we had 17 subjects assessing the quality of these videos after one, two, and three consecutive viewings. Eye movements were measured during the experiments. Results showed that, although offline foveation prior to encoding with H.264 yielded data reductions up to 52% (20% average) on the tested videos, it had little or no effect on where people looked, their intersubject dispersion, fixation duration, saccade amplitude, or the experienced quality during first-time viewing. However, seeing the videos more than once increased the intersubject dispersion and decreased the subjective quality. In view of these results, we discuss the usage of offline foveated video in practical applications.

References

  1. Bergström, P. 2003. Doctoral dissertation. Ph.D. thesis, Linköping University, Sweden.Google ScholarGoogle Scholar
  2. Buswell, G. 1935. How People Look at Pictures. University of Chicago Press, Chicago, IL.Google ScholarGoogle Scholar
  3. Duchowski, A. 2000. Acuity matching resolution degradation through wavelet coefficient scaling. IEEE Trans. Image Process. 9, 8, 1437--1440. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Duchowski, A. and McCormick, B. 1998. Gaze-Contingent video resolution degradation. In Human Vision and Electronic Imaging, B. Rogowitz and T. Pappas, Eds. Vol. 3299. 318--329.Google ScholarGoogle Scholar
  5. Geisler, W. and Perry, J. 1998. A real-time foveated multi-resolution system for low-bandwidth video communication. In Human Vision and Electronic Imaging, B. Rogowitz and T. Pappas, Eds. Vol. 3299. 294--305.Google ScholarGoogle Scholar
  6. Geisler, W. and Perry, J. 1999. Variable-Resolution displays for visual communication and simulation. Soc. Inf. Display 30, 1, 420--423.Google ScholarGoogle Scholar
  7. Henderson, J. 2003. Human gaze control during real-world scene perception. Trends Cogn. Sci. 7, 11, 498--504.Google ScholarGoogle ScholarCross RefCross Ref
  8. Itti, L. 2004. Automatic foveation for video compression using a neurobiological model for visual attention. IEEE Trans. Image Process. 13, 10, 1304--1318. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Johannesson, E. 2005. Master's thesis, Lund University.Google ScholarGoogle Scholar
  10. Kortum, P. and Geisler, W. 1996. Implementation of a foveated image coding system for image bandwidth reduction. In Human Vision and Electronic Imaging, B. Rogowitz and J. Allebach, Eds. Vol. 2657. SPIE, San Jose, CA, 350--360.Google ScholarGoogle Scholar
  11. Le Meur, O., Le Callet, P., and Barba, D. 2007. Predicting visual fixations on video based on low-level visual features. Vis. Res. 46, 2483--2498.Google ScholarGoogle ScholarCross RefCross Ref
  12. Nyström, M. and Holmqvist, K. 2007. Deriving and evaluating eye-tracking controlled volumes of interest for variable-resolution video compression. J. Electron. Imag. 16, 1, 013006.Google ScholarGoogle Scholar
  13. Nyström, M. and Holmqvist, K. 2008. Semantic override of low-level features in image viewing—Both initially and overall. J. Eye Movem. Res. 2, 2, 2:1--2:11.Google ScholarGoogle Scholar
  14. Nyström, M., Novak, M., and Holmqvist, K. 2004. A novel approach to image coding using off-line foveation controlled by multiple eye-tracking measurements. In Proceedings of the Picture Coding Symposium.Google ScholarGoogle Scholar
  15. Parkhurst, D., Law, K., and Niebur, E. 2002. Modeling the role of salience in the allocation of overt visual attention. Vis. Res. 42, 107--123.Google ScholarGoogle Scholar
  16. Pedersen, B. and Spivey, M. 2006. Offline tracking of eyes and more with a simple Webcam. In Proceedings of the 28th Annual Meeting of the Cognitive Science Society.Google ScholarGoogle Scholar
  17. Rajashekar, U., Cormack, L., and Bovik, A. 2004. Point of gaze analysis reveals visual search strategies. In Human Vision and Electronic Imaging. Vol. 5292. 296--306.Google ScholarGoogle Scholar
  18. Sheikh, H. R., Liu, S., Evans, B. L., and Bovik, A. C. 2001. Real-Time foveation techniques for h.263 video encoding in software. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 3. 1781--1784. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Stelmach, L. B. and Tam, W. 1994. Processing image sequences based on eye movements. In Human Vision, Visual Processing and Digital Display. Vol. 2179. 90--98.Google ScholarGoogle ScholarCross RefCross Ref
  20. Tatler, B., Baddeley, R., and Gilchrist, I. 2005. Visual correlates of fixation selection: Effects of scale and time. Vis. Res. 45, 5, 643--659.Google ScholarGoogle ScholarCross RefCross Ref
  21. VQEG. 2003. Final report from the video quality experts group on the validation of objective models of video quality assessment, phase II. http://www.its.bldrdoc.gov/vqeg/projects/frtv_phaseII/downloads/VQEGII_Final_Report.pdf.Google ScholarGoogle Scholar
  22. Wang, Z. and Bovik, A. 2001. Embedded foveation image coding. IEEE Trans. Image Process. 10, 10, 1397--1410. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Wang, Z., Lu, L., and Bovik, A. 2003. Foveation scalable video coding with automatic fixation selection. IEEE Trans. Image Process. 12, 2, 243--254. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Yarbus, A. 1967. Eye Movements and Vision. Plenum Press, New York.Google ScholarGoogle Scholar

Index Terms

  1. Effect of compressed offline foveated video on viewing behavior and subjective quality

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Transactions on Multimedia Computing, Communications, and Applications
          ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 6, Issue 1
          February 2010
          138 pages
          ISSN:1551-6857
          EISSN:1551-6865
          DOI:10.1145/1671954
          Issue’s Table of Contents

          Copyright © 2010 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 22 February 2010
          • Accepted: 1 October 2008
          • Revised: 1 September 2008
          • Received: 1 March 2008
          Published in tomm Volume 6, Issue 1

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Research
          • Refereed

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!