Abstract
Offline foveation is a technique to improve the compression efficiency of digitized video. The general idea behind offline foveation is to blur video regions where no or a small number of previewers look without decreasing the subjective quality for later viewers. It relies on the fact that peripheral vision is reduced compared to central vision, and the observation that during free-viewing humans' gaze positions generally coincide when watching video. In this article, we conduct two experiments to assess how offline foveation affects viewing behavior and subjective quality. In the first experiment, 15 subjects free-viewed six video clips before and after offline foveation whereas in the second experiment we had 17 subjects assessing the quality of these videos after one, two, and three consecutive viewings. Eye movements were measured during the experiments. Results showed that, although offline foveation prior to encoding with H.264 yielded data reductions up to 52% (20% average) on the tested videos, it had little or no effect on where people looked, their intersubject dispersion, fixation duration, saccade amplitude, or the experienced quality during first-time viewing. However, seeing the videos more than once increased the intersubject dispersion and decreased the subjective quality. In view of these results, we discuss the usage of offline foveated video in practical applications.
- Bergström, P. 2003. Doctoral dissertation. Ph.D. thesis, Linköping University, Sweden.Google Scholar
- Buswell, G. 1935. How People Look at Pictures. University of Chicago Press, Chicago, IL.Google Scholar
- Duchowski, A. 2000. Acuity matching resolution degradation through wavelet coefficient scaling. IEEE Trans. Image Process. 9, 8, 1437--1440. Google Scholar
Digital Library
- Duchowski, A. and McCormick, B. 1998. Gaze-Contingent video resolution degradation. In Human Vision and Electronic Imaging, B. Rogowitz and T. Pappas, Eds. Vol. 3299. 318--329.Google Scholar
- Geisler, W. and Perry, J. 1998. A real-time foveated multi-resolution system for low-bandwidth video communication. In Human Vision and Electronic Imaging, B. Rogowitz and T. Pappas, Eds. Vol. 3299. 294--305.Google Scholar
- Geisler, W. and Perry, J. 1999. Variable-Resolution displays for visual communication and simulation. Soc. Inf. Display 30, 1, 420--423.Google Scholar
- Henderson, J. 2003. Human gaze control during real-world scene perception. Trends Cogn. Sci. 7, 11, 498--504.Google Scholar
Cross Ref
- Itti, L. 2004. Automatic foveation for video compression using a neurobiological model for visual attention. IEEE Trans. Image Process. 13, 10, 1304--1318. Google Scholar
Digital Library
- Johannesson, E. 2005. Master's thesis, Lund University.Google Scholar
- Kortum, P. and Geisler, W. 1996. Implementation of a foveated image coding system for image bandwidth reduction. In Human Vision and Electronic Imaging, B. Rogowitz and J. Allebach, Eds. Vol. 2657. SPIE, San Jose, CA, 350--360.Google Scholar
- Le Meur, O., Le Callet, P., and Barba, D. 2007. Predicting visual fixations on video based on low-level visual features. Vis. Res. 46, 2483--2498.Google Scholar
Cross Ref
- Nyström, M. and Holmqvist, K. 2007. Deriving and evaluating eye-tracking controlled volumes of interest for variable-resolution video compression. J. Electron. Imag. 16, 1, 013006.Google Scholar
- Nyström, M. and Holmqvist, K. 2008. Semantic override of low-level features in image viewing—Both initially and overall. J. Eye Movem. Res. 2, 2, 2:1--2:11.Google Scholar
- Nyström, M., Novak, M., and Holmqvist, K. 2004. A novel approach to image coding using off-line foveation controlled by multiple eye-tracking measurements. In Proceedings of the Picture Coding Symposium.Google Scholar
- Parkhurst, D., Law, K., and Niebur, E. 2002. Modeling the role of salience in the allocation of overt visual attention. Vis. Res. 42, 107--123.Google Scholar
- Pedersen, B. and Spivey, M. 2006. Offline tracking of eyes and more with a simple Webcam. In Proceedings of the 28th Annual Meeting of the Cognitive Science Society.Google Scholar
- Rajashekar, U., Cormack, L., and Bovik, A. 2004. Point of gaze analysis reveals visual search strategies. In Human Vision and Electronic Imaging. Vol. 5292. 296--306.Google Scholar
- Sheikh, H. R., Liu, S., Evans, B. L., and Bovik, A. C. 2001. Real-Time foveation techniques for h.263 video encoding in software. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 3. 1781--1784. Google Scholar
Digital Library
- Stelmach, L. B. and Tam, W. 1994. Processing image sequences based on eye movements. In Human Vision, Visual Processing and Digital Display. Vol. 2179. 90--98.Google Scholar
Cross Ref
- Tatler, B., Baddeley, R., and Gilchrist, I. 2005. Visual correlates of fixation selection: Effects of scale and time. Vis. Res. 45, 5, 643--659.Google Scholar
Cross Ref
- VQEG. 2003. Final report from the video quality experts group on the validation of objective models of video quality assessment, phase II. http://www.its.bldrdoc.gov/vqeg/projects/frtv_phaseII/downloads/VQEGII_Final_Report.pdf.Google Scholar
- Wang, Z. and Bovik, A. 2001. Embedded foveation image coding. IEEE Trans. Image Process. 10, 10, 1397--1410. Google Scholar
Digital Library
- Wang, Z., Lu, L., and Bovik, A. 2003. Foveation scalable video coding with automatic fixation selection. IEEE Trans. Image Process. 12, 2, 243--254. Google Scholar
Digital Library
- Yarbus, A. 1967. Eye Movements and Vision. Plenum Press, New York.Google Scholar
Index Terms
Effect of compressed offline foveated video on viewing behavior and subjective quality
Recommendations
Latency Requirements for Foveated Rendering in Virtual Reality
Special Issue SAP 2017Foveated rendering is a performance optimization based on the well-known degradation of peripheral visual acuity. It reduces computational costs by showing a high-quality image in the user’s central (foveal) vision and a lower quality image in the ...
High quality, low delay foveated visual communications over mobile channels
Foveated video streams are created by selectively varying the spatio-temporal resolution of video data according to the assumed or measured fixation patterns of the intended viewers. The significant potential of foveated video lies in the considerable ...
Foveated mean squared error--a novel video quality metric
Efficiency of a video coding process, as well as accuracy of an objective video quality evaluation can be significantly improved by introduction of the human visual system (HVS) characteristics. In this paper we analyze one of these characteristics; ...






Comments