skip to main content
research-article

Using geometric properties of topographic manifold to detect and track eyes for human-computer interaction

Authors Info & Claims
Published:12 December 2007Publication History
Skip Abstract Section

Abstract

Automatic eye detection and tracking is an important component for advanced human-computer interface design. Accurate eye localization can help develop a successful system for face recognition and emotion identification. In this article, we propose a novel approach to detect and track eyes using geometric surface features on topographic manifold of eye images. First, in the joint spatial-intensity domain, a facial image is treated as a 3D terrain surface or image topographic manifold. In particular, eye regions exhibit certain intrinsic geometric traits on this topographic manifold, namely, the pit-labeled center and hillside-like surround regions. Applying a terrain classification procedure on the topographic manifold of facial images, each location of the manifold can be labeled to generate a terrain map. We use the distribution of terrain labels to represent the eye terrain pattern. The Bhattacharyya affinity is employed to measure the distribution similarity between two topographic manifolds. Based on the Bhattacharyya kernel, a support vector machine is applied for selecting proper eye pairs from the pit-labeled candidates. Second, given detected eyes on the first frame of a video sequence, a mutual-information-based fitting function is defined to describe the similarity between two terrain surfaces of neighboring frames. By optimizing the fitting function, eye locations are updated for subsequent frames. The distinction of the proposed approach lies in that both eye detection and eye tracking are performed on the derived topographic manifold, rather than on an original-intensity image domain. The robustness of the approach is demonstrated under various imaging conditions and with different facial appearances, using both static images and video sequences without background constraints.

References

  1. Besl, P. and Jain, R. 1988. Segmentation through variable-order surface fitting. IEEE Trans. Pattern Anal. Mach. Intell. 10, 2, 167--192. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Bhattacharyya, A. 1943. On a measure of divergence between two statistical populations defined by their probability distributions. Bull. Calcutta Math. Soc. 35, 99--109.Google ScholarGoogle Scholar
  3. Bradski, G. 1998. Real time face and object tracking as a coponent of a perceptual user interface. In Proceedings of the IEEE Workshop on Applications of Computer Vision (WACV). Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Burges, C. 1998. A tutorial on support vector machines for pattern recognition. Data Mining Knowl. Discov. 2, 2, 121--167. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Comaniciu, D. and Meer, P. 1999. Mean shift analysis and applications. In Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV). Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Comaniciu, D., Ramesh, V., and Meer, P. Real-Time tracking of non-rigid objects using mean shift. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarGoogle ScholarCross RefCross Ref
  7. Cross, A. and Hancock, E. 1999. Scale space vector fields for symmetry detection. Image Vision Comput. 17, 337--345.Google ScholarGoogle ScholarCross RefCross Ref
  8. Djeraba, C. 2005. State of art of bye tracking, Tech. Rep., www.lifl.fr/evenements/publications/2005-07.pdf. LIFL, UMR CNRS-USTL.Google ScholarGoogle Scholar
  9. Djeraba, C., Stanislas, L., Simovici, D., Mongy, S., and Ihaddadene, N. 2006. Eye/Gaze tracking in web, image and video documents. In Proceedings of the 14th Annual International Conference on ACM Multimedia, Santa Barbara, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Gulliver, S. and Ghinea, G. 2004. Stars in their eyes: What eye-tracking reveals about multimedia perceptual quality. IEEE Trans. Syst. Man Cybernet. A 34, 4, 472--482. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Gulliver, S. and Ghinea, G. 2005. Commercial uses of eyetracking. In Proceedings of the Conference on Human-Computer Interaction (HCI), Edinburgh, UK.Google ScholarGoogle Scholar
  12. Haralick, R., Watson, L., and Laffey, T. 1983. The topographic primal sketch. Int. J. Robotics Res. 2, 2, 50--72.Google ScholarGoogle ScholarCross RefCross Ref
  13. Haro, A., Flickner, M., and Essa, I. 2000. Detecting and tracking eyes by using their physiological properties dynamics and appearcance. In Proceedings of the IEEE International Conference on Computer Vision (CVPR).Google ScholarGoogle Scholar
  14. Huang, J. and Wechsleris, H. 2000. Visual routines for eye location using learing and evolution. IEEE Trans. Evol. Comput. 4, 1, 73--82. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jebara, T., Kondor, R., and Howard, A. 2004. Probability product kernels. J. Mach. Learn. Res. 5, 819--844. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jesorsky, O., Kirchberg, K., and Frischholz, R. 2001. Robust face detection using the Hausdorff distance. In Proceedings of the 3rd International Conference on Audio- and Video-Based Biomatric Person Authentication (AVBPA), 90--95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Ji, Q., Wechsler, H., Duchowski, A., and Flickner, M. 2005. Eye detection and tracking. Comput. Vision Image Understand. 98, 1, 1--3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Kartikeyan, B. and Sarkar, A. 1995. The assignment of topographic labels by a statistical model-based approach. Signal Process. 42, 1, 71--86. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Kondor, R. and Jebara, T. 2003. A kernel between sets of vectors. Proceedings of the 10th International Conference on Machine Learning (ICML).Google ScholarGoogle Scholar
  20. Lindeberg, T. 1993. Detecting salient blob-like image structures and their scales with a scale-space primal sketch: A method for focus-of-attention. Int. J. Comput. Vision 11, 3, 1573--1405. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Lyons, M., Budynek, J., and Akamatsu, S. 1999. Automatic classification of single facial images. IEEE Trans. Pattern Anal. Mach. Intell. 21, 12, 1357--1362. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Magee, J., Scott, M., Waber, B., and Betke, M. 2004. Eyekeys: A real-time vision interface based on gaze detection from a low-grade video camera. In Proceedings of the IEEE CVPR Workshop on Real-Time Vision for Human Computer Interaction. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Meer, P. and Weiss, I. 1992. Smoothed differentiation filters for images. J. Visual Commun. Image Rep. 3, 1.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Meth, R. and Chellappa, R. 1996. Automatic classification of targets in synthetic aperture radar imagery using topographic features. In Proceedings of SPIE, citeseer.ist.psu.edu/9394.html. 186--193.Google ScholarGoogle Scholar
  25. Morimoto, C., Koons, D., Amir, A., and Flickner, M. 1998. Real-time detection of eyes and faces. In Workshop on Perceptural User Interfaces.Google ScholarGoogle Scholar
  26. Morimoto, C. and Mimica, M. 2005. Eye gaze tracking techniques for interactive applications. Comput. Vision Image Understand. 98, 1, 4--24. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Nielsen Norman Group. 2006. Eyetracking research. http://www.useit.com/eyetracking/.Google ScholarGoogle Scholar
  28. Pentland, A., Moghaddam, B., and Starner, T. 1994. View-Based and modular Eigenspaces for face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarGoogle Scholar
  29. Phillips, P., Moon, H., Rizvi, S., and Rauss, P. 2000. The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22, 10, 1090--1104. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Pong, T., Shapiro, L., and Haralick, R. 1985. Shape estimation from topographic primal sketch. Pattern Recogn. 18, 5, 333--347.Google ScholarGoogle ScholarCross RefCross Ref
  31. Ruddarraju, R., Haro, A., Nagel, K., Tran, Q. T., Essa, I. A., Abowd, G., and Mynatt, E. D. 2002. Perceptual user interfaces using vision-based eye tracking. In 5th International Conference on Multimodel Interfaces. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Scheiderman, H. 2004. Learning a restricted Bayesian network for object detection. In the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Sukanya, P., Takamatsu, R., and Sato, M. 1998. Image classification using the surface-shape operator and multiscale features. In Proceedings of the IAPR International Conference on Pattern Recognition, 334--337. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Trier, O., Taxt, T., and Jain, A. 1995. Data capture from maps based on gray scale topographic analysis. In Proceedings of the International Conference on Document Analysis and Recognition (ICDAR). Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Wang, J. and Yin, L. 2005. Detecting and tracking eyes through dynamic terrain feature matching. In the IEEE CVPR Workshop on Vision For Human Computer Interaction, San Diago, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Wang, J. and Yin, L. 2007. Static topographic modeling for facial expression recognition and analysis. Comput. Vision Image Understand. to appear.Google ScholarGoogle Scholar
  37. Wang, J., Yin, L., Wei, X., and Sun, Y. 2006. 3D facial expression recognition based on primitive surface feature distribution. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Vol. 2, 1399--1406. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Wang, L. and Pavlidis, T. 1993. Direct gray-scale extraction of features for character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 15, 10, 1053--1067. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Yin, L. and Basu, A. 2001. Generating realistic facial expressions with wrinkles for model based coding. Comput. Vision Image Understand. 84, 2, 201--240. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Yuille, A., Hallinan, P., and Cohen, D. 1992. Feature extraction from faces using deformable templates. Int. J. Comput. Vision. 8, 2. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Zhou, Z. and Geng, X. 2004. Projection functions for eye detection. Pattern Recogn. 37, 5.Google ScholarGoogle ScholarCross RefCross Ref
  42. Zhu, Z., Fujimura, K., and Ji, Q. 2002. Real-Time eye detection and tracking under various light conditions. In ACM SIGCHI Symposium on Eye Tracking Research and Application. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Using geometric properties of topographic manifold to detect and track eyes for human-computer interaction

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!