Abstract
Automatic eye detection and tracking is an important component for advanced human-computer interface design. Accurate eye localization can help develop a successful system for face recognition and emotion identification. In this article, we propose a novel approach to detect and track eyes using geometric surface features on topographic manifold of eye images. First, in the joint spatial-intensity domain, a facial image is treated as a 3D terrain surface or image topographic manifold. In particular, eye regions exhibit certain intrinsic geometric traits on this topographic manifold, namely, the pit-labeled center and hillside-like surround regions. Applying a terrain classification procedure on the topographic manifold of facial images, each location of the manifold can be labeled to generate a terrain map. We use the distribution of terrain labels to represent the eye terrain pattern. The Bhattacharyya affinity is employed to measure the distribution similarity between two topographic manifolds. Based on the Bhattacharyya kernel, a support vector machine is applied for selecting proper eye pairs from the pit-labeled candidates. Second, given detected eyes on the first frame of a video sequence, a mutual-information-based fitting function is defined to describe the similarity between two terrain surfaces of neighboring frames. By optimizing the fitting function, eye locations are updated for subsequent frames. The distinction of the proposed approach lies in that both eye detection and eye tracking are performed on the derived topographic manifold, rather than on an original-intensity image domain. The robustness of the approach is demonstrated under various imaging conditions and with different facial appearances, using both static images and video sequences without background constraints.
- Besl, P. and Jain, R. 1988. Segmentation through variable-order surface fitting. IEEE Trans. Pattern Anal. Mach. Intell. 10, 2, 167--192. Google Scholar
Digital Library
- Bhattacharyya, A. 1943. On a measure of divergence between two statistical populations defined by their probability distributions. Bull. Calcutta Math. Soc. 35, 99--109.Google Scholar
- Bradski, G. 1998. Real time face and object tracking as a coponent of a perceptual user interface. In Proceedings of the IEEE Workshop on Applications of Computer Vision (WACV). Google Scholar
Digital Library
- Burges, C. 1998. A tutorial on support vector machines for pattern recognition. Data Mining Knowl. Discov. 2, 2, 121--167. Google Scholar
Digital Library
- Comaniciu, D. and Meer, P. 1999. Mean shift analysis and applications. In Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV). Google Scholar
Digital Library
- Comaniciu, D., Ramesh, V., and Meer, P. Real-Time tracking of non-rigid objects using mean shift. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
Cross Ref
- Cross, A. and Hancock, E. 1999. Scale space vector fields for symmetry detection. Image Vision Comput. 17, 337--345.Google Scholar
Cross Ref
- Djeraba, C. 2005. State of art of bye tracking, Tech. Rep., www.lifl.fr/evenements/publications/2005-07.pdf. LIFL, UMR CNRS-USTL.Google Scholar
- Djeraba, C., Stanislas, L., Simovici, D., Mongy, S., and Ihaddadene, N. 2006. Eye/Gaze tracking in web, image and video documents. In Proceedings of the 14th Annual International Conference on ACM Multimedia, Santa Barbara, CA. Google Scholar
Digital Library
- Gulliver, S. and Ghinea, G. 2004. Stars in their eyes: What eye-tracking reveals about multimedia perceptual quality. IEEE Trans. Syst. Man Cybernet. A 34, 4, 472--482. Google Scholar
Digital Library
- Gulliver, S. and Ghinea, G. 2005. Commercial uses of eyetracking. In Proceedings of the Conference on Human-Computer Interaction (HCI), Edinburgh, UK.Google Scholar
- Haralick, R., Watson, L., and Laffey, T. 1983. The topographic primal sketch. Int. J. Robotics Res. 2, 2, 50--72.Google Scholar
Cross Ref
- Haro, A., Flickner, M., and Essa, I. 2000. Detecting and tracking eyes by using their physiological properties dynamics and appearcance. In Proceedings of the IEEE International Conference on Computer Vision (CVPR).Google Scholar
- Huang, J. and Wechsleris, H. 2000. Visual routines for eye location using learing and evolution. IEEE Trans. Evol. Comput. 4, 1, 73--82. Google Scholar
Digital Library
- Jebara, T., Kondor, R., and Howard, A. 2004. Probability product kernels. J. Mach. Learn. Res. 5, 819--844. Google Scholar
Digital Library
- Jesorsky, O., Kirchberg, K., and Frischholz, R. 2001. Robust face detection using the Hausdorff distance. In Proceedings of the 3rd International Conference on Audio- and Video-Based Biomatric Person Authentication (AVBPA), 90--95. Google Scholar
Digital Library
- Ji, Q., Wechsler, H., Duchowski, A., and Flickner, M. 2005. Eye detection and tracking. Comput. Vision Image Understand. 98, 1, 1--3. Google Scholar
Digital Library
- Kartikeyan, B. and Sarkar, A. 1995. The assignment of topographic labels by a statistical model-based approach. Signal Process. 42, 1, 71--86. Google Scholar
Digital Library
- Kondor, R. and Jebara, T. 2003. A kernel between sets of vectors. Proceedings of the 10th International Conference on Machine Learning (ICML).Google Scholar
- Lindeberg, T. 1993. Detecting salient blob-like image structures and their scales with a scale-space primal sketch: A method for focus-of-attention. Int. J. Comput. Vision 11, 3, 1573--1405. Google Scholar
Digital Library
- Lyons, M., Budynek, J., and Akamatsu, S. 1999. Automatic classification of single facial images. IEEE Trans. Pattern Anal. Mach. Intell. 21, 12, 1357--1362. Google Scholar
Digital Library
- Magee, J., Scott, M., Waber, B., and Betke, M. 2004. Eyekeys: A real-time vision interface based on gaze detection from a low-grade video camera. In Proceedings of the IEEE CVPR Workshop on Real-Time Vision for Human Computer Interaction. Google Scholar
Digital Library
- Meer, P. and Weiss, I. 1992. Smoothed differentiation filters for images. J. Visual Commun. Image Rep. 3, 1.Google Scholar
Digital Library
- Meth, R. and Chellappa, R. 1996. Automatic classification of targets in synthetic aperture radar imagery using topographic features. In Proceedings of SPIE, citeseer.ist.psu.edu/9394.html. 186--193.Google Scholar
- Morimoto, C., Koons, D., Amir, A., and Flickner, M. 1998. Real-time detection of eyes and faces. In Workshop on Perceptural User Interfaces.Google Scholar
- Morimoto, C. and Mimica, M. 2005. Eye gaze tracking techniques for interactive applications. Comput. Vision Image Understand. 98, 1, 4--24. Google Scholar
Digital Library
- Nielsen Norman Group. 2006. Eyetracking research. http://www.useit.com/eyetracking/.Google Scholar
- Pentland, A., Moghaddam, B., and Starner, T. 1994. View-Based and modular Eigenspaces for face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Phillips, P., Moon, H., Rizvi, S., and Rauss, P. 2000. The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22, 10, 1090--1104. Google Scholar
Digital Library
- Pong, T., Shapiro, L., and Haralick, R. 1985. Shape estimation from topographic primal sketch. Pattern Recogn. 18, 5, 333--347.Google Scholar
Cross Ref
- Ruddarraju, R., Haro, A., Nagel, K., Tran, Q. T., Essa, I. A., Abowd, G., and Mynatt, E. D. 2002. Perceptual user interfaces using vision-based eye tracking. In 5th International Conference on Multimodel Interfaces. Google Scholar
Digital Library
- Scheiderman, H. 2004. Learning a restricted Bayesian network for object detection. In the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Google Scholar
Digital Library
- Sukanya, P., Takamatsu, R., and Sato, M. 1998. Image classification using the surface-shape operator and multiscale features. In Proceedings of the IAPR International Conference on Pattern Recognition, 334--337. Google Scholar
Digital Library
- Trier, O., Taxt, T., and Jain, A. 1995. Data capture from maps based on gray scale topographic analysis. In Proceedings of the International Conference on Document Analysis and Recognition (ICDAR). Google Scholar
Digital Library
- Wang, J. and Yin, L. 2005. Detecting and tracking eyes through dynamic terrain feature matching. In the IEEE CVPR Workshop on Vision For Human Computer Interaction, San Diago, CA. Google Scholar
Digital Library
- Wang, J. and Yin, L. 2007. Static topographic modeling for facial expression recognition and analysis. Comput. Vision Image Understand. to appear.Google Scholar
- Wang, J., Yin, L., Wei, X., and Sun, Y. 2006. 3D facial expression recognition based on primitive surface feature distribution. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Vol. 2, 1399--1406. Google Scholar
Digital Library
- Wang, L. and Pavlidis, T. 1993. Direct gray-scale extraction of features for character recognition. IEEE Trans. Pattern Anal. Mach. Intell. 15, 10, 1053--1067. Google Scholar
Digital Library
- Yin, L. and Basu, A. 2001. Generating realistic facial expressions with wrinkles for model based coding. Comput. Vision Image Understand. 84, 2, 201--240. Google Scholar
Digital Library
- Yuille, A., Hallinan, P., and Cohen, D. 1992. Feature extraction from faces using deformable templates. Int. J. Comput. Vision. 8, 2. Google Scholar
Digital Library
- Zhou, Z. and Geng, X. 2004. Projection functions for eye detection. Pattern Recogn. 37, 5.Google Scholar
Cross Ref
- Zhu, Z., Fujimura, K., and Ji, Q. 2002. Real-Time eye detection and tracking under various light conditions. In ACM SIGCHI Symposium on Eye Tracking Research and Application. Google Scholar
Digital Library
Index Terms
Using geometric properties of topographic manifold to detect and track eyes for human-computer interaction
Recommendations
Real-time eye-gaze estimation using a low-resolution webcam
Eye detection and gaze estimation play an important role in many applications, e.g., the eye-controlled mouse in the assisting system for disabled or elderly persons, eye fixation and saccade in psychological analysis, or iris recognition in the ...
Eye Tracking in Human-Computer Interaction and Usability Research
INTERACT '09: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part IIThe objective of the tutorial is to give an overview on how eye tracking is currently used and how it can be used as a method in human computer interaction research and especially in usability research. An eye tracking system records how the eyes move ...
Reliable detection of eye features and eyes in color facial images using ternary eye-verifier
Eye detection plays an important role in applications related to face recognition. The position of eyes can be used as a reliable reference for other facial feature detection. This paper presents a novel approach for the precise and reliable detection ...






Comments