Nonlinear dimensionality reduction for face recognition

ABSTRACT

Principal component analysis (PCA) has long been a dominating linear technique for dimensionality reduction. Many nonlinear methods and neural networks have been proposed to extend PCA for complex nonlinear data. They include kernel PCA, local linear embedding, isomap, self-organising map (SOM), and visualization induced SOM (ViSOM), a variant of SOM for a faithful and metric-preserving mapping. In this paper, we investigate these nonlinear manifold methods for face recognition, and compare their performances with linear PCA. Results from the experiments on real-world face databases show that although nonlinear methods have greater capability than PCA, the differences in classification rate of most nonlinear methods and PCA are insignificant. However, ViSOM has produced marked improvement over PCA and other nonlinear methods. A nonlinearity measure is used to quantify the degree of nonlinearity of a data set in the reduced subspace. It can be used to indicate the effectiveness of nonlinear or linear dimensionality reduction.

References

1. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, Chichester (2001).
2. Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive Neuroscience 3, 71-86 (1991).
3. Scholköpf, B., Smola, A., Müller, K.-R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299-1319 (1998).
4. Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290, 2323-2326 (2000).
5. Tenenbaum, J.B., de Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290, 2319-2323 (2000).
6. Demartines, P., Hérault, J.: Curvilinear component analysis: a self-organizing neural network for nonlinear mapping of data sets. IEEE Trans. on Neural Networks 8, 148-154 (1997).
7. Yin, H.: Nonlinear dimensionality reduction and data visualization: A review. Int. J. Automation and Computing 3, 294-303 (2007).
8. Pang, Y.H., Teoh, A.B.J., Wong, E.K., Abas, F.S.: Supervised locally linear embedding in face recognition. In: Int. Symp. on Biometrics and Security Technologies, pp. 1-6 (2008).
9. Yang, M.H., Ahuja, N., Kriegman, D.: Face recognition using kernel eigenfaces. In: IEEE Int. Conf. on Image Processing, vol. 1, pp. 37-40 (2000).
10. Yang, M.H.: Extended Isomap for pattern classification. In: Proc. National Conference on Artificial Intelligence, pp. 224-229 (2002).
11. Kohonen, T.: Self-Organizing Maps, 2nd edn. Springer, Heidelberg (1997).
12. Lawrence, S., Giles, C.L., Tsoi, A., Back, A.: Face recognition: Aconvolutional neural-network approach. IEEE Trans. Neural Networks 8, 98-113 (1997).
13. Tan, X., Chen, S., Zhou, Z., Zhang, F.: Recognizing partially occluded, expression variant faces from single training image per person with SOM and soft k-NN ensemble. IEEE Trans. on Neural Networks 16, 875-886 (2005).
14. Yin, H.: ViSOM - A novel method for multivariate data projection and structure visualization. IEEE Trans. on Neural Networks 13, 237-243 (2002).
15. Yin, H.: On multidimensional scaling and the embedding of self-organising maps. Neural Networks 21, 160-169 (2008).
16. Kim, M., Kim, D., Lee, S.: Face recognition using the embedded HMM with second-order block specific observations. Pattern Recognition 36, 2723-2735 (2003).
17. Yang, J., Zhang, D., Frangi, A.F., Yang, J.: Two-dimensional PCA: a new approach to appearance-based face representation and recogntion. IEEE Trans. on Pattern Analysis and Machine Intelligence 26, 1-7 (2004).
18. Von der Malsburg, C., Willshaw, D.J.: Self-organization of orientation sensitive cells in the striate cortex. Biological Cybernetic 14, 85-100 (1973).
19. Burges, C.J.C.: A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2, 121-167 (1998).
20. Ji, S., Ye, J.: Generalized linear discriminant analysis: a unified framework and efficient model selection. IEEE Trans. on Neural Networks 19, 1768-1782 (2008).
21. Huang, W., Yin, H.: Linear and nonlinear dimensionality reduction for face recognition. In: IEEE Int. Conf. on Image Processing (to apear, 2009).

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Object recognition

  • Machine learning

    • Learning paradigms

      • Supervised learning

        • Supervised learning by classification

    • Machine learning algorithms

      • Feature selection