Efficient Implementation of a Dimensionality Reduction Method Using a Complex Moment-Based Subspace

ABSTRACT

Dimensionality reduction methods are widely used for processing data efficiently. Recently Imakura et al. proposed a novel dimensionality reduction method using a complex moment-based subspace. Their method can use more eigenvectors than the existing matrix trace optimization-based methods which explains its reported higher precision. However, the computational complexity is also higher than that of the existing methods, in particular for the nonlinear kernel version. To reduce the computational complexity, we propose a practical parallel implementation of the method by introducing the Nyström approximation. We evaluate the parallel performance of our implementation using the Oakforest-PACS supercomputer.

References

1. Lars Eldén and Haesun Park. 1999. A Procrustes problem on the Stiefel manifold. Numerische Mathematik 82, 4 (1999), 599–619. https://doi.org/10.1007/s002110050432
2. Ronald A. Fisher. 1936. The use of multiple measurements in taxonomic problems. Annals of Eugenics 7, 2 (1936), 179–188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x
3. Keinosuke Fukunaga. 1990. Introduction to Statistical Pattern Recognition (second ed.). Academic Press Professional, Inc., San Diego, CA, USA.
4. Yasunori Futamura. 2014. z-Pares: Parallel Eigenvalue Solver. https://zpares.cs.tsukuba.ac.jp/
5. Xiaofei He and Partha Niyogi. 2004. Locality preserving projections. In Advances in Neural Information Processing Systems, Vol. 16. MIT Press, Cambridge, MA, USA, 153–160.
6. Akira Imakura, Momo Matsuda, Xiucai Ye, and Tetsuya Sakurai. 2019. Complex moment-based supervised eigenmap for dimensionality reduction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. AAAI Press, Palo Alto, CA, USA, 3910–3918. https://doi.org/10.1609/aaai.v33i01.33013910
7. Shigeru Iwase, Yasunori Futamura, Akira Imakura, Tetsuya Sakurai, and Tomoya Ono. 2017. Efficient and scalable calculation of complex band structure using Sakurai-Sugiura method. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, Colorado) (SC ’17). Association for Computing Machinery, New York, NY, USA, Article 40, 12 pages. https://doi.org/10.1145/3126908.3126942
8. Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11 (1998), 2278–2324. https://doi.org/10.1109/5.726791
9. Xuelong Li, Mulin Chen, Feiping Nie, and Qi Wang. 2017. Locality adaptive discriminant analysis. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (Melbourne, Australia) (IJCAI’17). AAAI Press, Palo Alto, CA, USA, 2201–2207. https://doi.org/10.24963/ijcai.2017/306
10. Haesun Park. 1991. A parallel algorithm for the unbalanced orthogonal procrustes problem. Parallel Computing 17, 8 (1991), 913–923. https://doi.org/10.1016/S0167-8191(05)80075-4
11. Karl Pearson. 1901. LIII. On lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science 2, 11 (1901), 559–572. https://doi.org/10.1080/14786440109462720
12. Tetsuya Sakurai and Hiroshi Sugiura. 2003. A projection method for generalized eigenvalue problems using numerical integration. Journal of Computational and Applied Mathematics 159, 1(2003), 119–128. https://doi.org/10.1016/S0377-0427(03)00565-X
13. Bernhard Schölkopf, Alexander Smola, and Klaus-Robert Müller. 1998. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 5 (1998), 1299–1319. https://doi.org/10.1162/089976698300017467
14. Masashi Sugiyama. 2007. Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis. Journal of Machine Learning Research 8, 37 (2007), 1027–1061.
15. Christopher K. I. Williams and Matthias Seeger. 2001. Using the Nyström Method to Speed Up Kernel Machines. In Advances in Neural Information Processing Systems, Vol. 13. MIT Press, Cambridge, MA, USA, 682–688.
16. Haifeng Zhao, Zheng Wang, and Feiping Nie. 2016. Orthogonal least squares regression for feature extraction. Neurocomputing 216(2016), 200–207. https://doi.org/10.1016/j.neucom.2016.07.037

Index Terms

• Computing methodologies

  • Symbolic and algebraic manipulation

    • Symbolic and algebraic algorithms

      • Linear algebra algorithms

• Mathematics of computing

  • Mathematical analysis

    • Numerical analysis

      • Computations on matrices