research-article

Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization

Author:

Timothy A. DavisAuthors Info & Claims

ACM Transactions on Mathematical Software (TOMS), Volume 38, Issue 1

Article No.: 8, Pages 1 - 22

https://doi.org/10.1145/2049662.2049670

Published: 07 December 2011 Publication History

Abstract

SuiteSparseQR is a sparse QR factorization package based on the multifrontal method. Within each frontal matrix, LAPACK and the multithreaded BLAS enable the method to obtain high performance on multicore architectures. Parallelism across different frontal matrices is handled with Intel's Threading Building Blocks library. The symbolic analysis and ordering phase pre-eliminates singletons by permuting the input matrix A into the form [R₁₁ R₁₂; 0 A₂₂] where R₁₁ is upper triangular with diagonal entries above a given tolerance. Next, the fill-reducing ordering, column elimination tree, and frontal matrix structures are found without requiring the formation of the pattern of A^TA. Approximate rank-detection is performed within each frontal matrix using Heath's method. While Heath's method is not always exact, it has the advantage of not requiring column pivoting and thus does not interfere with the fill-reducing ordering. For sufficiently large problems, the resulting sparse QR factorization obtains a substantial fraction of the theoretical peak performance of a multicore computer.

Supplementary Material

ZIP File (915.zip)

Software for SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization

Download
3.07 MB

References

[1]

Amestoy, P. R., Davis, T. A., and Duff, I. S. 1996a. An approximate minimum degree ordering algorithm. SIAM J. Matrix Anal. Appl. 17, 4, 886--905.

Digital Library

[2]

Amestoy, P. R., Davis, T. A., and Duff, I. S. 2004. Algorithm 837: AMD, an approximate minimum degree ordering algorithm. ACM Trans. Math. Softw. 30, 3, 381--388.

Digital Library

[3]

Amestoy, P. R. and Duff, I. S. 1989. Vectorization of a multiprocessor multifrontal code. Intl. J. Supercomp. Appl. 3, 3, 41--59.

Digital Library

[4]

Amestoy, P. R., Duff, I. S., and Puglisi, C. 1996b. Multifrontal QR factorization in a multiprocessor environment. Numer. Linear Algebra Appl. 3, 4, 275--300.

[5]

Amestoy, P. R. and Puglisi, C. 2002. An unsymmetrized multifrontal LU factorization. SIAM J. Matrix Anal. Appl. 24, 553--569.

Digital Library

[6]

Anderson, E., Bai, Z., Bischof, C. H., Blackford, S., Demmel, J. W., Dongarra, J. J., Du Croz, J., Greenbaum, A., Hammarling, S., McKenney, A., and Sorensen, D. C. 1999. LAPACK Users' Guide 3rd Ed. SIAM, Philadelphia, PA.

Digital Library

[7]

Ashcraft, C. C. and Grimes, R. 1989. The influence of relaxed supernode partitions on the multifrontal method. ACM Trans. Math. Softw. 15, 4, 291--309.

Digital Library

[8]

Bischof, C. H., Lewis, J. G., and Pierce, D. J. 1990. Incremental condition estimation for sparse matrices. SIAM J. Matrix Anal. Appl. 11, 4, 644--659.

Digital Library

[9]

Bischof, C. H. and Van Loan, C. F. 1987. The WY representation for products of Householder vectors. SIAM J. Sci. Statist. Comput. 8, 1, s2--s13.

Digital Library

[10]

Björck, A. 1996. Numerical Methods for Least Squares Problems. SIAM, Philadelphia, PA.

[11]

Blackford, L. S., Choi, J., Cleary, A., D'Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammarling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., and Whaley, R. C. 1997. ScaLAPACK User's Guide. SIAM, Philadelphia, PA.

Digital Library

[12]

Chapman, B., Jost, G., and van der Pas, R. 2007. Using OpenMP: Portable Shared Memory Parallel Programming. MIT Press, Cambridge, MA.

Digital Library

[13]

Chen, Y., Davis, T. A., Hager, W. W., and Rajamanickam, S. 2008. Algorithm 887: CHOLMOD, supernodal sparse Cholesky factorization and update/downdate. ACM Trans. Math. Softw. 35, 3, 1--14.

Digital Library

[14]

Coleman, T. F., Edenbrandt, A., and Gilbert, J. R. 1986. Predicting fill for sparse orthogonal factorization. J. ACM 33, 517--532.

Digital Library

[15]

Davis, T. A. 2004. A column pre-ordering strategy for the unsymmetric-pattern multifrontal method. ACM Trans. Math. Softw. 30, 2, 165--195.

Digital Library

[16]

Davis, T. A. 2006. Direct Methods for Sparse Linear Systems. SIAM, Philadelphia, PA.

Digital Library

[17]

Davis, T. A. and Duff, I. S. 1997. An unsymmetric-pattern multifrontal method for sparse LU factorization. SIAM J. Matrix Anal. Appl. 18, 1, 140--158.

Digital Library

[18]

Davis, T. A., Gilbert, J. R., Larimore, S. I., and Ng, E. G. 2004a. Algorithm 836: COLAMD, a column approximate minimum degree ordering algorithm. ACM Trans. Math. Softw. 30, 3, 377--380.

Digital Library

[19]

Davis, T. A., Gilbert, J. R., Larimore, S. I., and Ng, E. G. 2004b. A column approximate minimum degree ordering algorithm. ACM Trans. Math. Softw. 30, 3, 353--376.

Digital Library

[20]

Davis, T. A. and Hager, W. W. 1999. Modifying a sparse Cholesky factorization. SIAM J. Matrix Anal. Appl. 20, 3, 606--627.

Digital Library

[21]

Davis, T. A. and Hager, W. W. 2001. Multiple-rank modifications of a sparse Cholesky factorization. SIAM J. Matrix Anal. Appl. 22, 997--1013.

Digital Library

[22]

Davis, T. A. and Hager, W. W. 2009. Dynamic supernodes in sparse cholesky update/downdate and triangular solves. ACM Trans. Math. Softw. 35, 4, 1--23.

Digital Library

[23]

Davis, T. A. and Hu, Y. 2011. The University of Florida sparse matrix collection. ACM Trans. Math. Softw. 28, 1.

Digital Library

[24]

Dongarra, J. J., Du Croz, J. J., Duff, I. S., and Hammarling, S. 1990. A set of level 3 basic linear algebra subprograms. ACM Trans. Math. Softw. 16, 1--17.

Digital Library

[25]

Duff, I. S. 1977. On permutations to block triangular form. J. Inst. Math. Appl. 19, 339--342.

[26]

Duff, I. S. 1981. On algorithms for obtaining a maximum transversal. ACM Trans. Math. Softw. 7, 1, 315--330.

Digital Library

[27]

Duff, I. S. 2004. MA57—A code for the solution of sparse symmetric definite and indefinite systems. ACM Trans. Math. Softw. 30, 2, 118--144.

Digital Library

[28]

Duff, I. S. and Reid, J. K. 1983. The multifrontal solution of indefinite sparse symmetric linear systems. ACM Trans. Math. Softw. 9, 302--325.

Digital Library

[29]

Duff, I. S. and Reid, J. K. 1984. The multifrontal solution of unsymmetric sets of linear equations. SIAM J. Sci. Statist. Comput. 5, 3, 633--641.

Digital Library

[30]

Edlund, O. 2002. A software package for sparse orthogonal factorization and updating. ACM Trans. Math. Softw. 28, 4, 448--482.

Digital Library

[31]

Foster, L. V. 2009. Calculating ranks, null spaces, and pseudoinverse solutions for sparse matrices using SuiteSparseQR. http://www.math.sjsu.edu/~foster/spqr_rank_LA09.pdf.

[32]

Geist, G. A. and Ng, E. G. 1989. Task scheduling for parallel sparse Cholesky factorization. Intl. J. Parall. Program. 18, 4, 291--314.

Digital Library

[33]

George, A. 1973. Nested dissection of a regular finite element mesh. SIAM J. Numer. Anal. 10, 2, 345--363.

Digital Library

[34]

George, A. and Heath, M. T. 1980. Solution of sparse linear least squares problems using Givens rotations. Linear Algebra Appl. 34, 69--83.

[35]

George, A. and Liu, J. W. H. 1981. Computer Solution of Large Sparse Positive Definite Systems. Prentice-Hall, Englewood Cliffs, NJ.

Digital Library

[36]

George, A., Liu, J. W. H., and Ng, E. G. 1988. A data structure for sparse QR and LU factorizations. SIAM J. Sci. Statist. Comput. 9, 1, 100--121.

Digital Library

[37]

George, A. and Ng, E. G. 1987. Symbolic factorization for sparse Gaussian elimination with partial pivoting. SIAM J. Sci. Statist. Comput. 8, 6, 877--898.

Digital Library

[38]

Gilbert, J. R., Li, X. S., Ng, E. G., and Peyton, B. W. 2001. Computing row and column counts for sparse QR and LU factorization. BIT 41, 4, 693--710.

Digital Library

[39]

Gilbert, J. R., Moler, C., and Schreiber, R. 1992. Sparse matrices in MATLAB: Design and implementation. SIAM J. Matrix Anal. Appl. 13, 1, 333--356.

Digital Library

[40]

Givens, W. 1958. Computation of plane unitary rotations transforming a general matrix to triangular form. SIAM J. Appl. Math. 6, 26--50.

[41]

Golub, G. H. 1965. Numerical methods for solving linear least squares problems. Numer. Math. 7, 206--216.

Digital Library

[42]

Goto, K. and van de Geijn, R. 2008. High performance implementation of the level-3 BLAS. ACM Trans. Math. Softw. 35, 1, 4. Article 4, 14 pages.

Digital Library

[43]

Grigori, L., Cosnard, M., and Ng, E. G. 2007. On the row merge tree for sparse LU factorization with partial pivoting. BIT 47, 1, 45--76.

[44]

Heath, M. T. 1982. Some extensions of an algorithm for sparse linear least squares problems. SIAM J. Sci. Statist. Comput. 3, 2, 223--237.

Digital Library

[45]

Heath, M. T. and Sorensen, D. C. 1986. A pipelined Givens method for computing the QR factorization of a sparse matrix. Linear Algebra Appl. 77, 189--203.

[46]

Householder, A. S. 1958. Unitary triangularization of a nonsymmetric matrix. J. ACM 5, 339--342.

Digital Library

[47]

Karypis, G. and Kumar, V. 1998. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20, 359--392.

Digital Library

[48]

Lipton, R. J., Rose, D. J., and Tarjan, R. E. 1979. Generalized nested dissection. SIAM J. Numer. Anal. 16, 346--358.

Digital Library

[49]

Liu, J. W. H. 1986. On general row merging schemes for sparse Givens transformations. SIAM J. Sci. Statist. Comput. 7, 4, 1190--1211.

Digital Library

[50]

Liu, J. W. H. 1989. The multifrontal method and paging in sparse Cholesky factorization. ACM Trans. Math. Softw. 15, 4, 310--325.

Digital Library

[51]

Lu, S. M. and Barlow, J. L. 1996. Multifrontal computation with the orthogonal factors of sparse matrices. SIAM J. Matrix Anal. Appl. 17, 3, 658--679.

Digital Library

[52]

Matstoms, P. 1994. Sparse QR factorization in MATLAB. ACM Trans. Math. Softw. 20, 1, 136--159.

Digital Library

[53]

Matstoms, P. 1995. Parallel sparse QR factorization on shared memory architectures. Parallel Comput. 21, 3, 473--486.

Digital Library

[54]

Oliveira, S. 2001. Exact prediction of QR fill-in by row-merge trees. SIAM J. Sci. Comput. 22, 6, 1962--1973.

Digital Library

[55]

Pierce, D. J. and Lewis, J. G. 1997. Sparse multifrontal rank revealing QR factorization. SIAM J. Matrix Anal. Appl. 18, 1, 159--180.

Digital Library

[56]

Pothen, A. and Fan, C. 1990. Computing the block triangular form of a sparse matrix. ACM Trans. Math. Softw. 16, 4, 303--324.

Digital Library

[57]

Reinders, J. 2007. Intel Threading Building Blocks. Outfitting C++ for Multi-core Processor Parallelism. O'Reilly Media, Sebastopol, CA.

Digital Library

[58]

Robison, A. 2008. Architect of Intel's threading building blocks. personal communication.

[59]

Schreiber, R. and Van Loan, C. F. 1989. A storage-efficient WY representation for products of Householder matrices. SIAM J. Sci. Statist. Comput. 10, 1, 53--57.

Digital Library

[60]

Stewart, G. W. 1998. Matrix Algorithms, Volume 1: Basic Decompositions. SIAM, Philadelphia, PA.

[61]

Sun, C. 1996. Parallel sparse orthogonal factorization on distributed-memory multiprocessors. SIAM J. Sci. Comput. 17, 3, 666--685.

Digital Library

[62]

van de Geijn, R. A. 1997. Using PLAPACK. Scientific and Engineering Computation Series. MIT Press, Cambridge, MA.

Cited By

Zhang XChenevert GSu YTariq FD’sa JBae SLee D(2025)ZSG-MPPI: Robust Model-Predictive Path-Integral Method for Disturbance HandlingIEEE Robotics and Automation Letters10.1109/LRA.2025.362299910:12(12437-12444)Online publication date: Dec-2025
https://doi.org/10.1109/LRA.2025.3622999
Marehalli Srinivas SEsposito MFreitas N(2025)Hypergraph-based models of random chemical reaction networks: Conservation laws, connectivity, and percolationThe Journal of Chemical Physics10.1063/5.0292914163:19Online publication date: 20-Nov-2025
https://doi.org/10.1063/5.0292914
Mejia-Domenzain LLourenco CMoreno-Centeno E(2025)Sparse integer-preserving Gram–Schmidt orthogonalization for REF QR factorizationOptimization Letters10.1007/s11590-025-02228-yOnline publication date: 14-Nov-2025
https://doi.org/10.1007/s11590-025-02228-y
Show More Cited By

Index Terms

Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization
1. Mathematics of computing
  1. Mathematical software

Recommendations

Algorithm 887: CHOLMOD, Supernodal Sparse Cholesky Factorization and Update/Downdate

CHOLMOD is a set of routines for factorizing sparse symmetric positive definite matrices of the form A or AA^T, updating/downdating a sparse Cholesky factorization, solving linear systems, updating/downdating the solution to the triangular system Lx = b, ...
Algorithm 980: Sparse QR Factorization on the GPU

Sparse matrix factorization involves a mix of regular and irregular computation, which is a particular challenge when trying to obtain high-performance on the highly parallel general-purpose computing cores available on graphics processing units (GPUs). ...
Algorithm 930: FACTORIZE: An object-oriented linear system solver for MATLAB

The MATLAB™ backslash (x=A\b) is an elegant and powerful interface to a suite of high-performance factorization methods for the direct solution of the linear system Ax = b and the least-squares problem min_x ‖b - Ax‖. It is a meta-algorithm that selects ...

Comments

View Issue’s Table of Contents