Abstract
We present a simple algorithm that approximates the product of n-by-n real matrices A and B. Let ‖AB‖F denote the Frobenius norm of AB, and b be a parameter determining the time/accuracy trade-off. Given 2-wise independent hash functions h1,h2: [n]→ [b], and s1,s2: [n]→ {−1,+1} the algorithm works by first “compressing” the matrix product into the polynomial p(x) = ∑k=1n \left(∑i=1n Aik s1(i) xh1(i)\right) \left(∑j=1n Bkj s2(j) xh2(j)\right). Using the fast Fourier transform to compute polynomial multiplication, we can compute c0,…,cb−1 such that ∑i ci xi = (p(x) mod xb) + (p(x) div xb) in time Õ(n2+ nb). An unbiased estimator of (AB)ij with variance at most ‖AB‖F2/b can then be computed as: Cij = s1(i)s2(j)c(h1(i)+h2(j)) mod b. Our approach also leads to an algorithm for computing AB exactly, with high probability, in time Õ(N + nb) in the case where A and B have at most N nonzero entries, and AB has at most b nonzero entries.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, Compressed matrix multiplication
- Alon, N., Gibbons, P. B., Matias, Y., and Szegedy, M. 2002. Tracking join and self-join sizes in limited storage. J. Comput. Syst. Sci 64, 3, 719--747.Google Scholar
Digital Library
- Alon, N., Matias, Y., and Szegedy, M. 1999. The space complexity of approximating the frequency moments. J. Comput. Syst. Sci 58, 1, 137--147. Google Scholar
Digital Library
- Braverman, V., Chung, K.-M., Liu, Z., Mitzenmacher, M., and Ostrovsky, R. 2010. AMS without 4-wise independence on product domains. In Proceedings of the 27th International Symposium on Theoretical Aspects of Computer Science (STACS). Leibniz International Proceedings in Informatics (LIPIcs) Series, vol. 5. 119--130.Google Scholar
- Carter, J. L. and Wegman, M. N. 1979. Universal classes of hash functions. J. Comput. Syst. Sci. 18, 2, 143--154.Google Scholar
Digital Library
- Charikar, M., Chen, K., and Farach-Colton, M. 2004. Finding frequent items in data streams. Theoret. Comput. Sci. 312, 1, 3--15. Google Scholar
Digital Library
- Clarkson, K. L. and Woodruff, D. P. 2009. Numerical linear algebra in the streaming model. In Proceedings of Symposium on Theory of Computing (STOC). ACM, 205--214. Google Scholar
Digital Library
- Cohen, E. 1998. Structure prediction and computation of sparse matrix products. J. Combin. Optimat. 2, 4, 307--332.Google Scholar
Cross Ref
- Cohen, E. and Lewis, D. D. 1999. Approximating matrix multiplication for pattern recognition tasks. J. Algor. 30, 2, 211--252. Google Scholar
Digital Library
- Cooley, J. W. and Tukey, J. W. 1965. An algorithm for the machine calculation of complex Fourier series. Math. Computat. 19, 90, 297--301.Google Scholar
Cross Ref
- Coppersmith, D. and Winograd, S. 1990. Matrix multiplication via arithmetic progressions. J. Symbol. Computat. 9, 3, 251--280. Google Scholar
Digital Library
- Dietzfelbinger, M., Gil, J., Matias, Y., and Pippenger, N. 1992. Polynomial hash functions are reliable (extended abstract). In Proceedings of the 19th International Colloquium on Automata, Languages and Programming. Lecture Notes in Computer Science Series, vol. 623, Springer, 235--246. Google Scholar
Digital Library
- Drineas, P., Kannan, R., and Mahoney, M. W. 2006. Fast Monte Carlo algorithms for matrices I: Approximating matrix multiplication. SIAM J. Comput. 36, 1, 132--157. Google Scholar
Digital Library
- Gilbert, A. and Indyk, P. 2010. Sparse recovery using sparse matrices. Proc. IEEE 98, 6, 937--947.Google Scholar
Cross Ref
- Gilbert, A. C., Li, Y., Porat, E., and Strauss, M. J. 2010. Approximate sparse recovery: optimizing time and measurements. In Proceedings of the 42nd ACM Symposium on Theory of Computing (STOC). ACM, 475--484. Google Scholar
Digital Library
- Goodman, L. A. 1960. On the exact variance of products. J. Amer. Statis. Assoc. 55, 292, 708--713.Google Scholar
Cross Ref
- Indyk, P. and McGregor, A. 2008. Declaring independence via the sketching of sketches. In Proceedings of the 19th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA). SIAM, 737--745. Google Scholar
Digital Library
- Iwen, M. A. and Spencer, C. V. 2009. A note on compressed sensing and the complexity of matrix multiplication. Inf. Process. Lett. 109, 10, 468--471. Google Scholar
Digital Library
- Kutzkov, K. 2013. Deterministic algorithms for skewed matrix products. In Proceedings of the 30th International Symposium on Theoretical Aspects of Computer Science (STACS). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, vol. 20, 466--477.Google Scholar
- Le Gall, F. 2012. Faster algorithms for rectangular matrix multiplication. In Proceedings of the 53rd Annual IEEE Symposium on Foundations of Computer Science (FOCS). IEEE, 514--523. Google Scholar
Digital Library
- Lingas, A. 2009. A fast output-sensitive algorithm for Boolean matrix multiplication. In Proceedings of 17th European Symposium on Algorithms (ESA). Lecture Notes in Computer Science Series, vol. 5757, Springer, 408--419.Google Scholar
Cross Ref
- Mahoney, M. W. 2012. Algorithmic and statistical perspectives on large-scale data analysis. In Combinatorial Scientific Computing. Chapman and Hall/CRC Press.Google Scholar
- Motwani, R. and Raghavan, P. 1995. Randomized Algorithms. Cambridge University Press. Google Scholar
Digital Library
- Pagh, R. 2012. Compressed matrix multiplication. In Proceedings of Innovations in Theoretical Computer Science (ITCS). ACM, 442--451. Google Scholar
Digital Library
- Paturi, R., Rajasekaran, S., and Reif, J. 1995. The light bulb problem. Inf. Computat. 117, 2, 187--192. Google Scholar
Digital Library
- Pǎtraşcu, M. and Thorup, M. 2011. The power of simple tabulation hashing. In Proceedings of the 43rd Annual ACM Symposium on Theory of Computing (STOC). ACM, 1--10. Google Scholar
Digital Library
- Resen Amossen, R., Campagna, A., and Pagh, R. 2010. Better size estimation for sparse matrix products. In Proceedings of 13th International Workshop on Approximation, Randomization, and Combinatorial Optimization (APPROX-RANDOM). Lecture Notes in Computer Science Series, vol. 6302, Springer, 406--419. Google Scholar
Digital Library
- Resen Amossen, R. and Pagh, R. 2009. Faster join-projects and sparse matrix multiplications. In Proceedings of 12th International Conference on Database Theory (ICDT). ACM International Conference Proceeding Series Series, vol. 361. ACM, 121--126. Google Scholar
Digital Library
- Sarlós, T. 2006. Improved approximation algorithms for large matrices via random projections. In Proceedings of the IEEE Symposium on Foundations of Computer Science (FOCS). IEEE Computer Society, 143--152. Google Scholar
Digital Library
- Sipser, M. and Spielman, D. A. 1996. Expander codes. IEEE Trans. Inf. Theory 42, 6, 1710--1722. Google Scholar
Cross Ref
- Valiant, G. 2012. Finding correlations in subquadratic time, with applications to learning parities and juntas. In Proceedings of Symposium on Foundations of Computer Science (FOCS). IEEE Computer Society, 11--20. Google Scholar
Digital Library
- Vassilevska Williams, V. 2012. Multiplying matrices faster than Coppersmith-Winograd. In Proceedings of the 44th Symposium on Theory of Computing Conference (STOC). ACM, 887--898. Google Scholar
Digital Library
- Yuster, R. and Zwick, U. 2005. Fast sparse matrix multiplication. ACM Trans. Algor. 1, 1, 2--13. Google Scholar
Digital Library
Index Terms
Compressed matrix multiplication
Recommendations
Fast sparse matrix multiplication
Let A and B two n×n matrices over a ring R (e.g., the reals or the integers) each containing at most m nonzero elements. We present a new algorithm that multiplies A and B using O(m0.7n1.2+n2+o(1)) algebraic operations (i.e., multiplications, additions ...
Faster Algorithms for Rectangular Matrix Multiplication
FOCS '12: Proceedings of the 2012 IEEE 53rd Annual Symposium on Foundations of Computer ScienceLet $\alpha$ be the maximal value such that the product of an $n\times n^\alpha$ matrix by an $n^\alpha\times n$ matrix can be computed with $n^{2+o(1)}$ arithmetic operations. In this paper we show that $\alpha>0.30298$, which improves the previous ...
Partial and Total Matrix Multiplication
In 1979 considerable progress was made in estimating the complexity of matrix multiplication. Here the new techniques and recent results are presented, based upon the notion of approximate rank and the observation that certain patterns of partial matrix ...








Comments