skip to main content
research-article
Public Access

On Characterizing the Data Access Complexity of Programs

Published:14 January 2015Publication History
Skip Abstract Section

Abstract

Technology trends will cause data movement to account for the majority of energy expenditure and execution time on emerging computers. Therefore, computational complexity will no longer be a sufficient metric for comparing algorithms, and a fundamental characterization of data access complexity will be increasingly important. The problem of developing lower bounds for data access complexity has been modeled using the formalism of Hong and Kung's red/blue pebble game for computational directed acyclic graphs (CDAGs). However, previously developed approaches to lower bounds analysis for the red/blue pebble game are very limited in effectiveness when applied to CDAGs of real programs, with computations comprised of multiple sub-computations with differing DAG structure. We address this problem by developing an approach for effectively composing lower bounds based on graph decomposition. We also develop a static analysis algorithm to derive the asymptotic data-access lower bounds of programs, as a function of the problem size and cache size.

Skip Supplemental Material Section

Supplemental Material

p567-sidebyside.mpg

References

  1. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz. Minimizing communication in numerical linear algebra. SIAM J. Matrix Analysis Applications, 32(3):866--901, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  2. G. Ballard, J. Demmel, O. Holtz, B. Lipshitz, and O. Schwartz. Brief announcement: Strong scaling of matrix multiplication algorithms and memory-independent communication lower bounds. In Proc. SPAA'12, pages 77--79, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. G. Ballard, J. Demmel, O. Holtz, and O. Schwartz. Graph expansion and communication costs of fast matrix multiplication. J. ACM, 59(6): 32, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. A. Barvinok. Computing the Ehrhart polynomial of a convex l attice polytope. Discrete and Computational Geometry, 12:35--48, 1994.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. J. Bennett, A. Carbery, M. Christ, and T. Tao. Finite bounds for Holder-Brascamp-Lieb multilinear inequalities. Mathematical Research Letters, 55(4):647--666, 2010.Google ScholarGoogle ScholarCross RefCross Ref
  6. K. Bergman, S. Borkar, et al. Exascale computing study: Technology challenges in achieving exascale systems. DARPA IPTO, Tech. Rep, 2008.Google ScholarGoogle Scholar
  7. G. Bilardi and E. Peserico. A characterization of temporal locality and its portability across memory hierarchies. Automata, Languages and Programming, pages 128--139, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. G. Bilardi and F. P. Preparata. Processor - Time Tradeoffs under Bounded-Speed Message Propagation: Part II, Lower Bounds. Theory Comput. Syst., 32(5):531--559, 1999.Google ScholarGoogle ScholarCross RefCross Ref
  9. G. Bilardi, A. Pietracaprina, and P. D'Alberto. On the space and access complexity of computation DAGs. In Graph-Theoretic Concepts in Computer Science, volume 1928 of LNCS, pages 81--92. 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. G. Bilardi, M. Scquizzato, and F. Silvestri. A lower bound technique for communication on bsp with application to the fft. In Euro-Par, pages 676--687, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. M. Christ, J. Demmel, N. Knight, T. Scanlon, and K. Yelick. Communication Lower Bounds and Optimal Algorithms for Programs That Reference Arrays Part 1. EECS Technical Report EECS'2013--6 1, UC Berkeley, May 2013.Google ScholarGoogle Scholar
  12. S. A. Cook. An observation on time-storage trade off. J. Comput. Syst. Sci., 9(3):308--316, 1974. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. J. Demmel, L. Grigori, M. Hoemmen, and J. Langou. Communication-optimal parallel and sequential QR and LU factorizations. SIAM J. Scientific Computing, 34(1), 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. V. Elango, F. Rastello, L.-N. Pouchet, J. Ramanujam, and P. Sadayappan. Data access complexity: The red/blue pebble game revisited. Technical report, OSU/INRIA/LSU/UCLA, Sept. 2013. OSU-CI SRC-7/13-TR16.Google ScholarGoogle Scholar
  15. V. Elango, F. Rastello, L. Pouchet, J. Ramanujam, and P. Sadayappan. On characterizing the data movement complexity of computational dags for parallel execution. In 26th ACM Symposium on Parallelism in Algorithms and Architectures, SPAA '14, pages 296--306, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. P. Feautrier. Parametric integer programming. RAIRO Recherche Operationnelle, 22(3):243--268, 1988.Google ScholarGoogle ScholarCross RefCross Ref
  17. S. H. Fuller and L. I. Millett. The Future of Computing Performance: Game Over or Next Level? The National Academies Press, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. S. Girbal, N. Vasilache, C. Bastoul, A. Cohen, D. Parello, M. Sigler, and O. Temam. Semi-automatic composition of loop transformations for deep parallelism and memory hierarchies. Intl. J. of Parallel Programming, 34(3), 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. G. Gupta and S. Rajopadhye. The Z-polyhedral model. In ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP'07), pages 237--248. ACM, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. J.-W. Hong and H. T. Kung. I/O complexity: The red-blue pebble game. In Proc. of the 13th annual ACM sympo. on Theory of computing (STOC'81), pages 326--333. ACM, 1981. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. D. Irony, S. Toledo, and A. Tiskin. Communication lower bounds for distributed-memory matrix multiplication. J. Parallel Distrib. Comput., 64(9):1017--1026, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. A. Ketterlin and P. Clauss. Profiling data-dependence to assist parallelization: Framework, scope, and optimization. In MICRO, pages 437--448, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. L. Loomis and H. Whitney. An inequality related to the isoperimetric inequality. Bull. Am. Math. Soc., 55:961--962, 1949.Google ScholarGoogle ScholarCross RefCross Ref
  24. D. Ranjan and M. Zubair. Vertex isoperimetric parameter of a computation graph. Int. J. Found. Comput. Sci., 23(4):941--964, 2012.Google ScholarGoogle ScholarCross RefCross Ref
  25. D. Ranjan, J. Savage, and M. Zubair. Strong I/O lower bounds for binomial and FFT computation graphs. In Computing and Combinatorics, volume 6842 of LNCS, pages 134--145. Springer, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. D. Ranjan, J. E. Savage, and M. Zubair. Upper and lower I/O bounds for pebbling r-pyramids. J. Discrete Algorithms, 14:2--12, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. J. Savage. Extending the Hong-Kung model to memory hierarchies. In Computing and Combinatorics, volume 959 of LNCS, pages 270--281. 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. E. Savage. Models of Computation. Addison-Wesley, 1998.Google ScholarGoogle Scholar
  29. J. E. Savage and M. Zubair. A unified model for multicore arc hi-tectures. In Proceedings of the 1st International Forum on Next-generation Multicore/Manycore Technologies, page 9. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. J. E. Savage and M. Zubair. Cache-optimal algorithms for option pricing. ACM Trans. Math. Softw., 37(1), 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. M. Scquizzato and F. Silvestri. Communication lower bounds for distributed-memory computations. CoRR, abs/1307.1805, 2013.Google ScholarGoogle Scholar
  32. J. Shalf, S. Dosanjh, and J. Morrison. Exascale computing technology challenges. High Performance Computing for Computational Science-VECPAR 2010, pages 1--25, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. E. Solomonik, A. Buluc, and J. Demmel. Minimizing communication in all-pairs shortest paths. In IPDPS, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. S. I. Valdimarsson. The Brascamp-Lieb polyhedron. Can. J. Math., 62(4):870--888, 2010.Google ScholarGoogle ScholarCross RefCross Ref
  35. L. G. Valiant. A bridging model for multi-core computing. J. Comput. Syst. Sci., 77:154--166, Jan. 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. S. Verdoolaege. isl: An integer set library for the poly hedral model. In Mathematical Software-ICMS 2010, pages 299--302. Springer, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. On Characterizing the Data Access Complexity of Programs

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 50, Issue 1
          POPL '15
          January 2015
          682 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/2775051
          • Editor:
          • Andy Gill
          Issue’s Table of Contents
          • cover image ACM Conferences
            POPL '15: Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages
            January 2015
            716 pages
            ISBN:9781450333009
            DOI:10.1145/2676726

          Copyright © 2015 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 14 January 2015

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!