skip to main content
article

Meta optimization: improving compiler heuristics with machine learning

Published:09 May 2003Publication History
Skip Abstract Section

Abstract

Compiler writers have crafted many heuristics over the years to approximately solve NP-hard problems efficiently. Finding a heuristic that performs well on a broad range of applications is a tedious and difficult process. This paper introduces Meta Optimization, a methodology for automatically fine-tuning compiler heuristics. Meta Optimization uses machine-learning techniques to automatically search the space of compiler heuristics. Our techniques reduce compiler design complexity by relieving compiler writers of the tedium of heuristic tuning. Our machine-learning system uses an evolutionary algorithm to automatically find effective compiler heuristics. We present promising experimental results. In one mode of operation Meta Optimization creates application-specific heuristics which often result in impressive speedups. For hyperblock formation, one optimization we present in this paper, we obtain an average speedup of 23% (up to 73%) for the applications in our suite. Furthermore, by evolving a compiler's heuristic over several benchmarks, we can create effective, general-purpose heuristics. The best general-purpose heuristic our system found for hyperblock formation improved performance by an average of 25% on our training set, and 9% on a completely unrelated test set. We demonstrate the efficacy of our techniques on three different optimizations in this paper: hyperblock formation, register allocation, and data prefetching.

References

  1. S. G. Abraham, V. Kathail, and B. L. Deitrich. Meld Scheduling: Relaxing Scheduling Constaints Across Region Boundaries. In Proceedings of the 29th Annual International Symposium on Microarchitecture (MICRO-29) pages 308--321,1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. W. Banzhaf, P. Nordin, R. Keller, and F. Francone. Genetic Programming : An Introduction : On the Automatic Evolution of Computer Programs and Its Applications Morgan Kaufmann, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. D. Bernstein, D. Goldin, and M. G. et. al. Spill Code Minimization Techniques for Optimizing Compilers. In Proceedings of the SIGPLAN '89 Conference on Programming Language Design and Implementation pages 258--263, 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. D. Bourgin. Losslessy compression schemes http://hpux.u-aizu.ac.jp/hppd/hpux-/Languages/codecs-1.0/.Google ScholarGoogle Scholar
  5. B. Calder, D. G.ad Michael Jones, D. Lindsay, J. Martin, M. Mozer, and B. Zorn. Evidence-Based Static Branch Prediction Using Machine Learning. In ACM Transactions on Programming Languages and Systems (ToPLaS-19) volume 19, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. P. Chang, D. Lavery, S. Mahlke, W. Chen, and W. Hwu. The Importance of Prepass Code Scheduling for Superscalar and Superpipelined processors. In IEEE Transactions on Computers volume 44, pages 353--370, March 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. F. C. Chow and J. L. Hennessey. The Priority-Based Coloring Approch to Register Allocation. In ACM Transactions on Programming Languages and Systems (ToPLaS-12) pages 501--536, 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. K. Cooper, P. Scheilke, and D. Subramanian. Optimizing for Reduced Code Space using Genetic Algorithms. In Languages, Compilers, Tools for Embedded Systems pages 1--9, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. C. Gathercole. An Investigation of Supervised Learning in Genetic Programming PhDthesis, University of Edinburgh, 1998.Google ScholarGoogle Scholar
  10. P. B. Gibbons and S. S. Muchnick. Efficient Instruction Scheduling for a Pipelined Architecture. In Proceedings of the ACM Symposium on Compiler Construction volume 21, pages 11--16, 1986. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. G. W. Grewal and C. T. Wilson. Mappping Reference Code to Irregular DSPs with the Retargetable, Optimizing Compiler COGEN(T). In International Symposium on Microarchitecture volume 34, pages 192--202, 2001 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. Kessler and T. Haynes. Depth-Fair Crossover in Genetic Programming. In Proceedings of the ACM Symposium on Applied Computing pages 319--323, February 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. J. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection TheMIT Press, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. C. Lee, M. Potkonjak, and W. H. Mangione-Smith. MediaBench: a tool for evaluating and synthesizing multimedia and communication systems. In International Symposium on Microarchitecture volume 30, pages 330--335, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. S. A. Mahlke. Exploiting instruction level parallelism in the presence of branches PhD thesis, University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. S. A. Mahlke, D. Lin, W. Chen, R. Hank, and R. Bringmann. Effective Compiler Support for Predicated Execution Using the Hyperblock.In International Symposium on Microarchitecture volume 25, pages 45--54, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. Monsifrot, F. Bodin, and R. Quiniou. A Machine Learning Approach to Automatic Production of Compiler Heuristics. In Artificial Intelligence: Methodology, Systems, Applications pages 41--50, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. T. C. Mowry. Tolerating Latency through Software-Controlled Data Prefetching PhDthesis, Stanford University,Department of Electrical Engineering, 1994 Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Open Research Compiler. http://ipf-orc.sourceforge.netGoogle ScholarGoogle Scholar
  20. E. Ozer, S. Banerjia, and T. Conte. Unified Assign and Schedule: A New Approach to Scheduling for Clustered Register Filee Microarchitectures. In Proceedings of the 27th Annual International Symposium on Microarchitecture (MICRO-24) pages 308--315, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J. C. H. Park and M. S. Schlansker. On Predicated Execution. Technical Report HPL-91-58, Hewlett Packard Laboratories, 1991.Google ScholarGoogle Scholar
  22. B. R. Rau. Iterative Modulo Scheduling: An Algorithm for Software Pipelining Loops. In Proceedings of the 27th Annual International Symposium on Microarchitecture (MICRO-24) November 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. Stephenson, S. Amarasinghe, U.-M. O 'Reilly, and M. Martin. Compiler Heuristic Design with Machine Learning. Technical Report TR-893, Massachusetts Institute of Technology, 2003.Google ScholarGoogle Scholar
  24. Trimaran. http://www.trimaran.orgGoogle ScholarGoogle Scholar
  25. N. Warter. Modulo Scheduling with Isomorphic Control Transformations PhD thesis, University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Meta optimization: improving compiler heuristics with machine learning

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 38, Issue 5
          May 2003
          349 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/780822
          Issue’s Table of Contents
          • cover image ACM Conferences
            PLDI '03: Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation
            June 2003
            360 pages
            ISBN:1581136625
            DOI:10.1145/781131

          Copyright © 2003 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 9 May 2003

          Check for updates

          Qualifiers

          • article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!