Abstract
Compiler writers have crafted many heuristics over the years to approximately solve NP-hard problems efficiently. Finding a heuristic that performs well on a broad range of applications is a tedious and difficult process. This paper introduces Meta Optimization, a methodology for automatically fine-tuning compiler heuristics. Meta Optimization uses machine-learning techniques to automatically search the space of compiler heuristics. Our techniques reduce compiler design complexity by relieving compiler writers of the tedium of heuristic tuning. Our machine-learning system uses an evolutionary algorithm to automatically find effective compiler heuristics. We present promising experimental results. In one mode of operation Meta Optimization creates application-specific heuristics which often result in impressive speedups. For hyperblock formation, one optimization we present in this paper, we obtain an average speedup of 23% (up to 73%) for the applications in our suite. Furthermore, by evolving a compiler's heuristic over several benchmarks, we can create effective, general-purpose heuristics. The best general-purpose heuristic our system found for hyperblock formation improved performance by an average of 25% on our training set, and 9% on a completely unrelated test set. We demonstrate the efficacy of our techniques on three different optimizations in this paper: hyperblock formation, register allocation, and data prefetching.
- S. G. Abraham, V. Kathail, and B. L. Deitrich. Meld Scheduling: Relaxing Scheduling Constaints Across Region Boundaries. In Proceedings of the 29th Annual International Symposium on Microarchitecture (MICRO-29) pages 308--321,1996. Google Scholar
Digital Library
- W. Banzhaf, P. Nordin, R. Keller, and F. Francone. Genetic Programming : An Introduction : On the Automatic Evolution of Computer Programs and Its Applications Morgan Kaufmann, 1998. Google Scholar
Digital Library
- D. Bernstein, D. Goldin, and M. G. et. al. Spill Code Minimization Techniques for Optimizing Compilers. In Proceedings of the SIGPLAN '89 Conference on Programming Language Design and Implementation pages 258--263, 1989. Google Scholar
Digital Library
- D. Bourgin. Losslessy compression schemes http://hpux.u-aizu.ac.jp/hppd/hpux-/Languages/codecs-1.0/.Google Scholar
- B. Calder, D. G.ad Michael Jones, D. Lindsay, J. Martin, M. Mozer, and B. Zorn. Evidence-Based Static Branch Prediction Using Machine Learning. In ACM Transactions on Programming Languages and Systems (ToPLaS-19) volume 19, 1997. Google Scholar
Digital Library
- P. Chang, D. Lavery, S. Mahlke, W. Chen, and W. Hwu. The Importance of Prepass Code Scheduling for Superscalar and Superpipelined processors. In IEEE Transactions on Computers volume 44, pages 353--370, March 1995. Google Scholar
Digital Library
- F. C. Chow and J. L. Hennessey. The Priority-Based Coloring Approch to Register Allocation. In ACM Transactions on Programming Languages and Systems (ToPLaS-12) pages 501--536, 1990. Google Scholar
Digital Library
- K. Cooper, P. Scheilke, and D. Subramanian. Optimizing for Reduced Code Space using Genetic Algorithms. In Languages, Compilers, Tools for Embedded Systems pages 1--9, 1999. Google Scholar
Digital Library
- C. Gathercole. An Investigation of Supervised Learning in Genetic Programming PhDthesis, University of Edinburgh, 1998.Google Scholar
- P. B. Gibbons and S. S. Muchnick. Efficient Instruction Scheduling for a Pipelined Architecture. In Proceedings of the ACM Symposium on Compiler Construction volume 21, pages 11--16, 1986. Google Scholar
Digital Library
- G. W. Grewal and C. T. Wilson. Mappping Reference Code to Irregular DSPs with the Retargetable, Optimizing Compiler COGEN(T). In International Symposium on Microarchitecture volume 34, pages 192--202, 2001 Google Scholar
Digital Library
- M. Kessler and T. Haynes. Depth-Fair Crossover in Genetic Programming. In Proceedings of the ACM Symposium on Applied Computing pages 319--323, February 1999. Google Scholar
Digital Library
- J. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection TheMIT Press, 1992. Google Scholar
Digital Library
- C. Lee, M. Potkonjak, and W. H. Mangione-Smith. MediaBench: a tool for evaluating and synthesizing multimedia and communication systems. In International Symposium on Microarchitecture volume 30, pages 330--335, 1997. Google Scholar
Digital Library
- S. A. Mahlke. Exploiting instruction level parallelism in the presence of branches PhD thesis, University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, 1996. Google Scholar
Digital Library
- S. A. Mahlke, D. Lin, W. Chen, R. Hank, and R. Bringmann. Effective Compiler Support for Predicated Execution Using the Hyperblock.In International Symposium on Microarchitecture volume 25, pages 45--54, 1992. Google Scholar
Digital Library
- A. Monsifrot, F. Bodin, and R. Quiniou. A Machine Learning Approach to Automatic Production of Compiler Heuristics. In Artificial Intelligence: Methodology, Systems, Applications pages 41--50, 2002. Google Scholar
Digital Library
- T. C. Mowry. Tolerating Latency through Software-Controlled Data Prefetching PhDthesis, Stanford University,Department of Electrical Engineering, 1994 Google Scholar
Digital Library
- Open Research Compiler. http://ipf-orc.sourceforge.netGoogle Scholar
- E. Ozer, S. Banerjia, and T. Conte. Unified Assign and Schedule: A New Approach to Scheduling for Clustered Register Filee Microarchitectures. In Proceedings of the 27th Annual International Symposium on Microarchitecture (MICRO-24) pages 308--315, 1998. Google Scholar
Digital Library
- J. C. H. Park and M. S. Schlansker. On Predicated Execution. Technical Report HPL-91-58, Hewlett Packard Laboratories, 1991.Google Scholar
- B. R. Rau. Iterative Modulo Scheduling: An Algorithm for Software Pipelining Loops. In Proceedings of the 27th Annual International Symposium on Microarchitecture (MICRO-24) November 1994. Google Scholar
Digital Library
- M. Stephenson, S. Amarasinghe, U.-M. O 'Reilly, and M. Martin. Compiler Heuristic Design with Machine Learning. Technical Report TR-893, Massachusetts Institute of Technology, 2003.Google Scholar
- Trimaran. http://www.trimaran.orgGoogle Scholar
- N. Warter. Modulo Scheduling with Isomorphic Control Transformations PhD thesis, University of Illinois at Urbana-Champaign, Department of Electrical and Computer Engineering, 1993. Google Scholar
Digital Library
Index Terms
Meta optimization: improving compiler heuristics with machine learning
Recommendations
Meta optimization: improving compiler heuristics with machine learning
PLDI '03: Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementationCompiler writers have crafted many heuristics over the years to approximately solve NP-hard problems efficiently. Finding a heuristic that performs well on a broad range of applications is a tedious and difficult process. This paper introduces Meta ...
Feature selection and policy optimization for distributed instruction placement using reinforcement learning
PACT '08: Proceedings of the 17th international conference on Parallel architectures and compilation techniquesCommunication overheads are one of the fundamental challenges in a multiprocessor system. As the number of processors on a chip increases, communication overheads and the distribution of computation and data become increasingly important performance ...
Integrating estimation of distribution algorithms versus Q-learning into Meta-RaPS for solving the 0-1 multidimensional knapsack problem
Highlights- EDA and Q-Learning are integrated in the memoryless metaheuristic Meta-RaPS.
- ...
AbstractFinding near-optimal solutions in an acceptable amount of time is a challenge when developing sophisticated approximate approaches. A powerful answer to this challenge might be reached by incorporating intelligence into metaheuristics. ...







Comments