Abstract
Today's compilers have a plethora of optimizations to choose from, and the correct choice of optimizations can have a significant impact on the performance of the code being optimized. Furthermore, choosing the correct order in which to apply those optimizations has been a long standing problem in compilation research. Each of these optimizations interacts with the code and in turn with all other optimizations in complicated ways. Traditional compilers typically apply the same set of optimization in a fixed order to all functions in a program, without regard the code being optimized.
Understanding the interactions of optimizations is very important in determining a good solution to the phase-ordering problem. This paper develops a new approach that automatically selects good optimization orderings on a per method basis within a dynamic compiler. Our approach formulates the phase-ordering problem as a Markov process and uses a characterization of the current state of the code being optimized to creating a better solution to the phase ordering problem. Our technique uses neuro-evolution to construct an artificial neural network that is capable of predicting beneficial optimization ordering for a piece of code that is being optimized. We implemented our technique in Jikes RVM and achieved significant improvements on a set of standard Java benchmarks over a well-engineered fixed order.
- Dacapo benchmark suite. URL http://dacapobench.org/benchmarks.html.Google Scholar
- F. Agakov, E. Bonilla, J. Cavazos, B. Franke, G. Fursin, M. F. P. O'Boyle, J. Thomson, M. Toussaint, and C. K. I. Williams. Using machine learning to focus iterative optimization. In CGO '06: Proceedings of the International Symposium on Code Generation and Optimization, pages 295--305, Washington, DC, USA, 2006. IEEE Computer Society. Google Scholar
Digital Library
- L. Almagor, K. D. Cooper, A. Grosul, T. J. Harvey, S. W. Reeves, D. Subramanian, L. Torczon, and T. Waterman. Finding effective compilation sequences. In Proceedings of the 2004 ACM SIGPLAN/SIGBED conference on Languages, compilers, and tools for embedded systems, LCTES '04, pages 231--239. ACM, 2004. Google Scholar
Digital Library
- B. Alpern, C. R. Attanasio, J. J. Barton,M. G. Burke, P.Cheng, J.-D. Choi, A. Cocchi, S. J. Fink, D. Grove, M. Hind, S. F. Hummel, D. Lieber, V. Litvinov, M. F. Mergen, T. Ngo, J. R. Russell, V. Sarkar, M. J. Serrano, J. C. Shepherd, S. E. Smith, V. C. Sreedhar, H. Srinivasan, and J. Whaley. The Jalapeño virtual machine. IBM Systems Jounrnal, 39(1), 2000. Google Scholar
Digital Library
- J. Cavazos and M. F. P. O'Boyle. Method-specific Dynamic Compilation Using Logistic Regression. In OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 229--240, New York, NY, USA, 2006. ACM Press. Google Scholar
Digital Library
- K. Cooper, P. Schielke, and D. Subramanian. Optimizing for reduced code space using genetic algorithms. In Proceedings of the ACM SIGPLAN 1999 workshop on Languages, compilers, and tools for embedded systems, pages 1--9. ACM, 1999. Google Scholar
Digital Library
- K. Cooper, D. Subramanian, and L. Torczon. Adaptive optimizing compilers for the 21st century. The Journal of Supercomputing, 23(1):7--22, 2001. Google Scholar
Digital Library
- K. Cooper, A. Grosul, T. Harvey, S. Reeves, D. Subramanian, L. Torczon, and T. Waterman. ACME: adaptive compilation made efficient. In LCTES '05: Proceedings of the 2005 ACM SIGPLAN/SIGBED conference on Languages, compilers, and tools for embedded systems, volume 40, pages 69--77. ACM, 2005. Google Scholar
Digital Library
- K. D. Cooper, A. Grosul, T. J. Harvey, S. Reeves, D. Subramanian, L. Torczon, and T. Waterman. Exploring the Structure of the Space of Compilation Sequences Using Randomized Search Algorithms. J. Supercomputing, 36(2):135--151, 2006. Google Scholar
Digital Library
- G. Fursin, C.Miranda, O. Temam,M. Namolaru, E. Yom-Tov, A. Zaks, B. Mendelson, P. Barnard, E. Ashton, E. Courtois, F. Bodin, E. Bonilla, J. Thomson, H. Leather, C. Williams, and M. O'Boyle. Milepost gcc: machine learning based research compiler. In Proceedings of the GCC Developers' Summit, June 2008.Google Scholar
- A. Ganapathi, K. Datta, A. Fox, and D. Patterson. A case for machine learning to optimize multicore performance. First USENIXWorkshop on Hot Topics in Parallelism (HotPar '09), 2009. Google Scholar
Digital Library
- S.-C. Han, F. Franchetti, and M. Püschel. Program Generation for the All-pairs Shortest Path Problem. In PACT '06: Proceedings of the 15th international conference on Parallel architectures and compilation techniques, pages 222--232, New York, NY, USA, 2006. ACM Press. Google Scholar
Digital Library
- M. R. Jantz and P. A. Kulkarni. Eliminating false phase interactions to reduce optimization phase order search space. In CASES, pages 187--196. ACM, 2010. Google Scholar
Digital Library
- P. Kulkarni, S. Hines, J. Hiser, D. Whalley, J. Davidson, and D. Jones. Fast Searches for Effective Optimization Phase Sequences. In PLDI '04: Proceedings of the ACM SIGPLAN 2004 conference on Programming language design and implementation, pages 171--182. ACM Press, 2004. Google Scholar
Digital Library
- P. A. Kulkarni, D. B. Whalley, G. S. Tyson, and J. W. Davidson. Exhaustive optimization phase order space exploration. In Fourth Annual IEEE/ACM Interational Conference on Code Generation and Optimization, pages 306--318, New York City, NY, March 2006. Google Scholar
Digital Library
- P. A. Kulkarni, D. B. Whalley, and G. S. Tyson. Evaluating heuristic optimization phase order search algorithms. In CGO, pages 157--169. IEEE Computer Society, 2007. Google Scholar
Digital Library
- P. A. Kulkarni, D. B. Whalley, G. S. Tyson, and J. W. Davidson. Practical exhaustive optimization phase order exploration and evaluation. TACO, 6(1), 2009. Google Scholar
Digital Library
- X. Li, M. J. Garzarán, and D. Padua. Optimizing Sorting with Genetic Algorithms. In In Proc. of the International Symposium on Code Generation and Optimization (CGO), pages 99--110, March 2005. Google Scholar
Digital Library
- C. Liao, D. J. Quinlan, R. W. Vuduc, and T. Panas. Effective source-to-source outlining to support whole program empirical optimization. In LCPC'09, pages 308--322, 2009. Google Scholar
Digital Library
- A. Monsifrot, F. Bodin, and R. Quiniou. A machine learning approach to automatic production of compiler heuristics. In AIMSA '02: Proceedings of the 10th International Conference on Artificial Intelligence:Methodology, Systems, and Applications, pages 41--50, London, UK, 2002. Springer-Verlag. Google Scholar
Digital Library
- M. Püschel, J. Moura, J. Johnson, D. Padua, M. Veloso, B. Singer, J. Xiong, F. Franchetti, A. Gacic, Y. Voronenko, K. Chen, R. W. Johnson, and N. Rizzolo. SPIRAL: Code Generation for DSP Transforms. In Proc. of the IEEE, special issue on Program Generation, Optimization, and Platform Adaptation, 93(2):232--275, February 2005.Google Scholar
- L. A. Smith, J. M. Bull, and J. Obdrzálek. A parallel Java Grande benchmark suite. In ACM, editor, SC2001: High Performance Networking and Computing. Denver, CO, November 10--16, 2001. ACM Press and IEEE Computer Society Press, 2001. ISBN 1-58113-293-X. Google Scholar
Digital Library
- K. O. Stanley and R. Miikkulainen. Efficient reinforcement learning through evolving neural network topologies. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2002), page 9, San Francisco, 2002. Morgan Kaufmann. URL http://nn.cs.utexas.edu/?stanley:gecco02b. Google Scholar
Digital Library
- M. Stephenson and S. Amarasinghe. Predicting Unroll Factors Using Supervised Classification. In CGO '05: Proceedings of the international symposium on Code generation and optimization, pages 123--134, Washington, DC, USA, 2005. IEEE Computer Society. Google Scholar
Digital Library
- M. Stephenson, S. Amarasinghe, M. Martin, and U.-M. O'Reilly. Meta Optimization: Improving Compiler Heuristics with Machine Learning. In Proc. of Programing Language Design and Implementation, June 2003. Google Scholar
Digital Library
- R. Vuduc, J. W. Demmel, and K. A. Yelick. OSKI: A Library of Automatically Tuned Sparse Matrix Kernels. Journal of Physics Conference Series, 16:521--530, Jan. 2005.Google Scholar
Cross Ref
- Website. Specjvm 2008. URL http://www.spec.org/jvm2008/.Google Scholar
- Website. Specjvm 98,. URL http://www.spec.org/jvm98/.Google Scholar
- Website. Java grande benchmarks,. URL http://www2.epcc.ed.ac.uk/computing/research activities/java grande/.Google Scholar
- R. C. Whaley, A. Petitet, and J. J. Dongarra. Automated Empirical Optimizations of Software and the ATLAS Project. Parallel Computing, 27(1-2):3--35, 2001.Google Scholar
Digital Library
- K. Yotov, X. Li, G. Ren, M. Cibulskis, G. DeJong, M. Garzaran, D. Padua, K. Pingali, P. Stodghill, and P. Wu. A Comparison of Empirical and Model-driven Optimization. In Proc. of Programing Language Design and Implementation, pages 63--76, June 2003. Google Scholar
Digital Library
Index Terms
Mitigating the compiler optimization phase-ordering problem using machine learning
Recommendations
A Survey on Compiler Autotuning using Machine Learning
Since the mid-1990s, researchers have been trying to use machine-learning-based approaches to solve a number of different compiler optimization problems. These techniques primarily enhance the quality of the obtained results and, more importantly, make ...
Mitigating the compiler optimization phase-ordering problem using machine learning
OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applicationsToday's compilers have a plethora of optimizations to choose from, and the correct choice of optimizations can have a significant impact on the performance of the code being optimized. Furthermore, choosing the correct order in which to apply those ...
Towards Compile-Time-Reducing Compiler Optimization Selection via Machine Learning
ICPP Workshops '21: 50th International Conference on Parallel Processing WorkshopCompilers come with a multitude of optimizations to choose from, and the chosen optimizations significantly affect the performance of the code being optimized. Selecting the optimal set of optimizations to apply and determining the order to run them is ...







Comments