skip to main content
research-article

Mitigating the compiler optimization phase-ordering problem using machine learning

Published:19 October 2012Publication History
Skip Abstract Section

Abstract

Today's compilers have a plethora of optimizations to choose from, and the correct choice of optimizations can have a significant impact on the performance of the code being optimized. Furthermore, choosing the correct order in which to apply those optimizations has been a long standing problem in compilation research. Each of these optimizations interacts with the code and in turn with all other optimizations in complicated ways. Traditional compilers typically apply the same set of optimization in a fixed order to all functions in a program, without regard the code being optimized.

Understanding the interactions of optimizations is very important in determining a good solution to the phase-ordering problem. This paper develops a new approach that automatically selects good optimization orderings on a per method basis within a dynamic compiler. Our approach formulates the phase-ordering problem as a Markov process and uses a characterization of the current state of the code being optimized to creating a better solution to the phase ordering problem. Our technique uses neuro-evolution to construct an artificial neural network that is capable of predicting beneficial optimization ordering for a piece of code that is being optimized. We implemented our technique in Jikes RVM and achieved significant improvements on a set of standard Java benchmarks over a well-engineered fixed order.

References

  1. Dacapo benchmark suite. URL http://dacapobench.org/benchmarks.html.Google ScholarGoogle Scholar
  2. F. Agakov, E. Bonilla, J. Cavazos, B. Franke, G. Fursin, M. F. P. O'Boyle, J. Thomson, M. Toussaint, and C. K. I. Williams. Using machine learning to focus iterative optimization. In CGO '06: Proceedings of the International Symposium on Code Generation and Optimization, pages 295--305, Washington, DC, USA, 2006. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. L. Almagor, K. D. Cooper, A. Grosul, T. J. Harvey, S. W. Reeves, D. Subramanian, L. Torczon, and T. Waterman. Finding effective compilation sequences. In Proceedings of the 2004 ACM SIGPLAN/SIGBED conference on Languages, compilers, and tools for embedded systems, LCTES '04, pages 231--239. ACM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. B. Alpern, C. R. Attanasio, J. J. Barton,M. G. Burke, P.Cheng, J.-D. Choi, A. Cocchi, S. J. Fink, D. Grove, M. Hind, S. F. Hummel, D. Lieber, V. Litvinov, M. F. Mergen, T. Ngo, J. R. Russell, V. Sarkar, M. J. Serrano, J. C. Shepherd, S. E. Smith, V. C. Sreedhar, H. Srinivasan, and J. Whaley. The Jalapeño virtual machine. IBM Systems Jounrnal, 39(1), 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. J. Cavazos and M. F. P. O'Boyle. Method-specific Dynamic Compilation Using Logistic Regression. In OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 229--240, New York, NY, USA, 2006. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. K. Cooper, P. Schielke, and D. Subramanian. Optimizing for reduced code space using genetic algorithms. In Proceedings of the ACM SIGPLAN 1999 workshop on Languages, compilers, and tools for embedded systems, pages 1--9. ACM, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. K. Cooper, D. Subramanian, and L. Torczon. Adaptive optimizing compilers for the 21st century. The Journal of Supercomputing, 23(1):7--22, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. K. Cooper, A. Grosul, T. Harvey, S. Reeves, D. Subramanian, L. Torczon, and T. Waterman. ACME: adaptive compilation made efficient. In LCTES '05: Proceedings of the 2005 ACM SIGPLAN/SIGBED conference on Languages, compilers, and tools for embedded systems, volume 40, pages 69--77. ACM, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. K. D. Cooper, A. Grosul, T. J. Harvey, S. Reeves, D. Subramanian, L. Torczon, and T. Waterman. Exploring the Structure of the Space of Compilation Sequences Using Randomized Search Algorithms. J. Supercomputing, 36(2):135--151, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. G. Fursin, C.Miranda, O. Temam,M. Namolaru, E. Yom-Tov, A. Zaks, B. Mendelson, P. Barnard, E. Ashton, E. Courtois, F. Bodin, E. Bonilla, J. Thomson, H. Leather, C. Williams, and M. O'Boyle. Milepost gcc: machine learning based research compiler. In Proceedings of the GCC Developers' Summit, June 2008.Google ScholarGoogle Scholar
  11. A. Ganapathi, K. Datta, A. Fox, and D. Patterson. A case for machine learning to optimize multicore performance. First USENIXWorkshop on Hot Topics in Parallelism (HotPar '09), 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. S.-C. Han, F. Franchetti, and M. Püschel. Program Generation for the All-pairs Shortest Path Problem. In PACT '06: Proceedings of the 15th international conference on Parallel architectures and compilation techniques, pages 222--232, New York, NY, USA, 2006. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. M. R. Jantz and P. A. Kulkarni. Eliminating false phase interactions to reduce optimization phase order search space. In CASES, pages 187--196. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. P. Kulkarni, S. Hines, J. Hiser, D. Whalley, J. Davidson, and D. Jones. Fast Searches for Effective Optimization Phase Sequences. In PLDI '04: Proceedings of the ACM SIGPLAN 2004 conference on Programming language design and implementation, pages 171--182. ACM Press, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. P. A. Kulkarni, D. B. Whalley, G. S. Tyson, and J. W. Davidson. Exhaustive optimization phase order space exploration. In Fourth Annual IEEE/ACM Interational Conference on Code Generation and Optimization, pages 306--318, New York City, NY, March 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. P. A. Kulkarni, D. B. Whalley, and G. S. Tyson. Evaluating heuristic optimization phase order search algorithms. In CGO, pages 157--169. IEEE Computer Society, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. P. A. Kulkarni, D. B. Whalley, G. S. Tyson, and J. W. Davidson. Practical exhaustive optimization phase order exploration and evaluation. TACO, 6(1), 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. X. Li, M. J. Garzarán, and D. Padua. Optimizing Sorting with Genetic Algorithms. In In Proc. of the International Symposium on Code Generation and Optimization (CGO), pages 99--110, March 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. C. Liao, D. J. Quinlan, R. W. Vuduc, and T. Panas. Effective source-to-source outlining to support whole program empirical optimization. In LCPC'09, pages 308--322, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Monsifrot, F. Bodin, and R. Quiniou. A machine learning approach to automatic production of compiler heuristics. In AIMSA '02: Proceedings of the 10th International Conference on Artificial Intelligence:Methodology, Systems, and Applications, pages 41--50, London, UK, 2002. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. M. Püschel, J. Moura, J. Johnson, D. Padua, M. Veloso, B. Singer, J. Xiong, F. Franchetti, A. Gacic, Y. Voronenko, K. Chen, R. W. Johnson, and N. Rizzolo. SPIRAL: Code Generation for DSP Transforms. In Proc. of the IEEE, special issue on Program Generation, Optimization, and Platform Adaptation, 93(2):232--275, February 2005.Google ScholarGoogle Scholar
  22. L. A. Smith, J. M. Bull, and J. Obdrzálek. A parallel Java Grande benchmark suite. In ACM, editor, SC2001: High Performance Networking and Computing. Denver, CO, November 10--16, 2001. ACM Press and IEEE Computer Society Press, 2001. ISBN 1-58113-293-X. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. K. O. Stanley and R. Miikkulainen. Efficient reinforcement learning through evolving neural network topologies. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2002), page 9, San Francisco, 2002. Morgan Kaufmann. URL http://nn.cs.utexas.edu/?stanley:gecco02b. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Stephenson and S. Amarasinghe. Predicting Unroll Factors Using Supervised Classification. In CGO '05: Proceedings of the international symposium on Code generation and optimization, pages 123--134, Washington, DC, USA, 2005. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. M. Stephenson, S. Amarasinghe, M. Martin, and U.-M. O'Reilly. Meta Optimization: Improving Compiler Heuristics with Machine Learning. In Proc. of Programing Language Design and Implementation, June 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. R. Vuduc, J. W. Demmel, and K. A. Yelick. OSKI: A Library of Automatically Tuned Sparse Matrix Kernels. Journal of Physics Conference Series, 16:521--530, Jan. 2005.Google ScholarGoogle ScholarCross RefCross Ref
  27. Website. Specjvm 2008. URL http://www.spec.org/jvm2008/.Google ScholarGoogle Scholar
  28. Website. Specjvm 98,. URL http://www.spec.org/jvm98/.Google ScholarGoogle Scholar
  29. Website. Java grande benchmarks,. URL http://www2.epcc.ed.ac.uk/computing/research activities/java grande/.Google ScholarGoogle Scholar
  30. R. C. Whaley, A. Petitet, and J. J. Dongarra. Automated Empirical Optimizations of Software and the ATLAS Project. Parallel Computing, 27(1-2):3--35, 2001.Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. K. Yotov, X. Li, G. Ren, M. Cibulskis, G. DeJong, M. Garzaran, D. Padua, K. Pingali, P. Stodghill, and P. Wu. A Comparison of Empirical and Model-driven Optimization. In Proc. of Programing Language Design and Implementation, pages 63--76, June 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Mitigating the compiler optimization phase-ordering problem using machine learning

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!