ABSTRACT
Determining the best set of optimizations to apply to a program has been a long standing problem for compiler writers. To reduce the complexity of this task, existing approaches typically apply the same set of optimizations to all procedures within a program, without regard to their particular structure. This paper develops a new method-specific approach that automatically selects the best optimizations on a per method basis within a dynamic compiler. Our approach uses the machine learning technique of logistic regression to automatically derive a predictive model that determines which optimizations to apply based on the features of a method. This technique is implemented in the Jikes RVM Java JIT compiler. Using this approach we reduce the average total execution time of the SPECjvm98 benchmarks by 29%. When the same heuristic is applied to the DaCapo+ benchmark suite, we obtain an average 33% reduction over the default level O2 setting.
- F. Agakov, E. Bonilla, J. Cavazos, B. Franke, G. Fursin, M. F. P. O'Boyle, J. Thomson, M. Toussiant, and C. K. I. Williams. Using machine learning to focus iterative optimization. In Fourth Annual IEEE/ACM Interational Conference on Code Generation and Optimization, pages 295--305, New York City, NY, March 2006. Google Scholar
Digital Library
- L. Almagor, K. Cooper, A. Grosul, T. Harvey, S. Reeves, D. Subramanian, L. Torczon, and T. Waterman. Finding effective compilation sequences. In Proceedings of the Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES), pages 231--239, 2004. Google Scholar
Digital Library
- B. Alpern, C. R. Attanasio, J. J. Barton, M. G. Burke, P.Cheng, J.-D.Choi, A. Cocchi, S. J. Fink, D. Grove, M. Hind, S. F. Hummel, D. Lieber, V. Litvinov, M. F. Mergen, T. Ngo, J. R. Russell, V. Sarkar, M. J. Serrano, J. C. Shepherd, S. E. Smith, V. C. Sreedhar, H. Srinivasan, and J. Whaley. The Jalapeño virtual machine. IBM Systems Journal, 39(1):211--238, Feb. 2000. Google Scholar
Digital Library
- C. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 2005. Google Scholar
Digital Library
- S. M. Blackburn, R. Garner, C. Hoffman, A. M. Khan, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, A. Phansalkar, D. Stefanović, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-Oriented Programing, Systems, Languages, and Applications, New York, NY, USA, Oct. 2006. ACM Press Google Scholar
Digital Library
- B. Calder, D. Grunwald, M. Jones, D. Lindsay, J. Martin, M. Mozer, and B. Zoren. Evidence-based static branch prediction using machine learning. ACM Transactions on Programming Languages and Systems, 19(1):188--222, January 1997. Google Scholar
Digital Library
- J. Cavazos and J. E. B. Moss. Inducing heuristics to decide whether to schedule. In Proceedings of the ACM SIGPLAN '04 Conference on Programming Language Design and Implementation, pages 183--194, Washington, D.C., June 2004. ACM Press. Google Scholar
Digital Library
- K. D. Cooper, A. Grosul, T. Harvey, S. Reeves, D. Subramanian, L. Torczon, and T. Waterman. Acme: adaptive compilation made efficient. In Proceedings of the Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES), 2005. Google Scholar
Digital Library
- K. D. Cooper, A. Grosul, T. Harvey, S. Reeves, D. Subramanian, L. Torczon, and T. Waterman. Searching for compilation sequences. Tech. report, Rice University, 2005.Google Scholar
- B. Franke, M. O'Boyle, J. Thomson, and G. Fursin. Probabilistic source-level optimisation of embedded programs. In Proceedings of the Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES), 2005. Google Scholar
Digital Library
- A. Georges, D. Buytaert, L. Eeckhout, and K. D. Bosschere. Method-level phase behavior in java workloads. In OOPSLA '04: Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, pages 270--287, New York, NY, USA, 2004. ACM Press. Google Scholar
Digital Library
- P. A. Kulkarni, D. B. Whalley, G. S. Tyson, and J. W. Davidson. Exhaustive optimization phase order space exploration. In Fourth Annual IEEE/ACM Interational Conference on Code Generation and Optimization, pages 306--318, New York City, NY, March 2006. Google Scholar
Digital Library
- M. G. Lagoudakis and M. L. Littman. Algorithm selection using reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning, pages 511--518, Stanford, CA, June 2000. Morgan Kaufmann. Google Scholar
Digital Library
- J. Lau, M. Arnold, M. Hind, and B. Calder. Online performance auditing: using hot optimizations without getting burned. In Proceedings of the ACM SIGPLAN '06 Conference on Programming Language Design and Implementation, pages 239--251, Washington, D.C., June 2006. ACM Press. Google Scholar
Digital Library
- A. Monsifrot and F. Bodin. A machine learning approach to automatic production of compiler heuristics. In Tenth International Conference on Artificial Intelligence: Methodology, Systems, Applications (AIMSA), pages 41--50, Varna, Bulgaria, September 2002. Springer Verlag. Google Scholar
Digital Library
- J. E. B. Moss, P. E. Utgoff, J. Cavazos, D. Precup, D. Stefanović, C. Brodley, and D. Scheeff. Learning to schedule straight-line code. In Proceedings of Neural Information Processing Systems 1997 (NIPS*97), Denver CO, Dec. 1997. Google Scholar
Digital Library
- Z. Pan and R. Eigenmann. Fast automatic procedure-level performance tuning. In IEEE PACT, Seattle, WA, September 2006. IEEE Computer Society. Google Scholar
Digital Library
- R. Pinkers, P. Knijnenburg, M. Haneda, and H. Wijshoff. Statistical selection of compiler options. In Proceedings of the IEEE International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS), pages 494--501, 2004. Google Scholar
Digital Library
- Standard Performance Evaluation Corporation (SPEC), Fairfax, VA. SPEC JVM98 Benchmarks, 1998.Google Scholar
- M. Stephenson, S. Amarasinghe, M. Martin, and U.-M. O'Reilly. Meta optimization: Improving compiler heuristics with machine learning. In Proceedings of the ACM SIGPLAN '03 Conference on Programming Language Design and Implementation, pages 77--90, San Diego, Ca, June 2003. ACM Press. Google Scholar
Digital Library
- M. Stephenson and S. P. Amarasinghe. Predicting unroll factors using supervised classification. In Third Annual IEEE /ACM Interational Conference on Code Generation and Optimization, pages 123--134, 2005. Google Scholar
Digital Library
- S. Triantafyllis, M. Vachharajani, N. Vachharajani, and D. August. Compiler optimization-space exploration. In First Annual IEEE/ACM Interational Conference on Code Generation and Optimization, pages 204--215, 2003. Google Scholar
Digital Library
- K. Yotov, X. Li, G. Ren, M. Cibulskis, G. DeJong, M. Garzaran, D. Padua, K. Pingali, P. Stodghill, and P. Wu. A comparison of empirical and model-driven optimization. In Proceedings of the ACM SIGPLAN '03 Conference on Programming Language Design and Implementation, pages 63--76, San Diego, Ca, June 2003. ACM Press. Google Scholar
Digital Library
- M. Zhao, B. R. Childers, and M. L. Soffa. A model-based framework: an approach for profit-driven optimization. In Third Annual IEEE/ACM Interational Conference on Code Generation and Optimization, pages 317--327, 2005. Google Scholar
Digital Library
Index Terms
Method-specific dynamic compilation using logistic regression
Recommendations
Method-specific dynamic compilation using logistic regression
Proceedings of the 2006 OOPSLA ConferenceDetermining the best set of optimizations to apply to a program has been a long standing problem for compiler writers. To reduce the complexity of this task, existing approaches typically apply the same set of optimizations to all procedures within a ...
Inducing heuristics to decide whether to schedule
PLDI '04: Proceedings of the ACM SIGPLAN 2004 conference on Programming language design and implementationInstruction scheduling is a compiler optimization that can improve program speed, sometimes by 10% or more, but it can also be expensive. Furthermore, time spent optimizing is more important in a Java just-in-time (JIT) compiler than in a traditional ...
Logistic regression diagnostics in ridge regression
The adverse effects of multicollinearity and unusual observations are seen in logistic regression and attention had been given in the literature to each of these problems separately. However, multicollinearity and unusual observations can arise ...







Comments