Abstract
On-stack replacement (OSR) is a technique that allows a virtual machine to interrupt running code during the execution of a function/method, to re-optimize the function on-the-fly using an optimizing JIT compiler, and then to resume the interrupted function at the point and state at which it was interrupted. OSR is particularly useful for programs with potentially long-running loops, as it allows dynamic optimization of those loops as soon as they become hot.
This paper presents a modular approach to implementing OSR for the LLVM compiler infrastructure. This is an important step forward because LLVM is gaining popular support, and adding the OSR capability allows compiler developers to develop new dynamic techniques. In particular, it will enable more sophisticated LLVM-based JIT compiler approaches. Indeed, other compiler/VM developers can use our approach because it is a clean modular addition to the standard LLVM distribution. Further, our approach is defined completely at the LLVM-IR level and thus does not require any modifications to the target code generation.
The OSR implementation can be used by different compilers to support a variety of dynamic optimizations. As a demonstration of our OSR approach, we have used it to support dynamic inlining in McVM. McVM is a virtual machine for MATLAB which uses a LLVM-based JIT compiler. MATLAB is a popular dynamic language for scientific and engineering applications that typically manipulate large matrices and often contain long-running loops, and is thus an ideal target for dynamic JIT compilation and OSRs. Using our McVM example, we demonstrate reasonable overheads for our benchmark set, and performance improvements when using it to perform dynamic inlining.
- LLVM. http://www.llvm.org/.Google Scholar
- A. Adl-Tabatabai, J. Bharadwaj, D. Chen, A. Ghuloum, V. Menon, B. Murphy, M. Serrano, and T. Shpeisman. StarJIT: A Dynamic Compiler for Managed Runtime Environments. Intel Technology Journal, 7(1):19--31, Feb 2003.Google Scholar
- B. Alpern, S. Augart, S. M. Blackburn, M. Butrico, A. Cocchi, P. Cheng, J. Dolby, S. Fink, D. Grove, M. Hind, K. S. McKinley, M. Mergen, J. E. B. Moss, T. Ngo, and V. Sarkar. The Jikes Research Virtual Machine Project: Building an Open-Source Research Community. IBM Syst. J., 44(2):399--417, 2005. Google Scholar
Digital Library
- M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. Adaptive Optimization in the Jalapeno JVM. In Proceedings of the 15th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA '00, pages 47--65, New York, USA, 2000. ACM. Google Scholar
Digital Library
- M. Arnold, M. Hind, and B. G. Ryder. Online Feedback-Directed Optimization of Java. In Proceedings of the 17th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA '02, pages 111--129, New York, USA, 2002. ACM. Google Scholar
Digital Library
- C. Chambers and D. Ungar. Making Pure Object-Oriented Languages Practical. In Conference Proceedings on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA '91, pages 1--15, New York, USA, 1991. ACM. Google Scholar
Digital Library
- M. Chevalier-Boisvert, L. Hendren, and C. Verbrugge. Optimizing MATLAB through Just-In-Time Specialization. In International Conference on Compiler Construction, pages 46--65, March 2010. Google Scholar
Digital Library
- M. Cierniak, G.-Y. Lueh, and J. M. Stichnoth. Practicing JUDO: Java Under Dynamic Optimizations. In Proceedings of the ACMSIGPLAN 2000 Conference on Programming Language Design and Implementation, PLDI '00, pages 13--26, New York, USA, 2000. ACM. Google Scholar
Digital Library
- Cleve Moler. Numerical Computing with MATLAB. SIAM, 2004.Google Scholar
- S. J. Fink and F. Qian. Design, Implementation and Evaluation of Adaptive Recompilation with On-stack Replacement. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization, CGO '03, pages 241--252, Washington, DC, USA, 2003. IEEE Computer Society. Google Scholar
Digital Library
- K. Hazelwood and D. Grove. Adaptive Online Context-Sensitive Inlining. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization, CGO '03, pages 253--264, Washington, DC, USA, 2003. IEEE Computer Society. Google Scholar
Digital Library
- U. Hölzle, C. Chambers, and D. Ungar. Debugging Optimized Code with Dynamic Deoptimization. In Proceedings of the ACM SIGPLAN 1992 Conference on Programming Language Design and Implementation, PLDI '92, pages 32--43, New York, NY, USA, 1992. ACM. Google Scholar
Digital Library
- U. Hölzle and D. Ungar. A Third-Generation SELF Implementation: Reconciling Responsiveness with Performance. In Proceedings of the Ninth Annual Conference on Object-Oriented Programming Systems, Language, and Applications, OOPSLA '94, pages 229--243, New York, NY, USA, 1994. ACM. Google Scholar
Digital Library
- C. Lattner and V. Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In CGO '04: Proceedings of the International Symposium on Code Generation and Optimization, pages 75--86, Washington, DC, USA, 2004. IEEE Computer Society. Google Scholar
Digital Library
- MathWorks. MATLAB Programming Fundamentals. The Math-Works, Inc., 2009.Google Scholar
- McLAB. The McVM Virtual Machine and its JIT Compiler, 2012. http://www.sable.mcgill.ca/mclab/mcvm_mcjit.html.Google Scholar
- C. Moler. The Growth of MATLAB"and TheMathWorks over Two Decades, 2006. http://www.mathworks.com/company/newsletters/news_notes/clevescorner/jan06.pdf.Google Scholar
- M. Paleczny, C. Vick, and C. Click. The Java HotSpot Server Compiler. In Proceedings of the 2001 Symposium on JavaTM Virtual Machine Research and Technology Symposium - Volume 1, JVM'01, pages 1--12, Berkeley, CA, USA, 2001. USENIX Association. Google Scholar
Digital Library
- Press, H. William and Teukolsky, A. Saul and Vetterling, T. William and Flannery, P. Brian. Numerical Recipes : the Art of Scientific Computing. Cambridge University Press, 1986. Google Scholar
Digital Library
- L. D. Rose, K. Gallivan, E. Gallopoulos, B. A. Marsolf, and D. A. Padua. FALCON: A MATLAB Interactive Restructuring Compiler. In LCPC '95: Proceedings of the 8th International Workshop on Languages and Compilers for Parallel Computing, pages 269--288, London, UK, 1996. Springer-Verlag. Google Scholar
Digital Library
- S. Soman and C. Krintz. Efficient and General On-Stack Replacement for Aggressive Program Specialization. In Software Engineering Research and Practice, pages 925--932, 2006.Google Scholar
- T. Suganuma, T. Yasue, and T. Nakatani. An Empirical Study of Method In-lining for a Java Just-In-Time Compiler. In Proceedings of the 2nd Java Virtual Machine Research and Technology Symposium, pages 91--104, Berkeley, CA, USA, 2002. USENIX Association. Google Scholar
Digital Library
- M. Süsskraut, T. Knauth, S. Weigert, U. Schiffel, M. Meinhold, and C. Fetzer. Prospect: A Compiler Framework for Speculative Parallelization. In Proceedings of the 8th Annual IEEE/ACM International Symposium on Code Generation and Optimization, pages 131--140, New York, USA, 2010. ACM. Google Scholar
Digital Library
Index Terms
A modular approach to on-stack replacement in LLVM
Recommendations
Flexible on-stack replacement in LLVM
CGO '16: Proceedings of the 2016 International Symposium on Code Generation and OptimizationOn-Stack Replacement (OSR) is a technique for dynamically transferring execution between different versions of a function at run time. OSR is typically used in virtual machines to interrupt a long-running function and recompile it at a higher ...
A modular approach to on-stack replacement in LLVM
VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environmentsOn-stack replacement (OSR) is a technique that allows a virtual machine to interrupt running code during the execution of a function/method, to re-optimize the function on-the-fly using an optimizing JIT compiler, and then to resume the interrupted ...
Optimizing MATLAB feval with dynamic techniques
DLS '13: Proceedings of the 9th symposium on Dynamic languagesMATLAB is a popular dynamic array-based language used by engineers, scientists and students worldwide. The built-in function feval is an important MATLAB feature for certain classes of numerical programs and solvers which benefit from having functions ...









Comments