skip to main content
research-article

A modular approach to on-stack replacement in LLVM

Published:16 March 2013Publication History
Skip Abstract Section

Abstract

On-stack replacement (OSR) is a technique that allows a virtual machine to interrupt running code during the execution of a function/method, to re-optimize the function on-the-fly using an optimizing JIT compiler, and then to resume the interrupted function at the point and state at which it was interrupted. OSR is particularly useful for programs with potentially long-running loops, as it allows dynamic optimization of those loops as soon as they become hot.

This paper presents a modular approach to implementing OSR for the LLVM compiler infrastructure. This is an important step forward because LLVM is gaining popular support, and adding the OSR capability allows compiler developers to develop new dynamic techniques. In particular, it will enable more sophisticated LLVM-based JIT compiler approaches. Indeed, other compiler/VM developers can use our approach because it is a clean modular addition to the standard LLVM distribution. Further, our approach is defined completely at the LLVM-IR level and thus does not require any modifications to the target code generation.

The OSR implementation can be used by different compilers to support a variety of dynamic optimizations. As a demonstration of our OSR approach, we have used it to support dynamic inlining in McVM. McVM is a virtual machine for MATLAB which uses a LLVM-based JIT compiler. MATLAB is a popular dynamic language for scientific and engineering applications that typically manipulate large matrices and often contain long-running loops, and is thus an ideal target for dynamic JIT compilation and OSRs. Using our McVM example, we demonstrate reasonable overheads for our benchmark set, and performance improvements when using it to perform dynamic inlining.

References

  1. LLVM. http://www.llvm.org/.Google ScholarGoogle Scholar
  2. A. Adl-Tabatabai, J. Bharadwaj, D. Chen, A. Ghuloum, V. Menon, B. Murphy, M. Serrano, and T. Shpeisman. StarJIT: A Dynamic Compiler for Managed Runtime Environments. Intel Technology Journal, 7(1):19--31, Feb 2003.Google ScholarGoogle Scholar
  3. B. Alpern, S. Augart, S. M. Blackburn, M. Butrico, A. Cocchi, P. Cheng, J. Dolby, S. Fink, D. Grove, M. Hind, K. S. McKinley, M. Mergen, J. E. B. Moss, T. Ngo, and V. Sarkar. The Jikes Research Virtual Machine Project: Building an Open-Source Research Community. IBM Syst. J., 44(2):399--417, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. Adaptive Optimization in the Jalapeno JVM. In Proceedings of the 15th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA '00, pages 47--65, New York, USA, 2000. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. M. Arnold, M. Hind, and B. G. Ryder. Online Feedback-Directed Optimization of Java. In Proceedings of the 17th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, OOPSLA '02, pages 111--129, New York, USA, 2002. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C. Chambers and D. Ungar. Making Pure Object-Oriented Languages Practical. In Conference Proceedings on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA '91, pages 1--15, New York, USA, 1991. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. M. Chevalier-Boisvert, L. Hendren, and C. Verbrugge. Optimizing MATLAB through Just-In-Time Specialization. In International Conference on Compiler Construction, pages 46--65, March 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. M. Cierniak, G.-Y. Lueh, and J. M. Stichnoth. Practicing JUDO: Java Under Dynamic Optimizations. In Proceedings of the ACMSIGPLAN 2000 Conference on Programming Language Design and Implementation, PLDI '00, pages 13--26, New York, USA, 2000. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Cleve Moler. Numerical Computing with MATLAB. SIAM, 2004.Google ScholarGoogle Scholar
  10. S. J. Fink and F. Qian. Design, Implementation and Evaluation of Adaptive Recompilation with On-stack Replacement. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization, CGO '03, pages 241--252, Washington, DC, USA, 2003. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. K. Hazelwood and D. Grove. Adaptive Online Context-Sensitive Inlining. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-Directed and Runtime Optimization, CGO '03, pages 253--264, Washington, DC, USA, 2003. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. U. Hölzle, C. Chambers, and D. Ungar. Debugging Optimized Code with Dynamic Deoptimization. In Proceedings of the ACM SIGPLAN 1992 Conference on Programming Language Design and Implementation, PLDI '92, pages 32--43, New York, NY, USA, 1992. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. U. Hölzle and D. Ungar. A Third-Generation SELF Implementation: Reconciling Responsiveness with Performance. In Proceedings of the Ninth Annual Conference on Object-Oriented Programming Systems, Language, and Applications, OOPSLA '94, pages 229--243, New York, NY, USA, 1994. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. C. Lattner and V. Adve. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In CGO '04: Proceedings of the International Symposium on Code Generation and Optimization, pages 75--86, Washington, DC, USA, 2004. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. MathWorks. MATLAB Programming Fundamentals. The Math-Works, Inc., 2009.Google ScholarGoogle Scholar
  16. McLAB. The McVM Virtual Machine and its JIT Compiler, 2012. http://www.sable.mcgill.ca/mclab/mcvm_mcjit.html.Google ScholarGoogle Scholar
  17. C. Moler. The Growth of MATLAB"and TheMathWorks over Two Decades, 2006. http://www.mathworks.com/company/newsletters/news_notes/clevescorner/jan06.pdf.Google ScholarGoogle Scholar
  18. M. Paleczny, C. Vick, and C. Click. The Java HotSpot Server Compiler. In Proceedings of the 2001 Symposium on JavaTM Virtual Machine Research and Technology Symposium - Volume 1, JVM'01, pages 1--12, Berkeley, CA, USA, 2001. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Press, H. William and Teukolsky, A. Saul and Vetterling, T. William and Flannery, P. Brian. Numerical Recipes : the Art of Scientific Computing. Cambridge University Press, 1986. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. L. D. Rose, K. Gallivan, E. Gallopoulos, B. A. Marsolf, and D. A. Padua. FALCON: A MATLAB Interactive Restructuring Compiler. In LCPC '95: Proceedings of the 8th International Workshop on Languages and Compilers for Parallel Computing, pages 269--288, London, UK, 1996. Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. S. Soman and C. Krintz. Efficient and General On-Stack Replacement for Aggressive Program Specialization. In Software Engineering Research and Practice, pages 925--932, 2006.Google ScholarGoogle Scholar
  22. T. Suganuma, T. Yasue, and T. Nakatani. An Empirical Study of Method In-lining for a Java Just-In-Time Compiler. In Proceedings of the 2nd Java Virtual Machine Research and Technology Symposium, pages 91--104, Berkeley, CA, USA, 2002. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. Süsskraut, T. Knauth, S. Weigert, U. Schiffel, M. Meinhold, and C. Fetzer. Prospect: A Compiler Framework for Speculative Parallelization. In Proceedings of the 8th Annual IEEE/ACM International Symposium on Code Generation and Optimization, pages 131--140, New York, USA, 2010. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A modular approach to on-stack replacement in LLVM

    Recommendations

    Reviews

    William M. Waite

    On-stack replacement is the process of interrupting a running program, optimizing some portion of that program, and then continuing execution from the point of interruption. It dynamically implements a mixed-code approach [1], in which part of the program is compact and slow, and part is expanded and fast. This approach takes advantage of the observation that programs tend to spend most of their time in a tiny fraction of their code. The authors of this paper demonstrate an implementation of on-stack replacement in the context of LLVM, a virtual machine whose instruction set is close to that of conventional hardware. Their explanation of the implementation is clear, with well-chosen flow graphs and code snippets. Given their goals, however, the implementation is complex and therefore the description demands careful attention. A reader will need to be familiar with the basic concepts of code generation and optimization to understand this paper. The implementation was evaluated on 16 programs, by measuring the differences in average running time with and without replacements. Six of the programs never actually performed a replacement, so these cases measured the overhead imposed by the technique (one percent or less). Of the remaining ten programs, three showed improvements of one to four percent and one was sped up by a factor of two. This indicates that the implementation is not particularly costly, and can sometimes provide a significant gain. Although the authors' description follows their implementation, their technique is quite general. It's easy to imagine using the same approach in a wide variety of compilation or execution contexts. Online Computing Reviews Service

    Access critical reviews of Computing literature here

    Become a reviewer for Computing Reviews.

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 48, Issue 7
      VEE '13
      July 2013
      194 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2517326
      Issue’s Table of Contents
      • cover image ACM Conferences
        VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
        March 2013
        210 pages
        ISBN:9781450312660
        DOI:10.1145/2451512

      Copyright © 2013 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 16 March 2013

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!