skip to main content
article
Public Access

Code cache management in managed language VMs to reduce memory consumption for embedded systems

Published:13 June 2016Publication History
Skip Abstract Section

Abstract

The compiled native code generated by a just-in-time (JIT) compiler in managed language virtual machines (VM) is placed in a region of memory called the code cache. Code cache management (CCM) in a VM is responsible to find and evict methods from the code cache to maintain execution correctness and manage program performance for a given code cache size or memory budget. Effective CCM can also boost program speed by enabling more aggressive JIT compilation, powerful optimizations, and improved hardware instruction cache and I-TLB performance. Though important, CCM is an overlooked component in VMs. We find that the default CCM policies in Oracle’s production-grade HotSpot VM perform poorly even at modest memory pressure. We develop a detailed simulation-based framework to model and evaluate the potential efficiency of many different CCM policies in a controlled and realistic, but VM-independent environment. We make the encouraging discovery that effective CCM policies can sustain high program performance even for very small cache sizes. Our simulation study provides the rationale and motivation to improve CCM strategies in existing VMs. We implement and study the properties of several CCM policies in HotSpot. We find that in spite of working within the bounds of the HotSpot VM’s current CCM sub-system, our best CCM policy implementation in HotSpot improves program performance over the default CCM algorithm by 39%, 41%, 55%, and 50% with code cache sizes that are 90%, 75%, 50%, and 25% of the desired cache size, on average.

References

  1. B. Alpern, C. R. Attanasio, A. Cocchi, D. Lieber, S. Smith, T. Ngo, J. J. Barton, S. F. Hummel, J. C. Sheperd, and M. Mergen. Implementing Jalape ˜ No in Java. In Proceedings of the 14th ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’99, pages 314–324, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. Adaptive optimization in the Jalape ˜ No JVM. In Proceedings of the 15th ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’00, pages 47–65, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. J. A. Baiocchi and B. R. Childers. Heterogeneous code cache: Using scratchpad and main memory in dynamic binary translators. In Proceedings of the 46th Annual Design Automation Conference, DAC ’09, pages 744–749, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. J. A. Baiocchi, B. R. Childers, J. W. Davidson, and J. D. Hiser. Reducing pressure in bounded DBT code caches. In Proceedings of the 2008 International Conference on Compilers, Architectures and Synthesis for Embedded Systems, CASES ’08, pages 109–118, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. V. Bala, E. Duesterwald, and S. Banerjia. Dynamo: A transparent dynamic optimization system. In Proceedings of the ACM SIGPLAN 2000 Conference on Programming Language Design and Implementation, PLDI ’00, pages 1–12, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, B. Moss, A. Phansalkar, D. Stefanovi´c, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, OOPSLA ’06, pages 169–190, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. D. Bruening, T. Garnett, and S. Amarasinghe. An infrastructure for adaptive dynamic optimization. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization, CGO ’03, pages 265–275, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. D. Bruening, V. Kiriansky, T. Garnett, and S. Banerji. Thread-shared software code caches. In Proceedings of the International Symposium on Code Generation and Optimization, CGO ’06, pages 28–38, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. A. Georges, D. Buytaert, and L. Eeckhout. Statistically rigorous java performance evaluation. In Proceedings of the conference on Objectoriented programming systems and applications, OOPSLA ’07, pages 57–76, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Google. Chrome V8 JavaScript VM, September 2012.Google ScholarGoogle Scholar
  11. https://developers.google.com/v8/intro.Google ScholarGoogle Scholar
  12. A. Guha, K. Hazelwood, and M. Soffa. Balancing memory and performance through selective flushing of software code caches. In Proceedings of the 2010 International Conference on Compilers, Architectures and Synthesis for Embedded Systems, CASES ’10, pages 1–10, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. T. Hartmann, A. Noll, and T. Gross. Efficient code management for dynamic multi-tiered compilation systems. In Proceedings of the 2014 International Conference on Principles and Practices of Programming on the Java Platform: Virtual Machines, Languages, and Tools, PPPJ ’14, pages 51–62, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. K. Hazelwood and J. E. Smith. Exploring code cache eviction granularities in dynamic optimization systems. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization, CGO ’04, pages 89–99, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. K. Hazelwood and M. D. Smith. Managing bounded code caches in dynamic binary optimization systems. ACM Transactions on Architecture and Code Optimization, 3(3):263–294, Sept. 2006. ISSN 1544- 3566. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. K. Hazelwood, G. Lueck, and R. Cohn. Scalable support for multithreaded applications on dynamic binary instrumentation systems. In Proceedings of the 2009 International Symposium on Memory Management, ISMM ’09, pages 20–29, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. U. Hölzle and D. Ungar. Reconciling responsiveness with performance in pure object-oriented languages. ACM Transactions on Programming Language Systems, 18(4):355–400, 1996. ISSN 0164-0925. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. X. Huang, S. M. Blackburn, K. S. McKinley, J. E. B. Moss, Z. Wang, and P. Cheng. The garbage collection advantage: Improving program locality. In Proceedings of the 19th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’04, pages 69–80, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. H. Inoue, H. Hayashizaki, P. Wu, and T. Nakatani. A trace-based Java JIT compiler retrofitted from a method-based compiler. In Proceedings of the 9th Annual IEEE/ACM International Symposium on Code Generation and Optimization, CGO ’11, pages 246–256, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. R. Jantz and P. A. Kulkarni. Exploring single and multilevel JIT compilation policy for modern machines. ACM Transactions on Architecture and Code Optimization, 10(4):22:1–22:29, Dec. 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. T. Kotzmann, C. Wimmer, H. Mössenböck, T. Rodriguez, K. Russell, and D. Cox. Design of the Java HotSpot™client compiler for Java 6. ACM Trans. Archit. Code Optim., 5(1), 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. C. Krintz, D. Grove, V. Sarkar, and B. Calder. Reducing the overhead of dynamic compilation. Software: Practice and Experience, 31(8): 717–738, December 2000.Google ScholarGoogle ScholarCross RefCross Ref
  23. M. Paleczny, C. Vick, and C. Click. The Java HotSpot™server compiler. In JVM’01: Proceedings of the 2001 Symposium on JavaTM Virtual Machine Research and Technology Symposium, pages 1–12, Berkeley, CA, USA, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Smith and R. Nair. Virtual Machines: Versatile Platforms for Systems and Processes. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2005. ISBN 1558609105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. C. Wimmer, M. Haupt, M. L. Van De Vanter, M. Jordan, L. Daynès, and D. Simon. Maxine: An approachable virtual machine for, and in, Java. ACM Transactions on Architecture and Code Optimization, 9(4): 30:1–30:24, Jan. 2013. ISSN 1544-3566. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. L. Zhang and C. Krintz. Adaptive code unloading for resourceconstrained JVMs. In Proceedings of the 2004 ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems, LCTES ’04, pages 155–164, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. L. Zhang and C. Krintz. Profile-driven code unloading for resourceconstrained JVMs. In Proceedings of the 3rd International Symposium on Principles and Practice of Programming in Java, PPPJ ’04, pages 83–90, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Code cache management in managed language VMs to reduce memory consumption for embedded systems

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 51, Issue 5
        LCTES '16
        May 2016
        122 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/2980930
        • Editor:
        • Andy Gill
        Issue’s Table of Contents
        • cover image ACM Conferences
          LCTES 2016: Proceedings of the 17th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, Tools, and Theory for Embedded Systems
          June 2016
          122 pages
          ISBN:9781450343169
          DOI:10.1145/2907950

        Copyright © 2016 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 13 June 2016

        Check for updates

        Qualifiers

        • article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!