Abstract
The compiled native code generated by a just-in-time (JIT) compiler in managed language virtual machines (VM) is placed in a region of memory called the code cache. Code cache management (CCM) in a VM is responsible to find and evict methods from the code cache to maintain execution correctness and manage program performance for a given code cache size or memory budget. Effective CCM can also boost program speed by enabling more aggressive JIT compilation, powerful optimizations, and improved hardware instruction cache and I-TLB performance. Though important, CCM is an overlooked component in VMs. We find that the default CCM policies in Oracle’s production-grade HotSpot VM perform poorly even at modest memory pressure. We develop a detailed simulation-based framework to model and evaluate the potential efficiency of many different CCM policies in a controlled and realistic, but VM-independent environment. We make the encouraging discovery that effective CCM policies can sustain high program performance even for very small cache sizes. Our simulation study provides the rationale and motivation to improve CCM strategies in existing VMs. We implement and study the properties of several CCM policies in HotSpot. We find that in spite of working within the bounds of the HotSpot VM’s current CCM sub-system, our best CCM policy implementation in HotSpot improves program performance over the default CCM algorithm by 39%, 41%, 55%, and 50% with code cache sizes that are 90%, 75%, 50%, and 25% of the desired cache size, on average.
- B. Alpern, C. R. Attanasio, A. Cocchi, D. Lieber, S. Smith, T. Ngo, J. J. Barton, S. F. Hummel, J. C. Sheperd, and M. Mergen. Implementing Jalape ˜ No in Java. In Proceedings of the 14th ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’99, pages 314–324, 1999. Google Scholar
Digital Library
- M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. Adaptive optimization in the Jalape ˜ No JVM. In Proceedings of the 15th ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’00, pages 47–65, 2000. Google Scholar
Digital Library
- J. A. Baiocchi and B. R. Childers. Heterogeneous code cache: Using scratchpad and main memory in dynamic binary translators. In Proceedings of the 46th Annual Design Automation Conference, DAC ’09, pages 744–749, 2009. Google Scholar
Digital Library
- J. A. Baiocchi, B. R. Childers, J. W. Davidson, and J. D. Hiser. Reducing pressure in bounded DBT code caches. In Proceedings of the 2008 International Conference on Compilers, Architectures and Synthesis for Embedded Systems, CASES ’08, pages 109–118, 2008. Google Scholar
Digital Library
- V. Bala, E. Duesterwald, and S. Banerjia. Dynamo: A transparent dynamic optimization system. In Proceedings of the ACM SIGPLAN 2000 Conference on Programming Language Design and Implementation, PLDI ’00, pages 1–12, 2000. Google Scholar
Digital Library
- S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, B. Moss, A. Phansalkar, D. Stefanovi´c, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, OOPSLA ’06, pages 169–190, 2006. Google Scholar
Digital Library
- D. Bruening, T. Garnett, and S. Amarasinghe. An infrastructure for adaptive dynamic optimization. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization, CGO ’03, pages 265–275, 2003. Google Scholar
Digital Library
- D. Bruening, V. Kiriansky, T. Garnett, and S. Banerji. Thread-shared software code caches. In Proceedings of the International Symposium on Code Generation and Optimization, CGO ’06, pages 28–38, 2006. Google Scholar
Digital Library
- A. Georges, D. Buytaert, and L. Eeckhout. Statistically rigorous java performance evaluation. In Proceedings of the conference on Objectoriented programming systems and applications, OOPSLA ’07, pages 57–76, 2007. Google Scholar
Digital Library
- Google. Chrome V8 JavaScript VM, September 2012.Google Scholar
- https://developers.google.com/v8/intro.Google Scholar
- A. Guha, K. Hazelwood, and M. Soffa. Balancing memory and performance through selective flushing of software code caches. In Proceedings of the 2010 International Conference on Compilers, Architectures and Synthesis for Embedded Systems, CASES ’10, pages 1–10, 2010. Google Scholar
Digital Library
- T. Hartmann, A. Noll, and T. Gross. Efficient code management for dynamic multi-tiered compilation systems. In Proceedings of the 2014 International Conference on Principles and Practices of Programming on the Java Platform: Virtual Machines, Languages, and Tools, PPPJ ’14, pages 51–62, 2014. Google Scholar
Digital Library
- K. Hazelwood and J. E. Smith. Exploring code cache eviction granularities in dynamic optimization systems. In Proceedings of the International Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization, CGO ’04, pages 89–99, 2004. Google Scholar
Digital Library
- K. Hazelwood and M. D. Smith. Managing bounded code caches in dynamic binary optimization systems. ACM Transactions on Architecture and Code Optimization, 3(3):263–294, Sept. 2006. ISSN 1544- 3566. Google Scholar
Digital Library
- K. Hazelwood, G. Lueck, and R. Cohn. Scalable support for multithreaded applications on dynamic binary instrumentation systems. In Proceedings of the 2009 International Symposium on Memory Management, ISMM ’09, pages 20–29, 2009. Google Scholar
Digital Library
- U. Hölzle and D. Ungar. Reconciling responsiveness with performance in pure object-oriented languages. ACM Transactions on Programming Language Systems, 18(4):355–400, 1996. ISSN 0164-0925. Google Scholar
Digital Library
- X. Huang, S. M. Blackburn, K. S. McKinley, J. E. B. Moss, Z. Wang, and P. Cheng. The garbage collection advantage: Improving program locality. In Proceedings of the 19th Annual ACM SIGPLAN Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’04, pages 69–80, 2004. Google Scholar
Digital Library
- H. Inoue, H. Hayashizaki, P. Wu, and T. Nakatani. A trace-based Java JIT compiler retrofitted from a method-based compiler. In Proceedings of the 9th Annual IEEE/ACM International Symposium on Code Generation and Optimization, CGO ’11, pages 246–256, 2011. Google Scholar
Digital Library
- M. R. Jantz and P. A. Kulkarni. Exploring single and multilevel JIT compilation policy for modern machines. ACM Transactions on Architecture and Code Optimization, 10(4):22:1–22:29, Dec. 2013. Google Scholar
Digital Library
- T. Kotzmann, C. Wimmer, H. Mössenböck, T. Rodriguez, K. Russell, and D. Cox. Design of the Java HotSpot™client compiler for Java 6. ACM Trans. Archit. Code Optim., 5(1), 2008. Google Scholar
Digital Library
- C. Krintz, D. Grove, V. Sarkar, and B. Calder. Reducing the overhead of dynamic compilation. Software: Practice and Experience, 31(8): 717–738, December 2000.Google Scholar
Cross Ref
- M. Paleczny, C. Vick, and C. Click. The Java HotSpot™server compiler. In JVM’01: Proceedings of the 2001 Symposium on JavaTM Virtual Machine Research and Technology Symposium, pages 1–12, Berkeley, CA, USA, 2001. Google Scholar
Digital Library
- J. Smith and R. Nair. Virtual Machines: Versatile Platforms for Systems and Processes. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2005. ISBN 1558609105. Google Scholar
Digital Library
- C. Wimmer, M. Haupt, M. L. Van De Vanter, M. Jordan, L. Daynès, and D. Simon. Maxine: An approachable virtual machine for, and in, Java. ACM Transactions on Architecture and Code Optimization, 9(4): 30:1–30:24, Jan. 2013. ISSN 1544-3566. Google Scholar
Digital Library
- L. Zhang and C. Krintz. Adaptive code unloading for resourceconstrained JVMs. In Proceedings of the 2004 ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems, LCTES ’04, pages 155–164, 2004. Google Scholar
Digital Library
- L. Zhang and C. Krintz. Profile-driven code unloading for resourceconstrained JVMs. In Proceedings of the 3rd International Symposium on Principles and Practice of Programming in Java, PPPJ ’04, pages 83–90, 2004. Google Scholar
Digital Library
Index Terms
Code cache management in managed language VMs to reduce memory consumption for embedded systems
Recommendations
Code cache management in managed language VMs to reduce memory consumption for embedded systems
LCTES 2016: Proceedings of the 17th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, Tools, and Theory for Embedded SystemsThe compiled native code generated by a just-in-time (JIT) compiler in managed language virtual machines (VM) is placed in a region of memory called the code cache. Code cache management (CCM) in a VM is responsible to find and evict methods from the ...







Comments