Abstract
Originally conceived as the target platform for Java alone, the Java Virtual Machine (JVM) has since been targeted by other languages, one of which is Scala. This trend, however, is not yet reflected by the benchmark suites commonly used in JVM research. In this paper, we thus present the design and analysis of the first full-fledged benchmark suite for Scala. We furthermore compare the benchmarks contained therein with those from the well-known DaCapo 9.12 benchmark suite and show where the differences are between Scala and Java code---and where not.
- B. Alpern, A. Cocchi, S. Fink, and D. Grove. Efficient implementation of Java interfaces: invoke interface considered harmless. In Proceedings of the 16th Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA), 2001. Google Scholar
Digital Library
- G. Ammons, T. Ball, and J. R. Larus. Exploiting hardware performance counters with flow and context sensitive profiling. In Proceedings of the 10th Conference on Programming Language Design and Implementation (PLDI), 1997. Google Scholar
Digital Library
- M. Bebenita, M. Chang, G. Wagner, A. Gal, C. Wimmer, and M. Franz. Trace-based compilation in execution environments without interpreters. In Proceedings of the 8th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2010. Google Scholar
Digital Library
- W. Binder. A portable and customizable profiling framework for Java based on bytecode instruction counting. In Proceedings of the 3rd Asian Symposium on Programming Languages and Systems (APLAS), 2005. Google Scholar
Digital Library
- W. Binder, J. Hulaas, and P. Moret. A quantitative evaluation of the contribution of native code to Java workloads. In Proceedings of the IEEE International Symposium on Workload Characterization (IISWC), 2006.Google Scholar
Cross Ref
- W. Binder, J. Hulaas, P. Moret, and A. Villazo ?n. Platform-independent profiling in a virtual execution environment. Software: Practice and Experience, 39(1):47--79, 2009. Google Scholar
Digital Library
- S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J.E.B.Moss,B.Moss,A.Phansalkar,D.Stefanovic, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA), 2006. Google Scholar
Digital Library
- E. Bodden, A. Sewe, J. Sinschek, M. Mezini, and H. Oueslati. Taming reflection: Aiding static analysis in the presence of reflection and custom class loaders. In Proceedings of the 33rd International Conference on Software Engineering (ICSE), 2011. Google Scholar
Digital Library
- M. Braux and J. Noye. Towards partially evaluating reflection in Java. In Proceedings of the Workshop on Partial Evaluation and Semantics-Based Program Manipulation (PEPM), 1999. Google Scholar
Digital Library
- J. M. Bull, L. A. Smith, M. D. Westhead, D. S. Henty, and R. A. Davey. A methodology for benchmarking Java Grande applications. In Proceedings of the ACM 1999 Conference on Java Grande, 1999. Google Scholar
Digital Library
- Y. Chiba. Redundant boxing elimination by a dynamic compiler for Java. In Proceedings of the 5th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2007. Google Scholar
Digital Library
- S. R. Chidamber and C. F. Kemerer. A metrics suite for object oriented design. IEEE Transactions on Software Engineering, 20(6):476--493, June 1994. ISSN 0098-5589. Google Scholar
Digital Library
- C. Collberg, G. Myles, and M. Stepp. An empirical study of Java bytecode programs. Software: Practice and Experience, 37(6):581--641, May 2007. ISSN 0038-0644. Google Scholar
Digital Library
- C. Daly, J. Horgan, J. Power, and J. Waldron. Platform independent dynamic Java virtual machine analysis: the Java Grande Forum benchmark suite. In Proceedings of the 2001 joint ACM-ISCOPE Conference on Java Grande, 2001. Google Scholar
Digital Library
- D. Detlefs and O. Agesen. Inlining of virtual methods. In Proceedings of the 13th European Conference on Object-oriented Programming (ECOOP), 1999. Google Scholar
Digital Library
- S. Dieckmann and U. Holzle. A study of the allocation behavior of the SPECjvm98 java benchmarks. In Proceedings of the 13th European Conference on Object-oriented Programming (ECOOP), 1999. Google Scholar
Digital Library
- I. Dragos and M. Odersky. Compiling generics through user-directed type specialization. In Proceedings of the 4th Workshop on the Implementation, Compilation, Optimization of Object-oriented Languages and Programming Systems (ICOOOLPS), 2009. Google Scholar
Digital Library
- G. Dubochet and M. Odersky. Compiling structural types on the JVM: a comparison of reflective and generative techniques from Scalas perspective. In Proceedings of the 4th Workshop on the Implementation, Compilation, Optimization of Object-oriented Languages and Programming Systems (ICOOOLPS), 2009. Google Scholar
Digital Library
- B. Dufour, K. Driesen, L. Hendren, and C. Verbrugge. Dynamic metrics for Java. In Proceedings of the 18th Conference on Object-oriented Programing, Systems, Languages, and Applications (OOPSLA), 2003. Google Scholar
Digital Library
- Common Language Infrastructure (CLI). ECMA International, 5th edition, December 2010.Google Scholar
- L. Eeckhout, A. Georges, and K. De Bosschere. How Java programs interact with virtual machines at the microarchitectural level. In Proceedings of the 18th Conference on Object-oriented Programing, Systems, Languages, and Applications (OOPSLA), 2003. Google Scholar
Digital Library
- D. Gregg, J. Power, and J. Waldron. A method-level comparison of the Java Grande and SPEC JVM98 benchmark suites. Concurrency and Computation: Practice and Experience, 17: 757--773, 2005. Google Scholar
Digital Library
- N. M. Hanish and W. E. Cohen. Hardware support for profiling Java programs. In Proceedings of the Workshop on Hardware Support for Objects and Microarchitectures for Java, 1999.Google Scholar
- R. E. Hank, W.-M. W. Hwu, and B. R. Rau. Region-based compilation: an introduction and motivation. In Proceedings of the 28th International Symposium on Microarchitecture (MICRO), 1995. Google Scholar
Digital Library
- U. Holzle, C. Chambers, and D. Ungar. Optimizing dynamically-typed object-oriented languages with polymorphic inline caches. In Proceedings of the 5th European Conference on Object-oriented Programming (ECOOP), 1991. Google Scholar
Digital Library
- K. Hoste and L. Eeckhout. Microarchitecture-independent workload characterization. IEEE Micro, 27:63--72, 2007. Google Scholar
Digital Library
- I. Jibaja, S. Blackburn, M. Haghighat, and K. McKinley. Deferred gratification: Engineering for high performance garbage collection from the get go. In Proceedings of the Workshop on Memory Systems Performance and Correctness (MSPC), 2011. Google Scholar
Digital Library
- R. E. Jones and C. Ryder. A study of Java object demographics. In Proceedings of the 7th International Symposium on Memory Management (ISMM), 2008. Google Scholar
Digital Library
- R. K. Karmani, A. Shali, and G. Agha. Actor frameworks for the JVM platform: a comparative analysis. In Proceedings of the 7th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2009. Google Scholar
Digital Library
- G. Lashari and S. Srinivas. Characterizing Java application performance. In Proceedings of the 17th International Parallel and Distributed Processing Symposium (IPDPS), 2003. Google Scholar
Digital Library
- T. Lindholm and F. Yellin. The Java Virtual Machine Specification. Addison-Wesley, 2nd edition, 1999. Google Scholar
Digital Library
- A. McCallum, K. Schultz, and S. Singh. FACTORIE: Probabilistic programming via imperatively defined factor graphs. Advances on Neural Information Processing Systems, 2009.Google Scholar
- P. Moret, W. Binder, and A. Villazon. CCCP: Complete calling context profiling in virtual execution environments. In Proceedings of the Workshop on Partial Evaluation and Program Manipulation (PEPM), 2009. Google Scholar
Digital Library
- M. Odersky, L. Spoon, and B. Venners. Programming in Scala. Artima Press, 2nd edition, 2010.Google Scholar
- K. Ogata, D. Mikurube, K. Kawachiya, S. Trent, and T. Onodera. A study of Javas non-Java memory. In Proceedings of the 25th Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA), 2010. Google Scholar
Digital Library
- K. Pearson. On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2:559--572, 1901.Google Scholar
- D. Ramage, E. Rosen, J. Chuang, C. D. Manning, and D. A. McFarland. Topic modeling for the social sciences. In NIPS Workshop on Applications for Topic Models: Text and Beyond, 2009.Google Scholar
- P. Ratanaworabhan, B. Livshits, and B. G. Zorn. JSMeter: comparing the behavior of JavaScript benchmarks with real Web applications. In Proceedings of the 2010 USENIX Conference on Web Application Development, 2010. Google Scholar
Digital Library
- G. Richards, S. Lebresne, B. Burg, and J. Vitek. An analysis of the dynamic behavior of JavaScript programs. In Proceedings of the Conference on Programming Language Design and Implementation (PLDI), 2010. Google Scholar
Digital Library
- I. Rogers, J. Zhao, and I. Watson. Approaches to reflective method invocation. In Proceedings of the 3rd Workshop on Implementation, Compilation, Optimization of Object-oriented Languages, Programs and Systems (ICOOOLPS), 2008.Google Scholar
- J. R. Rose. Bytecodes meet combinators: invokedynamic on the JVM. In Proceedings of the 3rd Workshop on Virtual Machines and Intermediate Languages (VMIL), 2009. Google Scholar
Digital Library
- A. Sarimbekov, P. Moret, W. Binder, A. Sewe, and M. Mezini. Complete and platform-independent calling context profiling for the Java virtual machine. In Proceedings of the 6th Workshop on Bytecode Semantics, Verification, Analysis and Transformation (BYTECODE), 2011. Google Scholar
Digital Library
- A. Sarimbekov, A. Sewe, W. Binder, P. Moret, M. Schoberl, and M. Mezini. Portable and accurate collection of calling-context-sensitive bytecode metrics for the java virtual machine. In Proceedings of the 9th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2011. Google Scholar
Digital Library
- M. Schinz. Compiling Scala for the Java Virtual Machine. PhD thesis, EPFL, Switzerland, September 2005.Google Scholar
- A. Sewe. Scala ð Java mod JVM. In Proceedings of the Work-in-Progress Session at the 8th International Conference on the Principles and Practice of Programming in Java (PPPJ), volume 692 of CEUR Workshop Proceedings, 2010.Google Scholar
- K. Shiv, K. Chow, Y. Wang, and D. Petrochenko. SPECjvm2008 performance characterization. In Proceedings of the SPEC Benchmark Workshop, 2009. Google Scholar
Digital Library
- A. M. Sloane. Experiences with domain-specific language embedding in Scala. In Proceedings of the 2nd International Workshop on Domain-Specific Program Development (DSPD), 2008.Google Scholar
- T. Suganuma, T. Yasue, and T. Nakatani. A region-based compilation technique for a Java just-in-time compiler. In Proceedings of the 16th Conference on Programming Language Design and Implementation (PLDI), 2003. Google Scholar
Digital Library
- C. Thalinger and J. Rose. Optimizing invokedynamic. In Proceedings of the 8th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2010. Google Scholar
Digital Library
Index Terms
Da capo con scala: design and analysis of a scala benchmark suite for the java virtual machine
Recommendations
A Comprehensive Java Benchmark Study on Memory and Garbage Collection Behavior of DaCapo, DaCapo Scala, and SPECjvm2008
ICPE '17: Proceedings of the 8th ACM/SPEC on International Conference on Performance EngineeringBenchmark suites are an indispensable part of scientific research to compare different approaches against each another. The diversity of benchmarks is an important asset to evaluate novel approaches for effectiveness and weaknesses. In this paper, we ...
Da capo con scala: design and analysis of a scala benchmark suite for the java virtual machine
OOPSLA '11: Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applicationsOriginally conceived as the target platform for Java alone, the Java Virtual Machine (JVM) has since been targeted by other languages, one of which is Scala. This trend, however, is not yet reflected by the benchmark suites commonly used in JVM ...
Performance and Energy-Efficiency of Scala on Mobile Devices
NGMAST '13: Proceedings of the 2013 Seventh International Conference on Next Generation Mobile Apps, Services and TechnologiesThe objective of this study was to compare how new languages like Scala perform on mobile environments compared to the classical languages. Since Scala also runs on the Java Virtual Machine (JVM), it is possible to run the code on Android devices and ...







Comments