skip to main content
research-article

Da capo con scala: design and analysis of a scala benchmark suite for the java virtual machine

Published:22 October 2011Publication History
Skip Abstract Section

Abstract

Originally conceived as the target platform for Java alone, the Java Virtual Machine (JVM) has since been targeted by other languages, one of which is Scala. This trend, however, is not yet reflected by the benchmark suites commonly used in JVM research. In this paper, we thus present the design and analysis of the first full-fledged benchmark suite for Scala. We furthermore compare the benchmarks contained therein with those from the well-known DaCapo 9.12 benchmark suite and show where the differences are between Scala and Java code---and where not.

References

  1. B. Alpern, A. Cocchi, S. Fink, and D. Grove. Efficient implementation of Java interfaces: invoke interface considered harmless. In Proceedings of the 16th Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA), 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. G. Ammons, T. Ball, and J. R. Larus. Exploiting hardware performance counters with flow and context sensitive profiling. In Proceedings of the 10th Conference on Programming Language Design and Implementation (PLDI), 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. M. Bebenita, M. Chang, G. Wagner, A. Gal, C. Wimmer, and M. Franz. Trace-based compilation in execution environments without interpreters. In Proceedings of the 8th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. W. Binder. A portable and customizable profiling framework for Java based on bytecode instruction counting. In Proceedings of the 3rd Asian Symposium on Programming Languages and Systems (APLAS), 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. W. Binder, J. Hulaas, and P. Moret. A quantitative evaluation of the contribution of native code to Java workloads. In Proceedings of the IEEE International Symposium on Workload Characterization (IISWC), 2006.Google ScholarGoogle ScholarCross RefCross Ref
  6. W. Binder, J. Hulaas, P. Moret, and A. Villazo ?n. Platform-independent profiling in a virtual execution environment. Software: Practice and Experience, 39(1):47--79, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J.E.B.Moss,B.Moss,A.Phansalkar,D.Stefanovic, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA), 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. E. Bodden, A. Sewe, J. Sinschek, M. Mezini, and H. Oueslati. Taming reflection: Aiding static analysis in the presence of reflection and custom class loaders. In Proceedings of the 33rd International Conference on Software Engineering (ICSE), 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. M. Braux and J. Noye. Towards partially evaluating reflection in Java. In Proceedings of the Workshop on Partial Evaluation and Semantics-Based Program Manipulation (PEPM), 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. J. M. Bull, L. A. Smith, M. D. Westhead, D. S. Henty, and R. A. Davey. A methodology for benchmarking Java Grande applications. In Proceedings of the ACM 1999 Conference on Java Grande, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Y. Chiba. Redundant boxing elimination by a dynamic compiler for Java. In Proceedings of the 5th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. S. R. Chidamber and C. F. Kemerer. A metrics suite for object oriented design. IEEE Transactions on Software Engineering, 20(6):476--493, June 1994. ISSN 0098-5589. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. C. Collberg, G. Myles, and M. Stepp. An empirical study of Java bytecode programs. Software: Practice and Experience, 37(6):581--641, May 2007. ISSN 0038-0644. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. C. Daly, J. Horgan, J. Power, and J. Waldron. Platform independent dynamic Java virtual machine analysis: the Java Grande Forum benchmark suite. In Proceedings of the 2001 joint ACM-ISCOPE Conference on Java Grande, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. D. Detlefs and O. Agesen. Inlining of virtual methods. In Proceedings of the 13th European Conference on Object-oriented Programming (ECOOP), 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. S. Dieckmann and U. Holzle. A study of the allocation behavior of the SPECjvm98 java benchmarks. In Proceedings of the 13th European Conference on Object-oriented Programming (ECOOP), 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. I. Dragos and M. Odersky. Compiling generics through user-directed type specialization. In Proceedings of the 4th Workshop on the Implementation, Compilation, Optimization of Object-oriented Languages and Programming Systems (ICOOOLPS), 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. G. Dubochet and M. Odersky. Compiling structural types on the JVM: a comparison of reflective and generative techniques from Scalas perspective. In Proceedings of the 4th Workshop on the Implementation, Compilation, Optimization of Object-oriented Languages and Programming Systems (ICOOOLPS), 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. B. Dufour, K. Driesen, L. Hendren, and C. Verbrugge. Dynamic metrics for Java. In Proceedings of the 18th Conference on Object-oriented Programing, Systems, Languages, and Applications (OOPSLA), 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Common Language Infrastructure (CLI). ECMA International, 5th edition, December 2010.Google ScholarGoogle Scholar
  21. L. Eeckhout, A. Georges, and K. De Bosschere. How Java programs interact with virtual machines at the microarchitectural level. In Proceedings of the 18th Conference on Object-oriented Programing, Systems, Languages, and Applications (OOPSLA), 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. D. Gregg, J. Power, and J. Waldron. A method-level comparison of the Java Grande and SPEC JVM98 benchmark suites. Concurrency and Computation: Practice and Experience, 17: 757--773, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. N. M. Hanish and W. E. Cohen. Hardware support for profiling Java programs. In Proceedings of the Workshop on Hardware Support for Objects and Microarchitectures for Java, 1999.Google ScholarGoogle Scholar
  24. R. E. Hank, W.-M. W. Hwu, and B. R. Rau. Region-based compilation: an introduction and motivation. In Proceedings of the 28th International Symposium on Microarchitecture (MICRO), 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. U. Holzle, C. Chambers, and D. Ungar. Optimizing dynamically-typed object-oriented languages with polymorphic inline caches. In Proceedings of the 5th European Conference on Object-oriented Programming (ECOOP), 1991. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. K. Hoste and L. Eeckhout. Microarchitecture-independent workload characterization. IEEE Micro, 27:63--72, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. I. Jibaja, S. Blackburn, M. Haghighat, and K. McKinley. Deferred gratification: Engineering for high performance garbage collection from the get go. In Proceedings of the Workshop on Memory Systems Performance and Correctness (MSPC), 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. R. E. Jones and C. Ryder. A study of Java object demographics. In Proceedings of the 7th International Symposium on Memory Management (ISMM), 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. R. K. Karmani, A. Shali, and G. Agha. Actor frameworks for the JVM platform: a comparative analysis. In Proceedings of the 7th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. G. Lashari and S. Srinivas. Characterizing Java application performance. In Proceedings of the 17th International Parallel and Distributed Processing Symposium (IPDPS), 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. T. Lindholm and F. Yellin. The Java Virtual Machine Specification. Addison-Wesley, 2nd edition, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. A. McCallum, K. Schultz, and S. Singh. FACTORIE: Probabilistic programming via imperatively defined factor graphs. Advances on Neural Information Processing Systems, 2009.Google ScholarGoogle Scholar
  33. P. Moret, W. Binder, and A. Villazon. CCCP: Complete calling context profiling in virtual execution environments. In Proceedings of the Workshop on Partial Evaluation and Program Manipulation (PEPM), 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. M. Odersky, L. Spoon, and B. Venners. Programming in Scala. Artima Press, 2nd edition, 2010.Google ScholarGoogle Scholar
  35. K. Ogata, D. Mikurube, K. Kawachiya, S. Trent, and T. Onodera. A study of Javas non-Java memory. In Proceedings of the 25th Conference on Object-oriented Programming, Systems, Languages, and Applications (OOPSLA), 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. K. Pearson. On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2:559--572, 1901.Google ScholarGoogle Scholar
  37. D. Ramage, E. Rosen, J. Chuang, C. D. Manning, and D. A. McFarland. Topic modeling for the social sciences. In NIPS Workshop on Applications for Topic Models: Text and Beyond, 2009.Google ScholarGoogle Scholar
  38. P. Ratanaworabhan, B. Livshits, and B. G. Zorn. JSMeter: comparing the behavior of JavaScript benchmarks with real Web applications. In Proceedings of the 2010 USENIX Conference on Web Application Development, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. G. Richards, S. Lebresne, B. Burg, and J. Vitek. An analysis of the dynamic behavior of JavaScript programs. In Proceedings of the Conference on Programming Language Design and Implementation (PLDI), 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. I. Rogers, J. Zhao, and I. Watson. Approaches to reflective method invocation. In Proceedings of the 3rd Workshop on Implementation, Compilation, Optimization of Object-oriented Languages, Programs and Systems (ICOOOLPS), 2008.Google ScholarGoogle Scholar
  41. J. R. Rose. Bytecodes meet combinators: invokedynamic on the JVM. In Proceedings of the 3rd Workshop on Virtual Machines and Intermediate Languages (VMIL), 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. A. Sarimbekov, P. Moret, W. Binder, A. Sewe, and M. Mezini. Complete and platform-independent calling context profiling for the Java virtual machine. In Proceedings of the 6th Workshop on Bytecode Semantics, Verification, Analysis and Transformation (BYTECODE), 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. A. Sarimbekov, A. Sewe, W. Binder, P. Moret, M. Schoberl, and M. Mezini. Portable and accurate collection of calling-context-sensitive bytecode metrics for the java virtual machine. In Proceedings of the 9th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. M. Schinz. Compiling Scala for the Java Virtual Machine. PhD thesis, EPFL, Switzerland, September 2005.Google ScholarGoogle Scholar
  45. A. Sewe. Scala ð Java mod JVM. In Proceedings of the Work-in-Progress Session at the 8th International Conference on the Principles and Practice of Programming in Java (PPPJ), volume 692 of CEUR Workshop Proceedings, 2010.Google ScholarGoogle Scholar
  46. K. Shiv, K. Chow, Y. Wang, and D. Petrochenko. SPECjvm2008 performance characterization. In Proceedings of the SPEC Benchmark Workshop, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. A. M. Sloane. Experiences with domain-specific language embedding in Scala. In Proceedings of the 2nd International Workshop on Domain-Specific Program Development (DSPD), 2008.Google ScholarGoogle Scholar
  48. T. Suganuma, T. Yasue, and T. Nakatani. A region-based compilation technique for a Java just-in-time compiler. In Proceedings of the 16th Conference on Programming Language Design and Implementation (PLDI), 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. C. Thalinger and J. Rose. Optimizing invokedynamic. In Proceedings of the 8th International Conference on the Principles and Practice of Programming in Java (PPPJ), 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Da capo con scala: design and analysis of a scala benchmark suite for the java virtual machine

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 46, Issue 10
          OOPSLA '11
          October 2011
          1063 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/2076021
          Issue’s Table of Contents
          • cover image ACM Conferences
            OOPSLA '11: Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applications
            October 2011
            1104 pages
            ISBN:9781450309400
            DOI:10.1145/2048066

          Copyright © 2011 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 22 October 2011

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!