Abstract
JavaScript is a highly dynamic language for web-based applications. Innovative implementation techniques for improving its speed and responsiveness have been developed in recent years. Industry benchmarks such as WebKit SunSpider are often cited as a measure of the efficacy of these techniques. However, recent studies have shown that these benchmarks fail to accurately represent the dynamic nature of modern JavaScript applications, and so may be poor predictors of real-world performance. Worse, they may guide the development of optimizations which are unhelpful for real applications. Our goal is to develop a tool and techniques to automate the creation of realistic and representative benchmarks from existing web applications. We propose a record-and-replay approach to capture JavaScript sessions which has sufficient fidelity to accurately recreate key characteristics of the original application, and at the same time is sufficiently flexible that a recording produced on one platform can be replayed on a different one. We describe JSBench, a flexible tool for workload capture and benchmark generation, and demonstrate its use in creating eight benchmarks based on popular sites. Using a variety of runtime metrics collected with instrumented versions of Firefox, Internet Explorer, and Safari, we show that workloads created by JSBench match the behavior of the original web applications.
- S. M. Blackburn et al. The DaCapo benchmarks: Java benchmarking development and analysis. In Conference on Object-Oriented Programming Systems Languages and Applications (OOPSLA), pages 169--190, 2006. Google Scholar
Digital Library
- F. Cornelis, A. Georges, M. Christiaens, M. Ronsse, T. Ghesquiere, and K. D. Bosschere. A taxonomy of execution replay systems. In Conference on Advances in Infrastructure for Electronic Business, Education, Science, Medicine, and Mobile Technologies on the Internet, 2003.Google Scholar
- J. de Halleux and N. Tillmann. Moles: Tool-assisted environment isolation with closures. In International Conference on Objects, Models, Components, Patterns (TOOLS), pages 253--270, 2010. Google Scholar
Digital Library
- S. Dieckmann and U. Hölzle. A study of the allocation behaviour of the SPECjvm98 Java benchmarks. In European Conference on Object Oriented Programming, (ECOOP), pages 92--115, 1999. Google Scholar
Digital Library
- C. Dionne, M. Feeley, and J. Desbien. A taxonomy of distributed debuggers based on execution replay. pages 203--214, 1996.Google Scholar
- A. Gal et al. Trace-based just-in-time type specialization for dynamic languages. In Conference on Programming Language Design and Implementation (PLDI), pages 465--478, 2009. Google Scholar
Digital Library
- E. Kiciman and B. Livshits. Ajaxscope: a platform for remotely monitoring the client-side behavior of Web 2.0 applications. In Symposium on Operating Systems Principles (SOSP), pages 17--30, 2007. Google Scholar
Digital Library
- B. Livshits and E. Kiciman. Doloto: code splitting for network-bound Web 2.0 applications. In Conference on Foundations of Sofware Engineering (FSE), pages 350--360, 2008. Google Scholar
Digital Library
- J. Mickens, J. Elson, and J. Howell. Mugshot: Deterministic capture and replay for JavaScript applications. In Symposium on Networked Systems Design and Implementation (NSDI), 2010. Google Scholar
Digital Library
- K. Pattabiraman and B. Zorn. DoDOM: Leveraging DOM invariants for Web 2.0 applications robustness testing. In International Symposium on Software Reliability Engineering (ISRE), 2010. Google Scholar
Digital Library
- P. Ratanaworabhan, B. Livshits, and B. Zorn. JSMeter: Comparing the behavior of JavaScript benchmarks with real Web applications. In Conference on Web Application Development, 2010. Google Scholar
Digital Library
- G. Richards, C. Hammer, B. Burg, and J. Vitek. The eval that men do: A large-scale study of the use of eval in JavaScript applications. In European Conference on Object-Oriented Programming (ECOOP), 2011. Google Scholar
Digital Library
- G. Richards, S. Lebresne, B. Burg, and J. Vitek. An analysis of the dynamic behavior of JavaScript programs. In Conference on Programming Language Design and Implementation (PLDI), pages 1--12, 2010. Google Scholar
Digital Library
- . Unger and R. B. Smith. Self: The power of simplicity. In Conference on Object-Oriented Programming Systems, Languages, and Applications (OOSPLA), pages 227--242, 1987. Google Scholar
Digital Library
- K. Vikram, A. Prateek, and B. Livshits. Ripley: automatically securing Web 2.0 applications through replicated execution. In Conference on Computer and Communications Security (CCS), pages 173--186, 2009. Google Scholar
Digital Library
Index Terms
Automated construction of JavaScript benchmarks
Recommendations
Automated construction of JavaScript benchmarks
OOPSLA '11: Proceedings of the 2011 ACM international conference on Object oriented programming systems languages and applicationsJavaScript is a highly dynamic language for web-based applications. Innovative implementation techniques for improving its speed and responsiveness have been developed in recent years. Industry benchmarks such as WebKit SunSpider are often cited as a ...
Large System Performance of SPEC OMP2001 Benchmarks
ISHPC '02: Proceedings of the 4th International Symposium on High Performance ComputingPerformance characteristics of application programs on large-scale systems are often significantly different from those on smaller systems. SPEC OMP2001 is a benchmark suite intended for measuring performance of modern shared memory parallel systems. ...
Large System Performance of SPEC OMP2001 Benchmarks
ISHPC '02: Proceedings of the 4th International Symposium on High Performance ComputingPerformance characteristics of application programs on large-scale systems are often significantly different from those on smaller systems. SPEC OMP2001 is a benchmark suite intended for measuring performance of modern shared memory parallel systems. ...







Comments