skip to main content
article

Numerical computing on the web: benchmarking for the future

Published:06 April 2020Publication History
Skip Abstract Section

Abstract

Recent advances in execution environments for JavaScript and WebAssembly that run on a broad range of devices, from workstations and mobile phones to IoT devices, provide new opportunities for portable and web-based numerical computing. Indeed, numerous numerical libraries and applications are emerging on the web, including Tensorflow.js, JSMapReduce, and the NLG Protein Viewer. This paper evaluates the current performance of numerical computing on the web, including both JavaScript and WebAssembly, over a wide range of devices from workstations to IoT devices. We developed a new benchmarking approach, which allowed us to perform centralized benchmarking, including benchmarking on mobile and IoT devices. Using this approach we performed four performance studies using the Ostrich benchmark suite, a collection of numerical programs representing the numerical dwarf categories identified by Colella. We studied the performance evolution of JavaScript, the relative performance of WebAssembly, the performance of server-side Node.js, and a comprehensive performance showdown for a wide range of devices.

References

  1. 1988. Gnu compiler collection (gcc) internals. Retrieved September 4th, 2018 from http://gcc.gnu.org/onlinedocs/gccint/Google ScholarGoogle Scholar
  2. 2010. Kraken JavaScript Benchmark (Version 1.1). Retrieved April 14, 2018 from http://krakenbenchmark.mozilla.org/Google ScholarGoogle Scholar
  3. 2012. Octane Benchmark Suite. Retrieved April 14, 2018 from https://developers.google.com/octane/Google ScholarGoogle Scholar
  4. 2014. Speedometer. Retrieved April 14, 2018 from http://browserbench.org/Speedometer/Google ScholarGoogle Scholar
  5. 2015. Ndarray - Multidimensional arrays for JavaScript. Retrieved June 29, 2018 from https://github.com/scijs/ndarrayGoogle ScholarGoogle Scholar
  6. 2017. Standard ECMA-262 ECMAScript Language Specification 8th Edition ( June 2017). Retrieved April 14, 2018 from http://www. ecma-international.org/publications/standards/Ecma-262.htmGoogle ScholarGoogle Scholar
  7. 2018. Framework - JavaScriptCore. Retrieved April 14, 2018 from https://developer.apple.com/documentation/javascriptcoreGoogle ScholarGoogle Scholar
  8. 2018. TensorFlow.js. Retrieved April 14, 2018 from https://js.tensorflow.org/Google ScholarGoogle Scholar
  9. Apache Cordova. 2011. A hybrid mobile app. Retrieved April 14, 2018 from https://cordova.apache.org/Google ScholarGoogle Scholar
  10. Lars Bak. 2018. WebAssembly Core Specification, W3C First Public Working Draft, 15 February 2018. Retrieved June 30, 2018 from https://www.w3.org/TR/wasm-core-1/lGoogle ScholarGoogle Scholar
  11. Benjamin Bouvier. 2013. Efficient float32 Arithmetic in JavaScript. Retrieved January 19, 2018 from https://blog.mozilla.org/javascript/ 2013/11/07/efficient-float32-arithmetic-in-javascript/Google ScholarGoogle Scholar
  12. Antoine Boutet, Davide Frey, Rachid Guerraoui, Anne-Marie Kermarrec, and Rhicheek Patra. 2014. HyRec: Leveraging Browsers for Scalable Recommenders. In International Middleware Conference. ACM, 85–96. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Phillip Colella. 2004. Defining software requirements for scientific computing. Retrieved April 14, 2018 from https://www.krellinst.org/doecsgf/conf/2013/pres/pcolella.pdfGoogle ScholarGoogle Scholar
  14. Laouratou Diallo, Aisha Hashim, Momoh Jimoh Eyiomika Salami, Sara Babiker Omer Elagib, and Abdullah Ahmad Zarir. 2017. The Rise of Internet of Things and Big Data on the Cloud: Challenges and Future Trends. International Journal of Future Generation Communication and Networking 10 (03 2017), 49–56.Google ScholarGoogle Scholar
  15. Tomasz Fabisiak and Arkadiusz Danilecki. 2017. Browser-based Harnessing of Voluntary Computational Power. Foundations of Computing and Decision Sciences 42, 1 (2017), 3–42.Google ScholarGoogle ScholarCross RefCross Ref
  16. Firebase. 2011. A Real-time Database on Cloud. Retrieved April 14, 2018 from https://firebase.google.com/Google ScholarGoogle Scholar
  17. Vincent Foley-Bourgon and Laurie Hendren. 2016. Efficiently Implementing the Copy Semantics of MATLAB’s Arrays in JavaScript. In Symposium on Dynamic Languages. 72–83. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Andreas Gal, Brendan Eich, Mike Shaver, David Anderson, David Mandelin, Mohammad R. Haghighat, Blake Kaplan, Graydon Hoare, Boris Zbarsky, Jason Orendorff, Jesse Ruderman, Edwin W. Smith, Rick Reitmaier, Michael Bebenita, Mason Chang, and Michael Franz. 2009. Trace-based Just-in-time Type Specialization for Dynamic Languages. In Conference on Programming Language Design and Implementation. ACM, 465–478. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Andy Georges, Dries Buytaert, and Lieven Eeckhout. 2007. Statistically Rigorous Java Performance Evaluation. In Conference on Object-oriented Programming Systems and Applications. 57–76. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Dominique Guinard and Vlad Trifa. 2016. Building the Web of Things: With Examples in Node.js and Raspberry Pi. Manning Publications Co. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Andreas Haas, Andreas Rossberg, Derek L Schuff, Ben L Titzer, Michael Holman, Dan Gohman, Luke Wagner, Alon Zakai, and JF Bastien. 2017. Bringing the Web up to Speed with WebAssembly. In Conference on Programming Language Design and Implementation. 185–200. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Apple Inc. 2017. About Face ID Advanced Technology. Retrieved April 14, 2018 from https://support.apple.com/en-ca/HT208108Google ScholarGoogle Scholar
  23. Peter Jensen, Ivan Jibaja, Ningxin Hu, Dan Gohman, and John McCutchan. 2015. SIMD in Javascript via C++ and Emscripten. In Workshop on Programming Models for SIMD/Vector Processing.Google ScholarGoogle Scholar
  24. Alexandr A. Kalinin, Selvam Palanimalai, and Ivo D. Dinov. 2017. SOCRAT Platform Design: A Web Architecture for Interactive Visual Analytics Applications. In Workshop on Human-In-the-Loop Data Analytics. Article 8, 6 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Faiz Khan, Vincent Foley-Bourgon, Sujay Kathrotia, Erick Lavoie, and Laurie J. Hendren. 2014. Using JavaScript and WebCL for Numerical Computations: A Comparative Study of Native and Web Technologies. In Symposium on Dynamic Languages. 91–102. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Chris Lattner and Vikram Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Symposium on Code Generation and Optimization: Feedback-directed and Runtime Optimization. IEEE, 75. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Erick Lavoie. 2016. Wu-Wei Handbook. Retrieved August 24, 2018 from https://github.com/Sable/wu-wei-handbookGoogle ScholarGoogle Scholar
  28. Erick Lavoie, Laurie Hendren, and Frédéric Desprez. 2017. Pando: A Volunteer Computing Platform for the Web. In Foundations and Applications of Self* Systems (FAS* W). IEEE, 387–388.Google ScholarGoogle Scholar
  29. Edward Meeds, Remco Hendriks, Said Al Faraby, Magiel Bruntink, and Max Welling. 2015. MLitB: Machine Learning in the Browser. PeerJ Computer Science 1 (2015), e11.Google ScholarGoogle ScholarCross RefCross Ref
  30. Franziska Hinkelmann Michael Hablich. 2016. How V8 Measures Real-world Performance. Retrieved April 14, 2018 from https://v8project.blogspot.ca/2016/Google ScholarGoogle Scholar
  31. Mozilla JavaScript. 2018. JavaScript Typed Arrays. Retrieved April 14, 2018 from https://developer.mozilla.org/en-US/docs/Web/JavaScript/ Typed_arraysGoogle ScholarGoogle Scholar
  32. Rudolph Pienaar, Ata Turk, Jorge Bernal-Rusiel, Nicolas Rannou, Daniel Haehn, P. Ellen Grant, and Orran Krieger. 2017. CHIPS – A Service for Collecting, Organizing, Processing, and Sharing Medical Image Data in the Cloud. Springer International Publishing, 29–35.Google ScholarGoogle Scholar
  33. Paruj Ratanaworabhan, Benjamin Livshits, and Benjamin G Zorn. 2010. JSMeter: Comparing the Behavior of JavaScript Benchmarks with Real Web Applications. USENIX Conference on Web Application Development 10 (2010), 3–3. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Gregor Richards. 2013. JSBench Benchmark Suite. Retrieved April 14, 2018 from https://plg.uwaterloo.ca/~dynjs/jsbench/Google ScholarGoogle Scholar
  35. Gregor Richards, Sylvain Lebresne, Brian Burg, and Jan Vitek. 2010. An Analysis of the Dynamic Behavior of JavaScript Programs. In Conference on Programming Language Design and Implementation. ACM, 1–12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Marija Selakovic and Michael Pradel. 2016. Performance Issues and Optimizations in Javascript: An Empirical Study. In International Conference on Software Engineering. ACM, 61–72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. 2016. Edge Computing: Vision and Challenges. Internet of Things Journal 3, 5 (2016), 637–646.Google ScholarGoogle ScholarCross RefCross Ref
  38. Katie Shilton. 2009. Four Billion Little Brothers?: Privacy, Mobile Phones, and Ubiquitous Data Collection. Commun. ACM 52, 11 (2009), 48–53. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. V8 Team. 2017. Launching Ignition and TurboFan. Retrieved April 14, 2018 from https://v8project.blogspot.ca/2017/ 05/launching-ignition-and-turbofan.htmlGoogle ScholarGoogle Scholar
  40. V8 Team. 2017. Retiring Octane. Retrieved April 14, 2018 from https://v8project.blogspot.ca/2017/04/retiring-octane.htmlGoogle ScholarGoogle Scholar
  41. W3C. 2018. Google Chrome’s Need for Speed. Retrieved April 14, 2018 from http://blog.chromium.org/2008/09/ google-chromes-need-for-speed_02.htmlGoogle ScholarGoogle Scholar
  42. WebAssembly. 2015. Features to Add after the MVP. Retrieved April 14, 2018 from http://webassembly.org/docs/future-features/Google ScholarGoogle Scholar
  43. WebAssembly. 2016. Non-Web Embeddings. Retrieved April 14, 2018 from http://webassembly.org/docs/non-web/Google ScholarGoogle Scholar
  44. I. Yaqoob, E. Ahmed, I. A. T. Hashem, A. I. A. Ahmed, A. Gani, M. Imran, and M. Guizani. 2017. Internet of Things Architecture: Recent Advances, Taxonomy, Requirements, and Open Challenges. IEEE Wireless Communications 24, 3 (June 2017), 10–16.Google ScholarGoogle ScholarCross RefCross Ref
  45. Alon Zakai. 2011. Emscripten: An LLVM-to-JavaScript Compiler. In ACM International Conference Companion on Object-oriented Programming Systems Languages and Applications Companion. 301–312. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Numerical computing on the web: benchmarking for the future

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!