skip to main content
article

AutoBench: comparing the time performance of Haskell programs

Published:17 September 2018Publication History
Skip Abstract Section

Abstract

Two fundamental goals in programming are correctness (producing the right results) and efficiency (using as few resources as possible). Property-based testing tools such as QuickCheck provide a lightweight means to check the correctness of Haskell programs, but what about their efficiency? In this article, we show how QuickCheck can be combined with the Criterion benchmarking library to give a lightweight means to compare the time performance of Haskell programs. We present the design and implementation of the AutoBench system, demonstrate its utility with a number of case studies, and find that many QuickCheck correctness properties are also efficiency improvements.

Skip Supplemental Material Section

Supplemental Material

References

  1. T. Arts, J. Hughes, J. Johansson, and U.T. Wiger. 2006. Testing Telecoms Software with Quviq QuickCheck. Erlang Workshop. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. D. Beardsley. 2018. Fake. https://hackage.haskell.org/package/fake .Google ScholarGoogle Scholar
  3. J. Bernardy, P. Jansson, and K. Claessen. 2010. Testing Polymorphic Properties. ESOP. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. N. Brown. 2010. Progression. https://hackage.haskell.org/package/ progression .Google ScholarGoogle Scholar
  5. J. Christiansen and S. Fischer. 2008. EasyCheck - Test Data for Free. FLOPS . Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. K. Claessen, J. Duregård, and M.H. Palka. 2015. Generating Constrained Random Data with Uniform Distribution. JFP.Google ScholarGoogle Scholar
  7. K. Claessen and J. Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. ICFP. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. K. Claessen and J. Hughes. 2002. Testing Monadic Code with QuickCheck. Haskell Workshop. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. K. Claessen, N. Smallbone, and J. Hughes. 2010. QuickSpec: Guessing Formal Specifications Using Testing. TAP. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. E. Coppa, C. Demetrescu, and I. Finocchi. 2012. Input-Sensitive Profiling. PLDI. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. E. Coppa, C. Demetrescu, I. Finocchi, and R. Marotta. 2014. Estimating the Empirical Cost Function of Routines with Dynamic Workloads. CGO . Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. T.H. Cormen, C.E. Leiserson, R.L. Rivest, and C. Stein. 2009. Introduction to Algorithms . MIT Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. T. Docker. 2006. Chart. http://hackage.haskell.org/package/Chart .Google ScholarGoogle Scholar
  14. V. Estivill-Castro and D.Wood. 1992. A Survey of Adaptive Sorting Algorithms. ACM Computing Surveys. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. C.McGeoch et al. 2002. Using Finite Experiments to Study Asymptotic Performance. In Experimental Algorithmics. Springer. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. C. Klein et al. 2012. Run Your Research: On the Effectiveness of Lightweight Mechanization. POPL. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. J. Fox. 1997. Applied Regression Analysis, Linear Models, and Related Methods . Sage Publications.Google ScholarGoogle Scholar
  18. S.F. Goldsmith, A.S.Aiken, and D.S. Wilkerson. 2007. Measuring Empirical Computational Complexity. ESEC-FSE. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. D. Gorin. 2015. Hint. https://hackage.haskell.org/package/hint .Google ScholarGoogle Scholar
  20. M.A.T. Handley. 2018. GitHub Repository for AutoBench. https: //github.com/mathandley/AutoBench .Google ScholarGoogle Scholar
  21. A.E. Hoerl and R.W. Kennard. 1970. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. J. Hughes. 2007. QuickCheck Testing for Fun and Profit. PADL. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. G. Hutton. 2016. Programming in Haskell. Cambridge University Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. D.G. Hyams. 2010. CurveExpert. http://www.curveexpert.net .Google ScholarGoogle Scholar
  25. M.P. Jones. 1999. Hugs. https://www.haskell.org/hugs .Google ScholarGoogle Scholar
  26. L.G.L.T. Meertens. 2004. Calculating the Sieve of Eratosthenes. JFP. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. A.K. Moran and D. Sands. 1999. Improvement in a Lazy Context: An Operational Theory for Call-By-Need. POPL. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. G.E. Moss. 2000. Benchmarking Purely Functional Data Structures. Ph.D. Dissertation. University of York.Google ScholarGoogle Scholar
  29. The University of Glasgow. 2001. Base: Data.List. http://hackage. haskell.org/package/base-4.11.1.0/docs/src/Data.OldList.html#sort .Google ScholarGoogle Scholar
  30. The University of Glasgow. 2014. Process. https://hackage.haskell. org/package/process .Google ScholarGoogle Scholar
  31. The University of Glasgow. 2015. Glasgow Haskell Compiler User’s Guide: Profiling. http://downloads.haskell.org/~ghc/latest/docs/html/ users_guide/profiling.html .Google ScholarGoogle Scholar
  32. R. O’Keefe. 1982. A Smooth Applicative Merge Sort. Department of Artificial Intelligence, University of Edinburgh.Google ScholarGoogle Scholar
  33. M.E. O’Neill. 2009. The Genuine Sieve of Eratosthenes. JFP. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. OriginLab. 2000. Origin. https://www.originlab.com .Google ScholarGoogle Scholar
  35. B. O’Sullivan. 2009. Criterion: Robust Reliable Performance Measurement and Analysis. http://www.serpentine.com/criterion .Google ScholarGoogle Scholar
  36. C. Runciman, M. Naylor, and F. Lindblad. 2008. Smallcheck and Lazy Smallcheck: Automatic Exhaustive Testing for Small Values. Haskell Symposium . Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Systat Software. 2002. TableCurve 2D. http://www.sigmaplot.co.uk .Google ScholarGoogle Scholar
  38. The GHC Team. 2017. GHC. https://hackage.haskell.org/package/ghc .Google ScholarGoogle Scholar
  39. P. Wadler. 1987. The Concatenate Vanishes. University of Glasgow.Google ScholarGoogle Scholar
  40. Q. Xu and Y. Liang. 2001. Monte Carlo Cross Validation. Chemometrics and Intelligent Laboratory Systems .Google ScholarGoogle Scholar
  41. D. Zaparanuks and M. Hauswirth. 2012. Algorithmic Profiling. PLDI. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. AutoBench: comparing the time performance of Haskell programs

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 53, Issue 7
        Haskell '18
        July 2018
        185 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/3299711
        Issue’s Table of Contents
        • cover image ACM Conferences
          Haskell 2018: Proceedings of the 11th ACM SIGPLAN International Symposium on Haskell
          September 2018
          185 pages
          ISBN:9781450358354
          DOI:10.1145/3242744

        Copyright © 2018 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 17 September 2018

        Check for updates

        Qualifiers

        • article
      • Article Metrics

        • Downloads (Last 12 months)19
        • Downloads (Last 6 weeks)3

        Other Metrics

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!