skip to main content

Dynaplex: analyzing program complexity using dynamically inferred recurrence relations

Published:15 October 2021Publication History
Skip Abstract Section

Abstract

Being able to detect program runtime complexity is useful in many tasks (e.g., checking expected performance and identifying potential security vulnerabilities). In this work, we introduce a new dynamic approach for inferring the asymptotic complexity bounds of recursive programs. From program execution traces, we learn recurrence relations and solve them using pattern matching to obtain closed-form solutions representing the complexity bounds of the program. This approach allows us to efficiently infer simple recurrence relations that represent nontrivial, potentially nonlinear polynomial and non-polynomial, complexity bounds.

We present Dynaplex, a tool that implements these ideas to automatically generate recurrence relations from execution traces. Our preliminary results on popular and challenging recursive programs show that Dynaplex can learn precise relations capturing worst-case complexity bounds (e.g., O(n logn) for mergesort, O(2n) for Tower of Hanoi and O(n1.58) for Karatsuba’s multiplication algorithm).

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

In this work, we introduce a new dynamic approach for inferring the asymptotic complexity bounds of recursive programs. From program execution traces, we learn recurrence relations and solve them using pattern matching to obtain closed-form solutions representing the complexity bounds of the program. This approach allows us to efficiently infer simple recurrence relations that represent nontrivial, potentially nonlinear polynomial and non-polynomial, complexity bounds.

References

  1. Elvira Albert, Puri Arenas, Samir Genaim, Miguel Gómez-Zamalloa, German Puebla, Diana Ramírez, G Román, and Damiano Zanardini. 2009. Termination and cost analysis with COSTA and its user interfaces. Electronic Notes in Theoretical Computer Science, 258, 1 (2009), 109–121.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Elvira Albert, Puri Arenas, Samir Genaim, and Germán Puebla. 2008. Automatic inference of upper bounds for recurrence relations in cost analysis. In International Static Analysis Symposium. 221–237.Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Elvira Albert, Puri Arenas, Samir Genaim, Germán Puebla, and Damiano Zanardini. 2007. Cost Analysis of Java Bytecode. In European Symposium on Programming. 157–172.Google ScholarGoogle Scholar
  4. Thomas Ball and Sriram K. Rajamani. 2001. Automatically Validating Temporal Safety Properties of Interfaces. In SPIN Symposium on Model Checking of Software. Springer, 103–122.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Jason Breck, John Cyphert, Zachary Kincaid, and Thomas Reps. 2020. Templates and Recurrences: Better Together. In Proceedings of the 41st ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2020). Association for Computing Machinery, New York, NY, USA. 688–702. isbn:9781450376136Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Marc Brockschmidt, Fabian Emmes, Stephan Falke, Carsten Fuhs, and Jürgen Giesl. 2016. Analyzing runtime and size complexity of integer programs. ACM Transactions on Programming Languages and Systems (TOPLAS), 38, 4 (2016), 1–50.Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Jacob Burnim, Sudeep Juvekar, and Koushik Sen. 2009. WISE: Automated test generation for worst-case complexity. In 2009 IEEE 31st International Conference on Software Engineering. 463–473.Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Krishnendu Chatterjee, Hongfei Fu, and Amir Kafshdar Goharshady. 2019. Non-polynomial worst-case analysis of recursive programs. ACM Transactions on Programming Languages and Systems, 41, 4 (2019), 1–52.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. 2009. Introduction to algorithms. MIT press.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Joseph W Cutler, Daniel R Licata, and Norman Danner. 2020. Denotational recurrence extraction for amortized analysis. Proceedings of the ACM on Programming Languages, 4, ICFP (2020), 1–29.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Manuvir Das, Sorin Lerner, and Mark Seigle. 2002. ESP: path-sensitive program verification in polynomial time. SIGPLAN Notices, 37, 5 (2002), 57–68.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In Tools and Algorithms for the Construction and Analysis of Systems. Springer, 337–340.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Steven de Oliveira, Saddek Bensalem, and Virgile Prevosto. 2016. Polynomial invariants by linear algebra. In International Symposium on Automated Technology for Verification and Analysis. 479–494.Google ScholarGoogle ScholarCross RefCross Ref
  14. Paul J Drongowski. 2007. Instruction-based sampling: A new performance analysis technique for AMD family 10h processors. Advanced Micro Devices.Google ScholarGoogle Scholar
  15. Manuel Eberl. 2019. Verified solving and asymptotics of linear recurrences. In Proceedings of the 8th ACM SIGPLAN International Conference on Certified Programs and Proofs. 27–37.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jeff Erickson. 2013. Solving Recurrences. http://jeffe.cs.illinois.edu/teaching/algorithms/notes/99-recurrences.pdfGoogle ScholarGoogle Scholar
  17. Michael D. Ernst. 2000. Dynamically detecting likely program invariants. Ph.D. Dissertation. University of Washington.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Michael D. Ernst, Jeff H. Perkins, Philip J. Guo, Stephen McCamant, Carlos Pacheco, Matthew S. Tschantz, and Chen Xiao. 2007. The Daikon system for dynamic detection of likely invariants. Science of Computer Programming, 35–45.Google ScholarGoogle Scholar
  19. Azadeh Farzan and Zachary Kincaid. 2015. Compositional recurrence analysis. In 2015 Formal Methods in Computer-Aided Design (FMCAD). 57–64.Google ScholarGoogle Scholar
  20. Philippe Flajolet, Bruno Salvy, and Paul Zimmermann. 1991. Automatic average-case analysis of algorithms. Theoretical Computer Science, 79, 1 (1991), 37–109.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Simon F Goldsmith, Alex S Aiken, and Daniel S Wilkerson. 2007. Measuring empirical computational complexity. In Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering. 395–404.Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Susan L Graham, Peter B Kessler, and Marshall K McKusick. 1982. Gprof: A call graph execution profiler. ACM Sigplan Notices, 17, 6 (1982), 120–126.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Bernd Grobauer. 2001. Cost recurrences for DML programs. ACM SIGPLAN Notices, 36, 10 (2001), 253–264.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Jerrold W Grossman and R Suzanne Zeitman. 1988. An inherently iterative computation of Ackermann’s function. Theoretical computer science, 57, 2-3 (1988), 327–330.Google ScholarGoogle Scholar
  25. Bhargav S Gulavani and Sumit Gulwani. 2008. A numerical abstract domain based on expression abstraction and max operator with application in timing analysis. In International Conference on Computer Aided Verification. 370–384.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Sumit Gulwani. 2009. SPEED: Symbolic Complexity Bound Analysis. In Computer Aided Verification. Springer-Verlag, 51–62.Google ScholarGoogle Scholar
  27. Sumit Gulwani, Sagar Jain, and Eric Koskinen. 2009. Control-flow Refinement and Progress Invariants for Bound Analysis. In Programming Language Design and Implementation. 375–385.Google ScholarGoogle Scholar
  28. Sumit Gulwani, Krishna K Mehra, and Trishul Chilimbi. 2009. Speed: precise and efficient static estimation of program computational complexity. ACM Sigplan Notices, 44, 1 (2009), 127–139.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Sumit Gulwani, Krishna K. Mehra, and Trishul M. Chilimbi. 2009. SPEED: precise and efficient static estimation of program computational complexity. In Principles of Programming Languages. ACM, 127–139.Google ScholarGoogle Scholar
  30. Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernàndez del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. 2020. Array programming with NumPy. Nature, 585, 7825 (2020), Sept., 357–362. https://doi.org/10.1038/s41586-020-2649-2 Google ScholarGoogle ScholarCross RefCross Ref
  31. Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar, and Gregoire Sutre. 2002. Lazy Abstraction. In Principles of Programming Languages. ACM, 58–70.Google ScholarGoogle Scholar
  32. Jan Hoffmann, Klaus Aehlig, and Martin Hofmann. 2011. Multivariate amortized resource analysis. In Principles of Programming Languages. 357–370.Google ScholarGoogle Scholar
  33. Jan Hoffmann and Martin Hofmann. 2010. Amortized resource analysis with polynomial potential. In European Symposium on Programming. 287–306.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Martin Hofmann and Steffen Jost. 2003. Static prediction of heap space usage for first-order functional programs. ACM SIGPLAN Notices, 38, 1 (2003), 185–197.Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Didier Ishimwe, Kim Hao Nguyen, and Thanhvu Nguyen. 2021. Software Artifact for the OOPSLA’21 Paper Titled "Dynaplex: Analyzing Program Complexity using Dynamically Inferred Recurrence Relations". https://doi.org/10.5281/zenodo.5421762 Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Anirudh Jayaraman. 2015. Karatsuba Multiplication Algorithm - Python Code. https://pythonandr.com/2015/10/13/karatsuba-multiplication-algorithm-python-code/Google ScholarGoogle Scholar
  37. Zachary Kincaid, Jason Breck, Ashkan Forouhi Boroujeni, and Thomas Reps. 2017. Compositional recurrence analysis revisited. ACM SIGPLAN Notices, 52, 6 (2017), 248–262.Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Zachary Kincaid, Jason Breck, John Cyphert, and Thomas Reps. 2019. Closed forms for numerical loops. Proceedings of the ACM on Programming Languages, 3, POPL (2019), 1–29.Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Ton Chanh Le, Timos Antonopoulos, Parisa Fathololumi, Eric Koskinen, and ThanhVu Nguyen. 2020. DynamiTe: dynamic termination and non-termination proofs. Proceedings of the ACM on Programming Languages, 4, OOPSLA (2020), 1–30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Ton Chanh Le, Guolong Zheng, and ThanhVu Nguyen. 2019. SLING: using dynamic analysis to infer program invariants in separation logic. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. 788–801.Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Caroline Lemieux, Rohan Padhye, Koushik Sen, and Dawn Song. 2018. Perffuzz: Automatically generating pathological inputs. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis. 254–265.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Xavier Leroy. 2006. Formal certification of a compiler back-end or: programming a compiler with a proof assistant. In Principles of Programming Languages. ACM, 42–54.Google ScholarGoogle Scholar
  43. Nicholas Nethercote and Julian Seward. 2003. Valgrind: A program supervision framework. Electronic notes in theoretical computer science, 89, 2 (2003), 44–66.Google ScholarGoogle ScholarCross RefCross Ref
  44. ThanhVu Nguyen, Timos Antonopoulos, Andrew Ruef, and Michael Hicks. 2017. A Counterexample-guided Approach to Finding Numerical Invariants. In Foundations of Software Engineering. ACM, 605–615.Google ScholarGoogle Scholar
  45. ThanhVu Nguyen, Matthew Dwyer, and William Visser. 2017. SymInfer: Inferring Program Invariants using Symbolic States. In Automated Software Engineering. IEEE, 804–814.Google ScholarGoogle Scholar
  46. ThanhVu Nguyen, Didier Ishimwe, Alexey Malyshev, Timos Antonopoulos, and Quoc-Sang Phan. 2020. Using dynamically inferred invariants to analyze program runtime complexity. In Proceedings of the 3rd ACM SIGSOFT International Workshop on Software Security from Design to Deployment. 11–14.Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. ThanhVu Nguyen, Deepak Kapur, Westley Weimer, and Stephanie Forrest. 2012. Using Dynamic Analysis to Discover Polynomial and Array Invariants. In International Conference on Software Engineering. IEEE, 683–693.Google ScholarGoogle ScholarCross RefCross Ref
  48. Ocaml.org. 2021. 99 Problems in OCaml. https://ocaml.org/learn/tutorials/99problems.htmlGoogle ScholarGoogle Scholar
  49. Theofilos Petsios, Jason Zhao, Angelos D Keromytis, and Suman Jana. 2017. Slowfuzz: Automated domain-independent detection of algorithmic complexity vulnerabilities. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2155–2168.Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Moritz Sinn, Florian Zuleger, and Helmut Veith. 2014. A simple and scalable static analysis for bound analysis and amortized complexity analysis. In International Conference on Computer Aided Verification. 745–761.Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Moritz Sinn, Florian Zuleger, and Helmut Veith. 2017. Complexity and resource bound analysis of imperative programs using difference constraints. Journal of automated reasoning, 59, 1 (2017), 3–45.Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Linhai Song and Shan Lu. 2014. Statistical debugging for real-world performance problems. ACM SIGPLAN Notices, 49, 10 (2014), 561–578.Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Akhilesh Srikanth, Burak Sahin, and William R Harris. 2017. Complexity verification using guided theorem enumeration. ACM SIGPLAN Notices, 52, 1 (2017), 639–652.Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Termination Competitions. 2021. Complexity Benchmarks. https://termcomp.github.io/Google ScholarGoogle Scholar
  55. Di Wang and Jan Hoffmann. 2019. Type-guided worst-case input generation. Proceedings of the ACM on Programming Languages, 3, POPL (2019), 1–30.Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Peng Wang, Di Wang, and Adam Chlipala. 2017. TiML: a functional language for practical complexity analysis with invariants. Proceedings of the ACM on Programming Languages, 1, OOPSLA (2017), 79.Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. Jianan Yao, Gabriel Ryan, Justin Wong, Suman Jana, and Ronghui Gu. 2020. Learning nonlinear loop invariants with gated continuous logic networks. In Programming Language Design and Implementation. ACM, 106–120.Google ScholarGoogle Scholar
  58. Dmitrijs Zaparanuks and Matthias Hauswirth. 2012. Algorithmic profiling. In Proceedings of the 33rd ACM SIGPLAN conference on Programming Language Design and Implementation. 67–76.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Dynaplex: analyzing program complexity using dynamically inferred recurrence relations

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!