skip to main content
research-article

Relatively complete counterexamples for higher-order programs

Published:03 June 2015Publication History
Skip Abstract Section

Abstract

In this paper, we study the problem of generating inputs to a higher-order program causing it to error. We first approach the problem in the setting of PCF, a typed, core functional language and contribute the first relatively complete method for constructing counterexamples for PCF programs. The method is relatively complete with respect to a first-order solver over the base types of PCF. In practice, this means an SMT solver can be used for the effective, automated generation of higher-order counterexamples for a large class of programs. We achieve this result by employing a novel form of symbolic execution for higher-order programs. The remarkable aspect of this symbolic execution is that even though symbolic higher-order inputs and values are considered, the path condition remains a first-order formula. Our handling of symbolic function application enables the reconstruction of higher-order counterexamples from this first-order formula. After establishing our main theoretical results, we sketch how to apply the approach to untyped, higher-order, stateful languages with first-class contracts and show how counterexample generation can be used to detect contract violations in this setting. To validate our approach, we implement a tool generating counterexamples for erroneous modules written in Racket.

References

  1. C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. EXE: automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and Communications Security. ACM, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation. USENIX, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. K. Claessen and J. Hughes. QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming. ACM, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. S. A. Cook. Soundness and completeness of an axiom system for program verification. In SIAM Journal of Computing, 1978.Google ScholarGoogle Scholar
  5. L. De Moura and N. Bjørner. Z3: an efficient SMT solver. In Proceedings of the Theory and practice of software, 14th international conference on Tools and algorithms for the construction and analysis of systems. Springer-Verlag, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C. Dimoulas, R. B. Findler, C. Flanagan, and M. Felleisen. Correct blame for contracts: no more scapegoating. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. C. Dimoulas, S. Tobin-Hochstadt, and M. Felleisen. Complete monitors for behavioral contracts. In 21st European Symposium on Programming. Springer Berlin Heidelberg, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. R. B. Findler and M. Felleisen. Contracts for higher-order functions. In Proceedings of the seventh ACM SIGPLAN International Conference on Functional Programming. ACM, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. C. Flanagan. Hybrid type checking. In Conference record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. Flatt and PLT. Reference: Racket. Technical report, PLT Inc., 2010.Google ScholarGoogle Scholar
  11. J. S. Foster, T. Terauchi, and A. Aiken. Flow-sensitive type qualifiers. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. M. Greenberg, B. C. Pierce, and S. Weirich. Contracts made manifest. In Proceedings of the 37th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. P. Heidegger and P. Thiemann. Contract-Driven testing of JavaScript code. In Objects, Models, Components, Patterns. Springer Berlin Heidelberg, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. R. Hinze, J. Jeuring, and A. Löh. Typed contracts for functional programming. In Functional and Logic Programming. Springer, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. J. Hughes. Personal communication, 2015.Google ScholarGoogle Scholar
  17. M. Kawaguchi, P. Rondon, and R. Jhala. Dsolve: Safety verification via liquid types. In Computer Aided Verification. Springer Berlin Heidelberg, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. C. Klein, M. Flatt, and R. B. Findler. Random testing for higherorder, stateful programs. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. K. Knowles and C. Flanagan. Hybrid type checking. ACM Trans. Program. Lang. Syst., 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. N. Kobayashi. Model checking Higher-Order programs. J. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. N. Kobayashi, R. Sato, and H. Unno. Predicate abstraction and CEGAR for higher-order model checking. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. P. C. Nguyễn and D. Van Horn. Relatively complete counterexamples for Higher-Order programs. CoRR, abs/1411.3967, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. P. C. Nguyễn, S. Tobin-Hochstadt, and D. Van Horn. Soft contract verification. In Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. P. M. Rondon, M. Kawaguci, and R. Jhala. Liquid types. In Proceedings of the 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. D. S. Scott. A type-theoretical alternative to ISWIM, CUCH, OWHY. Theoretical Computer Science, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. E. L. Seidel, N. Vazou, and R. Jhala. Type targeted testing. In 21st European Symposium on Programming. Springer Berlin Heidelberg, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. V. St-Amour, S. Tobin-Hochstadt, M. Flatt, and M. Felleisen. Typing the numeric tower. In Practical Aspects of Declarative Languages. Springer Berlin Heidelberg, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. T. Terauchi. Dependent types from counterexamples. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. S. Tobin-Hochstadt and M. Felleisen. Logical types for untyped languages. In Proceedings of the ACM International Conference on Functional Programming. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. S. Tobin-Hochstadt and D. Van Horn. Higher-order symbolic execution via contracts. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. D. Van Horn and M. Might. Abstracting abstract machines. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. D. Vytiniotis, S. Peyton Jones, K. Claessen, and D. Rosén. HALO: Haskell to logic through denotational semantics. In Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Y. Xie and A. Aiken. Scalable error detection using boolean satisfiability. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. D. N. Xu. Hybrid contract checking via symbolic simplification. In Proceedings of the ACM SIGPLAN 2012 Workshop on Partial Evaluation and Program Manipulation. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. D. N. Xu, S. Peyton Jones, and S. Claessen. Static contract checking for Haskell. In Proceedings of the 36th Annual ACM SIGPLANSIGACT Symposium on Principles of Programming Languages. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. J. Yang, P. Twohey, D. Engler, and M. Musuvathi. Using model checking to find serious file system errors. In Sixth Symposium on Operating Systems Design and Implementation. USENIX, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. H. Zhu and S. Jagannathan. Compositional and lightweight dependent type inference for ML. In Conference on Verification, Model-Checking and Abstract Interpretation, 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Relatively complete counterexamples for higher-order programs

                Recommendations

                Comments

                Login options

                Check if you have access through your login credentials or your institution to get full access on this article.

                Sign in

                Full Access

                • Published in

                  cover image ACM SIGPLAN Notices
                  ACM SIGPLAN Notices  Volume 50, Issue 6
                  PLDI '15
                  June 2015
                  630 pages
                  ISSN:0362-1340
                  EISSN:1558-1160
                  DOI:10.1145/2813885
                  • Editor:
                  • Andy Gill
                  Issue’s Table of Contents
                  • cover image ACM Conferences
                    PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation
                    June 2015
                    630 pages
                    ISBN:9781450334686
                    DOI:10.1145/2737924

                  Copyright © 2015 ACM

                  Publisher

                  Association for Computing Machinery

                  New York, NY, United States

                  Publication History

                  • Published: 3 June 2015

                  Check for updates

                  Qualifiers

                  • research-article

                PDF Format

                View or Download as a PDF file.

                PDF

                eReader

                View online with eReader.

                eReader
                About Cookies On This Site

                We use cookies to ensure that we give you the best experience on our website.

                Learn more

                Got it!