Abstract
In this paper, we study the problem of generating inputs to a higher-order program causing it to error. We first approach the problem in the setting of PCF, a typed, core functional language and contribute the first relatively complete method for constructing counterexamples for PCF programs. The method is relatively complete with respect to a first-order solver over the base types of PCF. In practice, this means an SMT solver can be used for the effective, automated generation of higher-order counterexamples for a large class of programs. We achieve this result by employing a novel form of symbolic execution for higher-order programs. The remarkable aspect of this symbolic execution is that even though symbolic higher-order inputs and values are considered, the path condition remains a first-order formula. Our handling of symbolic function application enables the reconstruction of higher-order counterexamples from this first-order formula. After establishing our main theoretical results, we sketch how to apply the approach to untyped, higher-order, stateful languages with first-class contracts and show how counterexample generation can be used to detect contract violations in this setting. To validate our approach, we implement a tool generating counterexamples for erroneous modules written in Racket.
- C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. EXE: automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and Communications Security. ACM, 2006. Google Scholar
Digital Library
- C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation. USENIX, 2008. Google Scholar
Digital Library
- K. Claessen and J. Hughes. QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming. ACM, 2000. Google Scholar
Digital Library
- S. A. Cook. Soundness and completeness of an axiom system for program verification. In SIAM Journal of Computing, 1978.Google Scholar
- L. De Moura and N. Bjørner. Z3: an efficient SMT solver. In Proceedings of the Theory and practice of software, 14th international conference on Tools and algorithms for the construction and analysis of systems. Springer-Verlag, 2008. Google Scholar
Digital Library
- C. Dimoulas, R. B. Findler, C. Flanagan, and M. Felleisen. Correct blame for contracts: no more scapegoating. In Proceedings of the 38th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2011. Google Scholar
Digital Library
- C. Dimoulas, S. Tobin-Hochstadt, and M. Felleisen. Complete monitors for behavioral contracts. In 21st European Symposium on Programming. Springer Berlin Heidelberg, 2012. Google Scholar
Digital Library
- R. B. Findler and M. Felleisen. Contracts for higher-order functions. In Proceedings of the seventh ACM SIGPLAN International Conference on Functional Programming. ACM, 2002. Google Scholar
Digital Library
- C. Flanagan. Hybrid type checking. In Conference record of the 33rd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2006. Google Scholar
Digital Library
- M. Flatt and PLT. Reference: Racket. Technical report, PLT Inc., 2010.Google Scholar
- J. S. Foster, T. Terauchi, and A. Aiken. Flow-sensitive type qualifiers. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2002. Google Scholar
Digital Library
- P. Godefroid, N. Klarlund, and K. Sen. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2005. Google Scholar
Digital Library
- M. Greenberg, B. C. Pierce, and S. Weirich. Contracts made manifest. In Proceedings of the 37th annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2010. Google Scholar
Digital Library
- P. Heidegger and P. Thiemann. Contract-Driven testing of JavaScript code. In Objects, Models, Components, Patterns. Springer Berlin Heidelberg, 2010. Google Scholar
Digital Library
- R. Hinze, J. Jeuring, and A. Löh. Typed contracts for functional programming. In Functional and Logic Programming. Springer, 2006. Google Scholar
Digital Library
- J. Hughes. Personal communication, 2015.Google Scholar
- M. Kawaguchi, P. Rondon, and R. Jhala. Dsolve: Safety verification via liquid types. In Computer Aided Verification. Springer Berlin Heidelberg, 2010. Google Scholar
Digital Library
- C. Klein, M. Flatt, and R. B. Findler. Random testing for higherorder, stateful programs. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, 2010. Google Scholar
Digital Library
- K. Knowles and C. Flanagan. Hybrid type checking. ACM Trans. Program. Lang. Syst., 2010. Google Scholar
Digital Library
- N. Kobayashi. Model checking Higher-Order programs. J. ACM, 2013. Google Scholar
Digital Library
- N. Kobayashi, R. Sato, and H. Unno. Predicate abstraction and CEGAR for higher-order model checking. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2011. Google Scholar
Digital Library
- P. C. Nguyễn and D. Van Horn. Relatively complete counterexamples for Higher-Order programs. CoRR, abs/1411.3967, 2015. Google Scholar
Digital Library
- P. C. Nguyễn, S. Tobin-Hochstadt, and D. Van Horn. Soft contract verification. In Proceedings of the 19th ACM SIGPLAN International Conference on Functional Programming. ACM, 2014. Google Scholar
Digital Library
- P. M. Rondon, M. Kawaguci, and R. Jhala. Liquid types. In Proceedings of the 2008 ACM SIGPLAN Conference on Programming Language Design and Implementation. ACM, 2008. Google Scholar
Digital Library
- D. S. Scott. A type-theoretical alternative to ISWIM, CUCH, OWHY. Theoretical Computer Science, 1993. Google Scholar
Digital Library
- E. L. Seidel, N. Vazou, and R. Jhala. Type targeted testing. In 21st European Symposium on Programming. Springer Berlin Heidelberg, 2015. Google Scholar
Digital Library
- V. St-Amour, S. Tobin-Hochstadt, M. Flatt, and M. Felleisen. Typing the numeric tower. In Practical Aspects of Declarative Languages. Springer Berlin Heidelberg, 2012. Google Scholar
Digital Library
- T. Terauchi. Dependent types from counterexamples. In Proceedings of the 37th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2010. Google Scholar
Digital Library
- S. Tobin-Hochstadt and M. Felleisen. Logical types for untyped languages. In Proceedings of the ACM International Conference on Functional Programming. ACM, 2010. Google Scholar
Digital Library
- S. Tobin-Hochstadt and D. Van Horn. Higher-order symbolic execution via contracts. In Proceedings of the ACM International Conference on Object Oriented Programming Systems Languages and Applications. ACM, 2012. Google Scholar
Digital Library
- D. Van Horn and M. Might. Abstracting abstract machines. In Proceedings of the 15th ACM SIGPLAN International Conference on Functional Programming. ACM, 2010. Google Scholar
Digital Library
- D. Vytiniotis, S. Peyton Jones, K. Claessen, and D. Rosén. HALO: Haskell to logic through denotational semantics. In Proceedings of the 40th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages. ACM, 2013. Google Scholar
Digital Library
- Y. Xie and A. Aiken. Scalable error detection using boolean satisfiability. In Proceedings of the 32nd ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages. ACM, 2005. Google Scholar
Digital Library
- D. N. Xu. Hybrid contract checking via symbolic simplification. In Proceedings of the ACM SIGPLAN 2012 Workshop on Partial Evaluation and Program Manipulation. ACM, 2012. Google Scholar
Digital Library
- D. N. Xu, S. Peyton Jones, and S. Claessen. Static contract checking for Haskell. In Proceedings of the 36th Annual ACM SIGPLANSIGACT Symposium on Principles of Programming Languages. ACM, 2009. Google Scholar
Digital Library
- J. Yang, P. Twohey, D. Engler, and M. Musuvathi. Using model checking to find serious file system errors. In Sixth Symposium on Operating Systems Design and Implementation. USENIX, 2004. Google Scholar
Digital Library
- H. Zhu and S. Jagannathan. Compositional and lightweight dependent type inference for ML. In Conference on Verification, Model-Checking and Abstract Interpretation, 2013.Google Scholar
Digital Library
Index Terms
Relatively complete counterexamples for higher-order programs
Recommendations
Relatively complete counterexamples for higher-order programs
PLDI '15: Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and ImplementationIn this paper, we study the problem of generating inputs to a higher-order program causing it to error. We first approach the problem in the setting of PCF, a typed, core functional language and contribute the first relatively complete method for ...
Automating relatively complete verification of higher-order functional programs
POPL '13We present an automated approach to relatively completely verifying safety (i.e., reachability) property of higher-order functional programs. Our contribution is two-fold. First, we extend the refinement type system framework employed in the recent work ...
Relatively complete refinement type system for verification of higher-order non-deterministic programs
This paper considers verification of non-deterministic higher-order functional programs. Our contribution is a novel type system in which the types are used to express and verify (conditional) safety, termination, non-safety, and non-termination ...






Comments