Abstract
Symbolic reasoning about large programs is bound to be imprecise. How to deal with this imprecision is a fundamental problem in program analysis. Imprecision forces approximation. Traditional static program verification builds "may" over-approximations of the program behaviors to check universal "for-all-paths" properties, while automatic test generation requires "must" under-approximations to check existential "for-some-path" properties.
In this paper, we introduce a new approach to test generation where tests are derived from validity proofs of first-order logic formulas, rather than satisfying assignments of quantifier-free first-order logic formulas as usual. Two key ingredients of this higher-order test generation are to (1) represent complex/unknown program functions/instructions causing imprecision in symbolic execution by uninterpreted functions, and (2) record uninterpreted function samples capturing input-output pairs observed at execution time for those functions. We show that higher-order test generation generalizes and is more precise than simplifying complex symbolic expressions using their concrete runtime values. We present several program examples where our approach can exercise program paths and find bugs missed by previous techniques. We discuss the implementability and applications of this approach. We also explain in what sense dynamic test generation is more powerful than static test generation.
- S. Anand, P. Godefroid, and N. Tillmann. Demand-Driven Compositional Symbolic Execution. In Proceedings of TACAS'2008, volume 4963 of Lecture Notes in Computer Science, pages 367--381, Budapest, April 2008. Springer-Verlag. Google Scholar
Digital Library
- S. Artzi, A. Kiezun, J. Dolby, F. Tip, D. Dig, A. M. Paradkar, and M. D. Ernst. Finding Bugs in Web Applications Using Dynamic Test Generation and Explicit-State Model Checking. IEEE Trans. Software Eng., 36(4):474--494, 2010. Google Scholar
Digital Library
- L. Bachmair and H. Ganzinger. Resolution Theorem Proving. In Handbook of Automated Reasoning, pages 19--99. 2001.Google Scholar
Cross Ref
- M. Barnett, B. E. Chang, R. DeLine, B. Jacobs, and K. R. M. Leino. Boogie: A modular reusable verifier for object-oriented programs. In Proceedings of FMCO'2005, volume 4111 of Lecture Notes in Computer Science, pages 364--387. Springer-Verlag, September 2006. Google Scholar
Digital Library
- A. Blass, Y. Gurevich, L. Nachmanson, and M. Veanes. Play to Test. In Proceedings of FATES'2005, Edinburgh, July 2005. Google Scholar
Digital Library
- C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. EXE: Automatically Generating Inputs of Death. In ACM CCS, 2006. Google Scholar
Digital Library
- C. Cadar, P. Godefroid, S. Khurshid, C.S. Pasareanu, K. Sen, N. Tillmann, and W. Visser. Symbolic Execution for Software Testing in Practice -- Preliminary Assessment. In Proceedings of ICSE'2011, Honolulu, May 2011. Google Scholar
Digital Library
- S. Chandra, S. J. Fink, and M. Sridharan. Snugglebug: A Powerful Approach to Weakest Preconditions. In Proceedings of PLDI'2009, Dublin, June 2009. Google Scholar
Digital Library
- L. de Moura and N. Bjorner. Z3: An Efficient SMT Solver. In Proceedings of TACAS'2008, volume 4963 of Lecture Notes in Computer Science, pages 337--340, Budapest, April 2008. Springer-Verlag. Google Scholar
Digital Library
- M. Emmi, R. Majumdar, and K. Sen. Dynamic Test Input Generation for Database Applications. In Proceedings of ISSTA'2007, pages 151--162, 2007. Google Scholar
Digital Library
- P. Godefroid. Compositional Dynamic Test Generation. In Proceedings of POPL'2007, pages 47--54, Nice, January 2007. Google Scholar
Digital Library
- P. Godefroid. Software Model Checking Improving Security of a Billion Computers. In Proceedings of SPIN'2009, volume 5578 of Lecture Notes in Computer Science, page 1, Grenoble, June 2009. Springer-Verlag. Google Scholar
Digital Library
- P. Godefroid, M. Huth, and R. Jagadeesan. Abstraction-based Model Checking using Modal Transition Systems. In Proceedings of CONCUR'2001, volume 2154 of Lecture Notes in Computer Science, pages 426--440, Aalborg, August 2001. Springer-Verlag. Google Scholar
Digital Library
- P. Godefroid, A. Kiezun, and M. Y. Levin. Grammar-based Whitebox Fuzzing. In Proceedings of PLDI'2008, pages 206--215, Tucson, June 2008. Google Scholar
Digital Library
- P. Godefroid, N. Klarlund, and K. Sen. DART: Directed Automated Random Testing. In Proceedings of PLDI'2005, pages 213--223, Chicago, June 2005. Google Scholar
Digital Library
- P. Godefroid, M.Y. Levin, and D. Molnar. Automated Whitebox Fuzz Testing. In Proceedings of NDSS'2008, pages 151--166, San Diego, February 2008.Google Scholar
- P. Godefroid, A.V. Nori, S.K. Rajamani, and S.D. Tetali. Compositional May-Must Program Analysis: Unleashing The Power of Alternation. In Proceedings of POPL'2010, pages 43--55, Madrid, January 2010. Google Scholar
Digital Library
- J. Hoenicke, K. R. M. Leino, A. Podelski, M. Schaf, and Th. Wies. It's doomed; we can prove it. In Proceedings of 2009 World Congress on Formal Methods, 2009. Google Scholar
Digital Library
- Sarfraz Khurshid, Corina S. Păsăreanu, and Willem Visser. Generalized Symbolic Execution for Model Checking and Testing. In Proceeding of TACAS'2003, April 2003. Google Scholar
Digital Library
- J. C. King. Symbolic Execution and Program Testing. Journal of the ACM, 19(7):385--394, 1976. Google Scholar
Digital Library
- B. Korel. A Dynamic Approach of Test Data Generation. In IEEE Conference on Software Maintenance, pages 311--317, San Diego, November 1990.Google Scholar
- D. Molnar, X. C. Li, and D. Wagner. Dynamic test generation to find integer bugs in x86 binary linux programs. In Proc. of the 18th Usenix Security Symposium, Aug 2009. Google Scholar
Digital Library
- P. Saxena, D. Akhawe, S. Hanna, F. Mao, S. McCamant, and D. Song. A Symbolic Execution Framework for JavaScript. In IEEE Symposium on Security and Privacy, pages 513--528, 2010. Google Scholar
Digital Library
- N. Tillmann and J. de Halleux. Pex - White Box Test Generation for .NET. In Proceedings of TAP'2008, volume 4966 of Lecture Notes in Computer Science, pages 134--153. Springer-Verlag, April 2008. Google Scholar
Digital Library
- M. Yannakakis. Testing, Optimization, and Games. In Proceedings of LICS'2004, pages 78--88, Turku, July 2004. Google Scholar
Digital Library
Index Terms
Higher-order test generation
Recommendations
Compositional dynamic test generation
Proceedings of the 2007 POPL ConferenceDynamic test generation is a form of dynamic program analysis that attempts to compute test inputs to drive a program along a specific program path. Directed Automated Random Testing, or DART for short, blends dynamic test generation with model checking ...
Higher-order test generation
PLDI '11: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and ImplementationSymbolic reasoning about large programs is bound to be imprecise. How to deal with this imprecision is a fundamental problem in program analysis. Imprecision forces approximation. Traditional static program verification builds "may" over-approximations ...
Automating relatively complete verification of higher-order functional programs
POPL '13We present an automated approach to relatively completely verifying safety (i.e., reachability) property of higher-order functional programs. Our contribution is two-fold. First, we extend the refinement type system framework employed in the recent work ...







Comments