skip to main content
10.1145/1190216.1190226acmconferencesArticle/Chapter ViewAbstractPublication PagespoplConference Proceedingsconference-collections
Article

Compositional dynamic test generation

Published:17 January 2007Publication History

ABSTRACT

Dynamic test generation is a form of dynamic program analysis that attempts to compute test inputs to drive a program along a specific program path. Directed Automated Random Testing, or DART for short, blends dynamic test generation with model checking techniques with the goal of systematically executing all feasible program paths of a program while detecting various types of errors using run-time checking tools (like Purify, for instance). Unfortunately, systematically executing all feasible program paths does not scale to large, realistic programs.This paper addresses this major limitation and proposes to perform dynamic test generation compositionally, by adapting known techniques for interprocedural static analysis. Specifically, we introduce a new algorithm, dubbed SMART for Systematic Modular Automated Random Testing, that extends DART by testing functions in isolation, encoding test results as function summaries expressed using input preconditions and output postconditions, and then re-using those summaries when testing higher-level functions. We show that, for a fixed reasoning capability, our compositional approach to dynamic test generation (SMART) is both sound and complete compared to monolithic dynamic test generation (DART). In other words, SMART can perform dynamic test generation compositionally without any reduction in program path coverage. We also show that, given a bound on the maximum number of feasible paths in individual program functions, the number of program executions explored by SMART is linear in that bound, while the number of program executions explored by DART can be exponential in that bound. We present examples of C programs and preliminary experimental results that illustrate and validate empirically these properties.

References

  1. R. Alur, M. Benedikt, K. Etessami, P. Godefroid, T. Reps, and M. Yannakakis. Analysis of Recursive State Machines. TOPLAS, 27(4):786--818, July 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. R. Alur and M. Yannakakis. Model Checking of Hierarchical State Machines. In FSE'98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. D. Beyer, A. J. Chlipala, T. A. Henzinger, R. Jhala, and R. Majumdar. Generating Tests from Counterexamples. In ICSE'2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. C. Boyapati, S. Khurshid, and D. Marinov. Korat: Automated testing based on Java predicates. In ISSTA'2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. W.R. Bush, J.D. Pincus, and D.J. Sielaff. A static analyzer for finding dynamic programming errors. Software Practice and Experience, 30(7):775--802, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. H. Chen, D. Dean, and D. Wagner. Model Checking One Million Lines of C Code. In NDSS'04.Google ScholarGoogle Scholar
  7. C. Cadar and D. Engler. Execution Generated Test Cases: How to Make Systems Code Crash Itself. In SPIN'2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. A. Chakrabarti and P. Godefroid. Software Partitioning for Effective Automated Unit Testing. In EMSOFT'2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. C. Csallner and Y. Smaragdakis. Check'n Crash: Combining Static Checking and Testing. In ICSE'2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. Das, S. Lerner, and M. Seigle. ESP: Path-Sensitive Program Verification in Polynomial Time. In PLDI'2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. J. Edvardsson. A Survey on Automatic Test Data Generation. In Proceedings of the 2nd Conference on Computer Science and Engineering, pages 21--28, Linkoping, October 1999.Google ScholarGoogle Scholar
  12. P. Godefroid and N. Klarlund. Software Model Checking: Searching for Computations in the Abstract or the Concrete (Invited Paper). In IFM'2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. P. Godefroid, N. Klarlund, and K. Sen. DART: Directed Automated Random Testing. In PLDI'2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. N. Gupta, A. P. Mathur, and M. L. Soffa. Generating Test Data for Branch Coverage. In ASE'2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. P. Godefroid. Model Checking for Programming Languages using VeriSoft. In POPL'97. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. P. Godefroid. The Soundness of Bugs is What Matters (Position Paper). In BUGS'2005.Google ScholarGoogle Scholar
  17. S. Hallem, B. Chelf, Y. Xie, and D. Engler. A System and Language for Building System-Specific Static Analyses. In PLDI'2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. C. King. Symbolic Execution and Program Testing. Journal of the ACM, 19(7):385--394, 1976. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. B. Korel. A Dynamic Approach of Test Data Generation. In IEEE Conference on Software Maintenance, 1990.Google ScholarGoogle ScholarCross RefCross Ref
  20. G. J. Myers. The Art of Software Testing. Wiley, 1979. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. T. Reps, S. Horwitz, and M. Sagiv. Precise interprocedural dataflow analysis via graph reachability. In POPL'95. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. K. Sen, D. Marinov, and G. Agha. CUTE: A Concolic Unit Testing Engine for C. In FSE'2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. W. Visser, C. Pasareanu, and S. Khurshid. Test Input Generation with Java PathFinder. In ISSTA'2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. T. Xie, D. Marinov, W. Schulte, and D. Notkin. Symstra: A Framework for Generating Object-Oriented Unit Tests Using Symbolic Execution. In TACAS'2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. J. Yang, C. Sar, P. Twohey, C. Cadar, and D. Engler. Automatically Generating Malicious Disks using Symbolic Execution. In Proceedings of IEEE Security and Privacy'2006, Oakland, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Compositional dynamic test generation

                Recommendations

                Comments

                Login options

                Check if you have access through your login credentials or your institution to get full access on this article.

                Sign in

                PDF Format

                View or Download as a PDF file.

                PDF

                eReader

                View online with eReader.

                eReader
                About Cookies On This Site

                We use cookies to ensure that we give you the best experience on our website.

                Learn more

                Got it!