Abstract
We present a novel technique for producing bounded exhaustive test suites from hybrid invariants, i.e., invariants that are expressed imperatively, declaratively, or as a combination of declarative and imperative predicates. Hybrid specifications are processed using known mechanisms for the imperative and declarative parts, but combined in a way that enables us to exploit information from the declarative side, such as tight bounds computed from the declarative specification, to improve the search both on the imperative and declarative sides. Moreover, our technique automatically evaluates different possible ways of processing the imperative side, and the alternative settings (imperative or declarative) for parts of the invariant available both declaratively and imperatively, to decide the most convenient invariant configuration with respect to efficiency in test generation. This is achieved by transcoping, i.e., by assessing the efficiency of the different alternatives on small scopes (where generation times are negligible), and then extrapolating the results to larger scopes.
We also show experiments involving collection classes that support the effectiveness of our technique, by demonstrating that (i) bounded exhaustive suites can be computed from hybrid invariants significantly more efficiently than doing so using state-of-the-art purely imperative and purely declarative approaches, and (ii) our technique is able to automatically determine efficient hybrid invariants, in the sense that they lead to an efficient computation of bounded exhaustive suites, using transcoping.
- P. Abad, N. Aguirre, V. S. Bengolea, D. Ciolek, M. F. Frias, J. P. Galeotti, T. Maibaum, M. Moscato, N. Rosner and I. Vissani, Improving Test Generation under Rich Contracts by Tight Bounds and Incremental SAT Solving, in ICST 2013. Google Scholar
Digital Library
- C. Boyapati, S. Khurshid and D. Marinov, Korat: Automated Testing based on Java Predicates, in ISSTA 2002. Google Scholar
Digital Library
- L. Burdy, Y. Cheon, D. R. Cok, M. D. Ernst, J. R. Kiniry, G. T. Leavens, K. Rustan M. Leino and E. Poll, An overview of JML tools and applications, in STTT 7(3), Springer, 2005. Google Scholar
Digital Library
- F. S.-H. Chang and D. Jackson. Symbolic model checking of declarative relational models. In ICSE 2006. Google Scholar
Digital Library
- K. Claessen and J. Hughes, QuickCheck: a lightweight tool for random testing of Haskell programs, in ICFP 2000. Google Scholar
Digital Library
- L. De Moura and N. Bjørner. Z3: An efficient SMT solver. In TACAS 2008. Google Scholar
Digital Library
- G. Dennis, F. Chang, D. Jackson. Verification of Code with SAT, in ISSTA 2006, ACM, 2006. Google Scholar
Digital Library
- B. Dutertre and L. D. Moura. The Yices SMT solver. Technical report, 2006.Google Scholar
- N. Een and N. Sorensson. An Extensible SAT-solver. In SAT 2003.Google Scholar
- M. Frias, J. Galeotti, C. Pombo, and N. Aguirre. DynAlloy: Upgrading Alloy with actions. In ICSE 2005. Google Scholar
Digital Library
- J.P. Galeotti, N. Rosner, C. López Pombo, M. Frias, TACO: Efficient SAT-Based Bounded Verification Using Symmetry Breaking and Tight Bounds. IEEE TSE 39(9): 1283--1307 (2013). Google Scholar
Digital Library
- S. Ganov, S. Khurshid, D. E. Perry. Annotations for Alloy: Automated Incremental Analysis Using Domain Specific Solvers. In ICFEM 2012. Google Scholar
Digital Library
- J. Geldenhuys, N. Aguirre, M. F. Frias and W. Visser, Bounded Lazy Initialization, in NFM 2013, LNCS, Springer, 2013.Google Scholar
Cross Ref
- M. Gligoric, T. Gvero, V. Jagannath, S. Khurshid, V. Kuncak and D. Marinov, Test generation through programming in UDITA, in ICSE 2010, Cape Town, South Africa. Google Scholar
Digital Library
- G. J. Holzmann. The model checker SPIN. IEEE TSE 1997. Google Scholar
Digital Library
- D. Jackson, Software Abstractions: Logic, Language and Analysis, The MIT Press, 2006. Google Scholar
Digital Library
- C. Kaner, J. Bach and B. Pettichord, Lessons Learned in Software Testing, Wiley, 2001. Google Scholar
Digital Library
- S. A. Khalek. Systematic Testing Using Test Summaries: Effective and Efficient Testing of Relational Applications. Ph.D. Thesis. University of Texas at Austin, 2011.Google Scholar
- S. A. Khalek, V. P. Narayanan, and S. Khurshid. Mixed constraints for test input generation - An initial exploration. In ASE 2011 (Short paper). Google Scholar
Digital Library
- S. A. Khalek, G. Yang, L. Zhang, D. Marinov and S. Khurshid, TestEra: A Tool for Testing Java Programs using Alloy Specifications, in ASE 2011, IEEE, 2011. Google Scholar
Digital Library
- S. Khurshid and D. Marinov, TestEra: Specification-Based Testing of Java Programs Using SAT, Automated Software Engineering 11(4), Springer, 2004. Google Scholar
Digital Library
- A. S. Koksal, V. Kuncak, and P. Suter. Constraints as Control. In POPL 2012. Google Scholar
Digital Library
- B. Liskov and J. Guttag, Program Development in Java: Abstraction, Specification, and Object-Oriented Design, Addison-Wesley, 2000. Google Scholar
Digital Library
- A. Milicevic, S. Misailovic, D. Marinov and S. Khurshid, Korat: A Tool for Generating Structurally Complex Test Inputs, in ICSE 2007, IEEE Press, 2007. Google Scholar
Digital Library
- A. Milicevic, D. Rayside, K. Yessenov, and D. Jackson. Unifying execution of imperative and declarative code. In ICSE 2011. Google Scholar
Digital Library
- C. Pacheco, S. K. Lahiri, M. D. Ernst and T. Ball, Feedback-Directed Random Test Generation, in ICSE 2007, IEEE, 2007. Google Scholar
Digital Library
- N. Rosner, C. G. López Pombo, N. Aguirre, A. Jaoua, A. Mili and M. F. Frias, Parallel Bounded Verification of Alloy Models by TranScoping, in VSTTE 2014, LNCS, Springer, 2013.Google Scholar
- H. Samimi, E. D. Aung, and T. D. Millstein. Falling back on executable specifications. In ECOOP 2010. Google Scholar
Digital Library
- J. Siddiqui and S. Khurshid, An Empirical Study of Structural Constraint Solving Techniques, in ICFEM 2009, LNCS, Springer, 2009. Google Scholar
Digital Library
- E. Uzuncaova. Efficient specification-based testing using incremental techniques. PhD thesis, [email protected], 2008. Google Scholar
Digital Library
- E. Uzuncaova and S. Khurshid. Constraint prioritization for efficient analysis of declarative models. In FM 2008. Google Scholar
Digital Library
- W. Visser, C. Pasareanu and S. Khurshid, Test input generation with java PathFinder, in ISSTA 2004, ACM, 2004. Google Scholar
Digital Library
- T. Xie, D. Marinov and D. Notkin, Rostra: A Framework for Detecting Redundant Object-Oriented Unit Tests, in ASE 2004, IEEE, 2004. Google Scholar
Digital Library
- R. N. Zaeem and S. Khurshid. Contract-based data structure repair using Alloy. In ECOOP 2010. Google Scholar
Digital Library
- The HyTeK distribution can be downloaded from http://bonnie.exp.dc.uba.ar/hytek-oopsla.tar.gzGoogle Scholar
- http://alloy.mit.edu/Google Scholar
Index Terms
Bounded exhaustive test input generation from hybrid invariants
Recommendations
Bounded exhaustive test input generation from hybrid invariants
OOPSLA '14: Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & ApplicationsWe present a novel technique for producing bounded exhaustive test suites from hybrid invariants, i.e., invariants that are expressed imperatively, declaratively, or as a combination of declarative and imperative predicates. Hybrid specifications are ...
RepOK-based reduction of bounded exhaustive testing
While the effectiveness of bounded exhaustive test suites increases as one increases the scope for the bounded exhaustive generation, both the time for test generation and the time for test execution grow exponentially with respect to the scope. In this ...
Bounded exhaustive test-input generation on GPUs
Bounded exhaustive testing is an effective methodology for detecting bugs in a wide range of applications. A well-known approach for bounded exhaustive testing is Korat. It generates all test inputs, up to a given small size, based on a formal ...







Comments