Abstract
We present an efficient, modular, and feature-rich framework for automated generation and validation of complex structures, suitable for tasks that explore a large space of structured values. Our framework is capable of exhaustive, incremental, parallel, and memoized enumeration from not only finite but also infinite domains, while providing fine-grained control over the process. Furthermore, the framework efficiently supports the inverse of enumeration (checking whether a structure can be generated and fast-forwarding to this structure to continue the enumeration) and lazy enumeration (achieving exhaustive testing without generating all structures). The foundation of efficient enumeration lies in both direct access to encoded structures, achieved with well-known and new pairing functions, and dependent enumeration, which embeds constraints into the enumeration to avoid backtracking. Our framework defines an algebra of enumerators, with combinators for their composition that preserve exhaustiveness and efficiency. We have implemented our framework as a domain-specific language in Scala. Our experiments demonstrate better performance and shorter specifications by up to a few orders of magnitude compared to existing approaches.
- SciFe website and repository. http://kaptoxic.github.io/SciFe.Google Scholar
- S. Abramov and R. Glück. Principles of Inverse Computation and the Universal Resolving Algorithm. The Essence of Computation, 2002. Google Scholar
Digital Library
- R. Blanc, V. Kuncak, E. Kneuss, and P. Suter. An overview of the Leon verification system. In Scala, 2013. Google Scholar
Digital Library
- C. Boyapati, S. Khurshid, and D. Marinov. Korat: Automated Testing Based on Java Predicates. In ISSTA, 2002. Google Scholar
Digital Library
- C. Brabrand, A. Møller, and M. I. Schwartzbach. Dual syntax for XML languages. In DBLP, 2005. Google Scholar
Digital Library
- L. Bulwahn. The New Quickcheck for Isabelle: Random, Exhaustive and Symbolic Testing under One Roof. In CPP, 2012. Google Scholar
Digital Library
- J. Christiansen and S. Fischer. EasyCheck – Test data for free. In FLOPS, 2008.Google Scholar
Cross Ref
- I. Ciupa, A. Leitner, M. Oriol, and B. Meyer. Experimental assessment of random testing for object-oriented software. In ISSTA, 2007. Google Scholar
Digital Library
- I. Ciupa, A. Leitner, M. Oriol, and B. Meyer. ARTOO: adaptive random testing for object-oriented software. In ICSE, 2008. Google Scholar
Digital Library
- K. Claessen and J. Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. In ICFP, 2000. Google Scholar
Digital Library
- B. Daniel, D. Dig, K. Garcia, and D. Marinov. Automated testing of refactoring engines. In FSE, 2007. Google Scholar
Digital Library
- M. Dénès, C. Hritcu, L. Lampropoulos, Z. Paraskevopoulou, and B. C. Pierce. QuickChick: Property-based testing for Coq. In The Coq Workshop, 2014.Google Scholar
- K. Dewey, L. Nichols, and B. Hardekopf. Automated Data Structure Generation: Refuting Common Wisdom. Unpublished manuscript, at: http://www.cs.ucsb.edu/ benh/research/papers.html, 2015.Google Scholar
- J. Duregard, P. Jansson, and M. Wang. Feat: Functional Enumeration of Algebraic Types. In Haskell, 2012. Google Scholar
Digital Library
- B. Elkarablieh, D. Marinov, and S. Khurshid. Efficient solving of structural constraints. In ISSTA, 2008. Google Scholar
Digital Library
- M. Erwig. Inductive graphs and functional graph algorithms. Journal of Functional Programming, 2001. Google Scholar
Digital Library
- P. Flajolet and B. Salvy. Computer Algebra Libraries for Combinatorial Structures. Journal of Symbolic Computation, 1995. Google Scholar
Digital Library
- M. Gligoric, T. Gvero, V. Jagannath, S. Khurshid, V. Kuncak, and D. Marinov. Test generation through programming in UDITA. In ICSE, 2010. Google Scholar
Digital Library
- J. Goodenough and S. Gerhart. Toward a Theory of Test Data Selection. IEEE Transactions on Software Engineering, 1975. Google Scholar
Digital Library
- J. Goubault. Implementing functional languages with fast equality, sets and maps: an exercise in hash consing. JFLA, 1994.Google Scholar
- R. E. Griswold, D. R. Hanson, and J. T. Korb. Generators in ICON. TOPLAS, 1981. Google Scholar
Digital Library
- T. Gvero, V. Kuncak, I. Kuraj, and R. Piskac. Complete completion using types and weights. In PLDI, 2013. Google Scholar
Digital Library
- J. Hughes. Generalizing monads to arrows. Science of Computer Programming, 2000. Google Scholar
Digital Library
- J. Hughes. QuickCheck testing for fun and profit. In PADL, 2007. Google Scholar
Digital Library
- D. Jackson. Alloy: a lightweight object modelling notation. ACM Transactions on Software Engineering and Methodology, 2002. Google Scholar
Digital Library
- A. Kanade and R. Alur. Representation dependence testing using program inversion. In FSE, 2010. Google Scholar
Digital Library
- S. Kawanaka and H. Hosoya. biXid: a bidirectional transformation language for XML. In ICFP, 2006. Google Scholar
Digital Library
- S. Khurshid and D. Marinov. TestEra: Specification-based testing of java programs using SAT. In ASE, 2004. Google Scholar
Digital Library
- O. Kiselyov, C.-c. Shan, D. P. Friedman, and A. Sabry. Backtracking, interleaving, and terminating monad transformers. In ICFP, 2005. Google Scholar
Digital Library
- E. Kneuss, I. Kuraj, V. Kuncak, and P. Suter. Synthesis modulo recursive functions. In OOPSLA, 2013. Google Scholar
Digital Library
- A. S. Köksal, V. Kuncak, and P. Suter. Constraints as control. In POPL, 2012. Google Scholar
Digital Library
- I. Kuraj. Interactive Code Generation. Master’s thesis, EPFL, Lausanne, 2013.Google Scholar
- I. Kuraj and V. Kuncak. SciFe: Scala Framework for Efficient Enumeration of Data Structures with Invariants. In Scala, 2014. Google Scholar
Digital Library
- H. Miller, P. Haller, E. Burmako, and M. Odersky. Instant Pickles: Generating Object-oriented Pickler Combinators for Fast and Extensible Serialization. In OOPSLA, 2013. Google Scholar
Digital Library
- S. Misailovic, A. Milicevic, N. Petrovic, S. Khurshid, and D. Marinov. Parallel Test Generation and Execution with Korat. In FSE, 2007. Google Scholar
Digital Library
- S.-C. Mu, Z. Hu, and M. Takeichi. An Injective Language for Reversible Computation. In MPC, 2004.Google Scholar
- M. Odersky et al. An Overview of the Scala Programming Language (2nd Edition). Technical report, EPFL, 2006.Google Scholar
- C. Pacheco and M. D. Ernst. Randoop: Feedback-Directed Random Testing for Java. In OOPSLA, 2007. Google Scholar
Digital Library
- A. Rosenberg. Efficient pairing functions–and why you should care. In IPDPS, 2003. Google Scholar
Digital Library
- N. Rosner, V. Bengolea, P. Ponzio, S. A. Khalek, N. Aguirre, M. F. Frias, and S. Khurshid. Bounded exhaustive test input generation from hybrid invariants. In OOPSLA, 2014. Google Scholar
Digital Library
- B. J. Ross. Running programs backwards: The logical inversion of imperative computation. Formal Aspects of Computing, 1997.Google Scholar
Cross Ref
- C. Runciman, M. Naylor, and F. Lindblad. Smallcheck and lazy smallcheck. In Haskell, 2008. Google Scholar
Digital Library
- V. Senni and F. Fioravanti. Generation of test data structures using constraint logic programming. TAP, 2012. Google Scholar
Digital Library
- R. Sharma, M. Gligoric, V. Jagannath, and D. Marinov. A Comparison of Constraint-Based and Sequence-Based Generation of Complex Input Data Structures. In ICSTW, 2010. Google Scholar
Digital Library
- K. Sullivan, J. Yang, D. Coppit, S. Khurshid, and D. Jackson. Software assurance by bounded exhaustive testing. In ISSTA, 2004. Google Scholar
Digital Library
- M. Szudzik. An elegant pairing function. In NKS, 2006.Google Scholar
- P. Tarau. Deriving a Fast Inverse of the Generalized Cantor N-tupling Bijection. In ICLP, 2012.Google Scholar
- W. Visser, C. S. Păsăreanu, and R. Pelánek. Test input generation for red-black trees using abstraction. In ASE, 2005. Google Scholar
Digital Library
- J. Wells and B. Yakobowski. Graph-based proof counting and enumeration with applications for program fragment synthesis. In LOPSTR, 2005. Google Scholar
Digital Library
- A. Yakushev and J. Jeuring. Enumerating well-typed terms generically. AAIP, 2010.Google Scholar
Index Terms
Programming with enumerable sets of structures
Recommendations
Programming with enumerable sets of structures
OOPSLA 2015: Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and ApplicationsWe present an efficient, modular, and feature-rich framework for automated generation and validation of complex structures, suitable for tasks that explore a large space of structured values. Our framework is capable of exhaustive, incremental, ...
Automatic program inversion using symbolic transducers
PLDI '17We propose a fully-automated technique for inverting functional programs that operate over lists such as string encoders and decoders. We consider programs that can be modeled using symbolic extended finite transducers (), an expressive model that can ...
Automatic program inversion using symbolic transducers
PLDI 2017: Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and ImplementationWe propose a fully-automated technique for inverting functional programs that operate over lists such as string encoders and decoders. We consider programs that can be modeled using symbolic extended finite transducers (), an expressive model that can ...






Comments