skip to main content
research-article

Programming with enumerable sets of structures

Published:23 October 2015Publication History
Skip Abstract Section

Abstract

We present an efficient, modular, and feature-rich framework for automated generation and validation of complex structures, suitable for tasks that explore a large space of structured values. Our framework is capable of exhaustive, incremental, parallel, and memoized enumeration from not only finite but also infinite domains, while providing fine-grained control over the process. Furthermore, the framework efficiently supports the inverse of enumeration (checking whether a structure can be generated and fast-forwarding to this structure to continue the enumeration) and lazy enumeration (achieving exhaustive testing without generating all structures). The foundation of efficient enumeration lies in both direct access to encoded structures, achieved with well-known and new pairing functions, and dependent enumeration, which embeds constraints into the enumeration to avoid backtracking. Our framework defines an algebra of enumerators, with combinators for their composition that preserve exhaustiveness and efficiency. We have implemented our framework as a domain-specific language in Scala. Our experiments demonstrate better performance and shorter specifications by up to a few orders of magnitude compared to existing approaches.

References

  1. SciFe website and repository. http://kaptoxic.github.io/SciFe.Google ScholarGoogle Scholar
  2. S. Abramov and R. Glück. Principles of Inverse Computation and the Universal Resolving Algorithm. The Essence of Computation, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. R. Blanc, V. Kuncak, E. Kneuss, and P. Suter. An overview of the Leon verification system. In Scala, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. C. Boyapati, S. Khurshid, and D. Marinov. Korat: Automated Testing Based on Java Predicates. In ISSTA, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. C. Brabrand, A. Møller, and M. I. Schwartzbach. Dual syntax for XML languages. In DBLP, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. L. Bulwahn. The New Quickcheck for Isabelle: Random, Exhaustive and Symbolic Testing under One Roof. In CPP, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. J. Christiansen and S. Fischer. EasyCheck – Test data for free. In FLOPS, 2008.Google ScholarGoogle ScholarCross RefCross Ref
  8. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer. Experimental assessment of random testing for object-oriented software. In ISSTA, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer. ARTOO: adaptive random testing for object-oriented software. In ICSE, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. K. Claessen and J. Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. In ICFP, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. B. Daniel, D. Dig, K. Garcia, and D. Marinov. Automated testing of refactoring engines. In FSE, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. Dénès, C. Hritcu, L. Lampropoulos, Z. Paraskevopoulou, and B. C. Pierce. QuickChick: Property-based testing for Coq. In The Coq Workshop, 2014.Google ScholarGoogle Scholar
  13. K. Dewey, L. Nichols, and B. Hardekopf. Automated Data Structure Generation: Refuting Common Wisdom. Unpublished manuscript, at: http://www.cs.ucsb.edu/ benh/research/papers.html, 2015.Google ScholarGoogle Scholar
  14. J. Duregard, P. Jansson, and M. Wang. Feat: Functional Enumeration of Algebraic Types. In Haskell, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. B. Elkarablieh, D. Marinov, and S. Khurshid. Efficient solving of structural constraints. In ISSTA, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. M. Erwig. Inductive graphs and functional graph algorithms. Journal of Functional Programming, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. P. Flajolet and B. Salvy. Computer Algebra Libraries for Combinatorial Structures. Journal of Symbolic Computation, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. M. Gligoric, T. Gvero, V. Jagannath, S. Khurshid, V. Kuncak, and D. Marinov. Test generation through programming in UDITA. In ICSE, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. J. Goodenough and S. Gerhart. Toward a Theory of Test Data Selection. IEEE Transactions on Software Engineering, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. J. Goubault. Implementing functional languages with fast equality, sets and maps: an exercise in hash consing. JFLA, 1994.Google ScholarGoogle Scholar
  21. R. E. Griswold, D. R. Hanson, and J. T. Korb. Generators in ICON. TOPLAS, 1981. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. T. Gvero, V. Kuncak, I. Kuraj, and R. Piskac. Complete completion using types and weights. In PLDI, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. J. Hughes. Generalizing monads to arrows. Science of Computer Programming, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Hughes. QuickCheck testing for fun and profit. In PADL, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. D. Jackson. Alloy: a lightweight object modelling notation. ACM Transactions on Software Engineering and Methodology, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. A. Kanade and R. Alur. Representation dependence testing using program inversion. In FSE, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. S. Kawanaka and H. Hosoya. biXid: a bidirectional transformation language for XML. In ICFP, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. S. Khurshid and D. Marinov. TestEra: Specification-based testing of java programs using SAT. In ASE, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. O. Kiselyov, C.-c. Shan, D. P. Friedman, and A. Sabry. Backtracking, interleaving, and terminating monad transformers. In ICFP, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. E. Kneuss, I. Kuraj, V. Kuncak, and P. Suter. Synthesis modulo recursive functions. In OOPSLA, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. A. S. Köksal, V. Kuncak, and P. Suter. Constraints as control. In POPL, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. I. Kuraj. Interactive Code Generation. Master’s thesis, EPFL, Lausanne, 2013.Google ScholarGoogle Scholar
  33. I. Kuraj and V. Kuncak. SciFe: Scala Framework for Efficient Enumeration of Data Structures with Invariants. In Scala, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. H. Miller, P. Haller, E. Burmako, and M. Odersky. Instant Pickles: Generating Object-oriented Pickler Combinators for Fast and Extensible Serialization. In OOPSLA, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. S. Misailovic, A. Milicevic, N. Petrovic, S. Khurshid, and D. Marinov. Parallel Test Generation and Execution with Korat. In FSE, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. S.-C. Mu, Z. Hu, and M. Takeichi. An Injective Language for Reversible Computation. In MPC, 2004.Google ScholarGoogle Scholar
  37. M. Odersky et al. An Overview of the Scala Programming Language (2nd Edition). Technical report, EPFL, 2006.Google ScholarGoogle Scholar
  38. C. Pacheco and M. D. Ernst. Randoop: Feedback-Directed Random Testing for Java. In OOPSLA, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. A. Rosenberg. Efficient pairing functions–and why you should care. In IPDPS, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. N. Rosner, V. Bengolea, P. Ponzio, S. A. Khalek, N. Aguirre, M. F. Frias, and S. Khurshid. Bounded exhaustive test input generation from hybrid invariants. In OOPSLA, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. B. J. Ross. Running programs backwards: The logical inversion of imperative computation. Formal Aspects of Computing, 1997.Google ScholarGoogle ScholarCross RefCross Ref
  42. C. Runciman, M. Naylor, and F. Lindblad. Smallcheck and lazy smallcheck. In Haskell, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. V. Senni and F. Fioravanti. Generation of test data structures using constraint logic programming. TAP, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. R. Sharma, M. Gligoric, V. Jagannath, and D. Marinov. A Comparison of Constraint-Based and Sequence-Based Generation of Complex Input Data Structures. In ICSTW, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. K. Sullivan, J. Yang, D. Coppit, S. Khurshid, and D. Jackson. Software assurance by bounded exhaustive testing. In ISSTA, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. M. Szudzik. An elegant pairing function. In NKS, 2006.Google ScholarGoogle Scholar
  47. P. Tarau. Deriving a Fast Inverse of the Generalized Cantor N-tupling Bijection. In ICLP, 2012.Google ScholarGoogle Scholar
  48. W. Visser, C. S. Păsăreanu, and R. Pelánek. Test input generation for red-black trees using abstraction. In ASE, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. J. Wells and B. Yakobowski. Graph-based proof counting and enumeration with applications for program fragment synthesis. In LOPSTR, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. A. Yakushev and J. Jeuring. Enumerating well-typed terms generically. AAIP, 2010.Google ScholarGoogle Scholar

Index Terms

  1. Programming with enumerable sets of structures

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!