skip to main content
10.1145/1621607.1621612acmconferencesArticle/Chapter ViewAbstractPublication PagesgpceConference Proceedingsconference-collections
research-article

The axioms strike back: testing with concepts and axioms in C++

Published:04 October 2009Publication History

ABSTRACT

Modern development practises encourage extensive testing of code while it is still under development, using unit tests to check individual code units in isolation. Such tests are typically case-based, checking a likely error scenario or an error that has previously been identified and fixed. Coming up with good test cases is challenging, and focusing on individual tests can distract from creating tests that cover the full functionality.

Axioms, known from program specification, allow for an alternative way of generating test cases, where the intended functionality is described as rules or equations that can be checked automatically. Axioms are proposed as part of the concept feature of the upcoming C++0x standard.

In this paper, we describe how tests may be generated automatically from axioms in C++ concepts, and supplied with appropriate test data to form effective automated unit tests.

References

  1. S. Antoy. Systematic design of algebraic specifications. In IWSSD '89: Proceedings of the 5th international workshop on Software specification and design, pages 278--280. ACM, New York, NY, USA, 1989. ISBN0-89791-305-1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. S. Antoy and R. G. Hamlet. Automatically checking an implementation against its formal specification. IEEE Trans. Software Eng., 26 (1): 55--69, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. A. H. Bagge, V. David, and M. Haveraaen. Axiom-based testing for C++. In OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications, pages 721--722. ACM, New York, NY, USA, 2008. ISBN 978-1-60558-220-7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. A. H. Bagge and M. Haveraaen. Axiom-based transformations: Optimisation and testing. In J. J. Vinju and A. Johnstone, editors, Eighth Workshop on Language Descriptions, Tools and Applications (LDTA 2008), Electronic Notes in Theoretical Computer Science. Elsevier, Budapest, Hungary, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. A. H. Bagge and M. Haveraaen. Interfacing concepts: Why declaration style shouldn't matter. In T. Ekman and J. J. Vinju, editors, Proceedings of the Ninth Workshop on Language Descriptions, Tools and Applications (LDTA '09), Electronic Notes in Theoretical Computer Science. Elsevier, York, UK, March 2009.Google ScholarGoogle Scholar
  6. K. Beck. Extreme programming: A humanistic discipline of software development. In Proceedings of the 1st International Conference on Fundamental Approaches to Software Engineering (FASE'98), pages 1--6. Springer Berlin / Heidelberg, 1998.Google ScholarGoogle ScholarCross RefCross Ref
  7. K. Beck. Test-Driven Development: By Example. Addison-Wesley, 2002. ISBN 0321146530. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. K. Beck and E. Gamma. JUnit -- Java Unit testing. http://www.junit.org and http://junit.sourceforge.net/ per 2007-03-15.Google ScholarGoogle Scholar
  9. P. Becker. Working draft, standard for programming language C++. Technical Report N2857=09-0047, JTC1/SC22/WG21 -- The C++ Standards Committee, March 23rd, 2009.Google ScholarGoogle Scholar
  10. A. Borghi, V. David, and A. Demaille. C-Transformers: A framework to write C program transformations. ACM Crossroads, 12(3), April 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. M. Bravenboer, K. T. Kalleberg, R. Vermaas, and E. Visser. Stratego/XT 0.16: Components for transformation systems. In F. Tip and J. Hatcliff, editors, PEPM'06: Workshop on Partial Evaluation and Program Manipulation. ACM Press, January 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. H. Y. Chen, T. H. Tse, F. T. Chan, and T. Y. Chen. In black and white: an integrated approach to class-level testing of object-oriented programs. ACM Trans. Softw. Eng. Methodol., 7(3):250--295, 1998. ISSN 1049-331X. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. H. Y. Chen, T. H. Tse, and T. Y. Chen. Taccle: a methodology for object-oriented software testing at the class and cluster levels. ACM Trans. Softw. Eng. Methodol., 10(1):56--109, 2001. ISSN 1049-331X. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. K. Claessen and J. Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. In ICFP '00: Proceedings of the fifth ACM SIGPLAN international conference on Functional programming, pages 268--279. ACM, New York, NY, USA, 2000. ISBN 1-58113-202-6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. R.-K. Doong. An approach to testing object-oriented programs. PhD thesis, Polytechnic University, Brooklyn, NY, USA, 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. R.-K. Doong and P. G. Frankl. Case studies on testing object-oriented programs. In TAV4: Proceedings of the symposium on Testing, analysis, and verification, pages 165--177. ACM Press, New York, NY, USA, 1991. ISBN 0-89791-449-X. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. R.-K. Doong and P. G. Frankl. The ASTOOT approach to testing object-oriented programs. ACM Trans. Softw. Eng. Methodol., 3(2): 101--130, 1994. ISSN 1049-331X. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. Gannon, P. McMullin, and R. Hamlet. Data abstraction, implementation, specification, and testing. ACM Trans. Program. Lang. Syst., 3(3):211--223, 1981. ISSN 0164-0925. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. M.-C. Gaudel. Testing can be formal, too. In P. D. Mosses, M. Nielsen, and M. I. Schwartzbach, editors, TAPSOFT, volume 915 of Lecture Notes in Computer Science, pages 82--96. Springer, 1995. ISBN 3-540-59293-8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M.-C. Gaudel and P. L. Gall. Testing data types implementations from algebraic specifications. In Formal Methods and Testing, volume 4949 of Lecture Notes in Computer Science, pages 209--239. Springer, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. J. Goguen, J. Thatcher, and E. Wagner. An initial algebra approach to the specification, correctness and implementation of abstract data types. In R. Yeh, editor, Current Trends in Programming Methodology, volume 4, pages 80--149. Prentice Hall, 1978.Google ScholarGoogle Scholar
  22. P. Gottschling. Fundamental algebraic concepts in concept-enabled C++. Technical Report TR639, Department of Computer Science, Indiana University, 2006.Google ScholarGoogle Scholar
  23. D. Gregor, J. Järvi, J. Siek, B. Stroustrup, G. D. Reis, and A. Lumsdaine. Concepts: linguistic support for generic programming in C++. In OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 291--310. ACM, New York, NY, USA, 2006. ISBN 1-59593-348-4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. D. Gregor and A. Lumsdaine. Core concepts for the C++0x standard library (revision 2). Technical Report N2621=08-0131, JTC1/SC22/WG21 -- The C++ Standards Committee, May 19th, 2008.Google ScholarGoogle Scholar
  25. W. J. Gutjahr. Partition testing vs. random testing: The influence of uncertainty. IEEE Transactions on Software Engineering, 25(5): 661--674, 1999. ISSN 0098-5589. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. J. V. Guttag. Notes on type abstraction (version 2). IEEE Trans. Softw. Eng., 6(1):13--23, 1980. ISSN 0098-5589. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. J. V. Guttag and J. J. Horning. The algebraic specification of abstract data types. Acta Inf., 10:27--52, 1978.Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. V. Guttag, E. Horowitz, and D. R. Musser. Abstract data types and software validation. Commun. ACM, 21(12):1048--1064, 1978. ISSN 0001-0782. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. D. Hamlet and R. Taylor. Partition testing does not inspire confidence. IEEE Trans. Softw. Eng., 16(12):1402--1411, 1990. ISSN 0098-5589. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. R. Hamlet. Random testing. In J. Marciniak, editor, Encyclopedia of Software Engineering, pages 970--978. Wiley, 1994.Google ScholarGoogle Scholar
  31. M. Haveraaen and E. Brkic. Structured testing in Sophus. In E. Coward, editor, Norsk informatikkonferanse NIK'2005, pages 43--54. Tapir akademisk forlag, Trondheim, Norway, 2005. URL http://www.nik.no/2005/.Google ScholarGoogle Scholar
  32. M. Haveraaen, H. A. Friis, and H. Munthe-Kaas. Computable scalar fields: a basis for PDE software. Journal of Logic and Algebraic Programming, 65(1):36--49, September-October 2005.Google ScholarGoogle ScholarCross RefCross Ref
  33. M. Haveraaen and K. T. Kalleberg. JAxT and JDI: the simplicity of JUnit applied to axioms and data invariants. In OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications, pages 731--732. ACM, New York, NY, USA, 2008. ISBN 978-1-60558-220-7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. M. Hughes and D. Stotts. Daistish: systematic algebraic testing for OO programs in the presence of side-effects. In ISSTA '96: Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis, pages 53--61. ACM Press, New York, NY, USA, 1996. ISBN 0-89791-787-1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. K. T. Kalleberg. Abstractions for Language-Independent Program Transformations, chapter 11. University of Bergen, Norway, Postboks 7800, 5020 Bergen, Norway, June 2007. ISBN 978-82-308-0441-4.Google ScholarGoogle Scholar
  36. B. Liskov and S. Zilles. Specification techniques for data abstractions. In Proceedings of the international conference on Reliable software, pages 72--87. ACM, New York, NY, USA, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. B. Liskov and S. Zilles. Specification techniques for data abstractions. In Proceedings of the international conference on Reliable software, pages 72--87. ACM, New York, NY, USA, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. Masood, E. Birkenes, K. T. Kalleberg, M. Haveraaen, and A. H. Bagge. Axiom-based testing of Java collections with JAxT. Technical Report 388, Department of Informatics, University of Bergen, P.O.Box 7803, N-5020 Bergen, Norway, August 2009. URL http://www.ii.uib.no/publikasjoner/texrap/.Google ScholarGoogle Scholar
  39. I. S. W. B. Prasetya, T. E. J. Vos, and A. Baars. Trace-based reflexive testing of OO programs. Technical Report UU-CS-2007-037, Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands, 2007. URL http://www.cs.uu.nl/research/techreps/repo/CS-2007/2007-037.pdf.Google ScholarGoogle Scholar
  40. D. Saff. Theory-infected: or how I learned to stop worrying and love universal quantification. In OOPSLA '07: Companion to the 22nd ACM SIGPLAN conference on Object oriented programming systems and applications companion, pages 846--847. ACM, New York, NY, USA, 2007. ISBN 978-1-59593-865-7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. K. Sen. Effective random testing of concurrent programs. In ASE'07: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, pages 323--332. ACM, New York, NY, USA, 2007. ISBN 978-1-59593-882-4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. J. G. Siek and A. Lumsdaine. The matrix template library: A unifying framework for numerical linear algebra. In ECOOP '98: Workshop on Object-Oriented Technology, pages 466--467. Springer-Verlag, London, UK, 1998. ISBN 3-540-65460-7. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. P. Sommerlad. C++ Unit Testing Easier, 2009. URL http://r2.ifs.hsr.ch/cute.Google ScholarGoogle Scholar
  44. P. D. Stotts, M. Lindsey, and A. Antley. An informal formal method for systematic JUnit test case generation. In D. Wells and L. A. Williams, editors, XP/Agile Universe, volume 2418 of Lecture Notes in Computer Science, pages 131--143. Springer, 2002. ISBN 3-540-44024-0. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. X. Tang and J. Järvi. Concept-based optimization. In LCSD '07: Proceedings of the 2007 Symposium on Library-Centric Software Design, pages 97--108. ACM, New York, NY, USA, 2007. ISBN 978-1-60558-086-9. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. The Transformers Group, LRDE, EPITA. Transformers, 2008. URL http://www.lrde.epita.fr/cgi-bin/twiki/view/Transformers/Transformers.Google ScholarGoogle Scholar
  47. B. Yu, L. Kong, Y. Zhang, and H. Zhu. Testing java components based on algebraic specifications. In ICST '08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, pages 190--199. IEEE Computer Society, Washington, DC, USA, 2008. ISBN 978-0-7695-3127-4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. H. Zhu. A note on test oracles and semantics of algebraic specifications. In QSIC '03: Proceedings of the Third International Conference on Quality Software, pages 91--98. IEEE Computer Society, Washington, DC, USA, 2003. ISBN 0-7695-2015-4. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. The axioms strike back: testing with concepts and axioms in C++

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!