ABSTRACT
Modern development practises encourage extensive testing of code while it is still under development, using unit tests to check individual code units in isolation. Such tests are typically case-based, checking a likely error scenario or an error that has previously been identified and fixed. Coming up with good test cases is challenging, and focusing on individual tests can distract from creating tests that cover the full functionality.
Axioms, known from program specification, allow for an alternative way of generating test cases, where the intended functionality is described as rules or equations that can be checked automatically. Axioms are proposed as part of the concept feature of the upcoming C++0x standard.
In this paper, we describe how tests may be generated automatically from axioms in C++ concepts, and supplied with appropriate test data to form effective automated unit tests.
- S. Antoy. Systematic design of algebraic specifications. In IWSSD '89: Proceedings of the 5th international workshop on Software specification and design, pages 278--280. ACM, New York, NY, USA, 1989. ISBN0-89791-305-1. Google Scholar
Digital Library
- S. Antoy and R. G. Hamlet. Automatically checking an implementation against its formal specification. IEEE Trans. Software Eng., 26 (1): 55--69, 2000. Google Scholar
Digital Library
- A. H. Bagge, V. David, and M. Haveraaen. Axiom-based testing for C++. In OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications, pages 721--722. ACM, New York, NY, USA, 2008. ISBN 978-1-60558-220-7. Google Scholar
Digital Library
- A. H. Bagge and M. Haveraaen. Axiom-based transformations: Optimisation and testing. In J. J. Vinju and A. Johnstone, editors, Eighth Workshop on Language Descriptions, Tools and Applications (LDTA 2008), Electronic Notes in Theoretical Computer Science. Elsevier, Budapest, Hungary, 2008. Google Scholar
Digital Library
- A. H. Bagge and M. Haveraaen. Interfacing concepts: Why declaration style shouldn't matter. In T. Ekman and J. J. Vinju, editors, Proceedings of the Ninth Workshop on Language Descriptions, Tools and Applications (LDTA '09), Electronic Notes in Theoretical Computer Science. Elsevier, York, UK, March 2009.Google Scholar
- K. Beck. Extreme programming: A humanistic discipline of software development. In Proceedings of the 1st International Conference on Fundamental Approaches to Software Engineering (FASE'98), pages 1--6. Springer Berlin / Heidelberg, 1998.Google Scholar
Cross Ref
- K. Beck. Test-Driven Development: By Example. Addison-Wesley, 2002. ISBN 0321146530. Google Scholar
Digital Library
- K. Beck and E. Gamma. JUnit -- Java Unit testing. http://www.junit.org and http://junit.sourceforge.net/ per 2007-03-15.Google Scholar
- P. Becker. Working draft, standard for programming language C++. Technical Report N2857=09-0047, JTC1/SC22/WG21 -- The C++ Standards Committee, March 23rd, 2009.Google Scholar
- A. Borghi, V. David, and A. Demaille. C-Transformers: A framework to write C program transformations. ACM Crossroads, 12(3), April 2006. Google Scholar
Digital Library
- M. Bravenboer, K. T. Kalleberg, R. Vermaas, and E. Visser. Stratego/XT 0.16: Components for transformation systems. In F. Tip and J. Hatcliff, editors, PEPM'06: Workshop on Partial Evaluation and Program Manipulation. ACM Press, January 2006. Google Scholar
Digital Library
- H. Y. Chen, T. H. Tse, F. T. Chan, and T. Y. Chen. In black and white: an integrated approach to class-level testing of object-oriented programs. ACM Trans. Softw. Eng. Methodol., 7(3):250--295, 1998. ISSN 1049-331X. Google Scholar
Digital Library
- H. Y. Chen, T. H. Tse, and T. Y. Chen. Taccle: a methodology for object-oriented software testing at the class and cluster levels. ACM Trans. Softw. Eng. Methodol., 10(1):56--109, 2001. ISSN 1049-331X. Google Scholar
Digital Library
- K. Claessen and J. Hughes. QuickCheck: a lightweight tool for random testing of Haskell programs. In ICFP '00: Proceedings of the fifth ACM SIGPLAN international conference on Functional programming, pages 268--279. ACM, New York, NY, USA, 2000. ISBN 1-58113-202-6. Google Scholar
Digital Library
- R.-K. Doong. An approach to testing object-oriented programs. PhD thesis, Polytechnic University, Brooklyn, NY, USA, 1993. Google Scholar
Digital Library
- R.-K. Doong and P. G. Frankl. Case studies on testing object-oriented programs. In TAV4: Proceedings of the symposium on Testing, analysis, and verification, pages 165--177. ACM Press, New York, NY, USA, 1991. ISBN 0-89791-449-X. Google Scholar
Digital Library
- R.-K. Doong and P. G. Frankl. The ASTOOT approach to testing object-oriented programs. ACM Trans. Softw. Eng. Methodol., 3(2): 101--130, 1994. ISSN 1049-331X. Google Scholar
Digital Library
- J. Gannon, P. McMullin, and R. Hamlet. Data abstraction, implementation, specification, and testing. ACM Trans. Program. Lang. Syst., 3(3):211--223, 1981. ISSN 0164-0925. Google Scholar
Digital Library
- M.-C. Gaudel. Testing can be formal, too. In P. D. Mosses, M. Nielsen, and M. I. Schwartzbach, editors, TAPSOFT, volume 915 of Lecture Notes in Computer Science, pages 82--96. Springer, 1995. ISBN 3-540-59293-8. Google Scholar
Digital Library
- M.-C. Gaudel and P. L. Gall. Testing data types implementations from algebraic specifications. In Formal Methods and Testing, volume 4949 of Lecture Notes in Computer Science, pages 209--239. Springer, 2008. Google Scholar
Digital Library
- J. Goguen, J. Thatcher, and E. Wagner. An initial algebra approach to the specification, correctness and implementation of abstract data types. In R. Yeh, editor, Current Trends in Programming Methodology, volume 4, pages 80--149. Prentice Hall, 1978.Google Scholar
- P. Gottschling. Fundamental algebraic concepts in concept-enabled C++. Technical Report TR639, Department of Computer Science, Indiana University, 2006.Google Scholar
- D. Gregor, J. Järvi, J. Siek, B. Stroustrup, G. D. Reis, and A. Lumsdaine. Concepts: linguistic support for generic programming in C++. In OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 291--310. ACM, New York, NY, USA, 2006. ISBN 1-59593-348-4. Google Scholar
Digital Library
- D. Gregor and A. Lumsdaine. Core concepts for the C++0x standard library (revision 2). Technical Report N2621=08-0131, JTC1/SC22/WG21 -- The C++ Standards Committee, May 19th, 2008.Google Scholar
- W. J. Gutjahr. Partition testing vs. random testing: The influence of uncertainty. IEEE Transactions on Software Engineering, 25(5): 661--674, 1999. ISSN 0098-5589. Google Scholar
Digital Library
- J. V. Guttag. Notes on type abstraction (version 2). IEEE Trans. Softw. Eng., 6(1):13--23, 1980. ISSN 0098-5589. Google Scholar
Digital Library
- J. V. Guttag and J. J. Horning. The algebraic specification of abstract data types. Acta Inf., 10:27--52, 1978.Google Scholar
Digital Library
- J. V. Guttag, E. Horowitz, and D. R. Musser. Abstract data types and software validation. Commun. ACM, 21(12):1048--1064, 1978. ISSN 0001-0782. Google Scholar
Digital Library
- D. Hamlet and R. Taylor. Partition testing does not inspire confidence. IEEE Trans. Softw. Eng., 16(12):1402--1411, 1990. ISSN 0098-5589. Google Scholar
Digital Library
- R. Hamlet. Random testing. In J. Marciniak, editor, Encyclopedia of Software Engineering, pages 970--978. Wiley, 1994.Google Scholar
- M. Haveraaen and E. Brkic. Structured testing in Sophus. In E. Coward, editor, Norsk informatikkonferanse NIK'2005, pages 43--54. Tapir akademisk forlag, Trondheim, Norway, 2005. URL http://www.nik.no/2005/.Google Scholar
- M. Haveraaen, H. A. Friis, and H. Munthe-Kaas. Computable scalar fields: a basis for PDE software. Journal of Logic and Algebraic Programming, 65(1):36--49, September-October 2005.Google Scholar
Cross Ref
- M. Haveraaen and K. T. Kalleberg. JAxT and JDI: the simplicity of JUnit applied to axioms and data invariants. In OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applications, pages 731--732. ACM, New York, NY, USA, 2008. ISBN 978-1-60558-220-7. Google Scholar
Digital Library
- M. Hughes and D. Stotts. Daistish: systematic algebraic testing for OO programs in the presence of side-effects. In ISSTA '96: Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis, pages 53--61. ACM Press, New York, NY, USA, 1996. ISBN 0-89791-787-1. Google Scholar
Digital Library
- K. T. Kalleberg. Abstractions for Language-Independent Program Transformations, chapter 11. University of Bergen, Norway, Postboks 7800, 5020 Bergen, Norway, June 2007. ISBN 978-82-308-0441-4.Google Scholar
- B. Liskov and S. Zilles. Specification techniques for data abstractions. In Proceedings of the international conference on Reliable software, pages 72--87. ACM, New York, NY, USA, 1975. Google Scholar
Digital Library
- B. Liskov and S. Zilles. Specification techniques for data abstractions. In Proceedings of the international conference on Reliable software, pages 72--87. ACM, New York, NY, USA, 1975. Google Scholar
Digital Library
- M. Masood, E. Birkenes, K. T. Kalleberg, M. Haveraaen, and A. H. Bagge. Axiom-based testing of Java collections with JAxT. Technical Report 388, Department of Informatics, University of Bergen, P.O.Box 7803, N-5020 Bergen, Norway, August 2009. URL http://www.ii.uib.no/publikasjoner/texrap/.Google Scholar
- I. S. W. B. Prasetya, T. E. J. Vos, and A. Baars. Trace-based reflexive testing of OO programs. Technical Report UU-CS-2007-037, Department of Information and Computing Sciences, Utrecht University, Utrecht, The Netherlands, 2007. URL http://www.cs.uu.nl/research/techreps/repo/CS-2007/2007-037.pdf.Google Scholar
- D. Saff. Theory-infected: or how I learned to stop worrying and love universal quantification. In OOPSLA '07: Companion to the 22nd ACM SIGPLAN conference on Object oriented programming systems and applications companion, pages 846--847. ACM, New York, NY, USA, 2007. ISBN 978-1-59593-865-7. Google Scholar
Digital Library
- K. Sen. Effective random testing of concurrent programs. In ASE'07: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, pages 323--332. ACM, New York, NY, USA, 2007. ISBN 978-1-59593-882-4. Google Scholar
Digital Library
- J. G. Siek and A. Lumsdaine. The matrix template library: A unifying framework for numerical linear algebra. In ECOOP '98: Workshop on Object-Oriented Technology, pages 466--467. Springer-Verlag, London, UK, 1998. ISBN 3-540-65460-7. Google Scholar
Digital Library
- P. Sommerlad. C++ Unit Testing Easier, 2009. URL http://r2.ifs.hsr.ch/cute.Google Scholar
- P. D. Stotts, M. Lindsey, and A. Antley. An informal formal method for systematic JUnit test case generation. In D. Wells and L. A. Williams, editors, XP/Agile Universe, volume 2418 of Lecture Notes in Computer Science, pages 131--143. Springer, 2002. ISBN 3-540-44024-0. Google Scholar
Digital Library
- X. Tang and J. Järvi. Concept-based optimization. In LCSD '07: Proceedings of the 2007 Symposium on Library-Centric Software Design, pages 97--108. ACM, New York, NY, USA, 2007. ISBN 978-1-60558-086-9. Google Scholar
Digital Library
- The Transformers Group, LRDE, EPITA. Transformers, 2008. URL http://www.lrde.epita.fr/cgi-bin/twiki/view/Transformers/Transformers.Google Scholar
- B. Yu, L. Kong, Y. Zhang, and H. Zhu. Testing java components based on algebraic specifications. In ICST '08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation, pages 190--199. IEEE Computer Society, Washington, DC, USA, 2008. ISBN 978-0-7695-3127-4. Google Scholar
Digital Library
- H. Zhu. A note on test oracles and semantics of algebraic specifications. In QSIC '03: Proceedings of the Third International Conference on Quality Software, pages 91--98. IEEE Computer Society, Washington, DC, USA, 2003. ISBN 0-7695-2015-4. Google Scholar
Digital Library
Index Terms
The axioms strike back: testing with concepts and axioms in C++
Recommendations
The axioms strike back: testing with concepts and axioms in C++
GPCE '09Modern development practises encourage extensive testing of code while it is still under development, using unit tests to check individual code units in isolation. Such tests are typically case-based, checking a likely error scenario or an error that ...
Testing with concepts and axioms in C++
OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applicationsUnit testing is a popular way of increasing software reliability. Axioms, known from program specification, allow functionality to be described as rules or equations. We show a method and prototype tool for using the proposed concept and axiom features ...
Axiom-based testing for C++
OOPSLA Companion '08: Companion to the 23rd ACM SIGPLAN conference on Object-oriented programming systems languages and applicationsAxioms, known from program specification, allow program functionality to be described as rules or equations. The draft C++0x standard introduces axioms as part of the new concept feature. We will demonstrate a tool that uses these features for automated ...







Comments