Test case generation from mutated task models

ABSTRACT

This paper describes an approach to the model-based testing of graphical user interfaces from task models. Starting from a task model of the system under test, oracles are generated whose behaviour is compared with the execution of the running system. The use of task models means that the effort of producing the test oracles is reduced. It does also mean, however, that the oracles are confined to the set of expected user behaviours for the system. The paper focuses on solving this problem. It shows how task mutations can be generated automatically, enabling a broader range of user behaviours to be considered. A tool, based on a classification of user errors, generates these mutations. A number of examples illustrate the approach.

References

1. ISO 9241-11. Ergonomic requirements for office work with visual display terminals (VDTs) -- Part 11: Guidance on usability. First Edition ed. 1998: International Organization for Standardization
2. Ivory, M.Y. and M.A. Hearst, The State of the Art in Automating Usability Evaluation of User Interfaces. ACM Computing Surveys, 2001. 33(4): p. 470--516.
3. Paiva, A.C.R., Automated Specification-Based Testing of Graphical User Interfaces, in Department of Electrical and Computer Engineering. 2007, Engineering Faculty of Porto University (Ph.D thesis): Porto. p. 228.
4. Silva, J.L., J.C. Campos, and A.C.R. Paiva. Modelbased user interface testing with Spec Explorer and ConcurTaskTrees. in 2nd International Workshop on Formal Methods for Interactive Systems. 2007. Lancaster, UK.
5. Paternò, F., Model-Based Design and Evaluation of Interactive Applications. 1999, London, UK: Springer- Verlag.
6. Mugridge, R. and W. Cunningham, Fit for Developing Software: Framework for Integrated Tests. 1st Edition ed. 2005: Prentice Hall. 384.
7. Card, S.K., T.P. Moran, and A. Newell, The Psychology of Human-Computer Interaction 1986: Lawrence Erlbaum Associates. 469.
8. Hamilton, F., Predictive evaluation using task knowledge structures, in Conference companion on Human factors in computing systems: common ground. 1996, ACM: Vancouver, British Columbia, Canada. p. 261--262.
9. Mori, G., F. Paternò, and C. Santoro, CTTE: Support for Developing and Analyzing Task Models for Interactive System Design. IEEE Transactions on Software Engineering, 2002. 28(9).
10. Bolognesi, T. and E. Brinksma, Introduction to the ISO specification language LOTOS. Computer Networks and ISDN Systems - Special Issue: Protocol Specification and Testing, 1987. 14(1).
11. Hartman, A. and K. Nagin. The AGEDIS Tools for Model Based Testing. in ISSTA'04. 2004. Boston, Massachusetts, USA: Springer.
12. Jacky, J., et al., Model-Based Software Testing and Analysis with C#. 2007: Cambridge University Press. 366.
13. Brooks, P.A. and A.M. Memon, Automated GUI testing guided by usage profiles, in Proceedings of the 22nd IEEE international conference on Automated software engineering (ASE'07). 2007, IEEE CS: Washington, DC, USA. p. 333--342.
14. Memon, A., I. Banerjee, and A. Nagarajan. GUI Ripping: Reverse Engineering of Graphical User Interfaces for Testing. in Proceedings of the 10th Working Conference on Reverse Engineering (WCRE'03). 2003. Washington, DC, USA: IEEE CS.
15. Paiva, A.C.R., et al. A Model-to-implementation Mapping Tool for Automated Model-based GUI Testing. in Proceedings of the 7th International Conferece on Formal Engineering Methods(ICFEM'05). 2005.
16. Moreira, R.M.L.M. and A.C.R. Paiva, Visual Abstract Notation for GUI Modelling and Testing: VAN4GUIM, in Proceedings of the 3rd International Conference on Software and Data Technologies (ICSOFT'08), J. Cordeiro, et al., Editors. 2008, INSTICC Press: Gaia, Portugal.
17. Cunha, M., et al., PETTool: A Pattern-Based GUI Testing Tool, in 2nd International Conference on Software Technology and Engineering (ICSTE'10). 2010. p. 202--206.
18. Reason, J., Human Error. 1990: Cambridge University Press.
19. Paternò F. Santoro C. Mäntyjärvi J., Mori G., Sansone S., Authoring pervasive multimodal user interfaces, International Journal of Web Engineering and Technology, vol. 4 pp. 235 - 261. Inderscience Enterprises Ltd, 2008.
20. Campos, J.C., J.L. Silva, and A.C.R. Paiva, Task models in the model-based testing of user interfaces. Technical Report, 2009, Universidade do Minho.
21. Veans, M., et al., Model-based testing of object-oriented reactive systems with Spec Explorer, in Formal Methods and testing: an outcome of the FORTEST network. 2008, Springer-Verlag. p. 39--76.
22. Barnett, M., K.R.M. Leino, and W. Schulte. The Spec# Programming System: An Overview. in CASSIS'04 - International workshop on Construction and Analysis of Safe, Secure and Interoperable Smart devices. 2004. Marseille.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Software defect analysis

        • Software testing and debugging