skip to main content
research-article

A More Intelligent Test Case Generation Approach through Task Models Manipulation

Authors Info & Claims
Published:30 June 2017Publication History
Skip Abstract Section

Abstract

Ensuring that an interactive application allows users to perform their activities and reach their goals is critical to the overall usability of the interactive application. Indeed, the effectiveness factor of usability directly refers to this capability. Assessing effectiveness is a real challenge for usability testing as usability tests only cover a very limited number of tasks and activities. This paper proposes an approach towards automated testing of effectiveness of interactive applications. To this end we resort to two main elements: an exhaustive description of users' activities and goals using task models, and the generation of scenarios (from the task models) to be tested over the application. However, the number of scenarios can be very high (beyond the computing capabilities of machines) and we might end up testing multiple similar scenarios. In order to overcome these problems, we propose strategies based on task models manipulations (e.g., manipulating task nodes, operator nodes, information...) resulting in a more intelligent test case generation approach. For each strategy, we investigate its relevance (both in terms of test case generation and in terms of validity compared to the original task models) and we illustrate it with a small example. Finally, the proposed strategies are applied on a real-size case study demonstrating their relevance and validity to test interactive applications.

References

  1. 1998. ISO 9241--11: Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11 : Guidance on usability (1 ed.). Technical Report. International Organization for Standardization, Geneva.Google ScholarGoogle Scholar
  2. Thomas A. Alspaugh, Debra J. Richardson, and Thomas A. Standish. 2005. Scenarios, State Machines and Purpose-driven Testing. In Proceedings of the Fourth International Workshop on Scenarios and State Machines: Models, Algorithms and Tools (SCESM '05). ACM, New York, NY, USA, 1--5. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Domenico Amalfitano, Anna Rita Fasolino, and Porfirio Tramontana. 2011. A GUI Crawling-Based Technique for Android Mobile Application Testing. In Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops (ICSTW '11). IEEE Computer Society, Washington, DC, USA, 252--261. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Judy Bowen and Steve Reeves. 2011. UI-driven Test-first Development of Interactive Systems. In Proceedings of the 3rd ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '11). ACM, New York, NY, USA, 165--174. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. José C. Campos, Camille Fayollas, Célia Martinie, David Navarre, Philippe Palanque, and Miguel Pinto. 2016. Systematic Automation of Scenario-based Testing of User Interfaces. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '16). ACM, New York, NY, USA, 138--148. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Stuart K. Card, Thomas P. Moran, and Allen Newell. 1980. The Keystroke-level Model for User Performance Time with Interactive Systems. Commun. ACM 23, 7 (July 1980), 396--410. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. John P. Chin, Virginia A. Diehl, and Kent L. Norman. 1988. Development of an Instrument Measuring User Satisfaction of the Humancomputer Interface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '88). ACM, New York, NY, USA, 213--218. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Camille Fayollas, Célia Martinie, David Navarre, and Philippe Palanque. 2016. Engineering Mixed-criticality Interactive Applications. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '16). ACM, New York, NY, USA, 108--119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. P. M. Fitts. 1954. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental PSychology 74 (1954), 381--391.Google ScholarGoogle ScholarCross RefCross Ref
  10. Andy Gimblett and Harold Thimbleby. 2010. User Interface Model Discovery: Towards a Generic Approach. In Proceedings of the 2nd ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '10). ACM, New York, NY, USA, 145--154. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Yves Guiard. 2009. The Problem of Consistency in the Design of Fitts' Law Experiments: Consider Either Target Distance and Width or Movement Form and Scale. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'09). ACM, New York, NY, USA, 1809--1818. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Marc Hassenzahl. 2004. Funology. Kluwer Academic Publishers, Norwell, MA, USA, Chapter The Thing and I: Understanding the Relationship Between User and Product, 31--42. http://dl.acm.org/citation.cfm?id=1139008.1139015 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. ISO/IEC. 2010. ISO/IEC 25010 - Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models. Technical Report.Google ScholarGoogle Scholar
  14. Bonnie E. John and David E. Kieras. 1996. Using GOMS for User Interface Design and Evaluation: Which Technique? ACM Trans. Comput.-Hum. Interact. 3, 4 (Dec. 1996), 287--319. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Barbara A. Kitchenham, Tore Dyba, and Magne Jorgensen. 2004. Evidence-Based Software Engineering. In Proceedings of the 26th International Conference on Software Engineering (ICSE '04). IEEE Computer Society, Washington, DC, USA, 273--281. http://dl.acm.org/citation.cfm?id=998675.999432 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Valéria Lelli, Arnaud Blouin, Benoit Baudry, and Fabien Coulon. 2015. On Model-Based Testing Advanced GUIs. In Proceedings of the 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW) -- Workshop on Advances in Model Based Testing (A-MOST 2015). Graz, Austria.Google ScholarGoogle ScholarCross RefCross Ref
  17. Jane Li, Laurie Wilson, Stuart Stapleton, and Patrick Cregan. 2006. Design of an Advanced Telemedicine System for Emergency Care. In Proceedings of the 18th Australia Conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments (OZCHI '06). ACM, New York, NY, USA, 413--416. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Célia Martinie, David Navarre, Philippe Palanque, and Camille Fayollas. 2015. A Generic Tool-supported Framework for Coupling Task Models and Interactive Applications. In Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '15). ACM, New York, NY, USA, 244--253. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Célia Martinie, Philippe Palanque, and Marco Winckler. 2011. Structuring and Composition Mechanisms to Address Scalability Issues in Task Models. Springer Berlin Heidelberg, Berlin, Heidelberg, 589--609.Google ScholarGoogle Scholar
  20. Atif M. Memon. 2001. A comprehensive framework for testing graphical user interfaces. Ph.D. University of Pittsburgh. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Atif M. Memon, Mary Lou Soffa, and Martha E. Pollack. 2001. Coverage Criteria for GUI Testing. In Proceedings of the 8th European Software Engineering Conference Held Jointly with 9th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-9). ACM, New York, NY, USA, 256--267. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Rodrigo M.L.M. Moreira and Ana C.R. Paiva. 2014. PBGT Tool: An Integrated Modeling and Testing Environment for Pattern-based GUI Testing. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering (ASE '14). ACM, New York, NY, USA, 863--866. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Inês Coimbra Morgado. 2016. Automated Pattern-Based Testing of Mobile Applications. Ph.D. Faculdade de Engenharia da Universidade do Porto.Google ScholarGoogle Scholar
  24. Glenford J. Myers and Corey Sandler. 2004. The Art of Software Testing. John Wiley & Sons. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. David Navarre, Philippe A. Palanque, Fabio Paternò, Carmen Santoro, and Rémi Bastide. 2001. A Tool Suite for Integrating Task and System Models Through Scenarios. In Proceedings of the 8th International Workshop on Interactive Systems: Design, Specification, and Verification-Revised Papers (DSV-IS '01). Springer-Verlag, London, UK, UK, 88--113. http://dl.acm.org/citation.cfm?id=646166.680841 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Bao N. Nguyen, Bryan Robbins, Ishan Banerjee, and Atif Memon. 2014. GUITAR: an innovative tool for automated testing of GUI-driven software. Automated Software Engineering 21, 1 (2014), 65--105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Simeon Ntafos. 1998. On Random and Partition Testing. In Proceedings of the 1998 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA '98). ACM, New York, NY, USA, 42--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ana C. R. Paiva, João C. P. Faria, and Pedro M. C. Mendes. 2008. Reverse Engineered Formal Models for GUI Testing. Springer Berlin Heidelberg, Berlin, Heidelberg, 218--233. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Fabio Paternò. 2004. ConcurTaskTrees: An Engineered Notation for Task Models. In The Handbook of Task Analysis for Human-Computer Interaction, Dan Diaper and Neville Stanton (Eds.). Lawrence Erlbaum Associates, Chapter 24, 483--501.Google ScholarGoogle Scholar
  30. Carmen Santoro. 2005. A task model-based approach for design and evaluation of innovative user interfaces. Ph.D. Presses universitaires de Louvain.Google ScholarGoogle Scholar
  31. J.C. Silva, C. Silva, R. Goncalo, J. Saraiva, and J.C. Campos. 2010. The GUISurfer tool: towards a language independent approach to reverse engineering GUI code. In Proceedings of the 2nd ACM SIGCHI Symposium on Engineering interactive computing systems. ACM, 181--186. ISBN: 978--1--4503-0083--4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. J. L. Silva, J. C. Campos, and A. Paiva. 2008. Model-based user interface testing with Spec Explorer and ConcurTaskTrees. Electronic Notes in Theoretical Computer Science 208 (2008), 77--93. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Wan-Hui Tseng and Chin-Feng Fan. 2013. Systematic Scenario Test Case Generation for Nuclear Safety Systems. Inf. Softw. Technol. 55, 2 (Feb. 2013), 344--356. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Mark Utting, Alexander Pretschner, and Bruno Legeard. 2012. A Taxonomy of Model-based Testing Approaches. Software: Testing, Verification and Reliability 22, 5 (Aug. 2012), 297--312. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A More Intelligent Test Case Generation Approach through Task Models Manipulation

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!