Abstract
Regression testing is a form of software quality assurance (QA) that involves comparing the behavior of a newer version of a software artifact to its earlier correct behavior, and signaling the QA engineer when deviations are detected. Given the large potential in automated generation and execution of regression test cases for business process models in the context of running systems, powerful tools are required to make this practically feasible, more specifically to limit the potential impact on production systems, and to reduce the manual effort required from QA engineers.
In this paper, we present a regression testing automation framework that implements the capture & replay paradigm in the context of BPMN 2.0, a domain-specific language for modeling and executing business processes. The framework employs parallelization techniques and efficient communication patterns to reduce the performance overhead of capturing. Based on inputs from the QA engineer, it manipulates the BPMN2 model before executing tests for isolating the latter from external dependencies (e.g. human actors or expensive web services) and for avoiding undesired side-effects. Finally, it performs a regression detection algorithm and reports the results to the QA engineer.
We have implemented our framework on top of a BPMN2-compliant execution engine, namely jBPM, and performed functional validations and evaluations of its performance and fault-tolerance. The results, indicating 3.9% average capturing performance overhead, demonstrate that the implemented framework can be the foundation of a practical regression testing tool for BPMN 2.0, and a key enabler for continuous delivery of business process-driven applications and services.
- MongoDB. https://www.mongodb.org/. Accessed: 2016- 02-17.Google Scholar
- Business Process Model and Notation (BPMN). http:// www.omg.org/spec/BPMN/2.0/PDF/. Accessed: 2016-02- 17.Google Scholar
- RedHat JBoss jBPM. http://www.jbpm.org/. Accessed: 2016-02-17.Google Scholar
- Jackson JSON Processor Wiki. http://wiki.fasterxml. com/JacksonHome. Accessed: 2016-02-17.Google Scholar
- JSON Benchmark Data. https://github.com/bura/ json-benchmarks. Accessed: 2016-02-17.Google Scholar
- Wildfly JEE Server. https://docs.jboss.org/author/ display/WFLY8/Documentation. Accessed: 2016-02-17.Google Scholar
- iMinds-OCareCloudS: Organizing Home Care Using a Cloud-based Platform. http://www.iminds.be/en/ projects/ocareclouds, June 2016.Google Scholar
- M. Bruno, G. Canfora, M. Di Penta, G. Esposito, and V. Mazza. Using test cases as contract to ensure service compliance across releases. In Service-Oriented Computing-ICSOC 2005, pages 87–100. Springer, 2005. Google Scholar
Digital Library
- J.-D. Choi and H. Srinivasan. Deterministic replay of java multithreaded applications. In Proceedings of the SIGMETRICS symposium on Parallel and distributed tools, pages 48– 59. ACM, 1998. Google Scholar
Digital Library
- M. M. Eler, A. Bertolino, and P. C. Masiero. More testable service compositions by test metadata. In Service Oriented System Engineering (SOSE), 2011 IEEE 6th International Symposium on, pages 204–213. IEEE, 2011. Google Scholar
Digital Library
- S. Freeman, T. Mackinnon, N. Pryce, and J. Walnes. Mock roles, objects. In Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 236–246. ACM, 2004. Google Scholar
Digital Library
- G. Gröner, C. Wende, M. Boˇskovi´c, F. S. Parreiras, T. Walter, F. Heidenreich, D. Gaˇsevi´c, and S. Staab. Validation of families of business processes. In International Conference on Advanced Information Systems Engineering, pages 551–565. Springer, 2011. Google Scholar
Digital Library
- G. Gröner, M. BoˇsKovi´c, F. S. Parreiras, and D. GaˇsEvi´c. Modeling and validation of business process families. Information Systems, 38(5):709–726, 2013. Google Scholar
Digital Library
- S.-S. Hou, L. Zhang, T. Xie, and J.-S. Sun. Quota-constrained test-case prioritization for regression testing of service-centric systems. In Software Maintenance, 2008. ICSM 2008. IEEE International Conference on, pages 257–266. IEEE, 2008.Google Scholar
- J. Humble and D. Farley. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Pearson Education, 2010. Google Scholar
Digital Library
- S. Ilieva, I. Manova, and D. Petrova-Antonova. Towards a methodology for testing of business processes. In Computer Science and Information Systems (FedCSIS), 2012 Federated Conference on, pages 1315–1322. IEEE, 2012.Google Scholar
- R. K. Ko, S. S. Lee, and E. Wah Lee. Business process management (bpm) standards: a survey. Business Process Management Journal, 15(5):744–791, 2009.Google Scholar
Cross Ref
- J. M. Küster, C. Gerth, A. Förster, and G. Engels. Business Process Management: 6th International Conference, BPM 2008, Milan, Italy, September 2-4, 2008. Proceedings, chapter Detecting and Resolving Process Model Differences in the Absence of a Change Log, pages 244–260. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. Google Scholar
Digital Library
- F. Leymann. Bpel vs. bpmn 2.0: Should you care? In International Workshop on Business Process Modeling Notation, pages 8–13. Springer, 2010.Google Scholar
- B. Li, D. Qiu, S. Ji, and D. Wang. Automatic test case selection and generation for regression testing of composite service based on extensible bpel flow graph. In Software Maintenance (ICSM), 2010 IEEE International Conference on, pages 1–10. IEEE, 2010. Google Scholar
Digital Library
- Z. J. Li, H. Tan, H. Liu, J. Zhu, and N. M. Mitsumori. Business-process-driven gray-box soa testing. IBM Systems Journal, 47(3):457–472, 2008. Google Scholar
Digital Library
- M. Makki, D. Van Landuyt, S. Walraven, and W. Joosen. Scalable and manageable customization of workflows in multitenant saas offerings. In Proceedings of the 31st Annual ACM Symposium on Applied Computing, pages 432–439. ACM, 2016. Google Scholar
Digital Library
- L. Mei, Z. Zhang, W. Chan, and T. Tse. Test case prioritization for regression testing of service-oriented business applications. In Proceedings of the 18th international conference on World wide web, pages 901–910. ACM, 2009. Google Scholar
Digital Library
- L. Mei, W. Chan, T. Tse, B. Jiang, and K. Zhai. Preemptive regression testing of workflow-based web services. Services Computing, IEEE Transactions on, 8(5):740–754, 2015.Google Scholar
- G. J. Myers, C. Sandler, and T. Badgett. The art of software testing. John Wiley & Sons, 2011. Google Scholar
Digital Library
- A. Orso and B. Kennedy. Selective capture and replay of program executions. In ACM SIGSOFT Software Engineering Notes, volume 30, pages 1–7. ACM, 2005. Google Scholar
Digital Library
- D. Saff, S. Artzi, J. H. Perkins, and M. D. Ernst. Automatic test factoring for java. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, pages 114–123. ACM, 2005. Google Scholar
Digital Library
- J. Steven, P. Chandra, B. Fleck, and A. Podgurski. jRapture: A capture/replay tool for observation-based testing, volume 25. ACM, 2000. Google Scholar
Digital Library
- D. Thomas and A. Hunt. Mock objects. Software, IEEE, 19 (3):22–24, 2002. Google Scholar
Digital Library
- W. M. van der Aalst. Business process management: a comprehensive survey. ISRN Software Engineering, 2013, 2013.Google Scholar
- W. M. Van Der Aalst, M. Dumas, F. Gottschalk, A. H. Ter Hofstede, M. La Rosa, and J. Mendling. Correctnesspreserving configuration of business process models. In International Conference on Fundamental Approaches to Software Engineering, pages 46–61. Springer, 2008. Google Scholar
Digital Library
- W. M. van der Aalst, M. Dumas, F. Gottschalk, A. H. Ter Hofstede, M. La Rosa, and J. Mendling. Preserving correctness during business process model configuration. Formal Aspects of Computing, 22(3-4):459–482, 2010. Google Scholar
Cross Ref
- D. Van Landuyt, S. Walraven, and W. Joosen. Variability middleware for multi-tenant saas applications: a research roadmap for service lines. In Proceedings of the 19th International Conference on Software Product Line, pages 211–215. ACM, 2015. Google Scholar
Digital Library
- C. Ye and H.-A. Jacobsen. Whitening soa testing via event exposure. Software Engineering, IEEE Transactions on, 39 (10):1444–1465, 2013. Google Scholar
Digital Library
- U. Zdun. Systematic pattern selection using pattern language grammars and design space analysis. Software-Practice and Experience, 37(9):983, 2007. Google Scholar
Digital Library
Index Terms
Automated regression testing of BPMN 2.0 processes: a capture and replay framework for continuous delivery
Recommendations
Automated regression testing of BPMN 2.0 processes: a capture and replay framework for continuous delivery
GPCE 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and ExperiencesRegression testing is a form of software quality assurance (QA) that involves comparing the behavior of a newer version of a software artifact to its earlier correct behavior, and signaling the QA engineer when deviations are detected. Given the large ...
Test Case Prioritization for Continuous Regression Testing: An Industrial Case Study
ICSM '13: Proceedings of the 2013 IEEE International Conference on Software MaintenanceRegression testing in continuous integration environment is bounded by tight time constraints. To satisfy time constraints and achieve testing goals, test cases must be efficiently ordered in execution. Prioritization techniques are commonly used to ...
Dependent-test-aware regression testing techniques
ISSTA 2020: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and AnalysisDevelopers typically rely on regression testing techniques to ensure that their changes do not break existing functionality. Unfortunately, these techniques suffer from flaky tests, which can both pass and fail when run multiple times on the same ...







Comments