skip to main content
article

Automated regression testing of BPMN 2.0 processes: a capture and replay framework for continuous delivery

Published:20 October 2016Publication History
Skip Abstract Section

Abstract

Regression testing is a form of software quality assurance (QA) that involves comparing the behavior of a newer version of a software artifact to its earlier correct behavior, and signaling the QA engineer when deviations are detected. Given the large potential in automated generation and execution of regression test cases for business process models in the context of running systems, powerful tools are required to make this practically feasible, more specifically to limit the potential impact on production systems, and to reduce the manual effort required from QA engineers.

In this paper, we present a regression testing automation framework that implements the capture & replay paradigm in the context of BPMN 2.0, a domain-specific language for modeling and executing business processes. The framework employs parallelization techniques and efficient communication patterns to reduce the performance overhead of capturing. Based on inputs from the QA engineer, it manipulates the BPMN2 model before executing tests for isolating the latter from external dependencies (e.g. human actors or expensive web services) and for avoiding undesired side-effects. Finally, it performs a regression detection algorithm and reports the results to the QA engineer.

We have implemented our framework on top of a BPMN2-compliant execution engine, namely jBPM, and performed functional validations and evaluations of its performance and fault-tolerance. The results, indicating 3.9% average capturing performance overhead, demonstrate that the implemented framework can be the foundation of a practical regression testing tool for BPMN 2.0, and a key enabler for continuous delivery of business process-driven applications and services.

References

  1. MongoDB. https://www.mongodb.org/. Accessed: 2016- 02-17.Google ScholarGoogle Scholar
  2. Business Process Model and Notation (BPMN). http:// www.omg.org/spec/BPMN/2.0/PDF/. Accessed: 2016-02- 17.Google ScholarGoogle Scholar
  3. RedHat JBoss jBPM. http://www.jbpm.org/. Accessed: 2016-02-17.Google ScholarGoogle Scholar
  4. Jackson JSON Processor Wiki. http://wiki.fasterxml. com/JacksonHome. Accessed: 2016-02-17.Google ScholarGoogle Scholar
  5. JSON Benchmark Data. https://github.com/bura/ json-benchmarks. Accessed: 2016-02-17.Google ScholarGoogle Scholar
  6. Wildfly JEE Server. https://docs.jboss.org/author/ display/WFLY8/Documentation. Accessed: 2016-02-17.Google ScholarGoogle Scholar
  7. iMinds-OCareCloudS: Organizing Home Care Using a Cloud-based Platform. http://www.iminds.be/en/ projects/ocareclouds, June 2016.Google ScholarGoogle Scholar
  8. M. Bruno, G. Canfora, M. Di Penta, G. Esposito, and V. Mazza. Using test cases as contract to ensure service compliance across releases. In Service-Oriented Computing-ICSOC 2005, pages 87–100. Springer, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. J.-D. Choi and H. Srinivasan. Deterministic replay of java multithreaded applications. In Proceedings of the SIGMETRICS symposium on Parallel and distributed tools, pages 48– 59. ACM, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. M. M. Eler, A. Bertolino, and P. C. Masiero. More testable service compositions by test metadata. In Service Oriented System Engineering (SOSE), 2011 IEEE 6th International Symposium on, pages 204–213. IEEE, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. Freeman, T. Mackinnon, N. Pryce, and J. Walnes. Mock roles, objects. In Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pages 236–246. ACM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. G. Gröner, C. Wende, M. Boˇskovi´c, F. S. Parreiras, T. Walter, F. Heidenreich, D. Gaˇsevi´c, and S. Staab. Validation of families of business processes. In International Conference on Advanced Information Systems Engineering, pages 551–565. Springer, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. G. Gröner, M. BoˇsKovi´c, F. S. Parreiras, and D. GaˇsEvi´c. Modeling and validation of business process families. Information Systems, 38(5):709–726, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. S.-S. Hou, L. Zhang, T. Xie, and J.-S. Sun. Quota-constrained test-case prioritization for regression testing of service-centric systems. In Software Maintenance, 2008. ICSM 2008. IEEE International Conference on, pages 257–266. IEEE, 2008.Google ScholarGoogle Scholar
  15. J. Humble and D. Farley. Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation. Pearson Education, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. S. Ilieva, I. Manova, and D. Petrova-Antonova. Towards a methodology for testing of business processes. In Computer Science and Information Systems (FedCSIS), 2012 Federated Conference on, pages 1315–1322. IEEE, 2012.Google ScholarGoogle Scholar
  17. R. K. Ko, S. S. Lee, and E. Wah Lee. Business process management (bpm) standards: a survey. Business Process Management Journal, 15(5):744–791, 2009.Google ScholarGoogle ScholarCross RefCross Ref
  18. J. M. Küster, C. Gerth, A. Förster, and G. Engels. Business Process Management: 6th International Conference, BPM 2008, Milan, Italy, September 2-4, 2008. Proceedings, chapter Detecting and Resolving Process Model Differences in the Absence of a Change Log, pages 244–260. Springer Berlin Heidelberg, Berlin, Heidelberg, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. F. Leymann. Bpel vs. bpmn 2.0: Should you care? In International Workshop on Business Process Modeling Notation, pages 8–13. Springer, 2010.Google ScholarGoogle Scholar
  20. B. Li, D. Qiu, S. Ji, and D. Wang. Automatic test case selection and generation for regression testing of composite service based on extensible bpel flow graph. In Software Maintenance (ICSM), 2010 IEEE International Conference on, pages 1–10. IEEE, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Z. J. Li, H. Tan, H. Liu, J. Zhu, and N. M. Mitsumori. Business-process-driven gray-box soa testing. IBM Systems Journal, 47(3):457–472, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. M. Makki, D. Van Landuyt, S. Walraven, and W. Joosen. Scalable and manageable customization of workflows in multitenant saas offerings. In Proceedings of the 31st Annual ACM Symposium on Applied Computing, pages 432–439. ACM, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. L. Mei, Z. Zhang, W. Chan, and T. Tse. Test case prioritization for regression testing of service-oriented business applications. In Proceedings of the 18th international conference on World wide web, pages 901–910. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. L. Mei, W. Chan, T. Tse, B. Jiang, and K. Zhai. Preemptive regression testing of workflow-based web services. Services Computing, IEEE Transactions on, 8(5):740–754, 2015.Google ScholarGoogle Scholar
  25. G. J. Myers, C. Sandler, and T. Badgett. The art of software testing. John Wiley & Sons, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. A. Orso and B. Kennedy. Selective capture and replay of program executions. In ACM SIGSOFT Software Engineering Notes, volume 30, pages 1–7. ACM, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. D. Saff, S. Artzi, J. H. Perkins, and M. D. Ernst. Automatic test factoring for java. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, pages 114–123. ACM, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. J. Steven, P. Chandra, B. Fleck, and A. Podgurski. jRapture: A capture/replay tool for observation-based testing, volume 25. ACM, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. D. Thomas and A. Hunt. Mock objects. Software, IEEE, 19 (3):22–24, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. W. M. van der Aalst. Business process management: a comprehensive survey. ISRN Software Engineering, 2013, 2013.Google ScholarGoogle Scholar
  31. W. M. Van Der Aalst, M. Dumas, F. Gottschalk, A. H. Ter Hofstede, M. La Rosa, and J. Mendling. Correctnesspreserving configuration of business process models. In International Conference on Fundamental Approaches to Software Engineering, pages 46–61. Springer, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. W. M. van der Aalst, M. Dumas, F. Gottschalk, A. H. Ter Hofstede, M. La Rosa, and J. Mendling. Preserving correctness during business process model configuration. Formal Aspects of Computing, 22(3-4):459–482, 2010. Google ScholarGoogle ScholarCross RefCross Ref
  33. D. Van Landuyt, S. Walraven, and W. Joosen. Variability middleware for multi-tenant saas applications: a research roadmap for service lines. In Proceedings of the 19th International Conference on Software Product Line, pages 211–215. ACM, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. C. Ye and H.-A. Jacobsen. Whitening soa testing via event exposure. Software Engineering, IEEE Transactions on, 39 (10):1444–1465, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. U. Zdun. Systematic pattern selection using pattern language grammars and design space analysis. Software-Practice and Experience, 37(9):983, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Automated regression testing of BPMN 2.0 processes: a capture and replay framework for continuous delivery

                  Recommendations

                  Comments

                  Login options

                  Check if you have access through your login credentials or your institution to get full access on this article.

                  Sign in

                  Full Access

                  PDF Format

                  View or Download as a PDF file.

                  PDF

                  eReader

                  View online with eReader.

                  eReader
                  About Cookies On This Site

                  We use cookies to ensure that we give you the best experience on our website.

                  Learn more

                  Got it!