Test2Feature: feature-based test traceability tool for highly configurable software

ABSTRACT

To ensure the quality of Highly Configurable Software (HCS) in an evolution and maintenance scenario is a challenging task. As HCSs evolve, new features are added, changed, or removed, which hampers the selection and evolution of test cases. The use of test traceability reports can help in this task, but there is a lack of studies addressing HCS test-to-feature traceability. Existing work usually are based on the variability model, which is not always available or updated. Some tools only link test cases to code lines. Considering this gap, this paper introduces Test2Feature, a tool that traces test cases to features using the source code of annotated HCSs, written in C/C++. The tool produces the following outputs: the code lines that correspond to each feature, the lines that correspond to each test case, and the test cases that are linked to each feature. Test2Feature is based only on the static analysis of the code. The traceability report produced can be used to ease different tasks related, for instance, to regression testing, feature management, and HCS evolution and maintenance.

References

1. M. Al-Hajjaji, J. Meinicke, S. Krieter, R. Schröter, T. Thüm, T. Leich, and G. Saake. 2016. Tool demo: testing configurable systems with featureIDE. In GPCE. 173--177.
2. S. Apel, H. Speidel, P. Wendler, A. Von Rhein, and D. Beyer. 2011. Detection of feature interactions using feature-aware verification. In ASE. IEEE, 372--375.
3. D. Benavides, S. Segura, P. Trinidad, and A. Ruiz-Cortés. 2006. Using Java CSP Solvers in the Automated Analyses of Feature Models. Springer, 399--408.
4. G. Cavarlé, A. Plantec, S. Costiou, and V. Ribaud. 2018. A feature-oriented model-driven engineering approach for the early validation of feature-based applications. Science of Computer Programming 161 (2018), 18--33.
5. F. Ferreira, J. P. Diniz, C. Silva, and E. Figueiredo. 2019. Testing tools for configurable software systems: A review-based empirical study. In VAMOS. 1--10.
6. C. Henard, M. Papadakis, and Y. L. Traon. 2014. Mutation-based generation of software product line test configurations. In SSBSE. Springer, 92--106.
7. Martin Fagereng Johansen, Øystein Haugen, and Franck Fleurey. 2011. Properties of realistic feature models make combinatorial testing of product lines feasible. In Model Driven Engineering Languages and Systems. Springer, 638--652.
8. C. H. P. Kim, D. S. Batory, and S. Khurshid. 2011. Reducing combinatorics in testing product lines. In AOSD. 57--68.
9. Chang Hwan Peter Kim, Sarfraz Khurshid, and Don Batory. 2012. Shared Execution for Efficiently Testing Product Lines. In 23rd SSRE. 221--230.
10. K. Ludwig, J. Krüger, and T. Leich. 2019. Covert and phantom features in annotations: do they impact variability analysis?. In SPLC. ACM, 31:1--31:13.
11. D. Marijan, A. Gotlieb, S. Sen, and A. Hervieu. 2013. Practical pairwise testing for software product lines. In 17th SPLC. 227--235.
12. J. Meinicke, C. Wong, C. Kästner, T. Thüm, and G. Saake. 2016. On essential configuration complexity: Measuring interactions in highly-configurable systems. In ASE. 483--494.
13. G. K. Michelon, W. K. G. Assunção, D. Obermann, L. Linsbauer, P. Grünbacher, and A. Egyed. 2021. The life cycle of features in highly-configurable software systems evolving in space and time. In GPCE. ACM, 2--15.
14. G. K. Michelon, D Obermann, W. K. G. Assunção, L. Linsbauer, P. Grünbacher, and A. Egyed. 2020. Mining Feature Revisions in Highly-Configurable Software Systems. In SPLC. ACM, 74--78.
15. G. K. Michelon, D. Obermann, W. K. G. Assunção, L. Linsbauer, P. Grünbacher, S. Fischer, R. E. Lopez-Herrejon, and A. Egyed. 2022. Evolving software system families in space and time with feature revisions. Empirical Software Engineering 27, 5 (May 2022).
16. M. Mukelabai, D. Nešiundefined, S. Maro, T. Berger, and J. Steghöfer. 2018. Tackling Combinatorial Explosion: A Study of Industrial Needs and Practices for Analyzing Highly Configurable Systems. In ASE. ACM, 155--166.
17. T. Schiex and S. de Givry (Eds.). 2019. Principles and Practice of Constraint Programming. LNCS '19, Vol. 11802. Springer.
18. S. Souto, M. d'Amorim, and R. Gheyi. 2017. Balancing soundness and efficiency for practical testing of configurable systems. In ICSE. IEEE, 632--642.
19. H. Tufail, M. F. Masood, B. Zeb, F. Azam, and M. W. Anwar. 2017. A systematic review of requirement traceability techniques and tools. In 2nd ICSRS. 450--454.
20. A. von Rhein, A. Grebhahn, S. Apel, N. Siegmund, D. Beyer, and T. Berger. 2015. Presence-condition Simplification in Highly Configurable Systems. In ICSE. IEEE, 178--188.

Index Terms

• Software and its engineering

  • Software creation and management

    • Software development techniques

      • Reusability

        • Software product lines

    • Software verification and validation

      • Process validation

        • Traceability

      • Software defect analysis

        • Software testing and debugging

  • Software notations and tools

    • Software maintenance tools