Abstract
Most software systems are designed to provide custom functionality using configuration options. Testing such systems is challenging as running tests of a single configuration is often not sufficient, because defects may appear in other configurations. Ideally, all configurations of a software system should be tested, which is usually not applicable in practice due to the combinatorial explosion with respect to the configuration options. Multiple sampling strategies aim to reduce the set of tested configurations to a feasible amount, such as T-wise sampling, random configurations, and user-defined configurations. However, these strategies are often not applied in practice as they require manual effort or a specialized testing framework. Within our tool FeatureIDE, we integrate all aforementioned strategies and reduce the manual effort by automating the process of generating and testing configurations. Furthermore, we provide support for unit testing to avoid redundant test executions and for variability-aware testing. With this extension of FeatureIDE, we aim to make recent testing techniques for configurable systems applicable in practice.
- M. Al-Hajjaji, S. Krieter, T. Thüm, M. Lochau, and G. Saake. IncLing: Efficient Product-Line Testing Using Incremental Pairwise Sampling. In GPCE, 2016. To appear. Google Scholar
Digital Library
- M. Al-Hajjaji, T. Thüm, J. Meinicke, M. Lochau, and G. Saake. Similarity-Based Prioritization in Software Product-Line Testing. In SPLC, pp. 197–206. ACM, 2014. Google Scholar
Digital Library
- S. Apel, D. Batory, C. Kästner, and G. Saake. Feature-Oriented Software Product Lines: Concepts and Implementation. Springer, 2013. Google Scholar
Digital Library
- S. Apel, C. Kästner, and C. Lengauer. Language-Independent and Automated Software Composition: The FeatureHouse Experience. TSE, 39(1):63–79, 2013. Google Scholar
Digital Library
- S. Apel, A. von Rhein, P. Wendler, A. Größlinger, and D. Beyer. Strategies for Product-Line Verification: Case Studies and Experiments. In ICSE, pp. 482–491. IEEE, 2013. Google Scholar
Digital Library
- D. Batory. Feature Models, Grammars, and Propositional Formulas. In SPLC, pp. 7–20. Springer, 2005. Google Scholar
Digital Library
- D. Batory, J. N. Sarvela, and A. Rauschmayer. Scaling Step-Wise Refinement. TSE, 30(6):355–371, 2004. Google Scholar
Digital Library
- B. Beckert, R. Hähnle, and P. Schmitt. Verification of Object-Oriented Software: The KeY Approach. Springer, 2007. Google Scholar
Digital Library
- D. Benavides, S. Segura, and A. Ruiz-Cortés. Automated Analysis of Feature Models 20 Years Later: A Literature Review. Information Systems, 35(6):615–708, 2010. Google Scholar
Digital Library
- J. Bürdek, M. Lochau, S. Bauregger, A. Holzer, A. von Rhein, S. Apel, and D. Beyer. Facilitating Reuse in Multi-goal Test-Suite Generation for Software Product Lines. In FASE, pp. 84–99. Springer, 2015.Google Scholar
- V. Chvatal. A Greedy Heuristic for the Set-Covering Problem. MOR, 4(3):233–235, 1979. Google Scholar
Digital Library
- B. J. Garvin, M. B. Cohen, and M. B. Dwyer. Evaluating Improvements to a Meta-Heuristic Search for Constrained Interaction Testing. EMSE, 16(1):61–102, 2011. Google Scholar
Digital Library
- K. Havelund and T. Pressburger. Model Checking Java Programs Using Java PathFinder. STTT, 2(4):366–381, 2000.Google Scholar
- C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. L. Traon. PLEDGE: A Product Line Editor and Test Generation Tool. In SPLC, pp. 126–129. ACM, 2013. Google Scholar
Digital Library
- K. J. Hoffman, P. Eugster, and S. Jagannathan. Semanticsaware Trace Analysis. In PLDI, pp. 453–464. ACM, 2009. Google Scholar
Digital Library
- M. Jackson and P. Zave. Distributed Feature Composition: A Virtual Architecture for Telecommunications Services. TSE, 24(10):831–847, 1998. Google Scholar
Digital Library
- M. F. Johansen, Ø. Haugen, and F. Fleurey. Properties of Realistic Feature Models Make Combinatorial Testing of Product Lines Feasible. In MODELS, pp. 638–652. Springer, 2011. Google Scholar
Digital Library
- M. F. Johansen, Ø. Haugen, and F. Fleurey. An Algorithm for Generating T-Wise Covering Arrays from Large Feature Models. In SPLC, pp. 46–55. ACM, 2012. Google Scholar
Digital Library
- G. Kiczales, J. Lamping, A. Mendhekar, C. Maeda, C. Lopes, J.-M. Loingtier, and J. Irwin. Aspect-Oriented Programming. In ECOOP, pp. 220–242. Springer, 1997.Google Scholar
- C. H. P. Kim, D. Marinov, S. Khurshid, D. Batory, S. Souto, P. Barros, and M. d’Amorim. SPLat: Lightweight Dynamic Analysis for Reducing Combinatorics in Testing Configurable Systems. In ESEC/FSE, pp. 257–267. ACM, 2013. Google Scholar
Digital Library
- D. R. Kuhn, D. R. Wallace, and A. M. Gallo Jr. Software Fault Interactions and Implications for Software Testing. TSE, 30(6):418–421, 2004. Google Scholar
Digital Library
- D. Le Berre and A. Parrain. The sat4j Library, Release 2.2, System Description. JSAT, 7:59–64, 2010.Google Scholar
- J. McGregor. Testing a Software Product Line. In Testing Techniques in Software Engineering, pp. 104–140. Springer, 2010.Google Scholar
- F. Medeiros, C. Kästner, M. Ribeiro, S. Nadi, and R. Gheyi. The Love/Hate Relationship with the C Preprocessor: An Interview Study. In ECOOP, pp. 495–518. Schloss Dagstuhl– Leibniz-Zentrum fuer Informatik, 2015.Google Scholar
- J. Meinicke, T. Thüm, R. Schröter, S. Krieter, F. Benduhn, G. Saake, and T. Leich. FeatureIDE: Taming the Preprocessor Wilderness. In ICSE, pp. 629–632. ACM, 2016. Google Scholar
Digital Library
- J. Meinicke, C.-P. Wong, C. Kästner, T. Thüm, and G. Saake. On Essential Configuration Complexity: Measuring Interactions in Highly-configurable Systems. In ASE, pp. 483–494. ACM, 2016. Google Scholar
Digital Library
- J. A. Pereira, S. Krieter, J. Meinicke, R. Schröter, G. Saake, and T. Leich. FeatureIDE: Scalable Product Configuration of Variable Systems. In ICSR, pp. 397–401. Springer, 2016. Google Scholar
Digital Library
- G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. Le Traon. Automated and Scalable T-Wise Test Case Generation Strategies for Software Product Lines. In ICST, pp. 459–468. IEEE, 2010. Google Scholar
Digital Library
- M. Steffens, S. Oster, M. Lochau, and T. Fogdal. Industrial Evaluation of Pairwise SPL Testing with MoSo-PoLiTe. In VaMoS, pp. 55–62. ACM, 2012. Google Scholar
Digital Library
- R. Tartler, D. Lohmann, J. Sincero, and W. Schröder-Preikschat. Feature Consistency in Compile-Time-Configurable System Software: Facing the Linux 10,000 Feature Problem. In EuroSys, pp. 47–60. ACM, 2011. Google Scholar
Digital Library
- T. Thüm, S. Apel, C. Kästner, I. Schaefer, and G. Saake. A Classification and Survey of Analysis Strategies for Software Product Lines. CSUR, 47(1):6:1–6:45, 2014. Google Scholar
Digital Library
- T. Thüm, C. Kästner, F. Benduhn, J. Meinicke, G. Saake, and T. Leich. FeatureIDE: An Extensible Framework for Feature-Oriented Software Development. SCP, 79(0):70–85, 2014. Google Scholar
Digital Library
- T. Thüm, J. Meinicke, F. Benduhn, M. Hentschel, A. von Rhein, and G. Saake. Potential Synergies of Theorem Proving and Model Checking for Software Product Lines. In SPLC, pp. 177–186. ACM, 2014. Google Scholar
Digital Library
- A. von Rhein, S. Apel, and F. Raimondi. Introducing Binary Decision Diagrams in the Explicit-State Verification of Java code. In JavaPathfinder Workshop, 2011.Google Scholar
- B. Xin, W. N. Sumner, and X. Zhang. Efficient Program Execution Indexing. In PLDI, pp. 238–248. ACM, 2008. Google Scholar
Digital Library
Index Terms
Tool demo: testing configurable systems with FeatureIDE
Recommendations
Tool demo: testing configurable systems with FeatureIDE
GPCE 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and ExperiencesMost software systems are designed to provide custom functionality using configuration options. Testing such systems is challenging as running tests of a single configuration is often not sufficient, because defects may appear in other configurations. ...
Automating Test Reuse for Highly Configurable Software
SPLC '19: Proceedings of the 23rd International Systems and Software Product Line Conference - Volume ADealing with highly configurable systems is generally very complex. Hundreds of different analysis techniques have been conceived to deal with different aspects of configurable systems. One large focal point is the testing of configurable software. This ...
A Configurable Test Case Prioritization Technique for Early Fault Detection and Low Test Case Spreading
SBES '22: Proceedings of the XXXVI Brazilian Symposium on Software EngineeringDevelopers often use test suites as safety nets to avoid functionality regression. However, regression testing can be costly and time-consuming. Test case prioritization (TCP) techniques try to reduce this burden by reordering the tests of a given ...







Comments