Abstract
A software product line comprises a family of software products that share a common set of features. It enables customers to compose software systems from a managed set of features. Testing every product of a product line individually is often infeasible due to the exponential number of possible products in the number of features. Several approaches have been proposed to restrict the number of products to be tested by sampling a subset of products achieving sufficient combinatorial interaction coverage. However, existing sampling algorithms do not scale well to large product lines, as they require a considerable amount of time to generate the samples. Moreover, samples are not available until a sampling algorithm completely terminates. As testing time is usually limited, we propose an incremental approach of product sampling for pairwise interaction testing (called IncLing), which enables developers to generate samples on demand in a step-wise manner. Furthermore, IncLing uses heuristics to efficiently achieve pairwise interaction coverage with a reasonable number of products. We evaluated IncLing by comparing it against existing sampling algorithms using feature models of different sizes. The results of our approach indicate efficiency improvements for product-line testing.
- M. Al-Hajjaji. Scalable Sampling and Prioritization for Product-Line Testing. In Proc. Software Engineering (SE), pages 295–298. Gesellschaft für Informatik (GI), 2015.Google Scholar
- M. Al-Hajjaji, J. Meinicke, S. Krieter, R. Schröter, T. Thüm, T. Leich, and G. Saake. Tool Demo: Testing Configurable Systems with FeatureIDE. In Proc. Int’l Conf. Generative Programming and Component Engineering (GPCE), 2016. To appear. Google Scholar
Digital Library
- M. Al-Hajjaji, T. Thüm, J. Meinicke, M. Lochau, and G. Saake. Similarity-Based Prioritization in Software Product-Line Testing. In Proc. Int’l Software Product Line Conf. (SPLC), pages 197–206. ACM, 2014. Google Scholar
Digital Library
- S. Apel, D. Batory, C. Kästner, and G. Saake. Feature-Oriented Software Product Lines: Concepts and Implementation. Springer, 2013. Google Scholar
Digital Library
- D. Batory, J. N. Sarvela, and A. Rauschmayer. Scaling Step-Wise Refinement. IEEE Trans. Software Engineering (TSE), 30(6):355–371, 2004. Google Scholar
Digital Library
- D. Benavides, S. Segura, and A. Ruiz-Cortés. Automated Analysis of Feature Models 20 Years Later: A Literature Review. Information Systems, 35(6):615–708, 2010. Google Scholar
Digital Library
- I. D. Carmo Machado, J. D. McGregor, Y. a. C. Cavalcanti, and E. S. De Almeida. On Strategies for Testing Software Product Lines: A Systematic Literature Review. J. Information and Software Technology (IST), 56(10):1183–1199, 2014. Google Scholar
Digital Library
- V. Chvatal. A Greedy Heuristic for the Set-Covering Problem. Mathematics of operations research, 4(3):233– 235, 1979. Google Scholar
Digital Library
- P. Clements and L. Northrop. Software Product Lines: Practices and Patterns. Addison-Wesley, 2001. Google Scholar
Digital Library
- M. B. Cohen, M. B. Dwyer, and J. Shi. Coverage and Adequacy in Software Product Line Testing. In Proc. Int’l Symposium in Software Testing and Analysis (ISSTA), pages 53–63. ACM, 2006. Google Scholar
Digital Library
- M. B. Cohen, M. B. Dwyer, and J. Shi. Interaction Testing of Highly-Configurable Systems in the Presence of Constraints. In Proc. Int’l Symposium in Software Testing and Analysis (ISSTA), pages 129–139. ACM, 2007. Google Scholar
Digital Library
- S. A. Cook. The Complexity of Theorem-proving Procedures. In Proc. of ACM Symposium on Theory of Computing (STOC), pages 151–158. ACM, 1971. Google Scholar
Digital Library
- K. Czarnecki and U. Eisenecker. Generative Programming: Methods, Tools, and Applications. ACM/Addison-Wesley, 2000. Google Scholar
Digital Library
- L. Engebretsen. The Nonapproximability of Non-Boolean Predicates. SIAM Journal on Discrete Mathematics (JDM), 18(1):114–129, 2005. Google Scholar
Digital Library
- B. J. Garvin, M. B. Cohen, and M. B. Dwyer. Evaluating Improvements to a Meta-Heuristic Search for Constrained Interaction Testing. Empirical Software Engineering (EMSE), 16(1):61–102, 2011. Google Scholar
Digital Library
- C. Henard, M. Papadakis, G. Perrouin, J. Klein, P. Heymans, and Y. L. Le Traon. Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-Wise Test Configurations for Software Product Lines. IEEE Trans. Software Engineering (TSE), 40(7):650– 670, 2014. Google Scholar
Digital Library
- M. Jackson and P. Zave. Distributed Feature Composition: A Virtual Architecture for Telecommunications Services. IEEE Trans. Software Engineering (TSE), 24(10):831–847, 1998. Google Scholar
Digital Library
- M. F. Johansen, Ø. Haugen, and F. Fleurey. Properties of Realistic Feature Models Make Combinatorial Testing of Product Lines Feasible. In Proc. Int’l Conf. Model Driven Engineering Languages and Systems (MODELS), pages 638–652. Springer, 2011. Google Scholar
Digital Library
- M. F. Johansen, Ø. Haugen, and F. Fleurey. An Algorithm for Generating T-Wise Covering Arrays from Large Feature Models. In Proc. Int’l Software Product Line Conf. (SPLC), pages 46–55. ACM, 2012. Google Scholar
Digital Library
- K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and A. S. Peterson. Feature-Oriented Domain Analysis (FODA) Feasibility Study. Technical Report CMU/SEI- 90-TR-21, Software Engineering Institute, 1990.Google Scholar
Cross Ref
- C. H. P. Kim, D. Batory, and S. Khurshid. Reducing Combinatorics in Testing Product Lines. In Proc. Int’l Conf. Aspect-Oriented Software Development (AOSD), pages 57—68. ACM, 2011. Google Scholar
Digital Library
- M. Kowal, S. Schulze, and I. Schaefer. Towards Efficient SPL Testing by Variant Reduction. In Proce. Int’l workshop on Variability & composition (VariComp), pages 1–6. ACM, 2013. Google Scholar
Digital Library
- D. R. Kuhn, D. R. Wallace, and A. M. Gallo Jr. Software Fault Interactions and Implications for Software Testing. IEEE Trans. Software Engineering (TSE), 30(6):418– 421, 2004. Google Scholar
Digital Library
- Y. Lei, R. N. Kacker, D. R. Kuhn, V. Okun, and J. Lawrence. IPOG: A General Strategy for T-Way Software Testing. In Proc. Int’l Conf. Engineering of Computer-Based Systems (ECBS), pages 549–556. IEEE, 2007. Google Scholar
Digital Library
- J. Liebig, A. von Rhein, C. Kästner, S. Apel, J. Dörre, and C. Lengauer. Scalable Analysis of Variable Software. In Proc. Europ. Software Engineering Conf./Foundations of Software Engineering (ESEC/FSE), pages 81–91. ACM, 2013. Google Scholar
Digital Library
- M. Lochau, I. Schaefer, J. Kamischke, and S. Lity. Incremental Model-Based Testing of Delta-oriented Software Product Lines. In Proc. of Int’l Conf. on Tests and Proofs (TAP), pages 67–82. Springer, 2012. Google Scholar
Digital Library
- F. Medeiros, C. Kästner, M. Ribeiro, R. Gheyi, and S. Apel. A Comparison of 10 Sampling Algorithms for Configurable Systems. In Proc. Int’l Conf. Software Engineering (ICSE), ICSE ’16, pages 643–654. ACM, 2016. Google Scholar
Digital Library
- J. Melo, E. Flesborg, C. Brabrand, and A. Wasowski. A Quantitative Analysis of Variability Warnings in Linux. In Proc. Int’l Workshop Variability Modelling of Software-Intensive Systems (VaMoS), VaMoS ’16, pages 3–8. ACM, 2016. Google Scholar
Digital Library
- S. Oster, F. Markert, and P. Ritter. Automated Incremental Pairwise Testing of Software Product Lines. In Proc. Int’l Software Product Line Conf. (SPLC), pages 196–210. Springer, 2010. Google Scholar
Digital Library
- S. Oster, I. Zorcic, F. Markert, and M. Lochau. MoSo-PoLiTe - Tool Support for Pairwise and Model-Based Software Product Line Testing. In Proc. Int’l Workshop Variability Modelling of Software-Intensive Systems (VaMoS), pages 79–82. ACM, 2011. Google Scholar
Digital Library
- G. Perrouin, S. Oster, S. Sen, J. Klein, B. Baudry, and Y. Le Traon. Pairwise Testing for Software Product Lines: Comparison of Two Approaches. Software Quality Journal (SQJ), 20(3-4):605–643, 2012. Google Scholar
Digital Library
- G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. Le Traon. Automated and Scalable T-Wise Test Case Generation Strategies for Software Product Lines. In Proc. Int’l Conf. Software Testing, Verification and Validation (ICST), pages 459–468. IEEE, 2010. Google Scholar
Digital Library
- K. Pohl, G. Böckle, and F. J. van der Linden. Software Product Line Engineering: Foundations, Principles and Techniques. Springer, 2005. Google Scholar
Digital Library
- J. Shi, M. B. Cohen, and M. B. Dwyer. Integration Testing of Software Product Lines Using Compositional Symbolic Execution. In Proc. Int’l Conf. Fundamental Approaches to Software Engineering (FASE), pages 270– 284. Springer, 2012. Google Scholar
Digital Library
- T. Thüm, C. Kästner, F. Benduhn, J. Meinicke, G. Saake, and T. Leich. FeatureIDE: An Extensible Framework for Feature-Oriented Software Development. Science of Computer Programming (SCP), 79(0):70–85, 2014. Google Scholar
Digital Library
- D. M. Weiss. The Product Line Hall of Fame. In Proc. Int’l Software Product Line Conf. (SPLC), page 395. IEEE, 2008. Google Scholar
Digital Library
Index Terms
IncLing: efficient product-line testing using incremental pairwise sampling
Recommendations
Similarity-based prioritization in software product-line testing
SPLC '14: Proceedings of the 18th International Software Product Line Conference - Volume 1Exhaustively testing every product of a software product line (SPL) is a difficult task due to the combinatorial explosion of the number of products. Combinatorial interaction testing is a technique to reduce the number of products under test. However, ...
IncLing: efficient product-line testing using incremental pairwise sampling
GPCE 2016: Proceedings of the 2016 ACM SIGPLAN International Conference on Generative Programming: Concepts and ExperiencesA software product line comprises a family of software products that share a common set of features. It enables customers to compose software systems from a managed set of features. Testing every product of a product line individually is often ...
Effective product-line testing using similarity-based product prioritization
A software product line comprises a family of software products that share a common set of features. Testing an entire product-line product-by-product is infeasible due to the potentially exponential number of products in the number of features. ...







Comments