Abstract
The variability of configurable systems may lead to configuration-related issues (i.e., faults and warnings) that appear only when we select certain configuration options. Previous studies found that issues related to configurability are harder to detect than issues that appear in all configurations, because variability increases the complexity. However, little effort has been put into understanding configuration-related faults (e.g., undeclared functions and variables) and warnings (e.g., unused functions and variables). To better understand the peculiarities of configuration-related undeclared/unused variables and functions, in this paper we perform an empirical study of 15 systems to answer research questions related to how developers introduce these issues, the number of configuration options involved, and the time that these issues remain in source files. To make the analysis of several projects feasible, we propose a strategy that minimizes the initial setup problems of variability-aware tools. We detect and confirm 2 undeclared variables, 14 undeclared functions, 16 unused variables, and 7 unused functions related to configurability. We submit 30 patches to fix issues not fixed by developers. Our findings support the effectiveness of sampling (i.e., analysis of only a subset of valid configurations) because most issues involve two or less configuration options. Nevertheless, by analyzing the version history of the projects, we observe that a number of issues remain in the code for several years. Furthermore, the corpus of undeclared/unused variables and functions gathered is a valuable source to study these issues, compare sampling algorithms, and test and improve variability-aware tools.
- I. Abal, C. Brabrand, and A. Wasowski. 42 variability bugs in the Linux Kernel: A qualitative analysis. In Proceedings of the International Conference on Automated Software Engineering, ASE. IEEE/ACM, 2014. Google Scholar
Digital Library
- S. Apel, C. Kästner, A. Grösslinger, and C. Lengauer. Type safety for feature-oriented product lines. Journal of Automated Software Engineering, 17:251–300, September 2010. Google Scholar
Digital Library
- S. Apel, W. Scholz, C. Lengauer, and C. Kästner. Languageindependent reference checking in software product lines. In Proceedings of the 2nd International Workshop on Feature-Oriented Software Development, pages 65–71, 2010. ISBN 978-1-4503-0208-1. Google Scholar
Digital Library
- S. Apel, A. v. Rhein, P. Wendler, A. Grösslinger, and D. Beyer. Strategies for product-line verification: Case studies and experiments. In Proceedings of the International Conference on Software Engineering, ICSE. IEEE, 2013. Google Scholar
Digital Library
- I. Baxter. Design maintenance systems. Communication of the ACM, 35(4):73–89, 1992. Google Scholar
Digital Library
- R. C. Bryce and C. J. Colbourn. Prioritized interaction testing for pairwise coverage with seeding and constraints. Information and Software Technology, 48(10):960–970, 2006.Google Scholar
Cross Ref
- B. Delaware, W. Cook, and D. Batory. Fitting the Pieces Together: a machine-checked model of safe composition. In Proceedings of the Joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2009. Google Scholar
Digital Library
- M. Ernst, G. Badros, and D. Notkin. An empirical analysis of C preprocessor use. IEEE Transactions on Software Engineering, 28: 1146–1170, 2002. Google Scholar
Digital Library
- M. Forbes, J. Lawrence, Y. Lei, R. N. Kacker, and R. Kuhn. Refining the in-parameter-order strategy for constructing covering arrays. Journal of Research of the National Institute of Standards and Technology, 2008.Google Scholar
Cross Ref
- A. Garrido and R. Johnson. Analyzing multiple configurations of a C program. In Proceedings of the International Conference on Software Maintenance, ICSM. IEEE, 2005. Google Scholar
Digital Library
- B. J. Garvin and M. B. Cohen. Feature interaction faults revisited: An exploratory study. In Proceeding of the International Symposium on Software Reliability Engineering, ISSRE, 2011. Google Scholar
Digital Library
- B. J. Garvin, M. B. Cohen, and M. B. Dwyer. Using feature locality: Can we leverage history to avoid failures during reconfiguration? In Proceedings of the Workshop on Assurances for Self-adaptive Systems, ASAS. ACM, 2011. Google Scholar
Digital Library
- P. Gazzillo and R. Grimm. SuperC: parsing all of C by taming the preprocessor. In Proceedings of the programming language design and implementation, PLDI. ACM, 2012. Google Scholar
Digital Library
- C. Hunsen, B. Zhang, J. Siegmund, C. Kästner, O. Lebenich, M. Becker, and S. Apel. Preprocessor-based variability in open-source and industrial software systems: An empirical study. Journal of Empirical Software Engineering, 2015.Google Scholar
- M. F. Johansen, O. Haugen, and F. Fleurey. An algorithm for generating t-wise covering arrays from large feature models. In Proceedings of the International Software Product Line Conference, SPLC, 2012. Google Scholar
Digital Library
- K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and A. S. Peterson. Feature-oriented domain analysis (foda) feasibility study. Technical report, Carnegie Mellon University, Software Engineering Institute, November 1990.Google Scholar
Cross Ref
- C. Kästner, P. Giarrusso, T. Rendel, S. Erdweg, K. Ostermann, and T. Berger. Variability-aware parsing in the presence of lexical macros and conditional compilation. In Proceedings of the ACM SIGPLAN Object-oriented programming systems languages and applications, OOPSLA. ACM, 2011. Google Scholar
Digital Library
- C. Kästner, S. Apel, T. Thüm, and G. Saake. Type checking annotation-based product lines. ACM Transactions on Software Engineering and Methodology, 21(3):14:1–14:39, July 2012. Google Scholar
Digital Library
- C. Kästner, K. Ostermann, and S. Erdweg. A variability-aware module system. In Proceedings of ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). ACM, 2012. Google Scholar
Digital Library
- S. Kolesnikov, A. von Rhein, C. Hunsen, and S. Apel. A comparison of product-based, feature-based, and family-based type checking. In Proceedings of the International Conference on Generative Programming: Concepts & Experiences, GPCE. ACM, 2013. Google Scholar
Digital Library
- D. R. Kuhn, D. R. Wallace, and A. M. Gallo, Jr. Software fault interactions and implications for software testing. IEEE Transactions on Software Engineering, 30(6):418–421, June 2004. Google Scholar
Digital Library
- D. Larochelle and D. Evans. Statically detecting likely buffer overflow vulnerabilities. In Proceedings of the Conference on USENIX Security Symposium, SSYM. USENIX Association, 2001. Google Scholar
Digital Library
- J. Liebig, S. Apel, C. Lengauer, C. Kästner, and M. Schulze. An analysis of the variability in forty preprocessor-based software product lines. In Proceedings of the International Conference on Software Engineering, ICSE. ACM, 2010. Google Scholar
Digital Library
- J. Liebig, C. Kästner, and S. Apel. Analyzing the discipline of preprocessor annotations in 30 million lines of C code. In Proceedings of the 10th Aspect-Oriented Software Development, AOSD. ACM, 2011. Google Scholar
Digital Library
- J. Liebig, A. von Rhein, C. Kästner, S. Apel, J. Dörre, and C. Lengauer. Scalable analysis of variable software. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE, pages 81–91. ACM, 2013. Google Scholar
Digital Library
- J. Liebig, A. Janker, F. Garbe, S. Apel, and C. Lengauer. Morpheus: Variability-aware refactoring in the wild. In Proceedings of the International Conference on Software Engineering, ICSE. IEEE, 2015. Google Scholar
Digital Library
- F. Medeiros, M. Ribeiro, and R. Gheyi. Investigating preprocessorbased syntax errors. In Proceedings of the International Conference on Generative Programming: Concepts & Experiences, GPCE. ACM, 2013. Google Scholar
Digital Library
- F. Medeiros, C. Kästner, M. Ribeiro, S. Nadi, and R. Gheyi. The love/hate relationship with the c preprocessor: An interview study. In Proceedings of the European Conference on Object-Oriented Programming, ECOOP, 2015.Google Scholar
- C. Nie and H. Leung. A survey of combinatorial testing. ACM Computing Surveys, 43(2):11:1–11:29, 2011. Google Scholar
Digital Library
- S. Oster, F. Markert, and P. Ritter. Automated incremental pairwise testing of software product lines. In J. Bosch and J. Lee, editors, Software Product Lines: Going Beyond, volume 6287 of Lecture Notes in Computer Science, pages 196–210. Springer Berlin Heidelberg, 2010. Google Scholar
Digital Library
- G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. Le Traon. Automated and scalable t-wise test case generation strategies for software product lines. In Proceedings of the International Conference on Software Testing, Verification and Validation, ICST, 2010. Google Scholar
Digital Library
- M. Ribeiro, F. Queiroz, P. Borba, T. Tolˆedo, C. Brabrand, and S. Soares. On the impact of feature dependencies when maintaining preprocessor-based software product lines. In Proceedings of the Generative Programming and Component Engineering, GPCE. ACM, 2011. Google Scholar
Digital Library
- M. Ribeiro, P. Borba, and C. Kästner. Feature maintenance with emergent interfaces. In Proceedings of the International Conference on Software Engineering, ICSE, 2014. Google Scholar
Digital Library
- I. Schaefer, L. Bettini, and F. Damiani. Compositional type-checking for delta-oriented programming. In Proceedings of the International Conference on Aspect-Oriented Software Development, 2011. Google Scholar
Digital Library
- H. Spencer and G. Collyer. Ifdef considered harmful, or portability experience with C news. In USENIX Annual Technical Conference, pages 185–197, 1992.Google Scholar
- R. Tartler, D. Lohmann, C. Dietrich, C. Egger, and J. Sincero. Configuration coverage in the analysis of large-scale system software. In Proceedings of the Workshop on Programming Languages and Operating Systems, PLOS, 2011. Google Scholar
Digital Library
- R. Tartler, C. Dietrich, J. Sincero, W. Schröder-Preikschat, and D. Lohmann. Static analysis of variability in system software: The 90,000 #ifdefs issue. In 2014 USENIX Annual Technical Conference, USENIX, 2014. Google Scholar
Digital Library
- S. Thaker, D. Batory, D. Kitchin, and W. Cook. Safe Composition of Product Lines. In Proceedings of the 6th International Conference on Generative Programming and Component Engineering, pages 95– 104, 2007. Google Scholar
Digital Library
- T. Thüm, S. Apel, C. Kästner, I. Schaefer, and G. Saake. A classification and survey of analysis strategies for software product lines. ACM Computing Surveys, 47(1):6:1–6:45, June 2014. Google Scholar
Digital Library
- A. von Rhein, A. Grebhahn, S. Apel, N. Siegmund, D. Beyer, and T. Berger. Presence-condition simplification in highly configurable systems. In Proceedings of the International Conference on Software Engineering, ICSE. IEEE, 2015. Google Scholar
Digital Library
- C. Yilmaz, M. B. Cohen, and A. Porter. Covering arrays for efficient fault characterization in complex configuration spaces. In Proceedings of ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA. ACM, 2004. Google Scholar
Digital Library
Index Terms
An empirical study on configuration-related issues: investigating undeclared and unused identifiers
Recommendations
An empirical study on configuration-related issues: investigating undeclared and unused identifiers
GPCE 2015: Proceedings of the 2015 ACM SIGPLAN International Conference on Generative Programming: Concepts and ExperiencesThe variability of configurable systems may lead to configuration-related issues (i.e., faults and warnings) that appear only when we select certain configuration options. Previous studies found that issues related to configurability are harder to ...
An Empirical Study on Configuration-Related Code Weaknesses
SBES '20: Proceedings of the XXXIV Brazilian Symposium on Software EngineeringDevelopers often use the C preprocessor to handle variability and portability. However, many researchers and practitioners criticize the use of preprocessor directives because of their negative effect on code understanding, maintainability, and error ...
Safely evolving configurable systems
SPLASH Companion 2015: Companion Proceedings of the 2015 ACM SIGPLAN International Conference on Systems, Programming, Languages and Applications: Software for HumanityDevelopers use configuration options to tailor systems to different platforms. This configurability leads to exponential configuration spaces and traditional tools (e.g., gcc) check only one configuration at a time. As a result, developers introduce ...






Comments