skip to main content
research-article

An empirical study on configuration-related issues: investigating undeclared and unused identifiers

Published:26 October 2015Publication History
Skip Abstract Section

Abstract

The variability of configurable systems may lead to configuration-related issues (i.e., faults and warnings) that appear only when we select certain configuration options. Previous studies found that issues related to configurability are harder to detect than issues that appear in all configurations, because variability increases the complexity. However, little effort has been put into understanding configuration-related faults (e.g., undeclared functions and variables) and warnings (e.g., unused functions and variables). To better understand the peculiarities of configuration-related undeclared/unused variables and functions, in this paper we perform an empirical study of 15 systems to answer research questions related to how developers introduce these issues, the number of configuration options involved, and the time that these issues remain in source files. To make the analysis of several projects feasible, we propose a strategy that minimizes the initial setup problems of variability-aware tools. We detect and confirm 2 undeclared variables, 14 undeclared functions, 16 unused variables, and 7 unused functions related to configurability. We submit 30 patches to fix issues not fixed by developers. Our findings support the effectiveness of sampling (i.e., analysis of only a subset of valid configurations) because most issues involve two or less configuration options. Nevertheless, by analyzing the version history of the projects, we observe that a number of issues remain in the code for several years. Furthermore, the corpus of undeclared/unused variables and functions gathered is a valuable source to study these issues, compare sampling algorithms, and test and improve variability-aware tools.

References

  1. I. Abal, C. Brabrand, and A. Wasowski. 42 variability bugs in the Linux Kernel: A qualitative analysis. In Proceedings of the International Conference on Automated Software Engineering, ASE. IEEE/ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. S. Apel, C. Kästner, A. Grösslinger, and C. Lengauer. Type safety for feature-oriented product lines. Journal of Automated Software Engineering, 17:251–300, September 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. S. Apel, W. Scholz, C. Lengauer, and C. Kästner. Languageindependent reference checking in software product lines. In Proceedings of the 2nd International Workshop on Feature-Oriented Software Development, pages 65–71, 2010. ISBN 978-1-4503-0208-1. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. S. Apel, A. v. Rhein, P. Wendler, A. Grösslinger, and D. Beyer. Strategies for product-line verification: Case studies and experiments. In Proceedings of the International Conference on Software Engineering, ICSE. IEEE, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. I. Baxter. Design maintenance systems. Communication of the ACM, 35(4):73–89, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. R. C. Bryce and C. J. Colbourn. Prioritized interaction testing for pairwise coverage with seeding and constraints. Information and Software Technology, 48(10):960–970, 2006.Google ScholarGoogle ScholarCross RefCross Ref
  7. B. Delaware, W. Cook, and D. Batory. Fitting the Pieces Together: a machine-checked model of safe composition. In Proceedings of the Joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. M. Ernst, G. Badros, and D. Notkin. An empirical analysis of C preprocessor use. IEEE Transactions on Software Engineering, 28: 1146–1170, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. M. Forbes, J. Lawrence, Y. Lei, R. N. Kacker, and R. Kuhn. Refining the in-parameter-order strategy for constructing covering arrays. Journal of Research of the National Institute of Standards and Technology, 2008.Google ScholarGoogle ScholarCross RefCross Ref
  10. A. Garrido and R. Johnson. Analyzing multiple configurations of a C program. In Proceedings of the International Conference on Software Maintenance, ICSM. IEEE, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. B. J. Garvin and M. B. Cohen. Feature interaction faults revisited: An exploratory study. In Proceeding of the International Symposium on Software Reliability Engineering, ISSRE, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. B. J. Garvin, M. B. Cohen, and M. B. Dwyer. Using feature locality: Can we leverage history to avoid failures during reconfiguration? In Proceedings of the Workshop on Assurances for Self-adaptive Systems, ASAS. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. P. Gazzillo and R. Grimm. SuperC: parsing all of C by taming the preprocessor. In Proceedings of the programming language design and implementation, PLDI. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. C. Hunsen, B. Zhang, J. Siegmund, C. Kästner, O. Lebenich, M. Becker, and S. Apel. Preprocessor-based variability in open-source and industrial software systems: An empirical study. Journal of Empirical Software Engineering, 2015.Google ScholarGoogle Scholar
  15. M. F. Johansen, O. Haugen, and F. Fleurey. An algorithm for generating t-wise covering arrays from large feature models. In Proceedings of the International Software Product Line Conference, SPLC, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and A. S. Peterson. Feature-oriented domain analysis (foda) feasibility study. Technical report, Carnegie Mellon University, Software Engineering Institute, November 1990.Google ScholarGoogle ScholarCross RefCross Ref
  17. C. Kästner, P. Giarrusso, T. Rendel, S. Erdweg, K. Ostermann, and T. Berger. Variability-aware parsing in the presence of lexical macros and conditional compilation. In Proceedings of the ACM SIGPLAN Object-oriented programming systems languages and applications, OOPSLA. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. C. Kästner, S. Apel, T. Thüm, and G. Saake. Type checking annotation-based product lines. ACM Transactions on Software Engineering and Methodology, 21(3):14:1–14:39, July 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. C. Kästner, K. Ostermann, and S. Erdweg. A variability-aware module system. In Proceedings of ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA). ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. S. Kolesnikov, A. von Rhein, C. Hunsen, and S. Apel. A comparison of product-based, feature-based, and family-based type checking. In Proceedings of the International Conference on Generative Programming: Concepts & Experiences, GPCE. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. D. R. Kuhn, D. R. Wallace, and A. M. Gallo, Jr. Software fault interactions and implications for software testing. IEEE Transactions on Software Engineering, 30(6):418–421, June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. D. Larochelle and D. Evans. Statically detecting likely buffer overflow vulnerabilities. In Proceedings of the Conference on USENIX Security Symposium, SSYM. USENIX Association, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. J. Liebig, S. Apel, C. Lengauer, C. Kästner, and M. Schulze. An analysis of the variability in forty preprocessor-based software product lines. In Proceedings of the International Conference on Software Engineering, ICSE. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Liebig, C. Kästner, and S. Apel. Analyzing the discipline of preprocessor annotations in 30 million lines of C code. In Proceedings of the 10th Aspect-Oriented Software Development, AOSD. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. J. Liebig, A. von Rhein, C. Kästner, S. Apel, J. Dörre, and C. Lengauer. Scalable analysis of variable software. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC/FSE, pages 81–91. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. J. Liebig, A. Janker, F. Garbe, S. Apel, and C. Lengauer. Morpheus: Variability-aware refactoring in the wild. In Proceedings of the International Conference on Software Engineering, ICSE. IEEE, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. F. Medeiros, M. Ribeiro, and R. Gheyi. Investigating preprocessorbased syntax errors. In Proceedings of the International Conference on Generative Programming: Concepts & Experiences, GPCE. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. F. Medeiros, C. Kästner, M. Ribeiro, S. Nadi, and R. Gheyi. The love/hate relationship with the c preprocessor: An interview study. In Proceedings of the European Conference on Object-Oriented Programming, ECOOP, 2015.Google ScholarGoogle Scholar
  29. C. Nie and H. Leung. A survey of combinatorial testing. ACM Computing Surveys, 43(2):11:1–11:29, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. S. Oster, F. Markert, and P. Ritter. Automated incremental pairwise testing of software product lines. In J. Bosch and J. Lee, editors, Software Product Lines: Going Beyond, volume 6287 of Lecture Notes in Computer Science, pages 196–210. Springer Berlin Heidelberg, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. Le Traon. Automated and scalable t-wise test case generation strategies for software product lines. In Proceedings of the International Conference on Software Testing, Verification and Validation, ICST, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. M. Ribeiro, F. Queiroz, P. Borba, T. Tolˆedo, C. Brabrand, and S. Soares. On the impact of feature dependencies when maintaining preprocessor-based software product lines. In Proceedings of the Generative Programming and Component Engineering, GPCE. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. M. Ribeiro, P. Borba, and C. Kästner. Feature maintenance with emergent interfaces. In Proceedings of the International Conference on Software Engineering, ICSE, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. I. Schaefer, L. Bettini, and F. Damiani. Compositional type-checking for delta-oriented programming. In Proceedings of the International Conference on Aspect-Oriented Software Development, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. H. Spencer and G. Collyer. Ifdef considered harmful, or portability experience with C news. In USENIX Annual Technical Conference, pages 185–197, 1992.Google ScholarGoogle Scholar
  36. R. Tartler, D. Lohmann, C. Dietrich, C. Egger, and J. Sincero. Configuration coverage in the analysis of large-scale system software. In Proceedings of the Workshop on Programming Languages and Operating Systems, PLOS, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. R. Tartler, C. Dietrich, J. Sincero, W. Schröder-Preikschat, and D. Lohmann. Static analysis of variability in system software: The 90,000 #ifdefs issue. In 2014 USENIX Annual Technical Conference, USENIX, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. S. Thaker, D. Batory, D. Kitchin, and W. Cook. Safe Composition of Product Lines. In Proceedings of the 6th International Conference on Generative Programming and Component Engineering, pages 95– 104, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. T. Thüm, S. Apel, C. Kästner, I. Schaefer, and G. Saake. A classification and survey of analysis strategies for software product lines. ACM Computing Surveys, 47(1):6:1–6:45, June 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. A. von Rhein, A. Grebhahn, S. Apel, N. Siegmund, D. Beyer, and T. Berger. Presence-condition simplification in highly configurable systems. In Proceedings of the International Conference on Software Engineering, ICSE. IEEE, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. C. Yilmaz, M. B. Cohen, and A. Porter. Covering arrays for efficient fault characterization in complex configuration spaces. In Proceedings of ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA. ACM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. An empirical study on configuration-related issues: investigating undeclared and unused identifiers

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 51, Issue 3
      GPCE '15
      March 2016
      184 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2936314
      • Editor:
      • Andy Gill
      Issue’s Table of Contents
      • cover image ACM Conferences
        GPCE 2015: Proceedings of the 2015 ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences
        October 2015
        184 pages
        ISBN:9781450336871
        DOI:10.1145/2814204

      Copyright © 2015 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 26 October 2015

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!