skip to main content
article

Exploring feature interactions without specifications: a controlled experiment

Published:07 April 2020Publication History
Skip Abstract Section

Abstract

In highly configurable systems, features may interact unexpectedly and produce faulty behavior. Those faults are not easily identified from the analysis of each feature separately, especially when feature specifications are missing. We propose VarXplorer, a dynamic and iterative approach to detect suspicious interactions. It provides information on how features impact the control and data flow of the program. VarXplorer supports developers with a graph that visualizes this information, mainly showing suppress and require relations between features. To evaluate whether VarXplorer helps improve the performance of identifying suspicious interactions, we perform a controlled study with 24 subjects. We find that with our proposed feature-interaction graphs, participants are able to identify suspicious interactions more than 3 times faster compared to the state-of-the-art tool.

References

  1. F. Angerer, A. Grimmer, H. Prähofer, and P. Grünbacher. ConfigurationAware Change Impact Analysis. In ASE, pages 385–395. IEEE, 2015.Google ScholarGoogle Scholar
  2. S. Apel and C. Kästner. An Overview of Feature-Oriented Software Development. JOT, 8(5):49–84, 2009.Google ScholarGoogle Scholar
  3. S. Apel, S. Kolesnikov, N. Siegmund, C. Kästner, and B. Garvin. Exploring Feature Interactions in the Wild: The New Feature-interaction Challenge. In FOSD, pages 1–8. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. S. Apel, A. v. Rhein, P. Wendler, A. Größ linger, and D. Beyer. Strategies for product-line verification: Case studies and experiments. In Proceedings of the 2013 International Conference on Software Engineering, ICSE, pages 482–491. IEEE Press, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Apel, H. Speidel, P. Wendler, A. von Rhein, and D. Beyer. Detection of Feature Interactions Using Feature-Aware Verification. In ASE, pages 372–375. IEEE, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. Apel, A. Von Rhein, T. Thüm, and C. Kästner. Feature-interaction detection based on feature-based specifications. Comput. Netw., 57(12):2399–2409, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. S. Arzt, S. Rasthofer, C. Fritz, E. Bodden, A. Bartel, J. Klein, Y. Le Traon, D. Octeau, and P. McDaniel. Flowdroid: Precise Context, Flow, Field, Object-Sensitive and Lifecycle-Aware Taint Analysis for Android Apps. PLDI, 49(6):259–269, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. J. M. Atlee, U. Fahrenberg, and A. Legay. Measuring behaviour interactions between product-line features. In Formalise, pages 20–25. IEEE Press, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. T. H. Austin and C. Flanagan. Multiple Facets for Dynamic Information Flow. SIGPLAN Not., 47(1):165–178, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Ben-David, B. Sterin, J. M. Atlee, and S. Beidu. Symbolic model checking of product-line requirements using sat-based methods. In Proceedings of the 37th International Conference on Software Engineering - Volume 1, ICSE, pages 189–199. IEEE Press, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. H. Beyer and K. Holtzblatt. Contextual Design: Defining CustomerCentered Systems. Elsevier, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. M. Böhme, B. C. d. S. Oliveira, and A. Roychoudhury. Regression Tests to Expose Change Interaction Errors. In ESEC/FSE, pages 334–344. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. T. F. Bowen, F. S. Dworack, C. H. Chow, N. Griffeth, G. E. Herman, and Y. J. Lin. The feature interaction problem in telecommunications systems. In SETSS, pages 59–62, 1989.Google ScholarGoogle Scholar
  14. G. E. Box, J. S. Hunter, and W. G. Hunter. Statistics for experimenters: design, innovation, and discovery, volume 2. Wiley-Interscience New York, 2005.Google ScholarGoogle Scholar
  15. J. R. Burch, E. M. Clarke, K. L. McMillan, D. L. Dill, and L.-J. Hwang. Symbolic Model Checking: 10 20 States and Beyond. In LICS, pages 428–439. IEEE, 1990.Google ScholarGoogle Scholar
  16. M. Calder, M. Kolberg, E. H. Magill, and S. Reiff-Marganiec. Feature interaction: a critical review and considered forecast. Computer Networks, 41(1):115 – 141, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. A. Classen, P. Heymans, P.-Y. Schobbens, and A. Legay. Symbolic Model Checking of Software Product Lines. In ICSE, pages 321–330. ACM, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. M. B. Cohen, M. B. Dwyer, and J. Shi. Interaction Testing of HighlyConfigurable Systems in the Presence of Constraints. In ISSTA, pages 129–139. ACM, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. M. B. Cohen, M. B. Dwyer, and J. Shi. Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach. TSE, 34(5):633–650, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Dean. Experimental design: Overview. In N. J. Smelser and P. B. Baltes, editors, International Encyclopedia of the Social Behavioral Sciences, pages 5090 – 5096. Pergamon, 2001.Google ScholarGoogle Scholar
  21. N. Griffeth, R. Blumenthal, J.-C. Gregoire, and T. Ohta. Feature interaction detection contest of the fifth international workshop on feature interactions. Computer Networks, 32(4):487 – 510, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. R. J. Hall. Feature combination and interaction detection via foreground/background models. Computer Networks, 32(4):449 – 469, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. G. W. Heiman. Basic statistics for the behavioral sciences. Cengage Learning, 2013.Google ScholarGoogle Scholar
  24. M. Hentschel, R. Hähnle, and R. Bubel. The Interactive Verification Debugger: Effective Understanding of Interactive Proof Attempts. In ASE, pages 846–851, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. C. H. P. Kim, D. Batory, and S. Khurshid. Eliminating Products to Test in a Software Product Line. In ASE, pages 139–142. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. C. H. P. Kim, S. Khurshid, and D. Batory. Shared Execution for Efficiently Testing Product Lines. In ISSRE, pages 221–230. IEEE, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. C. H. P. Kim, D. Marinov, S. Khurshid, D. Batory, S. Souto, P. Barros, and M. d’Amorim. SPLat: Lightweight Dynamic Analysis for Reducing Combinatorics in Testing Configurable Systems. In ESEC/FSE, pages 257–267. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. H. Li, S. Krishnamurthi, and K. Fisler. Modular Verification of Open Features Using Three-Valued Model Checking. ASE, 12(3):349–382, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. M. Lillack, C. Kästner, and E. Bodden. Tracking Load-Time Configuration Options. TSE, 2017.Google ScholarGoogle Scholar
  30. R. Lotufo, S. She, T. Berger, K. Czarnecki, and A. Wąsowski. Evolution of the linux kernel variability model. SPLC, pages 136–150. SpringerVerlag, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. F. Medeiros, C. Kästner, M. Ribeiro, R. Gheyi, and S. Apel. A Comparison of 10 Sampling Algorithms for Configurable Systems. In ICSE, pages 664–675. ACM, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. J. Meinicke, C.-P. Wong, C. Kästner, and G. Saake. Understanding Differences among Executions with Variational Traces. ArXiv e-prints, 2018.Google ScholarGoogle Scholar
  33. J. Meinicke, C. P. Wong, C. Kästner, T. Thüm, and G. Saake. On Essential Configuration Complexity : Measuring Interactions in HighlyConfigurable Systems. In ASE, number 2, pages 483–494, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. J. Melo, C. Brabrand, and A. Wąsowski. How Does the Degree of Variability Affect Bug Finding? In ICSE, pages 679–690. ACM, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. S. Nadi, T. Berger, C. Kästner, and K. Czarnecki. Mining configuration constraints: Static analyses and empirical results. ICSE, pages 140–151. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. H. V. Nguyen, C. Kästner, and T. N. Nguyen. Exploring VariabilityAware Execution for Testing Plugin-Based Web Applications. In ICSE, pages 907–918. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. H. V. Nguyen, C. Kästner, and T. N. Nguyen. Exploring variabilityaware execution for testing plugin-based web applications. In Proceedings of the 36th International Conference on Software Engineering, ICSE, pages 907–918. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. C. Nie and H. Leung. A Survey of Combinatorial Testing. CSUR, 43(2):11:1–11:29, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. J. A. Parejo, A. B. Sánchez, S. Segura, A. Ruiz-Cortés, R. E. LopezHerrejon, and A. Egyed. Multi-objective test case prioritization in highly configurable systems: A case study. Journal of Systems and Software, 122:287 – 310, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. M. Plath and M. Ryan. Feature Integration Using a Feature Construct. SCP, 41(1):53–84, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. E. Reisner, C. Song, K.-K. Ma, J. S. Foster, and A. Porter. Using Symbolic Evaluation to Understand Behavior in Configurable Software Systems. In ICSE, pages 445–454. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. V. Rothberg, N. Dintzner, A. Ziegler, and D. Lohmann. Feature models in linux: From symbols to semantics. VaMoS, pages 65–72. ACM, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. L. R. Soares. Varxplorer: Reasoning about feature interactions. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings, ICSE, pages 500–502. ACM, 2018. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. L. R. Soares, J. Meinicke, S. Nadi, C. Kästner, and E. S. de Almeida. VarXplorer: Lightweight Process for Dynamic Analysis of Feature Interactions. In Proceedings of the International Workshop on Variability Modelling of Software-Intensive Systems (VaMoS), pages 59–66. ACM, 2018. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. L. R. Soares, P.-Y. Schobbens, I. do Carmo Machado, and E. S. de Almeida. Feature interaction in software product line engineering: A systematic mapping study. Information and Software Technology, pages –, 2018.Google ScholarGoogle Scholar
  46. S. Souto, M. d’Amorim, and R. Gheyi. Balancing Soundness and Efficiency for Practical Testing of Configurable Systems. In ICSE, pages 632–642. IEEE Press, 2017. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. W. N. Sumner and X. Zhang. Algorithms for automatically computing the causal paths of failures. In FSE, pages 355–369, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. W. N. Sumner and X. Zhang. Comparative causality: Explaining the differences between executions. In ICSE, pages 272–281. IEEE Press, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. T. Thüm, S. Apel, C. Kästner, I. Schaefer, and G. Saake. A Classification and Survey of Analysis Strategies for Software Product Lines. CSUR, 47(1):6:1–6:45, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. A. von Rhein, S. Apel, and F. Raimondi. Introducing Binary Decision Diagrams in the Explicit-State Verification of Java Code. In JPF Workshop, 2011.Google ScholarGoogle Scholar
  51. A. von Rhein, A. Grebhahn, S. Apel, N. Siegmund, D. Beyer, and T. Berger. Presence-condition simplification in highly configurable systems. ICSE, pages 178–188. IEEE Press, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. C.-P. Wong, J. Meinicke, L. Lazarek, and C. Kästner. Faster Variational Execution with Transparent Bytecode Transformation. In OOPSLA. ACM, 2018. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. P. Zave. Software Requirements and Design: The Work of Michael Jackson, chapter Modularity in Distributed Feature Composition, pages 267–290. Good Friends Publishing Company, 2009.Google ScholarGoogle Scholar
  54. A. Zeller. Isolating Cause-Effect Chains From Computer Programs. In FSE, pages 1–10. ACM, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Exploring feature interactions without specifications: a controlled experiment

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 53, Issue 9
          GPCE '18
          September 2018
          214 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/3393934
          Issue’s Table of Contents
          • cover image ACM Conferences
            GPCE 2018: Proceedings of the 17th ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences
            November 2018
            214 pages
            ISBN:9781450360456
            DOI:10.1145/3278122

          Copyright © 2018 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 7 April 2020

          Check for updates

          Qualifiers

          • article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!