Abstract
The C preprocessor (CPP) is a simple and language-independent tool, widely used to implement variable software systems using conditional compilation (i.e., by including or excluding annotated code). Although CPP provides powerful means to express variability, it has been criticized for allowing arbitrary annotations that break the underlying structure of the source code. We distinguish between disciplined annotations, which align with the structure of the source code, and undisciplined annotations, which do not. Several studies suggest that especially the latter type of annotations makes it hard to (automatically) analyze the code. However, little is known about whether the type of annotations has an effect on program comprehension. We address this issue by means of a controlled experiment with human subjects. We designed similar tasks for both, disciplined and undisciplined annotations, to measure program comprehension. Then, we measured the performance of the subjects regarding correctness and response time for solving the tasks. Our results suggest that there are no differences between disciplined and undisciplined annotations from a program-comprehension perspective. Nevertheless, we observed that finding and correcting errors is a time-consuming and tedious task in the presence of preprocessor annotations.
- T. Anderson and J. Finn. The New Statistical Analysis of Data. Springer, 1996.Google Scholar
Cross Ref
- B. Baker. On Finding Duplication and Near-Duplication in Large Software Systems. In Proc. Work. Conf. Reverse Engineering (WCRE), pages 86--95. IEEE, 1995. Google Scholar
Digital Library
- I. Baxter and M. Mehlich. Preprocessor Conditional Removal by Simple Partial Evaluation. In Proc. Work. Conf. Reverse Engineering (WCRE), pages 281--290. IEEE, 2001. Google Scholar
Digital Library
- G. Camilli and K. D. Hopkins. Applicability of Chi-square to 2 x 2 Contingency Tables with Small Expected Cell Frequencies. Psychological Bulletin, 85(1):163, 1978.Google Scholar
Cross Ref
- A. Dunsmore and M. Roper. A Comparative Evaluation of Program Comprehension Measures. Journal Sys. and Soft. (JSS), 52(3):121--129, 2000.Google Scholar
- M. Ernst, G. Badros, and D. Notkin. An Empirical Analysis of C Preprocessor Use. IEEE Trans. Software Engineering (TSE), 28(12):1146--1170, 2002. Google Scholar
Digital Library
- J.-M. Favre. The CPP Paradox. In Proc. European Workshop Software Maintenance, 1995. http://equipes-lig.imag.fr/adele/Les.Publications/intConferences/EWSM91995Fav.pdf.Google Scholar
- J.-M. Favre. Understanding-In-The-Large. In Int. Workshop Program Comprehension (IWPC), pages 29--38. IEEE, 1997. Google Scholar
Digital Library
- J. Feigenspan, C. Kästner, S. Apel, J. Liebig, M. Schulze, R. Dachselt, M. Papendieck, T. Leich, and G. Saake. Do background colors improve program comprehension in the #ifdef hell? Empirical Software Engineering, pages 1--47, 2012. Google Scholar
Digital Library
- J. Feigenspan, C. Kästner, J. Liebig, S. Apel, and S. Hanenberg. Measuring Programming Experience. In Proc. Int. Conf. Program Comprehension (ICPC), pages 73--82. IEEE, 2012.Google Scholar
Cross Ref
- J. Feigenspan, M. Schulze, M. Papendieck, C. Kästner, R. Dachselt, V. Köppen, and M. Frisch. Using Background Colors to Support Program Comprehension in Software Product Lines. In Proc. Int. Conf. Evaluation and Assessment in Software Engineering (EASE), pages 66--75. Institution of Engineering and Technology, 2011.Google Scholar
Cross Ref
- J. Feigenspan, M. Schulze, M. Papendieck, C. Kästner, R. Dachselt, V. Köppen, M. Frisch, and G. Saake. Supporting Program Comprehension in Large Preprocessor-Based Software Product Lines. IET Software, 6(6):488--501, 2012.Google Scholar
Cross Ref
- J. Feigenspan, N. Siegmund, A. Hasselberg, and M. Köppen. PROPHET: Tool Infrastructure to Support Program Comprehension Experiments. In Proc. Int. Symp. Empirical Software Engineering and Measurement (ESEM), 2011. Poster.Google Scholar
- A. Garrido and R. Johnson. Challenges of Refactoring C Programs. In Proc. Int. Workshop Principles of Software Evolution (IWPSE), pages 6--14. ACM, 2002. Google Scholar
Digital Library
- A. Garrido and R. Johnson. Refactoring C with Conditional Compilation. In Proc. Int. Conf. Automated Software Engineering (ASE), pages 323--326. IEEE, 2003.Google Scholar
Digital Library
- A. Garrido and R. Johnson. Analyzing Multiple Configurations of a C Program. In Proc. Int. Conf. Software Maintenance (ICSM), pages 379--388. IEEE, 2005. Google Scholar
Digital Library
- N. Göde and J. Harder. Clone Stability. In Proc. European Conf. Software Maintenance and Reengineering (CSMR), pages 65--74. IEEE, 2011. Google Scholar
Digital Library
- C. Goodwin. Research in Psychology: Methods and Design. Wiley Publishing, Inc., second edition, 1999.Google Scholar
- A. Jedlitschka, M. Ciolkowski, and D. Pfahl. Reporting Experiments in Software Engineering. In Guide to Advanced Empirical Software Engineering, pages 201--228. Springer, 2008.Google Scholar
Cross Ref
- E. Jürgens, F. Deissenböck, B. Hummel, and S. Wagner. Do Code Clones Matter? In Proc. Int. Conf. Software Engineering (ICSE), pages 485--495. IEEE, 2009. Google Scholar
Digital Library
- C. Kapser and M. W. Godfrey. "Cloning Considered Harmful" Considered Harmful. In Proc. Work. Conf. Reverse Engineering (WCRE), pages 19--28. IEEE, 2006. Google Scholar
Digital Library
- C. Kästner, S. Apel, and M. Kuhlemann. Granularity in Software Product Lines. In Proc. Int. Conf. Software Engineering (ICSE), pages 311--320. ACM, 2008. Google Scholar
Digital Library
- C. Kästner, S. Apel, S. Trujillo, M. Kuhlemann, and D. Batory. Guaranteeing Syntactic Correctness for all Product Line Variants: A Language-Independent Approach. In Proc. Int. Conf. Objects, Models, Components, Patterns (TOOLS), pages 174--194. Springer, 2009.Google Scholar
Cross Ref
- T. D. LaToza, G. Venolia, and R. DeLine. Maintaining Mental Models: A Study of Developer Work Habits. In Proc. Int. Conf. Software Engineering (ICSE), pages 492--501. ACM, 2006. Google Scholar
Digital Library
- D. Le, E. Walkingshaw, and M. Erwig. #ifdef Confirmed Harmful: Promoting Understandable Software Variation. In Proc. IEEE Symp. Visual Languages and Human-Centric Computing (VL/HCC), pages 143--150. IEEE, 2011.Google Scholar
- J. Liebig, S. Apel, C. Lengauer, C. Kästner, and M. Schulze. An Analysis of the Variability in Forty Preprocessor-Based Software Product Lines. In Proc. Int. Conf. Software Engineering (ICSE), pages 105--114. ACM, 2010. Google Scholar
Digital Library
- J. Liebig, C. Kästner, and S. Apel. Analyzing the Discipline of Preprocessor Annotations in 30 Million Lines of C Code. In Proc. Int. Conf. Aspect-Oriented Software Development (AOSD), pages 191--202. ACM, 2011. Google Scholar
Digital Library
- R. Likert. A Technique for the Measurement of Attitudes. Archives of Psychology, 140:1--55, 1932.Google Scholar
- F. Medeiros, M. Ribeiro, and R. Gheyi. Investigating Preprocessor-Based Syntax Errors. In Proc. Int. Conf. Generative Programming and Component Engineering (GPCE). ACM, 2013. to appear. Google Scholar
Digital Library
- C. Roy and J. Cordy. A Survey on Software Clone Detection Research. Technical Report 2007-541, Queen's University at Kingston, 2007.Google Scholar
- S. Schulze, E. Jürgens, and J. Feigenspan. Analyzing the Effect of Preprocessor Annotations on Code Clones. In Proc. Work. Conf. Source Code Analysis and Manipulation (SCAM), pages 115--124. IEEE, 2011. Google Scholar
Digital Library
- W. R. Shadish, T. D. Cook, and D. T. Campbell. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, 2002.Google Scholar
- J. Siegmund. Framework for Measuring Program Comprehension. PhD thesis, University of Magdeburg, 2012.Google Scholar
- H. Spencer and G. Collyer. #ifdef Considered Harmful, or Portability Experience with C News. In Proc. USENIX Technical Conf., pages 185--197. USENIX Association Berkeley, 1992.Google Scholar
- M. Svahnberg, A. Aurum, and C. Wohlin. Using Students as Subjects - An Empirical Evaluation. In Proc. Int. Symp. Empirical Software Engineering and Measurement (ESEM), pages 288--290. ACM, 2008. Google Scholar
Digital Library
- J. Yellott. Correction for Fast Guessing and the Speed Accuracy Trade-off in Choice Reaction Time. Journal of Mathematical Psychology, 8:159--199, 1971.Google Scholar
Cross Ref
Index Terms
Does the discipline of preprocessor annotations matter?: a controlled experiment
Recommendations
Analyzing the discipline of preprocessor annotations in 30 million lines of C code
AOSD '11: Proceedings of the tenth international conference on Aspect-oriented software developmentThe C preprocessor cpp is a widely used tool for implementing variable software. It enables programmers to express variable code (which may even crosscut the entire implementation) with conditional compilation. The C preprocessor relies on simple text ...
Does the discipline of preprocessor annotations matter?: a controlled experiment
GPCE '13: Proceedings of the 12th international conference on Generative programming: concepts & experiencesThe C preprocessor (CPP) is a simple and language-independent tool, widely used to implement variable software systems using conditional compilation (i.e., by including or excluding annotated code). Although CPP provides powerful means to express ...
Analyzing the Effect of Preprocessor Annotations on Code Clones
SCAM '11: Proceedings of the 2011 IEEE 11th International Working Conference on Source Code Analysis and ManipulationThe C preprocessor cpp is a powerful and language-independent tool, widely used to implement variable software in different programming languages (C, C++) using conditional compilation. Preprocessor annotations can used on different levels of ...







Comments