Abstract
Incremental and parallel builds are crucial features of modern build systems. Parallelism enables fast builds by running independent tasks simultaneously, while incrementality saves time and computing resources by processing the build operations that were affected by a particular code change. Writing build definitions that lead to error-free incremental and parallel builds is a challenging task. This is mainly because developers are often unable to predict the effects of build operations on the file system and how different build operations interact with each other. Faulty build scripts may seriously degrade the reliability of automated builds, as they cause build failures, and non-deterministic and incorrect outputs.
To reason about arbitrary build executions, we present BuildFS, a generally-applicable model that takes into account the specification (as declared in build scripts) and the actual behavior (low-level file system operation) of build operations. We then formally define different types of faults related to incremental and parallel builds in terms of the conditions under which a file system operation violates the specification of a build operation. Our testing approach, which relies on the proposed model, analyzes the execution of single full build, translates it into BuildFS, and uncovers faults by checking for corresponding violations.
We evaluate the effectiveness, efficiency, and applicability of our approach by examining 612 Make and Gradle projects. Notably, thanks to our treatment of build executions, our method is the first to handle JVM-oriented build systems. The results indicate that our approach is (1) able to uncover several important issues (247 issues found in 47 open-source projects have been confirmed and fixed by the upstream developers), and (2) much faster than a state-of-the-art tool for Make builds (the median and average speedup is 39X and 74X respectively).
Supplemental Material
- B. Adams, H. Tromp, K. de Schutter, and W. de Meuter. Design recovery and maintenance of build systems. In 2007 IEEE International Conference on Software Maintenance, pages 114-123, Oct 2007. doi: 10.1109/ICSM. 2007. 4362624.Google Scholar
Cross Ref
- J. Al-Kofahi, H. V. Nguyen, and T. N. Nguyen. Fault localization for build code errors in Makefiles. In Companion Proceedings of the 36th International Conference on Software Engineering, ICSE Companion 2014, pages 600-601, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2768-8. doi: 10.1145/2591062.2591135. URL http://doi.acm.org/10.1145/2591062.2591135. Google Scholar
Digital Library
- G. Ammons. Grexmk: Speeding up scripted builds. In Proceedings of the 2006 International Workshop on Dynamic Systems Analysis, WODA '06, pages 81-87, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595934006. doi: 10.1145/1138912.1138928. URL https://doi.org/10.1145/1138912.1138928. Google Scholar
Digital Library
- B. A. Avery Pennarun and P. Reinholdtsen. Debian popularity contest. https://popcon.debian.org/, 2020.Google Scholar
- Bazel. Build and test software of any size, quickly and reliably. https://bazel.build, 2020.Google Scholar
- C.-P. Bezemer, S. Mcintosh, B. Adams, D. M. German, and A. E. Hassan. An empirical study of unspecified dependencies in Make-based build systems. Empirical Software Engineering, 22 ( 6 ): 3117-3148, Dec. 2017. ISSN 1382-3256. doi: 10.1007/s10664-017-9510-8. URL https://doi.org/10.1007/s10664-017-9510-8. Google Scholar
Digital Library
- D. Bruening, Q. Zhao, and S. Amarasinghe. Transparent dynamic instrumentation. In Proceedings of the 8th ACM SIGPLAN/SIGOPS Conference on Virtual Execution Environments, VEE '12, pages 133-144, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1176-2. doi: 10.1145/2151024.2151043. URL http://doi.acm.org/10.1145/2151024.2151043. Google Scholar
Digital Library
- J. Calcote. Autotools: A Practitioner's Guide to GNU Autoconf, Automake, and Libtool. No Starch Press, 2020.Google Scholar
- A. Celik, A. Knaust, A. Milicevic, and M. Gligoric. Build system with lazy retrieval for Java projects. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, pages 643-654, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342186. doi: 10.1145/2950290.2950358. URL https://doi.org/10.1145/2950290.2950358. Google Scholar
Digital Library
- A. Celik, M. Vasic, A. Milicevic, and M. Gligoric. Regression test selection across JVM boundaries. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, pages 809-820, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450351058. doi: 10.1145/3106237.3106297. URL https: //doi.org/10.1145/3106237.3106297. Google Scholar
Digital Library
- M. Christakis, K. R. Leino, and W. Schulte. Formalizing and verifying a modern build language. In Proceedings of the 19th International Symposium on FM 2014: Formal Methods-Volume 8442, pages 643-657. Springer, 2014. ISBN 9783319064093. doi: 10.1007/978-3-319-06410-9_43. URL https://doi.org/10.1007/978-3-319-06410-9_43. Google Scholar
Digital Library
- D. Coetzee, A. Bhaskar, and G. Necula. apmake: A reliable parallel build manager. In 2011 USENIX Annual Technical Conference (USENIX), 2011.Google Scholar
- D. Dashenkov. Gradle task ordering constraints. https://github.com/SpineEventEngine/base/issues/516, 2020.Google Scholar
- Debian. sbuild. https://wiki.debian.org/sbuild, 2020a.Google Scholar
- Debian. Ultimatedebiandatabase. https://wiki.debian.org/UltimateDebianDatabase/, 2020b.Google Scholar
- E. Derr, S. Bugiel, S. Fahl, Y. Acar, and M. Backes. Keep me updated: An empirical study of third-party library updatability on Android. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS '17, pages 2187-2200, New York, NY, USA, 2017. ACM. ISBN 978-1-4503-4946-8. doi: 10.1145/3133956.3134059. URL http://doi.acm.org/10.1145/3133956.3134059. Google Scholar
Digital Library
- S. Erdweg, M. Lichter, and M. Weiel. A sound and optimal incremental build system with dynamic dependencies. SIGPLAN Not., 50 ( 10 ): 89-106, Oct. 2015. ISSN 0362-1340. doi: 10.1145/2858965.2814316. URL https://doi.org/10.1145/2858965. 2814316. Google Scholar
Digital Library
- S. I. Feldman. Make-a program for maintaining computer programs. Software: Practice & Experience, 9 ( 4 ): 255-265, 1979.Google Scholar
- M. Gligoric, W. Schulte, C. Prasad, D. van Velzen, I. Narasamdya, and B. Livshits. Automated migration of build scripts using dynamic analysis and search-based refactoring. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA ' 14, pages 599-616, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2585-1. doi: 10.1145/2660193.2660239. URL http://doi.acm.org/10.1145/2660193.2660239. Google Scholar
Digital Library
- M. Gligoric, L. Eloussi, and D. Marinov. Practical regression test selection with dynamic file dependencies. In Proceedings of the 2015 International Symposium on Software Testing and Analysis, ISSTA 2015, pages 211-222, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450336208. doi: 10.1145/2771783.2771784. URL https: //doi.org/10.1145/2771783.2771784. Google Scholar
Digital Library
- GNU Make. Generating prerequisites automatically. https://www.gnu.org/software/make/manual/html_node/AutomaticPrerequisites.html, 2020a.Google Scholar
- GNU Make. Handling tools that produce many outputs. https://www.gnu.org/software/automake/manual/html_node/ Multiple-Outputs.html, 2020b.Google Scholar
- Gradle Inc. Build cache. https://docs.gradle.org/current/userguide/build_cache.html, 2020a.Google Scholar
- Gradle Inc. Developing custom Gradle plugins. https://docs.gradle.org/current/userguide/custom_plugins.html, 2020b.Google Scholar
- Gradle Inc. Gradle vs Maven: Performance comparison. https://gradle.org /gradle-vs-maven-performance/, 2020c.Google Scholar
- S. van der Burg, E. Dolstra, S. McIntosh, J. Davies, D. M. German, and A. Hemel. Tracing software build processes to uncover license compliance inconsistencies. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering, ASE '14, pages 731-742, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-3013-8.Google Scholar
Digital Library
- J. Visser, S. Rigal, G. Wijnholds, and Z. Lubsen. Building Software Teams: Ten Best Practices for Efective Software Development. " O'Reilly Media, Inc.", 2016.Google Scholar
- K. Wang, C. Zhu, A. Celik, J. Kim, D. Batory, and M. Gligoric. Towards refactoring-aware regression test selection. In International Conference on Software Engineering, pages 233-244, 2018.Google Scholar
Digital Library
- L. Zhang. Hybrid regression test selection. In Proceedings of the 40th International Conference on Software Engineering, ICSE ' 18, pages 199-209, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450356381. doi: 10.1145/3180155.3180198. URL https://doi.org/10.1145/3180155.3180198. Google Scholar
Digital Library
Index Terms
A model for detecting faults in build specifications






Comments