skip to main content

A model for detecting faults in build specifications

Published:13 November 2020Publication History
Skip Abstract Section

Abstract

Incremental and parallel builds are crucial features of modern build systems. Parallelism enables fast builds by running independent tasks simultaneously, while incrementality saves time and computing resources by processing the build operations that were affected by a particular code change. Writing build definitions that lead to error-free incremental and parallel builds is a challenging task. This is mainly because developers are often unable to predict the effects of build operations on the file system and how different build operations interact with each other. Faulty build scripts may seriously degrade the reliability of automated builds, as they cause build failures, and non-deterministic and incorrect outputs.

To reason about arbitrary build executions, we present BuildFS, a generally-applicable model that takes into account the specification (as declared in build scripts) and the actual behavior (low-level file system operation) of build operations. We then formally define different types of faults related to incremental and parallel builds in terms of the conditions under which a file system operation violates the specification of a build operation. Our testing approach, which relies on the proposed model, analyzes the execution of single full build, translates it into BuildFS, and uncovers faults by checking for corresponding violations.

We evaluate the effectiveness, efficiency, and applicability of our approach by examining 612 Make and Gradle projects. Notably, thanks to our treatment of build executions, our method is the first to handle JVM-oriented build systems. The results indicate that our approach is (1) able to uncover several important issues (247 issues found in 47 open-source projects have been confirmed and fixed by the upstream developers), and (2) much faster than a state-of-the-art tool for Make builds (the median and average speedup is 39X and 74X respectively).

Skip Supplemental Material Section

Supplemental Material

Auxiliary Presentation Video

We present a method and a tool to detect three different types of faults in incremental and parallel builds. Our testing approach applies to both traditional (Make) and modern (Gradle) build systems without requiring extensive modifications to the underlying build scripts. We evaluate the effectiveness and efficiency of our approach by examining 612 Make and Gradle projects. Notably, our method is the first to handle JVM-based builds. The results indicate that our approach is (1) able to uncover several important issues (247 issues found in 47 open-source projects have been confirmed and fixed by the upstream developers), and (2) much faster than a state-of-the-art tool for Make builds. More details are available in our OOPSLA 2020 paper titled "A Model for Detecting Faults in Build Specifications" (https://doi.org/10.1145/3428212).

References

  1. B. Adams, H. Tromp, K. de Schutter, and W. de Meuter. Design recovery and maintenance of build systems. In 2007 IEEE International Conference on Software Maintenance, pages 114-123, Oct 2007. doi: 10.1109/ICSM. 2007. 4362624.Google ScholarGoogle ScholarCross RefCross Ref
  2. J. Al-Kofahi, H. V. Nguyen, and T. N. Nguyen. Fault localization for build code errors in Makefiles. In Companion Proceedings of the 36th International Conference on Software Engineering, ICSE Companion 2014, pages 600-601, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2768-8. doi: 10.1145/2591062.2591135. URL http://doi.acm.org/10.1145/2591062.2591135. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. G. Ammons. Grexmk: Speeding up scripted builds. In Proceedings of the 2006 International Workshop on Dynamic Systems Analysis, WODA '06, pages 81-87, New York, NY, USA, 2006. Association for Computing Machinery. ISBN 1595934006. doi: 10.1145/1138912.1138928. URL https://doi.org/10.1145/1138912.1138928. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. B. A. Avery Pennarun and P. Reinholdtsen. Debian popularity contest. https://popcon.debian.org/, 2020.Google ScholarGoogle Scholar
  5. Bazel. Build and test software of any size, quickly and reliably. https://bazel.build, 2020.Google ScholarGoogle Scholar
  6. C.-P. Bezemer, S. Mcintosh, B. Adams, D. M. German, and A. E. Hassan. An empirical study of unspecified dependencies in Make-based build systems. Empirical Software Engineering, 22 ( 6 ): 3117-3148, Dec. 2017. ISSN 1382-3256. doi: 10.1007/s10664-017-9510-8. URL https://doi.org/10.1007/s10664-017-9510-8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. D. Bruening, Q. Zhao, and S. Amarasinghe. Transparent dynamic instrumentation. In Proceedings of the 8th ACM SIGPLAN/SIGOPS Conference on Virtual Execution Environments, VEE '12, pages 133-144, New York, NY, USA, 2012. ACM. ISBN 978-1-4503-1176-2. doi: 10.1145/2151024.2151043. URL http://doi.acm.org/10.1145/2151024.2151043. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. J. Calcote. Autotools: A Practitioner's Guide to GNU Autoconf, Automake, and Libtool. No Starch Press, 2020.Google ScholarGoogle Scholar
  9. A. Celik, A. Knaust, A. Milicevic, and M. Gligoric. Build system with lazy retrieval for Java projects. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016, pages 643-654, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342186. doi: 10.1145/2950290.2950358. URL https://doi.org/10.1145/2950290.2950358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. A. Celik, M. Vasic, A. Milicevic, and M. Gligoric. Regression test selection across JVM boundaries. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2017, pages 809-820, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450351058. doi: 10.1145/3106237.3106297. URL https: //doi.org/10.1145/3106237.3106297. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. M. Christakis, K. R. Leino, and W. Schulte. Formalizing and verifying a modern build language. In Proceedings of the 19th International Symposium on FM 2014: Formal Methods-Volume 8442, pages 643-657. Springer, 2014. ISBN 9783319064093. doi: 10.1007/978-3-319-06410-9_43. URL https://doi.org/10.1007/978-3-319-06410-9_43. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. D. Coetzee, A. Bhaskar, and G. Necula. apmake: A reliable parallel build manager. In 2011 USENIX Annual Technical Conference (USENIX), 2011.Google ScholarGoogle Scholar
  13. D. Dashenkov. Gradle task ordering constraints. https://github.com/SpineEventEngine/base/issues/516, 2020.Google ScholarGoogle Scholar
  14. Debian. sbuild. https://wiki.debian.org/sbuild, 2020a.Google ScholarGoogle Scholar
  15. Debian. Ultimatedebiandatabase. https://wiki.debian.org/UltimateDebianDatabase/, 2020b.Google ScholarGoogle Scholar
  16. E. Derr, S. Bugiel, S. Fahl, Y. Acar, and M. Backes. Keep me updated: An empirical study of third-party library updatability on Android. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS '17, pages 2187-2200, New York, NY, USA, 2017. ACM. ISBN 978-1-4503-4946-8. doi: 10.1145/3133956.3134059. URL http://doi.acm.org/10.1145/3133956.3134059. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. S. Erdweg, M. Lichter, and M. Weiel. A sound and optimal incremental build system with dynamic dependencies. SIGPLAN Not., 50 ( 10 ): 89-106, Oct. 2015. ISSN 0362-1340. doi: 10.1145/2858965.2814316. URL https://doi.org/10.1145/2858965. 2814316. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. S. I. Feldman. Make-a program for maintaining computer programs. Software: Practice & Experience, 9 ( 4 ): 255-265, 1979.Google ScholarGoogle Scholar
  19. M. Gligoric, W. Schulte, C. Prasad, D. van Velzen, I. Narasamdya, and B. Livshits. Automated migration of build scripts using dynamic analysis and search-based refactoring. In Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications, OOPSLA ' 14, pages 599-616, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2585-1. doi: 10.1145/2660193.2660239. URL http://doi.acm.org/10.1145/2660193.2660239. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. M. Gligoric, L. Eloussi, and D. Marinov. Practical regression test selection with dynamic file dependencies. In Proceedings of the 2015 International Symposium on Software Testing and Analysis, ISSTA 2015, pages 211-222, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450336208. doi: 10.1145/2771783.2771784. URL https: //doi.org/10.1145/2771783.2771784. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. GNU Make. Generating prerequisites automatically. https://www.gnu.org/software/make/manual/html_node/AutomaticPrerequisites.html, 2020a.Google ScholarGoogle Scholar
  22. GNU Make. Handling tools that produce many outputs. https://www.gnu.org/software/automake/manual/html_node/ Multiple-Outputs.html, 2020b.Google ScholarGoogle Scholar
  23. Gradle Inc. Build cache. https://docs.gradle.org/current/userguide/build_cache.html, 2020a.Google ScholarGoogle Scholar
  24. Gradle Inc. Developing custom Gradle plugins. https://docs.gradle.org/current/userguide/custom_plugins.html, 2020b.Google ScholarGoogle Scholar
  25. Gradle Inc. Gradle vs Maven: Performance comparison. https://gradle.org /gradle-vs-maven-performance/, 2020c.Google ScholarGoogle Scholar
  26. S. van der Burg, E. Dolstra, S. McIntosh, J. Davies, D. M. German, and A. Hemel. Tracing software build processes to uncover license compliance inconsistencies. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering, ASE '14, pages 731-742, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-3013-8.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. J. Visser, S. Rigal, G. Wijnholds, and Z. Lubsen. Building Software Teams: Ten Best Practices for Efective Software Development. " O'Reilly Media, Inc.", 2016.Google ScholarGoogle Scholar
  28. K. Wang, C. Zhu, A. Celik, J. Kim, D. Batory, and M. Gligoric. Towards refactoring-aware regression test selection. In International Conference on Software Engineering, pages 233-244, 2018.Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. L. Zhang. Hybrid regression test selection. In Proceedings of the 40th International Conference on Software Engineering, ICSE ' 18, pages 199-209, New York, NY, USA, 2018. Association for Computing Machinery. ISBN 9781450356381. doi: 10.1145/3180155.3180198. URL https://doi.org/10.1145/3180155.3180198. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A model for detecting faults in build specifications

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!