Abstract
Build systems are used in all but the smallest software projects to invoke the right build tools on the right files in the right order. A build system must be sound (after a build, generated files consistently reflect the latest source files) and efficient (recheck and rebuild as few build units as possible). Contemporary build systems provide limited efficiency because they lack support for expressing fine-grained file dependencies. We present a build system called pluto that supports the definition of reusable, parameterized, interconnected builders. When run, a builder notifies the build system about dynamically required and produced files as well as about other builders whose results are needed. To support fine-grained file dependencies, we generalize the traditional notion of time stamps to allow builders to declare their actual requirements on a file's content. pluto collects the requirements and products of a builder with their stamps in a build summary. This enables pluto to provides provably sound and optimal incremental rebuilding. To support dynamic dependencies, our rebuild algorithm interleaves dependency analysis and builder execution and enforces invariants on the dependency graph through a dynamic analysis. We have developed pluto as a Java API and used it to implement more than 25 builders. We describe our experience with migrating a larger Ant build script to pluto and compare the respective build times.
Supplemental Material
Available for Download
This artifact contains a virtual-machine image of all software and benchmarking code of the paper "A Sound and Optimal Incremental Build System with Dynamic Dependencies". The artifact ZIP contains a detailed description of the artifact as a PDF. In particular, the artifact contains the source code of our build system pluto, the source code of the Java builder, the source code of the Latex builder, and the source code of the Spoofax builder. Moreover, the artifact contains an installation of the Eclipse IDE, which we use for evaluating the performance of the Spoofax builder. The artifact contains detailed instructions on how to reproduce the performance evaluation.
- D. Ancona, F. Damiani, S. Drossopoulou, and E. Zucca. Polymorphic bytecode: Compositional compilation for java-like languages. In Proceedings of Symposium on Principles of Programming Languages (POPL), pages 26–37. ACM, 2005. Google Scholar
Digital Library
- M. Christakis, K. R. M. Leino, and W. Schulte. Formalizing and verifying a modern build language. In Proceedings of Symposium on Formal Methods, volume 8442 of LNCS, pages 643–657. Springer, 2014.Google Scholar
- 12 http://bazel.io/Google Scholar
- S. Erdweg. Extensible Languages for Flexible and Principled Domain Abstraction. PhD thesis, Philipps-Universiät Marburg, 2013.Google Scholar
- S. Erdweg and K. Ostermann. Featherweight TeX and parser correctness. In Proceedings of Conference on Software Language Engineering (SLE), volume 6563 of LNCS, pages 397– 416. Springer, 2010. Google Scholar
Digital Library
- S. Erdweg and F. Rieger. A framework for extensible languages. In Proceedings of Conference on Generative Programming and Component Engineering (GPCE), pages 3–12. ACM, 2013. Google Scholar
Digital Library
- M. Gligoric, W. Schulte, C. Prasad, D. van Velzen, I. Narasamdya, and B. Livshits. Automated migration of build scripts using dynamic analysis and search-based refactoring. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 599–616. ACM, 2014. Google Scholar
Digital Library
- A. Heydon, R. Levin, T. Mann, and Y. Yu. The Vesta approach to software configuration management. Technical Report 168, Compaq SRC, 2001.Google Scholar
- A. Heydon, R. Levin, and Y. Yu. Caching function calls using precise dependencies. In Proceedings of Conference on Programming Language Design and Implementation (PLDI), pages 311–320. ACM, 2000. Google Scholar
Digital Library
- L. C. L. Kats and E. Visser. The Spoofax language workbench: Rules for declarative specification of languages and IDEs. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 444– 463. ACM, 2010. Google Scholar
Digital Library
- A. Kuhn, G. C. Murphy, and C. A. Thompson. An exploratory study of forces and frictions affecting large-scale model-driven development. In Proceedings of Conference on Model Driven Engineering Languages and Systems (MoDELS), volume 7590 of LNCS, pages 352–367. Springer, 2012. Google Scholar
Digital Library
- E. T. Kumfert G. Software in the DOE: The hidden overhead of "The Build". Technical report, Lawrence Livermore National Laboratory, 2002.Google Scholar
- S. McIntosh, B. Adams, and A. E. Hassan. The evolution of ANT build systems. In Proceedings of Conference on Mining Software Repositories (MSR), pages 42–51, 2010.Google Scholar
Cross Ref
- N. Mitchell. Shake before building: Replacing make with Haskell. In Proceedings of International Conference on Functional Programming (ICFP), pages 55–66. ACM, 2012. Google Scholar
Digital Library
- A. Neagu. What is wrong with Make? Available at http:// freecode.com/articles/what-is-wrong-with-make, 2005.Google Scholar
- M. Shal. Build system rules and algorithms. Available at http://gittup.org/tup/build_system_rules_and_ algorithms.pdf, 2009.Google Scholar
- R. M. Stallman, R. McGrath, and P. D. Smith. GNU Make Manual. Free Software Foundation, 2014.Google Scholar
- E. Visser, Z.-E.-A. Benaissa, and A. P. Tolmach. Building program optimizers with rewriting strategies. In Proceedings of International Conference on Functional Programming (ICFP), pages 13–26. ACM, 1998. Google Scholar
Digital Library
Index Terms
A sound and optimal incremental build system with dynamic dependencies
Recommendations
Scalable incremental building with dynamic task dependencies
ASE '18: Proceedings of the 33rd ACM/IEEE International Conference on Automated Software EngineeringIncremental build systems are essential for fast, reproducible software builds. Incremental build systems enable short feedback cycles when they capture dependencies precisely and selectively execute build tasks efficiently. A much overlooked feature of ...
A sound and optimal incremental build system with dynamic dependencies
OOPSLA 2015: Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and ApplicationsBuild systems are used in all but the smallest software projects to invoke the right build tools on the right files in the right order. A build system must be sound (after a build, generated files consistently reflect the latest source files) and ...
Accelerating Build Dependency Error Detection via Virtual Build
ASE '22: Proceedings of the 37th IEEE/ACM International Conference on Automated Software EngineeringBuild scripts play an important role in transforming the source code into executable artifacts. However, the development of build scripts is typically error-prone. As one kind of the most prevalent errors in build scripts, the dependency-related errors, ...






Comments