skip to main content
research-article

A sound and optimal incremental build system with dynamic dependencies

Published:23 October 2015Publication History
Skip Abstract Section

Abstract

Build systems are used in all but the smallest software projects to invoke the right build tools on the right files in the right order. A build system must be sound (after a build, generated files consistently reflect the latest source files) and efficient (recheck and rebuild as few build units as possible). Contemporary build systems provide limited efficiency because they lack support for expressing fine-grained file dependencies. We present a build system called pluto that supports the definition of reusable, parameterized, interconnected builders. When run, a builder notifies the build system about dynamically required and produced files as well as about other builders whose results are needed. To support fine-grained file dependencies, we generalize the traditional notion of time stamps to allow builders to declare their actual requirements on a file's content. pluto collects the requirements and products of a builder with their stamps in a build summary. This enables pluto to provides provably sound and optimal incremental rebuilding. To support dynamic dependencies, our rebuild algorithm interleaves dependency analysis and builder execution and enforces invariants on the dependency graph through a dynamic analysis. We have developed pluto as a Java API and used it to implement more than 25 builders. We describe our experience with migrating a larger Ant build script to pluto and compare the respective build times.

Skip Supplemental Material Section

Supplemental Material

References

  1. D. Ancona, F. Damiani, S. Drossopoulou, and E. Zucca. Polymorphic bytecode: Compositional compilation for java-like languages. In Proceedings of Symposium on Principles of Programming Languages (POPL), pages 26–37. ACM, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. M. Christakis, K. R. M. Leino, and W. Schulte. Formalizing and verifying a modern build language. In Proceedings of Symposium on Formal Methods, volume 8442 of LNCS, pages 643–657. Springer, 2014.Google ScholarGoogle Scholar
  3. 12 http://bazel.io/Google ScholarGoogle Scholar
  4. S. Erdweg. Extensible Languages for Flexible and Principled Domain Abstraction. PhD thesis, Philipps-Universiät Marburg, 2013.Google ScholarGoogle Scholar
  5. S. Erdweg and K. Ostermann. Featherweight TeX and parser correctness. In Proceedings of Conference on Software Language Engineering (SLE), volume 6563 of LNCS, pages 397– 416. Springer, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. S. Erdweg and F. Rieger. A framework for extensible languages. In Proceedings of Conference on Generative Programming and Component Engineering (GPCE), pages 3–12. ACM, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. M. Gligoric, W. Schulte, C. Prasad, D. van Velzen, I. Narasamdya, and B. Livshits. Automated migration of build scripts using dynamic analysis and search-based refactoring. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 599–616. ACM, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. A. Heydon, R. Levin, T. Mann, and Y. Yu. The Vesta approach to software configuration management. Technical Report 168, Compaq SRC, 2001.Google ScholarGoogle Scholar
  9. A. Heydon, R. Levin, and Y. Yu. Caching function calls using precise dependencies. In Proceedings of Conference on Programming Language Design and Implementation (PLDI), pages 311–320. ACM, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. L. C. L. Kats and E. Visser. The Spoofax language workbench: Rules for declarative specification of languages and IDEs. In Proceedings of Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA), pages 444– 463. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. A. Kuhn, G. C. Murphy, and C. A. Thompson. An exploratory study of forces and frictions affecting large-scale model-driven development. In Proceedings of Conference on Model Driven Engineering Languages and Systems (MoDELS), volume 7590 of LNCS, pages 352–367. Springer, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. E. T. Kumfert G. Software in the DOE: The hidden overhead of "The Build". Technical report, Lawrence Livermore National Laboratory, 2002.Google ScholarGoogle Scholar
  13. S. McIntosh, B. Adams, and A. E. Hassan. The evolution of ANT build systems. In Proceedings of Conference on Mining Software Repositories (MSR), pages 42–51, 2010.Google ScholarGoogle ScholarCross RefCross Ref
  14. N. Mitchell. Shake before building: Replacing make with Haskell. In Proceedings of International Conference on Functional Programming (ICFP), pages 55–66. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. A. Neagu. What is wrong with Make? Available at http:// freecode.com/articles/what-is-wrong-with-make, 2005.Google ScholarGoogle Scholar
  16. M. Shal. Build system rules and algorithms. Available at http://gittup.org/tup/build_system_rules_and_ algorithms.pdf, 2009.Google ScholarGoogle Scholar
  17. R. M. Stallman, R. McGrath, and P. D. Smith. GNU Make Manual. Free Software Foundation, 2014.Google ScholarGoogle Scholar
  18. E. Visser, Z.-E.-A. Benaissa, and A. P. Tolmach. Building program optimizers with rewriting strategies. In Proceedings of International Conference on Functional Programming (ICFP), pages 13–26. ACM, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A sound and optimal incremental build system with dynamic dependencies

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM SIGPLAN Notices
            ACM SIGPLAN Notices  Volume 50, Issue 10
            OOPSLA '15
            October 2015
            953 pages
            ISSN:0362-1340
            EISSN:1558-1160
            DOI:10.1145/2858965
            • Editor:
            • Andy Gill
            Issue’s Table of Contents
            • cover image ACM Conferences
              OOPSLA 2015: Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications
              October 2015
              953 pages
              ISBN:9781450336895
              DOI:10.1145/2814270

            Copyright © 2015 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 23 October 2015

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!