skip to main content
research-article

Automated migration of build scripts using dynamic analysis and search-based refactoring

Published:15 October 2014Publication History
Skip Abstract Section

Abstract

The efficiency of a build system is an important factor for developer productivity. As a result, developer teams have been increasingly adopting new build systems that allow higher build parallelization. However, migrating the existing legacy build scripts to new build systems is a tedious and error-prone process. Unfortunately, there is insufficient support for automated migration of build scripts, making the migration more problematic.

We propose the first dynamic approach for automated migration of build scripts to new build systems. Our approach works in two phases. First, from a set of execution traces, we synthesize build scripts that accurately capture the intent of the original build. The synthesized build scripts are typically long and hard to maintain. Second, we apply refactorings that raise the abstraction level of the synthesized scripts (e.g., introduce functions for similar fragments). As different refactoring sequences may lead to different build scripts, we use a search-based approach that explores various sequences to identify the best (e.g., shortest) build script. We optimize search-based refactoring with partial-order reduction to faster explore refactoring sequences. We implemented the proposed two phase migration approach in a tool called METAMORPHOSIS that has been recently used at Microsoft.

References

  1. Ant home page. http://ant.apache.org/.Google ScholarGoogle Scholar
  2. Buck home page. http://facebook.github.io/buck/.Google ScholarGoogle Scholar
  3. Fabricate home page. https://code.google.com/p/fabricate/.Google ScholarGoogle Scholar
  4. Build in the cloud. http://google-engtools.blogspot.com/2011/08/build-in-cloud-how-build-system-works.html.Google ScholarGoogle Scholar
  5. Gradle build init plugin. http://www.gradle.org/docs/current/userguide/build_init_plugin.html.Google ScholarGoogle Scholar
  6. MSBuild home page. http://msdn.microsoft.com/en-us/library/0k6kkbsd.aspx.Google ScholarGoogle Scholar
  7. Make home page. http://www.gnu.org/software/make/manual/html_node/index.html.Google ScholarGoogle Scholar
  8. Maven home page. http://maven.apache.org/.Google ScholarGoogle Scholar
  9. Memoize home page. https://github.com/kgaughan/memoize.py.Google ScholarGoogle Scholar
  10. NMake home page. http://msdn.microsoft.com/en-us/library/ms930369.aspx.Google ScholarGoogle Scholar
  11. Ninja home page. https://martine.github.io/ninja/.Google ScholarGoogle Scholar
  12. SCons home page. http://www.scons.org/.Google ScholarGoogle Scholar
  13. TPL home page. http://msdn.microsoft.com/en-us/library/dd537609.aspx.Google ScholarGoogle Scholar
  14. Vesta home page. http://www.vestasys.org/.Google ScholarGoogle Scholar
  15. H. Abdeen, S. Ducasse, H. Sahraoui, and I. Alloui. Automatic package coupling and cycle minimization. In WCRE, pages 103--112, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. B. Adams, H. Tromp, K. D. Schutter, and W. D. Meuter. Design recovery and maintenance of build systems. In ICSM, pages 114--123, 2007.Google ScholarGoogle ScholarCross RefCross Ref
  17. B. Adams, K. D. Schutter, H. Tromp, and W. D. Meuter. The evolution of the Linux build system. Electronic Communications of the ECEASST, 8:1--16, 2008.Google ScholarGoogle Scholar
  18. G. Antoniol, M. Di Penta, and M. Neteler. Moving to smaller libraries via clustering and genetic algorithms. In CSMR, page 307, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. M. Balazinska, E. Merlo, M. Dagenais, B. Lague, and K. Kontogiannis. Measuring clone based reengineering opportunities. In METRICS, pages 292--303, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. F. L. Bauer, R. Berghammer, M. Broy, W. Dosch, F. Geiselbrechtinger, R. Gnatz, E. Hangel, W. Hesse, B. Krieg-Brückner, A. Laut, T. Matzner, B. Möller, F. Nickl, H. Partsch, R. Pepper, K. Samelson, M. Wirsing, and H. Wössner, editors. The Munich Project CIP: The Wide Spectrum Language CIP-L, volume 183. 1985. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. T. Bodhuin, G. Canfora, and L. Troiano. SORMASA: A tool for suggesting model refactoring actions by metrics-led genetic algorithm. In WRT, pages 23--24, 2007.Google ScholarGoogle Scholar
  22. S. Bouktif, G. Antoniol, E. Merlo, and M. Neteler. A novel approach to optimize clone refactoring activity. In GECCO, pages 1885--1892, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. Bowman, L. Briand, and Y. Labiche. Solving the class responsibility assignment problem in object-oriented analysis with multi-objective genetic algorithms. Transactions on Software Engineering, 36(6): 817--837, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Christakis, K. R. M. Leino, and W. Schulte. Formalizing and verifying a modern build language. In FM, pages 643--657, 2014.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. D. Coetzee, A. Bhaskar, and G. Necula. A model and framework for reliable build systems. Technical Report UCB/EECS-2012-27, University of California at Berkeley, 2012.Google ScholarGoogle Scholar
  26. K. D. Cooper, D. Subramanian, and L. Torczon. Adaptive optimizing compilers for the 21st century. J. Supercomput., 23(1):7--22, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. B. Daniel, T. Gvero, and D. Marinov. On test repair using symbolic execution. In ISSTA, pages 207--218, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. D. Dig, J. Marrero, and M. D. Ernst. Refactoring sequential Java code for concurrency via concurrent libraries. In ICSE, pages 397--407, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. A. F. Donaldson and A. Miller. Extending symmetry reduction techniques to a realistic model of computation. Electronic Notes in Theoretical Computer Science, 185: 63--76, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. D. Doval, S. Mancoridis, and B. S. Mitchell. Automatic clustering of software systems using a genetic algorithm. In STEP, page 73, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. D. Fatiregun, M. Harman, and R. M. Hierons. Evolving transformation sequences using genetic algorithms. In SCAM, pages 66--75, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. D. Fatiregun, M. Harman, and R. M. Hierons. Search-based amorphous slicing. In WCRE, pages 3--12, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. A. Feldthaus and A. Mîller. Semi-automatic rename refactoring for JavaScript. In OOPSLA, pages 323--338, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. C. Flanagan and P. Godefroid. Dynamic partial-order reduction for model checking software. In POPL, pages 110--121, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. S. Forrest, T. Nguyen, W. Weimer, and C. Le Goues. A genetic programming approach to automated software repair. In GECCO, pages 947--954, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. W. Guttmann, H. Partsch, W. Schulte, and T. Vullinghs. Tool support for the interactive derivation of formally correct functional programs. Journal of Universal Computer Science, 9(2):173--188, 2003.Google ScholarGoogle Scholar
  37. R. Hardt and E. Munson. Ant build maintenance with Formiga. In RELENG, pages 13--16, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. Harman. Refactoring as testability transformation. In ICSTW, pages 414--421, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. M. Harman and L. Tratt. Pareto optimal search based refactoring at the design level. In GECCO, pages 1106--1113, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Y. Higo, S. Kusumoto, and K. Inoue. A metric-based approach to identifying refactoring opportunities for merging code clones in a Java software system. J. Softw. Maint. Evol., 20(6):435--461, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. G. Holzmann. The SPIN Model Checker: Primer and Reference Manual. Addison-Wesley Professional, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. G. Hunt and D. Brubacher. Detours: Binary interception ofWin32 functions. In USENIX Windows NT Symposium, pages 135--143, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. R. Iosif. Exploiting heap symmetries in explicit-state model checking of software. In ASE, pages 254--261, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. M. Kim, T. Zimmermann, and N. Nagappan. A field study of refactoring challenges and benefits. In FSE, pages 1--11, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. C. Kirsopp, M. J. Shepperd, and J. Hart. Search heuristics, case-based reasoning and software project effort prediction. In GECCO, pages 1367--1374, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. C. Le Goues, M. Dewey-Vogt, S. Forrest, and W. Weimer. A systematic study of automated program repair: fixing 55 out of 105 bugs for $8 each. In ICSE, pages 3--13, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. S. McIntosh, B. Adams, and A. E. Hassan. The evolution of ANT build systems. In MSR, pages 42--51, 2010.Google ScholarGoogle ScholarCross RefCross Ref
  48. S. McIntosh, B. Adams, T. H. Nguyen, Y. Kamei, and A. E. Hassan. An empirical study of build maintenance effort. In ICSE, pages 141--150, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. B. S. Mitchell and S. Mancoridis. Using heuristic search techniques to extract design abstractions from source code. In GECCO, pages 1375--1382, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. E. Murphy-Hill, C. Parnin, and A. P. Black. How we refactor, and how we know it. In ICSE, pages 287--297, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. A. Neagu. What is wrong with Make? Jul 2005. http://freecode.com/articles/what-is-wrong-with-make.Google ScholarGoogle Scholar
  52. A. Neundorf. Why the KDE project switched to CMake - and how. Apr 2012. http://lwn.net/Articles/188693/.Google ScholarGoogle Scholar
  53. A. Nisbet. GAPS: A compiler framework for genetic algorithm (GA) optimised parallelisation. In HPCN, pages 987--989, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. M. O'Keeffe and M. O'Cinneide. Search-based software maintenance. In CSMR, pages 249--260, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. M. O'Keeffe and M. O'Cinneide. Search-based refactoring: an empirical study. J. Softw. Maint. Evol., 20(5):345--364, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. W. F. Opdyke. Refactoring Object-Oriented Frame-works. PhD thesis, University of Illinois at Urbana-Champaign, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. O. Räihä. A survey on search-based software design. Computer Science Review, 4(4):203--249, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. V. Raychev, M. Schäfer, M. Sridharan, and M. Vechev. Refactoring with synthesis. In OOPSLA, pages 339--354, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. D. Roberts, J. Brant, and R. Johnson. A refactoring tool for Smalltalk. Theor. Pract. Object Syst., 3(4):253--263, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. M. Schäfer, M. Sridharan, J. Dolby, and F. Tip. Refactoring Java programs for exible locking. In ICSE, pages 71--80, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  61. O. Seng, J. Stammel, and D. Burkhart. Search-based determination of refactorings for improving the class structure of object-oriented systems. In GECCO, pages 1909--1916, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. R. Suvorov, M. Nagappan, A. Hassan, Y. Zou, and B. Adams. An empirical study of build system migrations in practice: Case studies on KDE and the Linux kernel. In ICSM, pages 160--169, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. R. Tairas and J. Gray. Increasing clone maintenance support by unifying clone detection and refactoring activities. Inf. Softw. Technol., 54(12):1297--1307, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. A. Tamrawi, H. A. Nguyen, H. V. Nguyen, and T. Nguyen. SYMake: a build code analysis and refactoring tool for makefiles. In ASE, pages 366--369, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  65. M. Vakilian, N. Chen, S. Negara, B. A. Rajkumar, B. P. Bailey, and R. E. Johnson. Use, disuse, and misuse of automated refactorings. In ICSE, pages 233--243, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. M. Vakilian, R. Sauciuc, Morgenthaler, J. D., and V. Mirrokni. Automated decomposition of build targets. Technical report, University of Illinois at Urbana-Champaign, 2014.Google ScholarGoogle Scholar
  67. D. R. White, A. Arcuri, and J. A. Clark. Evolutionary improvement of programs. Trans. Evol. Comp, 15(4): 515--538, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  68. D. Whitfield and M. L. Soffa. An approach for exploring code-improving transformations. Trans. Program. Lang. Syst., 19(6):1053--1084, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Automated migration of build scripts using dynamic analysis and search-based refactoring

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 49, Issue 10
      OOPSLA '14
      October 2014
      907 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/2714064
      • Editor:
      • Andy Gill
      Issue’s Table of Contents
      • cover image ACM Conferences
        OOPSLA '14: Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages & Applications
        October 2014
        946 pages
        ISBN:9781450325851
        DOI:10.1145/2660193

      Copyright © 2014 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 15 October 2014

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!