skip to main content
research-article
Open Access

Software multiplexing: share your libraries and statically link them too

Published:24 October 2018Publication History
Skip Abstract Section

Abstract

We describe a compiler strategy we call “Software Multiplexing” that achieves many benefits of both statically linked and dynamically linked libraries, and adds some additional advantages. Specifically, it achieves the code size benefits of dynamically linked libraries while eliminating the major disadvantages: unexpected failures due to missing dependences, slow startup times, reduced execution performance due to indirect references to globals, and the potential for security vulnerabilities. We design Software Multiplexing so that it works even in the common case where application build systems support only dynamic and not static linking; we have automatically built thousands of Linux software packages in this way. Software Multiplexing combines two ideas: Automatic Multicall, i.e., where multiple independent programs are automatically merged into a single executable, and Static Linking of Shared Libraries, which works by linking an IR-level version of application code and all its libraries, even if the libraries are normally compiled as shared, before native code generation. The benefits are achieved primarily through deduplication of libraries across the multiplexed programs, while using static linking, and secondly through more effective unused code elimination for statically linked shared libraries. Compared with equivalent dynamically linked programs, <span>allmux-optimized programs start more quickly and even have slightly lower memory usage and total disk size. Compared with equivalent statically linked programs, <span>allmux-optimized programs are much smaller in both aggregate size and memory usage, and have similar startup times and execution performance. We have implemented Software Multiplexing in a tool called <span>allmux, part of the open-source ALLVM project. Example results show that when the LLVM Compiler Infrastructure is optimized using allmux, the resulting binaries and libraries are 18.3% smaller and 30% faster than the default production version. For 74 other packages containing 2–166 programs each, multiplexing each package into one static binary reduces the aggregate package size by 39% (geometric mean) compared with dynamic linking.

Skip Supplemental Material Section

Supplemental Material

a154-dietz.webm

References

  1. Vikram Adve, Will Dietz, et al. 2016. ALLVM Project. http://allvm.orgGoogle ScholarGoogle Scholar
  2. Varun Agrawal, Abhiroop Dabral, Tapti Palit, Yongming Shen, and Michael Ferdman. 2015. Architectural Support for Dynamic Linking. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’15). ACM, New York, NY, USA, 691–702. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Gogul Balakrishnan and Thomas Reps. 2010. WYSINWYX: What You See is Not What You eXecute. ACM Trans. Program. Lang. Syst. 32, 6, Article 23 (Aug. 2010), 84 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Christian S Collberg, John H Hartman, Sridivya Babu, and Sharath K Udupa. 2004. Slinky - Static Linking Reloaded. Retrieved 2018 from http://slinky.cs.arizona.edu/Google ScholarGoogle Scholar
  5. Christian S Collberg, John H Hartman, Sridivya Babu, and Sharath K Udupa. 2005. SLINKY: Static Linking Reloaded.. In USENIX Annual Technical Conference, General Track. 309–322. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. CryoPID 2006. CryoPID. http://freecode.com/projects/cryopid/ . Original homepage is no longer available.Google ScholarGoogle Scholar
  7. Bruno De Bus, Daniel Kästner, Dominique Chanet, Ludo Van Put, and Bjorn De Sutter. 2003. Post-pass Compaction Techniques. Commun. ACM 46, 8 (Aug. 2003), 41–46. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Bjorn De Sutter, Bruno De Bus, and Koen De Bosschere. 2005. Link-time Binary Rewriting Techniques for Program Compaction. ACM Trans. Program. Lang. Syst. 27, 5 (Sept. 2005), 882–945. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Bjorn De Sutter, Hans Vandierendonck, Bruno De Bus, and Koen De Bosschere. 2003. On the Side-effects of Code Abstraction. In Proceedings of the 2003 ACM SIGPLAN Conference on Languages, Compilers and Tools for Embedded Systems (LCTES ’03). ACM, New York, NY, USA, 244–253. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Will Dietz, Joshua Cranmer, Nathan Dautenhahn, and Vikram Adve. 2015. Slipstream: Automatic Interprocess Communication Optimization. In 2015 USENIX Annual Technical Conference (USENIX ATC 15). USENIX Association, Santa Clara, CA, 431–443. https://www.usenix.org/conference/atc15/technical-session/presentation/dietz Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Ulrich Drepper. 2011. How To Write Shared Libraries. Retrieved July 2017 from http://people.redhat.com/drepper/dsohowto. pdfGoogle ScholarGoogle Scholar
  12. Tobias J.K. Edler von Koch, Björn Franke, Pranav Bhandarkar, and Anshuman Dasgupta. 2014. Exploiting Function Similarity for Code Size Reduction. In Proceedings of the 2014 SIGPLAN/SIGBED Conference on Languages, Compilers and Tools for Embedded Systems (LCTES ’14). ACM, New York, NY, USA, 85–94. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Cristiano Giuffrida, Clin Iorgulescu, Giordano Tamburrelli, and Andrew S. Tanenbaum. 2017. Automating Live Update for Generic Server Programs. IEEE Trans. Softw. Eng. 43, 3 (March 2017), 207–225. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. W. Wilson Ho, Wei-Chau Chang, and Lilian H. Leung. 1995. Optimizing the Performance of Dynamically-linked Programs. In Proceedings of the USENIX 1995 Technical Conference Proceedings (TCON’95). USENIX Association, Berkeley, CA, USA, 19–19. http://dl.acm.org/citation.cfm?id=1267411.1267430 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Blanchette Jasmin. 2008. C++ GUI Programming with Qt4, 2/e. Pearson Education India. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Jakub Jelinek. 2003. Prelink. Technical Report. Technical report, Red Hat, Inc., 2004. available at http://people. redhat. com/jakub/prelink. pdf.Google ScholarGoogle Scholar
  17. Neil E Johnson. 2004. Code size optimization for embedded processors. Technical Report. University of Cambridge, Computer Laboratory.Google ScholarGoogle Scholar
  18. Changhee Jung, Duk-Kyun Woo, Kanghee Kim, and Sung-Soo Lim. 2007. Performance Characterization of Prelinking and Preloading for Embedded Systems. In Proceedings of the 7th ACM &amp;Amp; IEEE International Conference on Embedded Software (EMSOFT ’07). ACM, New York, NY, USA, 213–220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. KSM 2009. Increasing memory density by using KSM. In Proceedings of the Linux Symposium. 19–28.Google ScholarGoogle Scholar
  20. Chris Lattner and Vikram Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis and Transformation. In Proc. Conf. on Code Generation and Optimization. San Jose, CA, USA, 75–88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. John R. Levine. 1999. Linkers and Loaders (1st ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Xiaozhu Meng and Barton P. Miller. 2016. Binary Code is Not Easy. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016). ACM, New York, NY, USA, 24–35. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. MITRE Corporation. 2017. CVE-2017-7494. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7494Google ScholarGoogle Scholar
  24. Matthew Moore. 2017. Distroless Docker: Containerizing Apps, Not VMs. (2017). https://swampup2017.sched.com/event/ A6CW/distroless-docker-containerizing-apps-not-vms SwampUP.Google ScholarGoogle Scholar
  25. Michael N. Nelson and Graham Hamilton. 1993. High Performance Dynamic Linking Through Caching. In Proceedings of the USENIX Summer 1993 Technical Conference - Volume 1 (Usenix-stc’93). USENIX Association, Berkeley, CA, USA, Article 17, 14 pages. http://dl.acm.org/citation.cfm?id=1361453.1361470 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Douglas B. Orr, Jay Lepreau, J. Bonn, and R. Mecklenburg. 1993. Fast and Flexible Shared Libraries. In Proceedings of the Summer 1993 USENIX Conference, Cincinnati, OH, USA, June 21-25, 1993. 237–252.Google ScholarGoogle Scholar
  27. PSS 2016. Proportional set size. Retrieved 2018 from https://en.wikipedia.org/wiki/Proportional_set_sizeGoogle ScholarGoogle Scholar
  28. Valery Reznic. 2016. ELF STATIFIER MAIN PAGE. Retrieved 2018 from http://statifier.sourceforge.net/Google ScholarGoogle Scholar
  29. Valery Reznic. 2018. Ermine: Linux Portable Application Creator. http://www.magicermine.com/ .Google ScholarGoogle Scholar
  30. Andreas Schwab. 2005. Re: Statically linking against a shared library. https://sourceware.org/ml/binutils/2005-03/msg00350. html Binutils mailing list.Google ScholarGoogle Scholar
  31. Carl A. Waldspurger. 2002. Memory Resource Management in VMware ESX Server. SIGOPS Oper. Syst. Rev. 36, SI (Dec. 2002), 181–194. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Hyungjo Yoon, Changwoo Min, and Young Ik Eom. 2014. Dynamic-prelink: An Enhanced Prelinking Mechanism without Modifying Shared Libraries. In Proceedings of the International Conference on Embedded Systems and Applications (ESA). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 1.Google ScholarGoogle Scholar

Index Terms

  1. Software multiplexing: share your libraries and statically link them too

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image Proceedings of the ACM on Programming Languages
        Proceedings of the ACM on Programming Languages  Volume 2, Issue OOPSLA
        November 2018
        1656 pages
        EISSN:2475-1421
        DOI:10.1145/3288538
        Issue’s Table of Contents

        Copyright © 2018 Owner/Author

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 24 October 2018
        Published in pacmpl Volume 2, Issue OOPSLA

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!