Abstract
We describe a compiler strategy we call “Software Multiplexing” that achieves many benefits of both statically linked and dynamically linked libraries, and adds some additional advantages. Specifically, it achieves the code size benefits of dynamically linked libraries while eliminating the major disadvantages: unexpected failures due to missing dependences, slow startup times, reduced execution performance due to indirect references to globals, and the potential for security vulnerabilities. We design Software Multiplexing so that it works even in the common case where application build systems support only dynamic and not static linking; we have automatically built thousands of Linux software packages in this way. Software Multiplexing combines two ideas: Automatic Multicall, i.e., where multiple independent programs are automatically merged into a single executable, and Static Linking of Shared Libraries, which works by linking an IR-level version of application code and all its libraries, even if the libraries are normally compiled as shared, before native code generation. The benefits are achieved primarily through deduplication of libraries across the multiplexed programs, while using static linking, and secondly through more effective unused code elimination for statically linked shared libraries. Compared with equivalent dynamically linked programs, <span>allmux-optimized programs start more quickly and even have slightly lower memory usage and total disk size. Compared with equivalent statically linked programs, <span>allmux-optimized programs are much smaller in both aggregate size and memory usage, and have similar startup times and execution performance. We have implemented Software Multiplexing in a tool called <span>allmux, part of the open-source ALLVM project. Example results show that when the LLVM Compiler Infrastructure is optimized using allmux, the resulting binaries and libraries are 18.3% smaller and 30% faster than the default production version. For 74 other packages containing 2–166 programs each, multiplexing each package into one static binary reduces the aggregate package size by 39% (geometric mean) compared with dynamic linking.
Supplemental Material
- Vikram Adve, Will Dietz, et al. 2016. ALLVM Project. http://allvm.orgGoogle Scholar
- Varun Agrawal, Abhiroop Dabral, Tapti Palit, Yongming Shen, and Michael Ferdman. 2015. Architectural Support for Dynamic Linking. In Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’15). ACM, New York, NY, USA, 691–702. Google Scholar
Digital Library
- Gogul Balakrishnan and Thomas Reps. 2010. WYSINWYX: What You See is Not What You eXecute. ACM Trans. Program. Lang. Syst. 32, 6, Article 23 (Aug. 2010), 84 pages. Google Scholar
Digital Library
- Christian S Collberg, John H Hartman, Sridivya Babu, and Sharath K Udupa. 2004. Slinky - Static Linking Reloaded. Retrieved 2018 from http://slinky.cs.arizona.edu/Google Scholar
- Christian S Collberg, John H Hartman, Sridivya Babu, and Sharath K Udupa. 2005. SLINKY: Static Linking Reloaded.. In USENIX Annual Technical Conference, General Track. 309–322. Google Scholar
Digital Library
- CryoPID 2006. CryoPID. http://freecode.com/projects/cryopid/ . Original homepage is no longer available.Google Scholar
- Bruno De Bus, Daniel Kästner, Dominique Chanet, Ludo Van Put, and Bjorn De Sutter. 2003. Post-pass Compaction Techniques. Commun. ACM 46, 8 (Aug. 2003), 41–46. Google Scholar
Digital Library
- Bjorn De Sutter, Bruno De Bus, and Koen De Bosschere. 2005. Link-time Binary Rewriting Techniques for Program Compaction. ACM Trans. Program. Lang. Syst. 27, 5 (Sept. 2005), 882–945. Google Scholar
Digital Library
- Bjorn De Sutter, Hans Vandierendonck, Bruno De Bus, and Koen De Bosschere. 2003. On the Side-effects of Code Abstraction. In Proceedings of the 2003 ACM SIGPLAN Conference on Languages, Compilers and Tools for Embedded Systems (LCTES ’03). ACM, New York, NY, USA, 244–253. Google Scholar
Digital Library
- Will Dietz, Joshua Cranmer, Nathan Dautenhahn, and Vikram Adve. 2015. Slipstream: Automatic Interprocess Communication Optimization. In 2015 USENIX Annual Technical Conference (USENIX ATC 15). USENIX Association, Santa Clara, CA, 431–443. https://www.usenix.org/conference/atc15/technical-session/presentation/dietz Google Scholar
Digital Library
- Ulrich Drepper. 2011. How To Write Shared Libraries. Retrieved July 2017 from http://people.redhat.com/drepper/dsohowto. pdfGoogle Scholar
- Tobias J.K. Edler von Koch, Björn Franke, Pranav Bhandarkar, and Anshuman Dasgupta. 2014. Exploiting Function Similarity for Code Size Reduction. In Proceedings of the 2014 SIGPLAN/SIGBED Conference on Languages, Compilers and Tools for Embedded Systems (LCTES ’14). ACM, New York, NY, USA, 85–94. Google Scholar
Digital Library
- Cristiano Giuffrida, Clin Iorgulescu, Giordano Tamburrelli, and Andrew S. Tanenbaum. 2017. Automating Live Update for Generic Server Programs. IEEE Trans. Softw. Eng. 43, 3 (March 2017), 207–225. Google Scholar
Digital Library
- W. Wilson Ho, Wei-Chau Chang, and Lilian H. Leung. 1995. Optimizing the Performance of Dynamically-linked Programs. In Proceedings of the USENIX 1995 Technical Conference Proceedings (TCON’95). USENIX Association, Berkeley, CA, USA, 19–19. http://dl.acm.org/citation.cfm?id=1267411.1267430 Google Scholar
Digital Library
- Blanchette Jasmin. 2008. C++ GUI Programming with Qt4, 2/e. Pearson Education India. Google Scholar
Digital Library
- Jakub Jelinek. 2003. Prelink. Technical Report. Technical report, Red Hat, Inc., 2004. available at http://people. redhat. com/jakub/prelink. pdf.Google Scholar
- Neil E Johnson. 2004. Code size optimization for embedded processors. Technical Report. University of Cambridge, Computer Laboratory.Google Scholar
- Changhee Jung, Duk-Kyun Woo, Kanghee Kim, and Sung-Soo Lim. 2007. Performance Characterization of Prelinking and Preloading for Embedded Systems. In Proceedings of the 7th ACM &Amp; IEEE International Conference on Embedded Software (EMSOFT ’07). ACM, New York, NY, USA, 213–220. Google Scholar
Digital Library
- KSM 2009. Increasing memory density by using KSM. In Proceedings of the Linux Symposium. 19–28.Google Scholar
- Chris Lattner and Vikram Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis and Transformation. In Proc. Conf. on Code Generation and Optimization. San Jose, CA, USA, 75–88. Google Scholar
Digital Library
- John R. Levine. 1999. Linkers and Loaders (1st ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. Google Scholar
Digital Library
- Xiaozhu Meng and Barton P. Miller. 2016. Binary Code is Not Easy. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA 2016). ACM, New York, NY, USA, 24–35. Google Scholar
Digital Library
- MITRE Corporation. 2017. CVE-2017-7494. https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-7494Google Scholar
- Matthew Moore. 2017. Distroless Docker: Containerizing Apps, Not VMs. (2017). https://swampup2017.sched.com/event/ A6CW/distroless-docker-containerizing-apps-not-vms SwampUP.Google Scholar
- Michael N. Nelson and Graham Hamilton. 1993. High Performance Dynamic Linking Through Caching. In Proceedings of the USENIX Summer 1993 Technical Conference - Volume 1 (Usenix-stc’93). USENIX Association, Berkeley, CA, USA, Article 17, 14 pages. http://dl.acm.org/citation.cfm?id=1361453.1361470 Google Scholar
Digital Library
- Douglas B. Orr, Jay Lepreau, J. Bonn, and R. Mecklenburg. 1993. Fast and Flexible Shared Libraries. In Proceedings of the Summer 1993 USENIX Conference, Cincinnati, OH, USA, June 21-25, 1993. 237–252.Google Scholar
- PSS 2016. Proportional set size. Retrieved 2018 from https://en.wikipedia.org/wiki/Proportional_set_sizeGoogle Scholar
- Valery Reznic. 2016. ELF STATIFIER MAIN PAGE. Retrieved 2018 from http://statifier.sourceforge.net/Google Scholar
- Valery Reznic. 2018. Ermine: Linux Portable Application Creator. http://www.magicermine.com/ .Google Scholar
- Andreas Schwab. 2005. Re: Statically linking against a shared library. https://sourceware.org/ml/binutils/2005-03/msg00350. html Binutils mailing list.Google Scholar
- Carl A. Waldspurger. 2002. Memory Resource Management in VMware ESX Server. SIGOPS Oper. Syst. Rev. 36, SI (Dec. 2002), 181–194. Google Scholar
Digital Library
- Hyungjo Yoon, Changwoo Min, and Young Ik Eom. 2014. Dynamic-prelink: An Enhanced Prelinking Mechanism without Modifying Shared Libraries. In Proceedings of the International Conference on Embedded Systems and Applications (ESA). The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp), 1.Google Scholar
Index Terms
Software multiplexing: share your libraries and statically link them too
Recommendations
Guided linking: dynamic linking without the costs
Dynamic linking is extremely common in modern software systems, thanks to the flexibility and space savings it offers. However, this flexibility comes at a cost: it’s impossible to perform interprocedural optimizations that involve calls to a dynamic ...
Implementation of a Wavelength Division Multiplexing and Bidirectional Optic Fiber Link iLab
ITNG '12: Proceedings of the 2012 Ninth International Conference on Information Technology - New GenerationsAn online laboratory that handles two critical concepts in Fiber Optic Communications: Bi-Directional Optic Fiber Link and Wavelength Division Multiplexing phenomena was developed at Makerere University. Designed using the EMONA Fiber Optics ...
Debugging native extensions of dynamic languages
ManLang '18: Proceedings of the 15th International Conference on Managed Languages & RuntimesMany dynamic programming languages such as Ruby and Python enable developers to use so called native extensions, code implemented in typically statically compiled languages like C and C++. However, debuggers for these dynamic languages usually lack ...






Comments