skip to main content
research-article

Making the common case the only case with anticipatory memory allocation

Published:02 February 2012Publication History
Skip Abstract Section

Abstract

We present anticipatory memory allocation (AMA), a new method to build kernel code that is robust to memory-allocation failures. AMA avoids the usual difficulties in handling allocation failures through a novel combination of static and dynamic techniques. Specifically, a developer, with assistance from AMA static analysis tools, determines how much memory a particular call into a kernel subsystem will need, and then preallocates said amount immediately upon entry to the kernel; subsequent allocation requests are serviced from the preallocated pool and thus guaranteed never to fail. We describe the static and runtime components of AMA, and then present a thorough evaluation of Linux ext2-mfr, a case study in which we transform the Linux ext2 file system into a memory-failure robust version of itself. Experiments reveal that ext2-mfr avoids memory-allocation failures successfully while incurring little space or time overhead.

References

  1. Albert, E., Genaim, S., and Gomez-Zamalloa, M. 2009. Live heap space analysis for languages for garbage collection. In Proceedings of the International Symposium on Memory Management (ISMM'09). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Arpaci-Dusseau, A. C., Arpaci-Dusseau, R. H., Burnett, N. C., Denehy, T. E., Engle, T. J., Gunawi, H. S., Nugent, J., and Popovici, F. I. 2003. Transforming policies into mechanisms with Infokernel. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP'03). ACM, New York, 90--105. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Austin, T. M., Breach, S. E., and Sohi, G. S. 2004. Efficient detection of all pointer and array access errors. In Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation (PLDI'04). ACM, New York, 290--301. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Bairavasundaram, L. N., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2006. Dependability analysis of virtual memory systems. In Proceedings of the International Conference on Dependable Systems and Networks (DSN'06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Bairavasundaram, L. N., Rungta, M., Agrawal, N., Arpaci-Dusseau, A. C., Arpaci-Dusseau, R. H., and Swift, M. M. 2008. Systematically benchmarking the effects of disk pointer corruption. In Proceedings of the International Conference on Dependable Systems and Networks (DSN'08).Google ScholarGoogle Scholar
  6. Berger, E. D. and Zorn, B. G. 2006. DieHard: Probabilistic memory safety for unsafe languages. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'06). ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Bernstein, P. A. and Vassos Hadzilacos, N. G. 1987. Concurrency Control and Recovery in Database Systems. Addison Wesley. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Bonwick, J. 1994. The slab allocator: An object-caching kernel memory allocator. In Proceedings of the USENIX Summer Technical Conference (USENIX'94). Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Bovet, D. P. and Cesati, M. 2006. Understanding the Linux Kernel. O'Reilly. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Braberman, V., Fernandez, F., Garbervetsky, D., and Yovine, S. 2008. Parametric prediction of heap memory requirements. In Proceedings of the International Symposium on Memory Management (ISMM'08). Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Brown, A. B. and Patterson, D. A. 2001. To err is human. In Proceedings of the 1ST Workshop on Evaluating and Architecting System Depandibility (EASY'01).Google ScholarGoogle Scholar
  12. Cadar, C., Ganesh, V., Pawlowski, P. M., Dill, D. L., and Engler, D. R. 2006. EXE: Automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and Communications Security (CCS'06). ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Chin,W.-N., Nguyen, H. H., Popeea, C., and Qin, S. 2008. Analysing memory resource bounds for low-level programs. In Proceedings of the International Symposium on Memory Management (ISMM'08). Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Chin, W.-N., Nguyen, H. H., Qin, S., and Rinard, M. 2005. Memory usage verification for OO programs. In Proceedings of the Static Analysis Symposium (SAS'05). Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Chou, A., Yang, J., Chelf, B., Hallem, S., and Engler, D. 2001. An empirical study of operating system errors. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP'01). ACM, New York, 73--88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Dhurjati, D., Kowshik, S., Adve, V., and Lattner, C. 2003. Memory safety without runtime checks or garbage collection. In Proceedings of the ACM SIGPLAN Conference on Language, Compiler, and Tool for Embedded Systems (LCTES'03). ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Dijkstra, E. W. 1977. EWD623: The Mathematics Behind The Bankers Algorithm. Selected Writings on Computing: A Personal Perspective, Springer, Berlin.Google ScholarGoogle Scholar
  18. Engler, D., Chen, D. Y., Hallem, S., Chou, A., and Chelf, B. 2001. Bugs as deviant behavior: A general approach to inferring errors in systems code. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP'01). ACM, New York, 57--72. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Engler, D. and Musuvathi, M. 2004. Static analysis versus software model checking for bug finding. In Proceedings of the 5th International Conference Verification, Model Checking and Abstract Interpretation (VMCAI'04).Google ScholarGoogle Scholar
  20. Engler, D. R., Kaashoek, M. F., and O'Toole, J. W. 1995. Exokernel: An operating system architecture for application-level resource management. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP'95). ACM, New York, 251--266. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Garbervetsky, D., Yovine, S., Braberman, V., Rouaux, M., and Taboada, A. 2009. On transforming Java-like programs into memory-predictable code. In Proceedings of the 7th International Workshop on Java Technologies for Real-Time and Embedded Systems (JTRES'09). ACM, New York, 140--149. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Gunawi, H. S., Prabhakaran, V., Krishnan, S., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2007. Improving file system reliability with I/O shepherding. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP'07). ACM, New York, 283--296. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Gunawi, H. S., Rubio-Gonzalez, C., Arpaci-Dusseau, A. C., Arpaci-Dusseau, R. H., and Liblit, B. 2008. EIO: Error handling is occasionally correct. In Proceedings of the 6th USENIX Symposium on File and Storage Technologies (FAST'08). 207--222. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Hallem, S., Chelf, B., Xie, Y., and Engler, D. R. 2002. A system and language for building system-specific, static analyses. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'02). ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Hofmann, M. and Jost, S. 2003. Static prediction of heap space usage for first order functional languages. In Proceedings of The 30th SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL'03). ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Hofmann, M. and Jost, S. 2006. Type-based amortised heap-space analysis. In Proceedings of the European Symposium on Programming (ESOP). Lecture Notes in Computer Science, vol. 3924, Springer, Berlin, 22--37. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Kleiman, S. R. 1986. Vnodes: An architecture for multiple file system types in Sun UNIX. In Proceedings of the USENIX Summer Technical Conference (USENIX'86). 238--247.Google ScholarGoogle Scholar
  28. Li, Z., Lu, S., Myagmar, S., and Zhou, Y. 2004. CP-Miner: A tool for finding copy-paste and related bugs in operating system code. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI'04). Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Lie, D., Chou, A., Engler, D., and Dill, D. L. 2001. A simple method for extracting models from protocol code. In Proceedings of the 28th Annual International Symposium on Computer Architecture (ISCA'01). Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Lu, S., Park, S., Seo, E., and Zhou, Y. 2008. Learning from mistakes---A comprehensive study on real world concurrency bug characteristics. In Proceedings of the 13th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XIII). Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Morton, A. 2007. Re: {patch} jbd slab cleanups. kerneltrap.org/mailarchive/linux-fsdevel/2007/9/19/322280/thread#mid-322280.Google ScholarGoogle Scholar
  32. Musuvathi, M., Park, D. Y., Chou, A., Engler, D. R., and Dill, D. L. 2002. CMC: A pragmatic approach to model checking real code. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI'02). Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Necula, G. C., McPeak, S., Rahul, S. P., and Weimer, W. 2002. Cil: An infrastructure for C program analysis and transformation. In Proceedings of the International Conference on Compiler Construction (CC'02). 213--228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Prabhakaran, V., Bairavasundaram, L. N., Agrawal, N., Gunawi, H. S., Arpaci-Dusseau, A. C., and Arpaci-Dusseau, R. H. 2005. IRON file systems. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP'05). ACM, New York, 206--220. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Qin, F., Tucek, J., Sundaresan, J., and Zhou, Y. 2005. Rx: Treating bugs as allergies. In Proceedings of the 20th ACM Symposium on Operating Systems Principles (SOSP'05). ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Rinard, M., Cadar, C., Dumitran, D., Roy, D. M., Leu, T., and Beebe, W. S. J. 2004. Enhancing server availability and security through failure-oblivious computing. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI'04). Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Rubio-Gonzalez, C., Gunawi, H. S., Liblit, B., Arpaci-Dusseau, R. H., and Arpaci-Dusseau, A. C. 2009. Error propagation analysis for file systems. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI'09). ACM, New York. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Sundararaman, S., Subramanian, S., Rajimwale, A., Arpaci-Dusseau, A. C., Arpaci-Dusseau, R. H., and Swift, M. M. 2010. Membrane: Operating system support for restartable file systems. In Proceedings of the 8th USENIX Symposium on File and Storage Technologies (FAST'10). Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. TUGS. 2010. StackAnalyzer stack usage analysis. http://www.absint.com/stackanalyzer/.Google ScholarGoogle Scholar
  40. Unnikrishnan, L. and Stoller, S. D. 2009. Parametric heap usage analysis for functional programs. In Proceedings of the International Symposium on Memory Management (ISMM'09). Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Weiser, M. 1981. Program slicing. In Proceedings of the International Conference on Software Engineering (ICSE'81). 439--449. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Yang, J., Sar, C., and Engler, D. 2006. EXPLODE: A lightweight, general system for finding serious storage system errors. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI'06). Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Yang, J., Sar, C., Twohey, P., Cadar, C., and Engler, D. 2006. Automatically generating malicious disks using symbolic execution. In Proceedings of the IEEE Conference on Security and Privacy (SP'06). IEEE, Los Alamitos, CA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Yang, J., Twohey, P., Engler, D., and Musuvathi, M. 2004. Using model checking to find serious file system errors. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI'04). Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Making the common case the only case with anticipatory memory allocation

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Storage
      ACM Transactions on Storage  Volume 7, Issue 4
      January 2012
      65 pages
      ISSN:1553-3077
      EISSN:1553-3093
      DOI:10.1145/2078861
      Issue’s Table of Contents

      Copyright © 2012 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 2 February 2012
      • Accepted: 1 August 2011
      • Received: 1 July 2011
      Published in tos Volume 7, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!