skip to main content
10.1145/1168857.1168861acmconferencesArticle/Chapter ViewAbstractPublication PagesasplosConference Proceedingsconference-collections
Article

Geiger: monitoring the buffer cache in a virtual machine environment

Published:20 October 2006Publication History

ABSTRACT

Virtualization is increasingly being used to address server management and administration issues like flexible resource allocation, service isolation and workload migration. In a virtualized environment, the virtual machine monitor (VMM) is the primary resource manager and is an attractive target for implementing system features like scheduling, caching, and monitoring. However, the lackof runtime information within the VMM about guest operating systems, sometimes called the semantic gap, is a significant obstacle to efficiently implementing some kinds of services.In this paper we explore techniques that can be used by a VMM to passively infer useful information about a guest operating system's unified buffer cache and virtual memory system. We have created a prototype implementation of these techniques inside the Xen VMM called Geiger and show that it can accurately infer when pages are inserted into and evicted from a system's buffer cache. We explore several nuances involved in passively implementing eviction detection that have not previously been addressed, such as the importance of tracking disk block liveness, the effect of file system journaling, and the importance of accounting for the unified caches found in modern operating systems.Using case studies we show that the information provided by Geiger enables a VMM to implement useful VMM-level services. We implement a novel working set size estimator which allows the VMM to make more informed memory allocation decisions. We also show that a VMM can be used to drastically improve the hit rate in remote storage caches by using eviction-based cache placement without modifying the application or operating system storage interface. Both case studies hint at a future where inference techniques enable a broad new class of VMM-level functionality.

References

  1. A.C. Arpaci-Dusseau and R.H. Arpaci-Dusseau. Information and Control in Gray-Box Systems. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), pages 43--56, Banff, Canada, October 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. L.N. Bairavasundaram, M. Sivathanu, A.C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. X-RAY: A Non-Invasive Exclusive Caching Mechanism for RAIDs. In Proceedings of the 31st Annual International Symposium on Computer Architecture (ISCA '04), Munich, Germany, June 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. S. Ballmer. Keynote address. Microsoft Management Summit, April 2005.Google ScholarGoogle Scholar
  4. S. Best. JFS Overview. www.ibm.com/developerworks/library/ljfs. html, 2000.Google ScholarGoogle Scholar
  5. T.C. Bressoud and F.B. Schneider. Hypervisor-based fault tolerance. In SOSP '95: Proceedings of the fifteenth ACM symposium on Operating systems principles, pages 1--11. ACM Press, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. E. Bugnion, S. Devine, and M. Rosenblum. Disco: Running commodity operating systems on scalable multiprocessors. In Proceedings of the 16th ACM Symposium on Operating Systems Principles (SOSP '97), pages 143--156, Saint-Malo, France, October 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. P.M. Chen and B.D. Noble. When Virtual Is Better Than Real. In HOTOS '01: Proceedings of the Eighth Workshop on Hot Topics in Operating Systems, page 133. IEEE Computer Society, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Z. Chen, Y. Ahang, Y. Zhou, H. Scott, and B. Schiefer. Empirical Evaluation of Multi-level Buffer Cache Collaboration for Storage Systems. In Proceedings of the 2005 ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '05), Banff, Canada, June 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Z. Chen, Y. Zhou, and K. Li. Eviction-based Placement for Storage Caches. In Proceedings of the USENIX Annual Technical Conference (USENIX '03), pages 269--282, San Antonio, Texas, June 2003.Google ScholarGoogle Scholar
  10. C. Clark, K. Fraser, S. Hand, J.G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live Migration of Virtual Machines. In Proceedings of the 2nd Symposium on Networked Systems Design and Implementation (NSDI '05), Boston, Massachusetts, May 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. P.J. Denning. The Working Set Model for Program Behavior. Communications of the ACM, 11(5): 323--333, May 1968. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. P.J. Denning. Working Sets: Past and Present. IEEE Transactions on Software Engineering, SE-6(1):64--84, January 1980.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, I. Pratt, A.Warfield, P. Barham, and R. Neugebauer. Xen and the Art of Virtualization. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP '03), Bolton Landing (Lake George), New York, October 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. R. Figueriredo, P. Dinda, and J. Fortes. A Case for Grid Computing on Virtual Machines. In Proceedings of the International Conference on Distributed Computing Systems (ICDCS), May 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. K. Fraser, S. Hand, R. Neugebauer, I. Pratt, A. Warfield, and M. Williamson. Safe Hardware Access with the Xen Virtual Machine Monitor. In OASIS ASPLOS 2004 workshop, 2004.Google ScholarGoogle Scholar
  16. T. Garfinkel, B. Pfaff, J. Chow, M. Rosenblum, and D. Boneh. Terra: A Virtual Machine-Based Platform for Trusted Computing. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP '03), Bolton Landing (Lake George), New York, October 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. R. Goldberg. Survey of Virtual Machine Research. IEEE Computer, 7(6):34--45, 1974.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. P. Gum. System/370 Extended Architecture: Facilities for Virtual Machines. IBM Journal of Research and Development, 27(6):530--544, November 1983.Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. ImageMagick Studio LLC. ImageMagick image processing software. http://www.imagemagick.org.Google ScholarGoogle Scholar
  20. Intel Corporation. Intel Virtualization Technology Specification. ftp://download.intel.com/technology/computing/vptech/C97063.pdf,2005.Google ScholarGoogle Scholar
  21. T. Johnson and D. Shasha. 2Q: A Low-Overhead High Performance Buffer Management Replacement Algorithm. In Proceedings of the 20th International Conference on Very Large Databases (VLDB 20), pages 439--450, Santiago, Chile, September 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. S.T. Jones, A.C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. Antfarm: Tracking Processes in a Virtual Machine Environment. In Proceedings of the USENIX Annual Technical Conference (USENIX '06), Boston, Massachusetts, June 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. N.P. Jouppi. Improving Direct-Mapped Cache Performance by the Addition of a Small Fully-Associative Cache and Prefetch Buffers. In Proceedings of the 17th Annual International Symposium on Computer Architecture (ISCA '90), pages 364--373, Seattle, Washington, May 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. S.T. King and P.M. Chen. Backtracking Intrusions. In Proceedings of the 18th ACM Symposium on Operating Systems Principles (SOSP '01), Banff, Canada, October 2001.Google ScholarGoogle Scholar
  25. D. Muntz and P. Honeyman. Multi-Level Caching in Distributed File Systems - or - Your Cache Ain't Nuthin' But Trash. Proceedings of the USENIX Winter Conference, pages 305--313, January 1992.Google ScholarGoogle Scholar
  26. Open Source Development Labs. OSDL Database Test Suite. http://www.osdl.org/lab activities/kernel testing/osdl database test suite.Google ScholarGoogle Scholar
  27. R. H. Patterson, G. A. Gibson, E. Ginting, D. Stodolsky, and J. Zelenka. Informed Prefetching and Caching. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP '95), pages 79--95, Copper Mountain Resort, Colorado, December 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. H. Reiser. ReiserFS. www.namesys.com, 2004.Google ScholarGoogle Scholar
  29. C.P. Sapuntzakis, R. Chandra, B. Pfaff, J. Chow, M.S. Lam, and M. Rosenblum. Optimizing the Migration of Virtual Computers. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI '02), pages 377--390, Boston, Massachusetts, December 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. Sivathanu, L.N. Bairavasundaram, A.C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. Life or Death at Block Level. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI '04), pages 379--394, San Francisco, California, December 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. M. Sivathanu, V. Prabhakaran, A C. Arpaci-Dusseau, and R.H. Arpaci-Dusseau. Improving Storage System Availability with DGRAID. In Proceedings of the Third USENIX Symposium on File and Storage Technologies (FAST '04), San Francisco, CA, March 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Storage Performance Council. SPC web search engine storage traces. http://traces.cs.umass.edu/storage.Google ScholarGoogle Scholar
  33. J. Sugerman, G. Venkitachalam, and B.-H. Lim. Virtualizing I/O Devices on VMware Workstation's Hosted Virtual Machine Monitor. In Proceedings of the USENIX Annual Technical Conference (USENIX '01), Boston, Massachusetts, June 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. A. Sweeney, D. Doucette, W. Hu, C. Anderson, M. Nishimoto, and G. Peck. Scalability in the XFS File System. In Proceedings of the USENIX Annual Technical Conference (USENIX '96), San Diego, California, January 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. A. Tridgell. Dbench filesystem benchmark. http://samba.org/ftp/tridge/dbench.Google ScholarGoogle Scholar
  36. T. Ts'o and S. Tweedie. Future Directions for the Ext2/3 Filesystem. In Proceedings of the USENIX Annual Technical Conference (FREENIX Track), Monterey, California, June 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. S.C. Tweedie. EXT3, Journaling File System. olstrans.sourceforge.net/release/OLS2000-ext3/OLS2000-ext3.html, July 2000.Google ScholarGoogle Scholar
  38. C.A. Waldspurger. Memory Resource Management in VMware ESX Server. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI '02), Boston, Massachusetts, December 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. A. Whitaker, M. Shaw, and S.D. Gribble. Scale and Performance in the Denali Isolation Kernel. In Proceedings of the 5th Symposium on Operating Systems Design and Implementation (OSDI '02), Boston, Massachusetts, December 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. T.M. Wong and J. Wilkes. My Cache or Yours? Making Storage More Exclusive. In Proceedings of the USENIX Annual Technical Conference (USENIX '02), Monterey, California, June 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. M. Zhao, J. Zhang, and R. Figueriredo. Distributed File System Support for Virtual Machines in Grid Computing. In Proceedings of High Performance Distributed Computing (HPDC), July 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Y. Zhou, J.F. Philbin, and K. Li. The Multi-Queue Replacement Algorithm for Second Level Buffer Caches. In Proceedings of the USENIX Annual Technical Conference (USENIX '01), pages 91--104, Boston, Massachusetts, June 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Geiger: monitoring the buffer cache in a virtual machine environment

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!