skip to main content
research-article

A lightweight VMM on many core for high performance computing

Published:16 March 2013Publication History
Skip Abstract Section

Abstract

Traditional Virtual Machine Monitor (VMM) virtualizes some devices and instructions, which induces performance overhead to guest operating systems. Furthermore, the virtualization contributes a large amount of codes to VMM, which makes a VMM prone to bugs and vulnerabilities.

On the other hand, in cloud computing, cloud service provider configures virtual machines based on requirements which are specified by customers in advance. As resources in a multi-core server increase to more than adequate in the future, virtualization is not necessary although it provides convenience for cloud computing. Based on the above observations, this paper presents an alternative way for constructing a VMM: configuring a booting interface instead of virtualization technology. A lightweight virtual machine monitor - OSV is proposed based on this idea. OSV can host multiple full functional Linux kernels with little performance overhead. There are only 6 hyper-calls in OSV. The Linux running on top of OSV is intercepted only for the inter-processor interrupts. The resource isolation is implemented with hardware-assist virtualization. The resource sharing is controlled by distributed protocols embedded in current operating systems.

We implement a prototype of OSV on AMD Opteron processor based 32-core servers with SVM and cache-coherent NUMA architectures. OSV can host up to 8 Linux kernels on the server with less than 10 lines of code modifications to Linux kernel. OSV has about 8000 lines of code which can be easily tuned and debugged. The experiment results show that OSV VMM has 23.7% performance improvement compared with Xen VMM.

References

  1. CVE-2010-0633. http://cve.mitre.org/cgibin/cvename.cgi?name=CVE-2010-0633, 2010.Google ScholarGoogle Scholar
  2. CVE-2010-4255. http://cve.mitre.org/cgibin/cvename.cgi?name=CVE-2010-4255, 2010.Google ScholarGoogle Scholar
  3. CVE-2010-4247. http://cve.mitre.org/cgibin/cvename.cgi?name=CVE-2010-4247, 2010.Google ScholarGoogle Scholar
  4. CVE-2012-0217: SYSRET 64-bit operating system privilege escalation vulnerability on Intel CPU hardware. http://cve.mitre.org/cgibin/cvename.cgi?name=CVE-2012-0217, 2012.Google ScholarGoogle Scholar
  5. HP ProLiant DL980 G7 Server Data Sheet. http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA1-5671ENW.pdf, 2012.Google ScholarGoogle Scholar
  6. AMD. Amd64 architecture programmer's manual volume 2: System programming. Publication No. 24593, September 2007.Google ScholarGoogle Scholar
  7. J. Appavoo, D. Silva, O. Krieger, M. Auslander, M. Ostrowski, B. Rosenburg, A. Waterland, R. Wisniewski, J. Xenidis, M. Stumm, et al. Experience distributing objects in an SMMP OS. ACM Transactions on Computer Systems (TOCS), 25(3):6--es, 2007. ISSN 0734-2071. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the Art of Virtualization. Proceedings of the 19th ACM symposium on Operating systems principles, pages 164--177, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. A. Baumann, P. Barham, P. Dagand, T. Harris, R. Isaacs, S. Peter, T. Roscoe, A. Schüpbach, and A. Singhania. The multikernel: a new OS architecture for scalable multicore systems. In Proceedings of the ACM SIGOPS 22nd symposium on Operating Systems Principles, pages 29--44. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Boyd-Wickizer, H. Chen, R. Chen, Y. Mao, F. Kaashoek, R. Morris, A. Pesterev, L. Stein, M.Wu, Y. Dai, et al. Corey: An operating system for many cores. In Proceedings of the 8th USENIX conference on Operating systems design and implementation, pages 43--57. USENIX Association, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. Boyd-Wickizer, A. Clements, O. Mao, A. Pesterev, M. Kaashoek, R. Morris, and N. Zeldovich. An analysis of Linux scalability to many cores. In Proceedings of the 9th USENIX conference on Operating systems design and implementation. USENIX Association, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. E. Bugnion, S. Devine, and M. Rosenblum. Disco: running commodity operating systems on scalable multiprocessors. ACM SIGOPS Operating Systems Review, 31(5):143--156, 1997. ISSN 0163-5980. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. D. Engler, M. Kaashoek, et al. Exokernel: An operating system architecture for application-level resource management. ACM SIGOPS Operating Systems Review, 29(5):251--266, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. B. Gamsa, O. Krieger, J. Appavoo, and M. Stumm. Tornado: Maximizing locality and concurrency in a shared memory multiprocessor operating system. Operating systems review, 33:87--100, 1998. ISSN 0163-5980.Google ScholarGoogle Scholar
  15. C. Gough, S. Siddha, and K. Chen. Kernel scalability--expanding the horizon beyond fine grain locks. In Proceedings of the Linux Symposium, pages 153--165, 2007.Google ScholarGoogle Scholar
  16. K. Govil, D. Teodosiu, Y. Huang, and M. Rosenblum. Cellular Disco: Resource management using virtual clusters on shared-memory multiprocessors. In Proceedings of the seventeenth ACM symposium on Operating systems principles, pages 154--169. ACM, 1999. ISBN 1581131402. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. J. Held, J. Bautista, and S. Koehl. White paper from a few cores to many: A tera-scale computing research review.Google ScholarGoogle Scholar
  18. Intel. The MultiProcessor Specification Version 1.4. http://download.intel.com/design/archives/processors/pro/docs/24201606.pdf, 1997.Google ScholarGoogle Scholar
  19. E. Keller, J. Szefer, J. Rexford, and R. B. Lee. Nohype: virtualized cloud infrastructure without the virtualization. In Proceedings of the 37th annual international symposium on Computer architecture, ISCA '10, pages 350--361, New York, NY, USA, 2010. ACM. ISBN 978-1-4503-0053-7. doi: 10.1145/1815961.1816010. URL http://doi.acm.org/10.1145/1815961.1816010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Klein. An NUMA API for Linux. http://www.firstfloor.org/andi/numa.html, August 2004.Google ScholarGoogle Scholar
  21. L. McVoy and C. Staelin. lmbench: Portable tools for performance analysis. In Proceedings of the 1996 annual conference on USENIX Annual Technical Conference, page 23. Usenix Association, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. E. Nightingale, O. Hodson, R. McIlroy, C. Hawblitzel, and G. Hunt. Helios: heterogeneous multiprocessing with satellite kernels. In Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles, pages 221--234. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. B. Saha, A. Adl-Tabatabai, A. Ghuloum, M. Rajagopalan, R. Hudson, L. Petersen, V. Menon, B. Murphy, T. Shpeisman, E. Sprangle, et al. Enabling scalability and performance in a large scale CMP environment. In ACM SIGOPS Operating Systems Review, volume 41, pages 73--86. ACM, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Szefer, E. Keller, R. B. Lee, and J. Rexford. Eliminating the hypervisor attack surface for a more secure cloud. In Proceedings of the 18th ACM conference on Computer and communications security, CCS '11, pages 401--412, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0948-6. doi: 10.1145/2046707.2046754. URL http://doi.acm.org/10.1145/2046707.2046754. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. B. Veal and A. Foong. Performance scalability of a multi-core web server. In Proceedings of the 3rd ACM/IEEE Symposium on Architecture for networking and communications systems, pages 57--66. ACM, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. B. Verghese, S. Devine, A. Gupta, and M. Rosenblum. Operating system support for improving data locality on CC-NUMA compute servers, volume 31. ACM, 1996. ISBN 0897917677. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. C. Waldspurger. Memory resource management in VMware ESX server. ACM SIGOPS Operating Systems Review, 36(SI):194, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A lightweight VMM on many core for high performance computing

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 48, Issue 7
          VEE '13
          July 2013
          194 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/2517326
          Issue’s Table of Contents
          • cover image ACM Conferences
            VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
            March 2013
            210 pages
            ISBN:9781450312660
            DOI:10.1145/2451512

          Copyright © 2013 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 16 March 2013

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!