skip to main content
research-article

CompSC: live migration with pass-through devices

Published:03 March 2012Publication History
Skip Abstract Section

Abstract

Live migration is one of the most important features of virtualization technology. With regard to recent virtualization techniques, performance of network I/O is critical. Current network I/O virtualization (e.g. Para-virtualized I/O, VMDq) has a significant performance gap with native network I/O. Pass-through network devices have near native performance, however, they have thus far prevented live migration. No existing methods solve the problem of live migration with pass-through devices perfectly.

In this paper, we propose CompSC: a solution of hardware state migration that will enable the live migration support of pass-through devices. We go on to apply CompSC to SR-IOV network interface controllers. We discuss the attributes of different hardware states in pass-through devices and migrate them with corresponding techniques. Our experiments show that CompSC enables live migration on an Intel 82599 VF with a throughput 282.66% higher than para-virtualized devices. In addition, service downtime during live migration is 42.9% less than para-virtualized devices.

References

  1. Kvm and linux source. http://git.kernel.org/.Google ScholarGoogle Scholar
  2. Network plugin architecture discussion on lkml. http://kerneltrap.org/mailarchive/linux-kernel/2010/5/4/4565952.Google ScholarGoogle Scholar
  3. Paravirtops. http://wiki.xen.org/xenwiki/XenParavirtOps.Google ScholarGoogle Scholar
  4. Spec web 2009. http://www.spec.org/web2009/.Google ScholarGoogle Scholar
  5. Virtual machine device queues. http://www.intel.com/content/www/us/en/network-adapters/gigabit-network-adapters/io-acceleration-technology-vmdq.html.Google ScholarGoogle Scholar
  6. Vmware esx. http://www.vmware.com/products/vsphere/esxi-and-esx/index.htm.Google ScholarGoogle Scholar
  7. D. Abramson, J. Jackson, S. Muthrasanallur, G. Neiger, G. Regnier, R. Sankran, I. Schoinas, R. Uhlig, B. Vembu, and J. Weigert. Intel® virtualization technology for directed i/o. Intel® Technology Journal, 10, Aug 2006. http://www.intel.com/technology/itj/2006/v10i3/.Google ScholarGoogle ScholarCross RefCross Ref
  8. K. Adams and O. Agesen. A comparison of software and hardware techniques for x86 virtualization. In Proceedings of the 12th international conference on Architectural support for programming languages and operating systems, ASPLOS-XII, pages 2--13, New York, NY, USA, 2006. ACM. ISBN 1--59593--451-0. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. Above the clouds: A berkeley view of cloud computing. Technical report, Feb. 2009.Google ScholarGoogle Scholar
  10. K. Avi. Kvm : The linux virtual machine monitor. Proceedings of the Ottawa Linux Symposium, 2007.Google ScholarGoogle Scholar
  11. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. SIGOPS Oper. Syst. Rev., 37: 164--177, October 2003. ISSN 0163--5980. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. S. Berger, R. Cáceres, K. A. Goldman, R. Perez, R. Sailer, and L. Doorn. vtpm: Virtualizing the trusted platform module. In USENIX Security, pages 305--320, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. P. M. Chen and B. D. Noble. When virtual is better than real. In Proceedings of the Eighth Workshop on Hot Topics in Operating Systems, pages 133--, Washington, DC, USA, 2001. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live migration of virtual machines. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2, NSDI'05, pages 273--286, Berkeley, CA, USA, 2005. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. B. Cully, G. Lefebvre, D. Meyer, M. Feeley, N. Hutchinson, and A. Warfield. Remus: High availability via asynchronous virtual machine replication. In Proc. NSDI, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Y. Dong, Z. Yu, and G. Rose. Sr-iov networking in xen: architecture, design and implementation. In Proceedings of the First conference on I/O virtualization, WIOV'08, pages 10--10, Berkeley, CA, USA, 2008. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Y. Dong, Y. Chen, Z. Pan, J. Dai, and Y. Jiang. Renic: Architecture extension to sr-iov network for efficient high availability replication. In Proceedings of the 7th International Conference on High Performance and Embedded Architectures and Compilers, HiPEAC'12, 2012.Google ScholarGoogle Scholar
  18. W. Huang, J. Liu, B. Abali, and D. K. Panda. A case for high performance computing with virtual machines. In Proceedings of the 20th annual international conference on Supercomputing, ICS '06, pages 125--134, New York, NY, USA, 2006. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Intel 82576 Gigabit Ethernet Controller Datasheet. Intel. http://download.intel.com/design/network/datashts/82576_Datasheet.pdf.Google ScholarGoogle Scholar
  20. Intel 82599 10 GbE Controller Datasheet. Intel. http://download.intel.com/design/network/datashts/82599_datasheet.pdf.Google ScholarGoogle Scholar
  21. A. Kadav and M. M. Swift. Live migration of direct-access devices. SIGOPS Oper. Syst. Rev., 43: 95--104, July 2009. ISSN 0163--5980. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. M. A. Kozuch, M. Kaminsky, and M. P. Ryan. Migration without virtualization. In Proceedings of the 12th conference on Hot topics in operating systems, HotOS'09, pages 10--10, Berkeley, CA, USA, 2009. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. G. Liao, D. Guo, L. Bhuyan, and S. R. King. Software techniques to improve virtualized i/o performance on multi-core systems. In Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems, ANCS '08, pages 161--170, New York, NY, USA, 2008. ACM. ISBN 978--1--60558--346--4. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Liu, W. Huang, B. Abali, and D. K. Panda. High performance vmm-bypass i/o in virtual machines. In Proceedings of the annual conference on USENIX '06 Annual Technical Conference, pages 3--3, Berkeley, CA, USA, 2006. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. A. Menon, J. R. Santos, Y. Turner, G. J. Janakiraman, and W. Zwaenepoel. Diagnosing performance overheads in the xen virtual machine environment. In Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments, VEE '05, pages 13--23, New York, NY, USA, 2005. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. A. Menon, A. L. Cox, and W. Zwaenepoel. Optimizing network virtualization in xen. In Proceedings of the annual conference on USENIX '06 Annual Technical Conference, pages 2--2, Berkeley, CA, USA, 2006. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. D. S. Milojicic, F. Douglis, Y. Paindaveine, R. Wheeler, and S. Zhou. Process migration. ACM Computing Surveys, 32: 2000, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. A. B. Nagarajan and F. Mueller. Proactive fault tolerance for hpc with xen virtualization. In Proceedings of the 21st Annual International Conference on Supercomputing (ICS'07, pages 23--32. ACM Press, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. J. R. Santos, Y. Turner, G. Janakiraman, and I. Pratt. Bridging the gap between software and hardware techniques for i/o virtualization. In USENIX 2008 Annual Technical Conference on Annual Technical Conference, pages 29--42, Berkeley, CA, USA, 2008. USENIX Association. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. J. Sugerman, G. Venkitachalam, and B.-H. Lim. Virtualizing i/o devices on vmware workstation's hosted virtual machine monitor. In Proceedings of the General Track: 2002 USENIX Annual Technical Conference, pages 1--14, Berkeley, CA, USA, 2001. USENIX Association. ISBN 1--880446-09-X. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. C. A. Waldspurger. Memory resource management in vmware esx server. SIGOPS Oper. Syst. Rev., 36: 181--194, December 2002. ISSN 0163--5980. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. E. Zhai, G. D. Cummings, and Y. Dong. Live migration with pass-through device for linux vm. In Ottawa Linux Symposium, 2008.Google ScholarGoogle Scholar

Index Terms

  1. CompSC: live migration with pass-through devices

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      • Published in

        cover image ACM SIGPLAN Notices
        ACM SIGPLAN Notices  Volume 47, Issue 7
        VEE '12
        July 2012
        229 pages
        ISSN:0362-1340
        EISSN:1558-1160
        DOI:10.1145/2365864
        Issue’s Table of Contents
        • cover image ACM Conferences
          VEE '12: Proceedings of the 8th ACM SIGPLAN/SIGOPS conference on Virtual Execution Environments
          March 2012
          248 pages
          ISBN:9781450311762
          DOI:10.1145/2151024

        Copyright © 2012 ACM

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 3 March 2012

        Check for updates

        Qualifiers

        • research-article

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!