skip to main content
research-article

Parallelizing live migration of virtual machines

Authors Info & Claims
Published:16 March 2013Publication History
Skip Abstract Section

Abstract

Live VM migration is one of the major primitive operations to manage virtualized cloud platforms. Such operation is usually mission-critical and disruptive to the running services, and thus should be completed as fast as possible. Unfortunately, with the increasing amount of resources configured to a VM, such operations are becoming increasingly time-consuming.

In this paper, we make a comprehensive analysis on the parallelization opportunities of live VM migration on two popular open-source VMMs (i.e., Xen and KVM). By leveraging abundant resources like CPU cores and NICs in contemporary server platforms, we design and implement a system called PMigrate that leverages data parallelism and pipeline parallelism to parallelize the operation. As the parallelization framework requires intensive mmap/munmap operations that tax the address space management system in an operating system, we further propose an abstraction called range lock, which improves scalability of concurrent mutation to the address space of an operating system (i.e., Linux) by selectively replacing the per-process address space lock inside kernel with dynamic and fine-grained range locks that exclude costly operations on the requesting address range from using the per-process lock. Evaluation with our working prototype on Xen and KVM shows that PMigrate accelerates the live VM migration ranging from 2.49X to 9.88X, and decreases the downtime ranging from 1.9X to 279.89X. Performance analysis shows that our integration of range lock to Linux significantly improves parallelism in mutating the address space in VM migration and thus boosts the performance ranging from 2.06X to 3.05X. We also show that PMigrate makes only small disruption to other co-hosted production VMs.

References

  1. Intel virtualization technology. http://www.intel.com/technology/virtualization/.Google ScholarGoogle Scholar
  2. TPC-B. http://www.tpc.org/tpcb/default.asp.Google ScholarGoogle Scholar
  3. Dbench. http://dbench.samba.org/.Google ScholarGoogle Scholar
  4. Instance Types of Amazon Elastic Compute Cloud (EC2). http://aws.amazon.com/ec2/#instance.Google ScholarGoogle Scholar
  5. LibMemcached. http://libmemcached.org/.Google ScholarGoogle Scholar
  6. Linux test project. http://ltp.sourceforge.net/.Google ScholarGoogle Scholar
  7. PostgreSQL. http://www.postgresql.org/.Google ScholarGoogle Scholar
  8. Quicklz. www.quicklz.com/.Google ScholarGoogle Scholar
  9. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proc. SOSP, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. S. Boyd-Wickizer, A. Clements, Y. Mao, A. Pesterev, M. Kaashoek, R. Morris, N. Zeldovich, et al. An analysis of linux scalability to many cores. In Proceedings of the 9th USENIX conference on Operating systems design and implementation, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. R. Bradford, E. Kotsovinos, A. Feldmann, and H. Schiöberg. Live wide-area migration of virtual machines including local persistent state. In Proceedings of the 3rd international conference on Virtual Execution Environments, pages 169--179, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. L. Cherkasova, D. Gupta, and A. Vahdat. Comparison of the three cpu schedulers in Xen. Performance Evaluation Review, 35 (2): 42, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. C. Clark, K. Fraser, S. Hand, J. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live migration of virtual machines. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation, pages 273--286, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. A. T. Clements, M. F. Kaashoek, and N. Zeldovich. Scalable address spaces using RCU balanced trees. In Proceedings of the 17th international conference on Architectural Support for Programming Languages and Operating Systems, pages 199--210, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. M. Hines and K. Gopalan. Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In Proceedings of the ACM SIGPLAN/SIGOPS international conference on Virtual Execution Environments, pages 51--60, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Kivity, Y. Kamay, D. Laor, U. Lublin, and A. Liguori. KVM: the linux virtual machine monitor. In Proceedings of the Linux Symposium, pages 225--230, 2007.Google ScholarGoogle Scholar
  17. H. Lagar-Cavilla, J. Whitney, A. Scannell, P. Patchin, S. Rumble, E. De Lara, M. Brudno, and M. Satyanarayanan. Snowflock: rapid virtual machine cloning for cloud computing. In Proceedings of the 4th ACM European conference on Computer systems, pages 1--12, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. R. LERNER. Memcached integration in rails. Linux Journal, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. A. Mashtizadeh, E. Celebi, T. Garfinkel, and M. Cai. The design and evolution of live storage migration in VMware ESX. In Proceedings of the USENIX Annual Technical Conference, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Nagarajan, F. Mueller, C. Engelmann, and S. Scott. Proactive fault tolerance for HPC with Xen virtualization. In Proceedings of the 21st annual international conference on Supercomputing, pages 23--32, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. R. Nathuji and K. Schwan. Virtualpower: coordinated power management in virtualized enterprise systems. In Proceedings of 21st ACM SIGOPS Symposium on Operating Systems Principles, pages 265--278, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. M. Nelson, B. Lim, and G. Hutchins. Fast transparent migration for virtual machines. In Proceedings of the USENIX Annual Technical Conference, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. and Elmorth(2011)}svard2011veeJ. T. Peter Svärd, Benoit Hudzia and E. Elmorth. Evaluation of delta compression techniques for efficient live migration of large virtual machines. In Proceeding of the ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. M. Powell and B. Miller. Process migration in DEMOS/MP. In Proceedings of the 9th Symposium on Operating System Principles, 1983. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. W. Pugh. Skip lists: a probabilistic alternative to balanced trees. Communications of the ACM, 33 (6): 668--676, 1990. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. M. Theimer, K. Lantz, and D. Cheriton. Preemptable remote execution facilities for the V-system. In Proceedings of the tenth ACM Symposium on Operating Systems Principles, 1985. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Parallelizing live migration of virtual machines

          Recommendations

          Comments

          Login options

          Check if you have access through your login credentials or your institution to get full access on this article.

          Sign in

          Full Access

          • Published in

            cover image ACM SIGPLAN Notices
            ACM SIGPLAN Notices  Volume 48, Issue 7
            VEE '13
            July 2013
            194 pages
            ISSN:0362-1340
            EISSN:1558-1160
            DOI:10.1145/2517326
            Issue’s Table of Contents
            • cover image ACM Conferences
              VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environments
              March 2013
              210 pages
              ISBN:9781450312660
              DOI:10.1145/2451512

            Copyright © 2013 ACM

            Publisher

            Association for Computing Machinery

            New York, NY, United States

            Publication History

            • Published: 16 March 2013

            Check for updates

            Qualifiers

            • research-article

          PDF Format

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader
          About Cookies On This Site

          We use cookies to ensure that we give you the best experience on our website.

          Learn more

          Got it!