Abstract
Live VM migration is one of the major primitive operations to manage virtualized cloud platforms. Such operation is usually mission-critical and disruptive to the running services, and thus should be completed as fast as possible. Unfortunately, with the increasing amount of resources configured to a VM, such operations are becoming increasingly time-consuming.
In this paper, we make a comprehensive analysis on the parallelization opportunities of live VM migration on two popular open-source VMMs (i.e., Xen and KVM). By leveraging abundant resources like CPU cores and NICs in contemporary server platforms, we design and implement a system called PMigrate that leverages data parallelism and pipeline parallelism to parallelize the operation. As the parallelization framework requires intensive mmap/munmap operations that tax the address space management system in an operating system, we further propose an abstraction called range lock, which improves scalability of concurrent mutation to the address space of an operating system (i.e., Linux) by selectively replacing the per-process address space lock inside kernel with dynamic and fine-grained range locks that exclude costly operations on the requesting address range from using the per-process lock. Evaluation with our working prototype on Xen and KVM shows that PMigrate accelerates the live VM migration ranging from 2.49X to 9.88X, and decreases the downtime ranging from 1.9X to 279.89X. Performance analysis shows that our integration of range lock to Linux significantly improves parallelism in mutating the address space in VM migration and thus boosts the performance ranging from 2.06X to 3.05X. We also show that PMigrate makes only small disruption to other co-hosted production VMs.
- Intel virtualization technology. http://www.intel.com/technology/virtualization/.Google Scholar
- TPC-B. http://www.tpc.org/tpcb/default.asp.Google Scholar
- Dbench. http://dbench.samba.org/.Google Scholar
- Instance Types of Amazon Elastic Compute Cloud (EC2). http://aws.amazon.com/ec2/#instance.Google Scholar
- LibMemcached. http://libmemcached.org/.Google Scholar
- Linux test project. http://ltp.sourceforge.net/.Google Scholar
- PostgreSQL. http://www.postgresql.org/.Google Scholar
- Quicklz. www.quicklz.com/.Google Scholar
- P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proc. SOSP, 2003. Google Scholar
Digital Library
- S. Boyd-Wickizer, A. Clements, Y. Mao, A. Pesterev, M. Kaashoek, R. Morris, N. Zeldovich, et al. An analysis of linux scalability to many cores. In Proceedings of the 9th USENIX conference on Operating systems design and implementation, 2010. Google Scholar
Digital Library
- R. Bradford, E. Kotsovinos, A. Feldmann, and H. Schiöberg. Live wide-area migration of virtual machines including local persistent state. In Proceedings of the 3rd international conference on Virtual Execution Environments, pages 169--179, 2007. Google Scholar
Digital Library
- L. Cherkasova, D. Gupta, and A. Vahdat. Comparison of the three cpu schedulers in Xen. Performance Evaluation Review, 35 (2): 42, 2007. Google Scholar
Digital Library
- C. Clark, K. Fraser, S. Hand, J. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live migration of virtual machines. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation, pages 273--286, 2005. Google Scholar
Digital Library
- A. T. Clements, M. F. Kaashoek, and N. Zeldovich. Scalable address spaces using RCU balanced trees. In Proceedings of the 17th international conference on Architectural Support for Programming Languages and Operating Systems, pages 199--210, 2012. Google Scholar
Digital Library
- M. Hines and K. Gopalan. Post-copy based live virtual machine migration using adaptive pre-paging and dynamic self-ballooning. In Proceedings of the ACM SIGPLAN/SIGOPS international conference on Virtual Execution Environments, pages 51--60, 2009. Google Scholar
Digital Library
- A. Kivity, Y. Kamay, D. Laor, U. Lublin, and A. Liguori. KVM: the linux virtual machine monitor. In Proceedings of the Linux Symposium, pages 225--230, 2007.Google Scholar
- H. Lagar-Cavilla, J. Whitney, A. Scannell, P. Patchin, S. Rumble, E. De Lara, M. Brudno, and M. Satyanarayanan. Snowflock: rapid virtual machine cloning for cloud computing. In Proceedings of the 4th ACM European conference on Computer systems, pages 1--12, 2009. Google Scholar
Digital Library
- R. LERNER. Memcached integration in rails. Linux Journal, 2009. Google Scholar
Digital Library
- A. Mashtizadeh, E. Celebi, T. Garfinkel, and M. Cai. The design and evolution of live storage migration in VMware ESX. In Proceedings of the USENIX Annual Technical Conference, 2011. Google Scholar
Digital Library
- A. Nagarajan, F. Mueller, C. Engelmann, and S. Scott. Proactive fault tolerance for HPC with Xen virtualization. In Proceedings of the 21st annual international conference on Supercomputing, pages 23--32, 2007. Google Scholar
Digital Library
- R. Nathuji and K. Schwan. Virtualpower: coordinated power management in virtualized enterprise systems. In Proceedings of 21st ACM SIGOPS Symposium on Operating Systems Principles, pages 265--278, 2007. Google Scholar
Digital Library
- M. Nelson, B. Lim, and G. Hutchins. Fast transparent migration for virtual machines. In Proceedings of the USENIX Annual Technical Conference, 2005. Google Scholar
Digital Library
- and Elmorth(2011)}svard2011veeJ. T. Peter Svärd, Benoit Hudzia and E. Elmorth. Evaluation of delta compression techniques for efficient live migration of large virtual machines. In Proceeding of the ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, 2011. Google Scholar
Digital Library
- M. Powell and B. Miller. Process migration in DEMOS/MP. In Proceedings of the 9th Symposium on Operating System Principles, 1983. Google Scholar
Digital Library
- W. Pugh. Skip lists: a probabilistic alternative to balanced trees. Communications of the ACM, 33 (6): 668--676, 1990. Google Scholar
Digital Library
- M. Theimer, K. Lantz, and D. Cheriton. Preemptable remote execution facilities for the V-system. In Proceedings of the tenth ACM Symposium on Operating Systems Principles, 1985. Google Scholar
Digital Library
Index Terms
Parallelizing live migration of virtual machines
Recommendations
Parallelizing live migration of virtual machines
VEE '13: Proceedings of the 9th ACM SIGPLAN/SIGOPS international conference on Virtual execution environmentsLive VM migration is one of the major primitive operations to manage virtualized cloud platforms. Such operation is usually mission-critical and disruptive to the running services, and thus should be completed as fast as possible. Unfortunately, with ...
Live gang migration of virtual machines
HPDC '11: Proceedings of the 20th international symposium on High performance distributed computingThis paper addresses the problem of simultaneously migrating a group of co-located and live virtual machines (VMs), i.e, VMs executing on the same physical machine. We refer to such a mass simultaneous migration of active VMs as "live gang migration". ...
Traffic-sensitive live migration of virtual machines
CCGRID '15: Proceedings of the 15th IEEE/ACM International Symposium on Cluster, Cloud, and Grid ComputingIn this paper we address the problem of network contention between the migration traffic and the Virtual Machine (VM) application traffic for the live migration of co-located Virtual Machines. When VMs are migrated with pre-copy, they run at the source ...







Comments