Abstract
Live migration is one of the most important features of virtualization technology. With regard to recent virtualization techniques, performance of network I/O is critical. Current network I/O virtualization (e.g. Para-virtualized I/O, VMDq) has a significant performance gap with native network I/O. Pass-through network devices have near native performance, however, they have thus far prevented live migration. No existing methods solve the problem of live migration with pass-through devices perfectly.
In this paper, we propose CompSC: a solution of hardware state migration that will enable the live migration support of pass-through devices. We go on to apply CompSC to SR-IOV network interface controllers. We discuss the attributes of different hardware states in pass-through devices and migrate them with corresponding techniques. Our experiments show that CompSC enables live migration on an Intel 82599 VF with a throughput 282.66% higher than para-virtualized devices. In addition, service downtime during live migration is 42.9% less than para-virtualized devices.
- Kvm and linux source. http://git.kernel.org/.Google Scholar
- Network plugin architecture discussion on lkml. http://kerneltrap.org/mailarchive/linux-kernel/2010/5/4/4565952.Google Scholar
- Paravirtops. http://wiki.xen.org/xenwiki/XenParavirtOps.Google Scholar
- Spec web 2009. http://www.spec.org/web2009/.Google Scholar
- Virtual machine device queues. http://www.intel.com/content/www/us/en/network-adapters/gigabit-network-adapters/io-acceleration-technology-vmdq.html.Google Scholar
- Vmware esx. http://www.vmware.com/products/vsphere/esxi-and-esx/index.htm.Google Scholar
- D. Abramson, J. Jackson, S. Muthrasanallur, G. Neiger, G. Regnier, R. Sankran, I. Schoinas, R. Uhlig, B. Vembu, and J. Weigert. Intel® virtualization technology for directed i/o. Intel® Technology Journal, 10, Aug 2006. http://www.intel.com/technology/itj/2006/v10i3/.Google Scholar
Cross Ref
- K. Adams and O. Agesen. A comparison of software and hardware techniques for x86 virtualization. In Proceedings of the 12th international conference on Architectural support for programming languages and operating systems, ASPLOS-XII, pages 2--13, New York, NY, USA, 2006. ACM. ISBN 1--59593--451-0. Google Scholar
Digital Library
- M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. Above the clouds: A berkeley view of cloud computing. Technical report, Feb. 2009.Google Scholar
- K. Avi. Kvm : The linux virtual machine monitor. Proceedings of the Ottawa Linux Symposium, 2007.Google Scholar
- P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. SIGOPS Oper. Syst. Rev., 37: 164--177, October 2003. ISSN 0163--5980. Google Scholar
Digital Library
- S. Berger, R. Cáceres, K. A. Goldman, R. Perez, R. Sailer, and L. Doorn. vtpm: Virtualizing the trusted platform module. In USENIX Security, pages 305--320, 2006. Google Scholar
Digital Library
- P. M. Chen and B. D. Noble. When virtual is better than real. In Proceedings of the Eighth Workshop on Hot Topics in Operating Systems, pages 133--, Washington, DC, USA, 2001. IEEE Computer Society. Google Scholar
Digital Library
- C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live migration of virtual machines. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation - Volume 2, NSDI'05, pages 273--286, Berkeley, CA, USA, 2005. USENIX Association. Google Scholar
Digital Library
- B. Cully, G. Lefebvre, D. Meyer, M. Feeley, N. Hutchinson, and A. Warfield. Remus: High availability via asynchronous virtual machine replication. In Proc. NSDI, 2008. Google Scholar
Digital Library
- Y. Dong, Z. Yu, and G. Rose. Sr-iov networking in xen: architecture, design and implementation. In Proceedings of the First conference on I/O virtualization, WIOV'08, pages 10--10, Berkeley, CA, USA, 2008. USENIX Association. Google Scholar
Digital Library
- Y. Dong, Y. Chen, Z. Pan, J. Dai, and Y. Jiang. Renic: Architecture extension to sr-iov network for efficient high availability replication. In Proceedings of the 7th International Conference on High Performance and Embedded Architectures and Compilers, HiPEAC'12, 2012.Google Scholar
- W. Huang, J. Liu, B. Abali, and D. K. Panda. A case for high performance computing with virtual machines. In Proceedings of the 20th annual international conference on Supercomputing, ICS '06, pages 125--134, New York, NY, USA, 2006. ACM. Google Scholar
Digital Library
- Intel 82576 Gigabit Ethernet Controller Datasheet. Intel. http://download.intel.com/design/network/datashts/82576_Datasheet.pdf.Google Scholar
- Intel 82599 10 GbE Controller Datasheet. Intel. http://download.intel.com/design/network/datashts/82599_datasheet.pdf.Google Scholar
- A. Kadav and M. M. Swift. Live migration of direct-access devices. SIGOPS Oper. Syst. Rev., 43: 95--104, July 2009. ISSN 0163--5980. Google Scholar
Digital Library
- M. A. Kozuch, M. Kaminsky, and M. P. Ryan. Migration without virtualization. In Proceedings of the 12th conference on Hot topics in operating systems, HotOS'09, pages 10--10, Berkeley, CA, USA, 2009. USENIX Association. Google Scholar
Digital Library
- G. Liao, D. Guo, L. Bhuyan, and S. R. King. Software techniques to improve virtualized i/o performance on multi-core systems. In Proceedings of the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems, ANCS '08, pages 161--170, New York, NY, USA, 2008. ACM. ISBN 978--1--60558--346--4. Google Scholar
Digital Library
- J. Liu, W. Huang, B. Abali, and D. K. Panda. High performance vmm-bypass i/o in virtual machines. In Proceedings of the annual conference on USENIX '06 Annual Technical Conference, pages 3--3, Berkeley, CA, USA, 2006. USENIX Association. Google Scholar
Digital Library
- A. Menon, J. R. Santos, Y. Turner, G. J. Janakiraman, and W. Zwaenepoel. Diagnosing performance overheads in the xen virtual machine environment. In Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments, VEE '05, pages 13--23, New York, NY, USA, 2005. ACM. Google Scholar
Digital Library
- A. Menon, A. L. Cox, and W. Zwaenepoel. Optimizing network virtualization in xen. In Proceedings of the annual conference on USENIX '06 Annual Technical Conference, pages 2--2, Berkeley, CA, USA, 2006. USENIX Association. Google Scholar
Digital Library
- D. S. Milojicic, F. Douglis, Y. Paindaveine, R. Wheeler, and S. Zhou. Process migration. ACM Computing Surveys, 32: 2000, 2000. Google Scholar
Digital Library
- A. B. Nagarajan and F. Mueller. Proactive fault tolerance for hpc with xen virtualization. In Proceedings of the 21st Annual International Conference on Supercomputing (ICS'07, pages 23--32. ACM Press, 2007. Google Scholar
Digital Library
- J. R. Santos, Y. Turner, G. Janakiraman, and I. Pratt. Bridging the gap between software and hardware techniques for i/o virtualization. In USENIX 2008 Annual Technical Conference on Annual Technical Conference, pages 29--42, Berkeley, CA, USA, 2008. USENIX Association. Google Scholar
Digital Library
- J. Sugerman, G. Venkitachalam, and B.-H. Lim. Virtualizing i/o devices on vmware workstation's hosted virtual machine monitor. In Proceedings of the General Track: 2002 USENIX Annual Technical Conference, pages 1--14, Berkeley, CA, USA, 2001. USENIX Association. ISBN 1--880446-09-X. Google Scholar
Digital Library
- C. A. Waldspurger. Memory resource management in vmware esx server. SIGOPS Oper. Syst. Rev., 36: 181--194, December 2002. ISSN 0163--5980. Google Scholar
Digital Library
- E. Zhai, G. D. Cummings, and Y. Dong. Live migration with pass-through device for linux vm. In Ottawa Linux Symposium, 2008.Google Scholar
Index Terms
CompSC: live migration with pass-through devices
Recommendations
A Hypervisor Approach to Enable Live Migration with Passthrough SR-IOV Network Devices
Special TopicsSingle-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (physical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly ...
SRVM: Hypervisor Support for Live Migration with Passthrough SR-IOV Network Devices
VEE '16Single-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (ysical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned ...
SRVM: Hypervisor Support for Live Migration with Passthrough SR-IOV Network Devices
VEE '16: Proceedings of the12th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution EnvironmentsSingle-Root I/O Virtualization (SR-IOV) is a specification that allows a single PCI Express (PCIe) device (ysical function or PF) to be used as multiple PCIe devices (virtual functions or VF). In a virtualization system, each VF can be directly assigned ...







Comments