skip to main content
research-article
Public Access

Urgent Virtual Machine Eviction with Enlightened Post-Copy

Published:25 March 2016Publication History
Skip Abstract Section

Abstract

Virtual machine (VM) migration demands distinct properties under resource oversubscription and workload surges. We present enlightened post-copy, a new mechanism for VMs under contention that evicts the target VM with fast execution transfer and short total duration. This design contrasts with common live migration, which uses the down time of the migrated VM as its primary metric; it instead focuses on recovering the aggregate performance of the VMs being affected. In enlightened post-copy, the guest OS identifies memory state that is expected to encompass the VM's working set. The hypervisor accordingly transfers its state, mitigating the performance impact on the migrated VM resulting from post-copy transfer. We show that our implementation, with modest instrumentation in guest Linux, resolves VM contention up to several times faster than live migration.

References

  1. AWStextbar Amazon Elastic Compute Cloud (EC2) - Scalable Cloud Hosting. http://aws.amazon.com/ec2.Google ScholarGoogle Scholar
  2. memcached - a distributed memory object caching system. http://memcached.org.Google ScholarGoogle Scholar
  3. OLTPBenchmark. http://oltpbenchmark.com/wiki.Google ScholarGoogle Scholar
  4. The Apache Cassandra Project. http://cassandra.apache.org.Google ScholarGoogle Scholar
  5. userfaultfd v4 [LWN.net]. https://lwn.net/Articles/644532.Google ScholarGoogle Scholar
  6. Virtio - KVM. http://www.linux-kvm.org/page/Virtio.Google ScholarGoogle Scholar
  7. VMware vSere vMotion Architecture, Performance and Best Practices in VMware vSere 5. http://www.vmware.com/files/pdf/vmotion-perf-vsere5.pdf.Google ScholarGoogle Scholar
  8. Yahoo! Cloud Serving Benchmark (YCSB). https://github.com/brianfrankcooper/YCSB/wiki.Google ScholarGoogle Scholar
  9. R. Bradford, E. Kotsovinos, A. Feldmann, and H. Schiöberg. Live Wide-Area Migration of Virtual Machines Including Local Persistent State. In Proceedings of the Third International Conference on Virtual Execution Environments (VEE '07), San Diego, CA, USA, June 2007.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. R. Bryant, A. Tumanov, O. Irzak, A. Scannell, K. Joshi, M. Hiltunen, A. Lagar-Cavilla, and E. de Lara. Kaleidoscope: Cloud Micro-elasticity via VM State Coloring. In Proceedings of the Sixth ACM European Conference on Computer Systems (EuroSys '11), Salzburg, Austria, April 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. R. Chandra, N. Zeldovich, C. Sapuntzakis, and M. S. Lam. The Collective: A Cache-based System Management Architecture. In Proceedings of the Second Conference on Symposium on Networked Systems Design & Implementation - Volume 2 (NSDI '05), Boston, MA, USA, May 2005.Google ScholarGoogle Scholar
  12. C. Clark, K. Fraser, S. Hand, J. G. Hansen, E. Jul, C. Limpach, I. Pratt, and A. Warfield. Live Migration of Virtual Machines. In Proceedings of the Second Conference on Symposium on Networked Systems Design & Implementation - Volume 2 (NSDI '05), Boston, MA, USA, May 2005.Google ScholarGoogle Scholar
  13. S. Das, D. Agrawal, and A. El Abbadi. ElasTraS: An Elastic, Scalable, and Self-managing Transactional Database for the Cloud. ACM Transactions on Database Systems, 38(1), April 2013.Google ScholarGoogle Scholar
  14. A. J. Elmore, S. Das, D. Agrawal, and A. El Abbadi. Zeyr: Live Migration in Shared Nothing Databases for Elastic Cloud Platforms. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data (SIGMOD '11), Athens, Greece, June 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. S. Hacking and B. Hudzia. Improving the Live Migration Process of Large Enterprise Applications. In Proceedings of the Third International Workshop on Virtualization Technologies in Distributed Computing (VTDC '09), Barcelona, Spain, June 2009.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. M. R. Hines, U. Deshpande, and K. Gopalan. Post-copy Live Migration of Virtual Machines. SIGOPS Operating Systems Review, 43(3), July 2009.Google ScholarGoogle Scholar
  17. M. R. Hines and K. Gopalan. Post-copy Based Live Virtual Machine Migration Using Adaptive Pre-paging and Dynamic Self-ballooning. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '09), Washington, DC, USA, March 2009.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. K.-Y. Hou, K. G. Shin, and J.-L. Sung. Application-assisted Live Migration of Virtual Machines with Java Applications. In Proceedings of the Tenth ACM European Conference on Computer Systems (EuroSys '15), Bordeaux, France, April 2015.Google ScholarGoogle Scholar
  19. K. Z. Ibrahim, S. Hofmeyr, C. Iancu, and E. Roman. Optimized Pre-copy Live Migration for Memory Intensive Applications. In Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC '11), Seattle, WA, USA, November 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. A. Koto, H. Yamada, K. Ohmura, and K. Kono. Towards Unobtrusive VM Live Migration for Cloud Computing Platforms. In Proceedings of the Third ACM SIGOPS Asia-Pacific Conference on Systems (APSys '12), Seoul, South Korea, July 2012.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. M. Kozuch, M. Satyanarayanan, T. Bressoud, and Y. Ke. Efficient State Transfer for Internet Suspend/Resume. Intel Research Pittsburgh Technical Report IRP-TR-02-03, May 2002.Google ScholarGoogle Scholar
  22. M. A. Kozuch and M. Satyanarayanan. Internet Suspend/Resume. In Proceedings of the Fourth IEEE Workshop on Mobile Computing Systems and Applications, Callicoon, NY, USA, June 2002.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. H. A. Lagar-Cavilla, J. A. Whitney, A. M. Scannell, P. Patchin, S. M. Rumble, E. de Lara, M. Brudno, and M. Satyanarayanan. SnowFlock: Rapid Virtual Machine Cloning for Cloud Computing. In Proceedings of the Fourth ACM European Conference on Computer Systems (EuroSys '09), Nuremberg, Germany, April 2009.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. H. Liu, H. Jin, X. Liao, L. Hu, and C. Yu. Live Migration of Virtual Machine Based on Full System Trace and Replay. In Proceedings of the Eighteenth ACM International Symposium on High Performance Distributed Computing (HPDC '09), Garching, Germany, June 2009.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. J. Lo, E. Wohlstadter, and A. Mesbah. Live Migration of JavaScript Web Apps. In Proceedings of the Twenty-Second International Conference on World Wide Web (WWW '13 Companion), Rio de Janeiro, Brazil, May 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. P. Lu, A. Barbalace, and B. Ravindran. HSG-LM: Hybrid-copy Speculative Guest OS Live Migration Without Hypervisor. In Proceedings of the Sixth International Systems and Storage Conference (SYSTOR '13), Haifa, Israel, June 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. A. J. Mashtizadeh, M. Cai, G. Tarasuk-Levin, R. Koller, T. Garfinkel, and S. Setty. XvMotion: Unified Virtual Machine Migration over Long Distance. In Proceedings of the 2014 USENIX Annual Technical Conference (USENIX ATC '14), iladelia, PA, USA, June 2014.Google ScholarGoogle Scholar
  28. G. Miłós, D. G. Murray, S. Hand, and M. A. Fetterman. Satori: Enlightened Page Sharing. In Proceedings of the 2009 USENIX Annual Technical Conference (USENIX ATC '09), San Diego, CA, USA, June 2009.Google ScholarGoogle Scholar
  29. T. Mishima and Y. Fujiwara. Madeus: Database Live Migration Middleware Under Heavy Workloads for Cloud Environment. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data (SIGMOD '15), Melbourne, Australia, May 2015.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. M. Nelson, B.-H. Lim, and G. Hutchins. Fast Transparent Migration for Virtual Machines. In Proceedings of the 2005 USENIX Annual Technical Conference (USENIX ATC '05), Anaheim, CA, USA, April 2005.Google ScholarGoogle Scholar
  31. C. Peng, M. Kim, Z. Zhang, and H. Lei. VDN: Virtual Machine Image Distribution Network for Cloud Data Centers. In Proceedings of INFOCOM 2012, Orlando, FL, USA, March 2012.Google ScholarGoogle ScholarCross RefCross Ref
  32. J. Reich, O. Laadan, E. Brosh, A. Sherman, V. Misra, J. Nieh, and D. Rubenstein. VMTorrent: Virtual Appliances On-demand. In Proceedings of the ACM SIGCOMM 2010 Conference (SIGCOMM '10), New Delhi, India, August 2010.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. J. Reich, O. Laadan, E. Brosh, A. Sherman, V. Misra, J. Nieh, and D. Rubenstein. VMTorrent: Scalable P2P Virtual Machine Streaming. In Proceedings of the Eighth International Conference on Emerging Networking Experiments and Technologies (CoNEXT '12), Nice, France, December 2012.Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. C. Sapuntzakis, D. Brumley, R. Chandra, N. Zeldovich, J. Chow, M. S. Lam, and M. Rosenblum. Virtual Appliances for Deploying and Maintaining Software. In Proceedings of the Seventeenth USENIX Conference on System Administration (LISA '03), San Diego, CA, USA, October 2003.Google ScholarGoogle Scholar
  35. C. Sapuntzakis and M. S. Lam. Virtual Appliances in the Collective: A Road to Hassle-free Computing. In Proceedings of the Ninth Conference on Hot Topics in Operating Systems - Volume 9 (HotOS '03), Lihue, HI, USA, May 2003.Google ScholarGoogle Scholar
  36. C. P. Sapuntzakis, R. Chandra, B. Pfaff, J. Chow, M. S. Lam, and M. Rosenblum. Optimizing the Migration of Virtual Computers. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation (OSDI '02), Boston, MA, USA, December 2002.Google ScholarGoogle ScholarCross RefCross Ref
  37. M. Satyanarayanan, B. Gilbert, M. Toups, N. Tolia, A. Surie, D. R. O'Hallaron, A. Wolbach, J. Harkes, A. Perrig, D. J. Farber, M. A. Kozuch, C. J. Helfrich, P. Nath, and H. A. Lagar-Cavilla. Pervasive Personal Computing in an Internet Suspend/Resume System. IEEE Internet Computing, 11(2), March 2007.Google ScholarGoogle Scholar
  38. A. Shribman and B. Hudzia. Pre-Copy and Post-copy VM Live Migration for Memory Intensive Applications. In Proceedings of the Eighteenth International Conference on Parallel Processing Workshops (Euro-Par '12), Rhodes Island, Greece, August 2012.Google ScholarGoogle Scholar
  39. X. Song, J. Shi, R. Liu, J. Yang, and H. Chen. Parallelizing Live Migration of Virtual Machines. In Proceedings of the Ninth ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '13), Houston, TX, USA, March 2013.Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. P. Sv\"ard, B. Hudzia, J. Tordsson, and E. Elmroth. Evaluation of Delta Compression Techniques for Efficient Live Migration of Large Virtual Machines. In Proceedings of the Seventh ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '11), Newport Beach, CA, USA, March 2011.Google ScholarGoogle Scholar
  41. C. A. Waldspurger. Memory Resource Management in VMware ESX Server. In Proceedings of the Fifth Symposium on Operating Systems Design and Implementation (OSDI '02), Boston, MA, USA, December 2002.Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. C. Wang, F. Mueller, C. Engelmann, and S. L. Scott. Proactive Process-level Live Migration and Back Migration in HPC Environments. Journal of Parallel and Distributed Computing, 72(2), February 2012.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. A. Whitaker, R. S. Cox, M. Shaw, and S. D. Gribble. Constructing Services with Interposable Virtual Hardware. In Proceedings of the First Conference on Symposium on Networked Systems Design and Implementation - Volume 1 (NSDI '04), San Francisco, CA, USA, March 2004.Google ScholarGoogle Scholar
  44. T. Wood, K. K. Ramakrishnan, P. Shenoy, and J. van der Merwe. CloudNet: Dynamic Pooling of Cloud Resources by Live WAN Migration of Virtual Machines. In Proceedings of the Seventh ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE '11), Newport Beach, CA, USA, March 2011.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Urgent Virtual Machine Eviction with Enlightened Post-Copy

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM SIGPLAN Notices
          ACM SIGPLAN Notices  Volume 51, Issue 7
          VEE '16
          July 2016
          167 pages
          ISSN:0362-1340
          EISSN:1558-1160
          DOI:10.1145/3007611
          Issue’s Table of Contents
          • cover image ACM Conferences
            VEE '16: Proceedings of the12th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments
            March 2016
            186 pages
            ISBN:9781450339476
            DOI:10.1145/2892242

          Copyright © 2016 ACM

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 25 March 2016

          Check for updates

          Qualifiers

          • research-article

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!