skip to main content
research-article

Demand-based coordinated scheduling for SMP VMs

Published:16 March 2013Publication History
Skip Abstract Section

Abstract

As processor architectures have been enhancing their computing capacity by increasing core counts, independent workloads can be consolidated on a single node for the sake of high resource efficiency in data centers. With the prevalence of virtualization technology, each individual workload can be hosted on a virtual machine for strong isolation between co-located workloads. Along with this trend, hosted applications have increasingly been multithreaded to take advantage of improved hardware parallelism. Although the performance of many multithreaded applications highly depends on communication (or synchronization) latency, existing schemes of virtual machine scheduling do not explicitly coordinate virtual CPUs based on their communication behaviors.

This paper presents a demand-based coordinated scheduling scheme for consolidated virtual machines that host multithreaded workloads. To this end, we propose communication-driven scheduling that controls time-sharing in response to inter-processor interrupts (IPIs) between virtual CPUs. On the basis of in-depth analysis on the relationship between IPI communications and coordination demands, we devise IPI-driven coscheduling and delayed preemption schemes, which effectively reduce synchronization latency and unnecessary CPU consumption. In addition, we introduce a load-conscious CPU allocation policy in order to address load imbalance in heterogeneously consolidated environments. The proposed schemes are evaluated with respect to various scenarios of mixed workloads using the PARSEC multithreaded applications. In the evaluation, our scheme improves the overall performance of consolidated workloads, especially communication-intensive applications, by reducing inefficient synchronization latency.

References

  1. AMD. Amd64 architecture programmer's manual volume 2: System programming, 2010.Google ScholarGoogle Scholar
  2. R. H. Arpaci, A. C. Dusseau, A. M. Vahdat, L. T. Liu, T. E. Anderson, and D. A. Patterson. The interaction of parallel and sequential workloads on a network of workstations. In Proc. SIGMETRICS, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. A. C. Arpaci-Dusseau. Implicit coscheduling: coordinated scheduling with implicit information in distributed systems. ACM TOCS, 19 (3): 283--331, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. A. C. Arpaci-Dusseau, D. E. Culler, and A. M. Mainwaring. Scheduling with implicit information in distributed systems. In Proc. SIGMETRICS, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer, I. Pratt, and A. Warfield. Xen and the art of virtualization. In Proc. SOSP, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. C. Bienia, S. Kumar, J. P. Singh, and K. Li. The PARSEC benchmark suite: characterization and architectural implications. In Proc. PACT, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. H. Chen, H. Jin, K. Hu, and J. Huang. Scheduling overcommitted VM: Behavior monitoring and dynamic switching-frequency scaling. FGCS, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. K. J. Duda and D. R. Cheriton. Borrowed-virtual-time (BVT) scheduling: supporting latency-sensitive threads in a general-purpose scheduler. In Proc. SOSP, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. A. C. Dusseau, R. H. Arpaci, and D. E. Culler. Effective distributed scheduling of parallel workloads. In Proc. SIGMETRICS, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. D. Feitelson. Gang scheduling performance benefits for fine-grain synchronization. JPDC, 16 (4): 306--318, 1992.Google ScholarGoogle ScholarCross RefCross Ref
  11. T. Friebel and S. Biemueller. How to deal with lock holder preemption. In Xen Summit, 2008.Google ScholarGoogle Scholar
  12. Intel. Intel 64 and ia-32 architectures software developer's manual. volume 3b: System programming guide, part 2, 2010.Google ScholarGoogle Scholar
  13. W. Jiang, Y. Zhou, Y. Cui, W. Feng, Y. Chen, Y. Shi, and Q. Wu. CFS optimizations to KVM threads on multi-core environment. In Proc. ICPADS, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. H. Kim, H. Lim, J. Jeong, H. Jo, and J. Lee. Task-aware virtual machine scheduling for I/O performance. In Proc. VEE, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. A. Kivity, U. Lublin, and A. Liguori. KVM: the Linux virtual machine monitor. In Proc. OLS, 2007.Google ScholarGoogle Scholar
  16. W. Lee, M. Frank, V. Lee, K. Mackenzie, and L. Rudolph. Implications of I/O for gang scheduled workloads. In Proc. IPPS, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Y. Lee, W. Son, S. Park, G. Lee, D. Howard, and D. Slezak. Design and implementation of a locking-aware scheduler for multiprocessor environments. Convergence and Hybrid Information Technology, 6935: 384--390, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. W. Mauerer. Professional Linux Kernel Architecture. Wrox Press Ltd., 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. P. B. Menage. Adding generic process containers to the Linux kernel. In Proc. OLS, 2007.Google ScholarGoogle Scholar
  20. J. Ousterhout. Scheduling Techniques for Concurrent Systems. In Proc. ICDCS, 1982.Google ScholarGoogle Scholar
  21. M. Russinovich and D. A. Solomon. Windows Internals: Including Windows Server 2008 and Windows Vista, Fifth Edition. Microsoft Press, 5th edition, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. P. Sobalvarro, S. Pakin, W. Weihl, and A. Chien. Dynamic coscheduling on workstation clusters. In Proc. IPPS, 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. P. G. Sobalvarro and W. E. Weihl. Demand-based coscheduling of parallel jobs on multiprogrammed multiprocessors. In Proc. IPPS, 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. O. Sukwong and H. S. Kim. Is co-scheduling too expensive for SMP VMs? In Proc. EuroSys, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. V. Uhlig, J. LeVasseur, E. Skoglund, and U. Dannowski. Towards scalable multiprocessor virtual machines. In Proc. VM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. VMWare. VMware, Inc. VMware vSphere 4: The CPU scheduler in VMware ESX 4.1. Technical report, 2010.Google ScholarGoogle Scholar
  27. C. Weng, Z. Wang, M. Li, and X. Lu. The hybrid scheduling framework for virtual machine systems. In Proc. VEE, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. C. Weng, Q. Liu, L. Yu, and M. Li. Dynamic adaptive scheduling for virtual machines. In Proc. HPDC, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Y. Wiseman and D. G. Feitelson. Paired gang scheduling. IEEE TPDS, 14 (6): 581--592, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Y. Yu, Y. Wang, H. Guo, and X. He. Hybrid co-scheduling optimizations for concurrent applications in virtualized environments. In Proc. ICNAS, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Demand-based coordinated scheduling for SMP VMs

Recommendations

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in

Full Access

  • Published in

    cover image ACM SIGPLAN Notices
    ACM SIGPLAN Notices  Volume 48, Issue 4
    ASPLOS '13
    April 2013
    540 pages
    ISSN:0362-1340
    EISSN:1558-1160
    DOI:10.1145/2499368
    Issue’s Table of Contents
    • cover image ACM Conferences
      ASPLOS '13: Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems
      March 2013
      574 pages
      ISBN:9781450318709
      DOI:10.1145/2451116

    Copyright © 2013 ACM

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    • Published: 16 March 2013

    Check for updates

    Qualifiers

    • research-article

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader
About Cookies On This Site

We use cookies to ensure that we give you the best experience on our website.

Learn more

Got it!