skip to main content
research-article

Efficient and scalable multiprocessor fair scheduling using distributed weighted round-robin

Published:14 February 2009Publication History
Skip Abstract Section

Abstract

Fairness is an essential requirement of any operating system scheduler. Unfortunately, existing fair scheduling algorithms are either inaccurate or inefficient and non-scalable for multiprocessors. This problem is becoming increasingly severe as the hardware industry continues to produce larger scale multi-core processors. This paper presents Distributed Weighted Round-Robin (DWRR), a new scheduling algorithm that solves this problem. With distributed thread queues and small additional overhead to the underlying scheduler, DWRR achieves high efficiency and scalability. Besides conventional priorities, DWRR enables users to specify weights to threads and achieve accurate proportional CPU sharing with constant error bounds. DWRR operates in concert with existing scheduler policies targeting other system attributes, such as latency and throughput. As a result, it provides a practical solution for various production OSes. To demonstrate the versatility of DWRR,we have implemented it in Linux kernels 2.6.22.15 and 2.6.24, which represent two vastly different scheduler designs. Our evaluation shows that DWRR achieves accurate proportional fairness and high performance for a diverse set of workloads.

References

  1. S. K. Baruah, N. K. Cohen, C. G. Plaxton, and D. A. Varvel. Proportionate progress: A notion of fairness in resource allocation. Algorithmica, 15(6):600--625, June 1996.Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. S. K. Baruah, J. E. Gehrke, C. G. Plaxton, I. Stoica, H. Abdel-Wahab, and K. Jeffay. Fair on-line scheduling of a dynamic set of tasks on a single resource. Information Processing Letters, 64(1):43--51, Oct. 1997.Google ScholarGoogle ScholarCross RefCross Ref
  3. J. C. R. Bennett and H. Zhang. WF 2 Q: Worst-case fair weighted fair queueing. In Proceedings of IEEE INFOCOM '96, pages 120--128, Mar. 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. J. M. Blanquer and B. Özden. Fair queuing for aggregated multiple links. In Proceedings of ACM SIGCOMM '01, pages 189--197, Aug. 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. J. Bruno, E. Gabber, B. Özden, and A. Silberschatz. The Eclipse operating system: Providing quality of service via reservation domains. In Proceedings of the 1998 USENIX Annual Technical Conference, pages 235--246, June 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. B. Caprita, W. C. Chan, J. Nieh, C. Stein, and H. Zheng. Group ratio round-robin: O(1) proportional share scheduling for uniprocessor and multiprocessor systems. In Proceedings of the 2005 USENIX Annual Technical Conference, pages 337--352, Apr. 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. B. Caprita, J. Nieh, and C. Stein. Grouped distributed queues: Distributed queue, proportional share multiprocessor scheduling. In Proceedings of the 25th ACM Symposium on Principles of Distributed Computing, pages 72--81, July 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. A. Chandra and P. Shenoy. Hierarchical scheduling for symmetric multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 19(3):418--431, Mar. 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. A. Chandra, M. Adler, P. Goyal, and P. Shenoy. Surplus fair scheduling: A proportional-share CPU scheduling algorithm for symmetric multiprocessors. In Proceedings of the 4th Symposium on Operating Systems Design and Implementation, pages 45--58, Oct. 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. A. Demers, S. Keshav, and S. Shenker. Analysis and simulation of a fair queueing algorithm. In Proceedings of ACM SIGCOMM '89, pages 1--12, Sept. 1989. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. K. J. Duda and D. R. Cheriton. Borrowed-virtual-time (BVT) scheduling: Supporting latency-sensitive threads in a general-purpose scheduler. In Proceedings of the 17th ACM Symposium on Operating System Principles, pages 261--276, Dec. 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. D. H. J. Epema. Decay-usage scheduling in multiprocessors. ACM Transactions on Computer Systems, 16(4):367--415, Nov. 1998. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. S. J. Golestani. A self-clocked fair queueing scheme for broadband applications. In Proceedings of IEEE INFOCOM '94, pages 636--646, June 1994.Google ScholarGoogle ScholarCross RefCross Ref
  14. P. Goyal, X. Guo, and H. M. Vin. A hierarchical CPU scheduler for multimedia operating systems. In Proceedings of the Second Symposium on Operating Systems Design and Implementation, pages 107--121, Oct. 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. P. Goyal, H. M. Vin, and H. Cheng. Start-time fair queueing: A scheduling algorithm for integrated services packet switching networks. IEEE/ACM Transactions on Networking, 5(5): 690--704, Oct. 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. G. Greenberg and N. Madras. How fair is fair queuing. Journal of the ACM, 39(3):568--598, July 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. C. Guo. SRR: An O(1) time complexity packet scheduler for flows in multi-service packet networks. IEEE/ACM Transactions on Networking, 12(6):1144--1155, Dec. 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. J. L. Hellerstein. Achieving service rate objectives with decay usage scheduling. IEEE Transactions on Software Engineering, 19(8):813--825, Aug. 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. I. Leslie, D. McAuley, R. Black, T. Roscoe, P. Barham, D. Evers, R. Fairbairns, and E. Hyden. The design and implementation of an operating system to support distributed multimedia applications. IEEE Journal on Selected Areas in Communications, 14(7):1280--1297, Sept. 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. T. Li, D. Baumberger, D. A. Koufaty, and S. Hahn. Efficient operating system scheduling for performance-asymmetric multi-core architectures. In Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, Nov. 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. C. L. Liu and J. W. Layland. Scheduling algorithms for multiprogramming in a hard-real-time environment. Journal of the ACM, 20(1):46--61, Jan. 1973. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. C. W. Mercer, S. Savage, and H. Tokuda. Processor capacity reserves: Operating system support for multimedia applications. In Proceedings of the First IEEE International Conference on Multimedia Computing and Systems, pages 90--99, May 1994.Google ScholarGoogle ScholarCross RefCross Ref
  23. J. B. Nagle. On packet switches with infinite storage. IEEE Transactions on Communications, 35(4):435--438, Apr. 1987.Google ScholarGoogle ScholarCross RefCross Ref
  24. J. Nieh and M. S. Lam. A SMART scheduler for multimedia applications. ACM Transactions on Computer Systems, 21(2): 117--163, May 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. J. Nieh, C. Vaill, and H. Zhong. Virtual-time round-robin: An O(1) proportional share scheduler. In Proceedings of the 2001 USENIX Annual Technical Conference, pages 245--259, June 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. A. K. Parekh and R. G. Gallager. A generalized processor sharing approach to flow control in integrated services net-works: The single-node case. IEEE/ACM Transactions on Networking, 1(3):344--357, June 1993. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. D. Petrou, J. W. Milford, and G. A. Gibson. Implementing lottery scheduling: Matching the specializations in traditional schedulers. In Proceedings of the 1999 USENIX Annual Technical Conference, pages 1--14, June 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. M. Shreedhar and G. Varghese. Efficient fair queueing using deficit round robin. IEEE/ACM Transactions on Networking, 4(3):375--385, June 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. A. Srinivasan. Efficient and Flexible Fair Scheduling of Real-time Tasks on Multiprocessors. PhD thesis, University of North Carolina at Chapel Hill, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. D. C. Steere, A. Goel, J. Gruenberg, D. McNamee, C. Pu, and J. Walpole. A feedback-driven proportion allocator for real-rate scheduling. In Proceedings of the Third Symposium on Operating Systems Design and Implementation, pages 145--158, Feb. 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. I. Stoica, H. Abdel-Wahab, K. Jeffay, S. K. Baruah, J. E. Gehrke, and C. G. Plaxton. A proportional share resource allocation algorithm for real-time, time-shared systems. In Proceedings of the 17th IEEE Real-Time Systems Symposium, pages 288--299, Dec. 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. C. A. Waldspurger. Lottery and Stride Scheduling: Flexible Proportional-Share Resource Management. PhD thesis, Massachusetts Institute of Technology, Sept. 1995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. C. A. Waldspurger and W. E. Weihl. Lottery scheduling: Flexible proportional-share resource management. In Proceedings of the First Symposium on Operating Systems Design and Implementation, pages 1--12, Nov. 1994. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Efficient and scalable multiprocessor fair scheduling using distributed weighted round-robin

Recommendations

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Sign in

Full Access

  • Published in

    cover image ACM SIGPLAN Notices
    ACM SIGPLAN Notices  Volume 44, Issue 4
    PPoPP '09
    April 2009
    294 pages
    ISSN:0362-1340
    EISSN:1558-1160
    DOI:10.1145/1594835
    Issue’s Table of Contents
    • cover image ACM Conferences
      PPoPP '09: Proceedings of the 14th ACM SIGPLAN symposium on Principles and practice of parallel programming
      February 2009
      322 pages
      ISBN:9781605583976
      DOI:10.1145/1504176

    Copyright © 2009 ACM

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    • Published: 14 February 2009

    Check for updates

    Qualifiers

    • research-article

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader
About Cookies On This Site

We use cookies to ensure that we give you the best experience on our website.

Learn more

Got it!