skip to main content
research-article
Open Access

Load Balancing Guardrails: Keeping Your Heavy Traffic on the Road to Low Response Times

Published:19 June 2019Publication History
Skip Abstract Section

Abstract

Load balancing systems, comprising a central dispatcher and a scheduling policy at each server, are widely used in practice, and their response time has been extensively studied in the theoretical literature. While much is known about the scenario where the scheduling at the servers is First-Come-First-Served (FCFS), to minimize mean response time we must use Shortest-Remaining-Processing-Time (SRPT) scheduling at the servers. Much less is known about dispatching polices when SRPT scheduling is used. Unfortunately, traditional dispatching policies that are used in practice in systems with FCFS servers often have poor performance in systems with SRPT servers. In this paper, we devise a simple fix that can be applied to any dispatching policy. This fix, called guardrails, ensures that the dispatching policy yields optimal mean response time under heavy traffic when used in a system with SRPT servers. Any dispatching policy, when augmented with guardrails, becomes heavy-traffic optimal. Our results yield the first analytical bounds on mean response time for load balancing systems with SRPT scheduling at the servers.

References

  1. Samuli Aalto, Urtzi Ayesta, and Rhonda Righter. 2009. On the Gittins index in the M/G/1 queue. Queueing Systems, Vol. 63, 1 (2009), 437--458. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. E. Altman, U. Ayesta, and B. J. Prabhu. 2011. Load balancing in processor sharing systems. Telecommunication Systems, Vol. 47, 1 (01 Jun 2011), 35--48. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Nir Avrahami and Yossi Azar. 2003. Minimizing Total Flow Time and Total Completion Time with Immediate Dispatching. In Proceedings of the Fifteenth Annual ACM Symposium on Parallel Algorithms and Architectures (SPAA '03). ACM, New York, NY, USA, 11--18. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Eitan Bachmat and Hagit Sarfati. 2008. Analysis of Size Interval Task Assignment Policies. SIGMETRICS Perform. Eval. Rev., Vol. 36, 2 (Aug. 2008), 107--109.Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. T. Bonald, M. Jonckheere, and A. Proutiére. 2004. Insensitive Load Balancing. In Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '04/Performance '04). ACM, New York, NY, USA, 367--377. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. F. Bonomi. 1990. On job assignment for a parallel system of processor sharing queues. IEEE Trans. Comput., Vol. 39, 7 (July 1990), 858--869. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Maury Bramson, Yi Lu, and Balaji Prabhakar. 2012. Asymptotic independence of queues under randomized load balancing. Queueing Systems, Vol. 71, 3 (01 Jul 2012), 247--292. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Rodolpho G. de Siqueira and Daniel R. Figueiredo. 2017. A Control-based Load Balancing Algorithm with Flow Control for Dynamic and Heterogeneous Servers. In Anais do XXXV Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos. SBC, Porto Alegre, RS, Brasil. http://portaldeconteudo.sbc.org.br/index.php/sbrc/article/view/2626Google ScholarGoogle Scholar
  9. Douglas G. Down and Rong Wu. 2006. Multi-layered round robin routing for parallel servers. Queueing Systems, Vol. 53, 4 (01 Aug 2006), 177--188. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Hanhua Feng, Vishal Misra, and Dan Rubenstein. 2005. Optimal state-free, size-aware dispatching for heterogeneous M/G/-type systems. Performance Evaluation, Vol. 62, 1 (2005), 475 -- 492. Performance 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. John C. Gittins, Kevin D. Glazebrook, and Richard Weber. 2011. Multi-armed Bandit Allocation Indices .John Wiley & Sons.Google ScholarGoogle Scholar
  12. Varun Gupta, Mor Harchol Balter, Karl Sigman, and Ward Whitt. 2007. Analysis of join-the-shortest-queue routing for web server farms. Performance Evaluation, Vol. 64, 9 (2007), 1062--1081. Performance 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Mor Harchol-Balter. 2013. Performance Modeling and Design of Computer Systems: Queueing Theory in Action 1st ed.). Cambridge University Press, New York, NY, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Mor Harchol-Balter, Mark E. Crovella, and Cristina D. Murta. 1999. On Choosing a Task Assignment Policy for a Distributed Server System. J. Parallel and Distrib. Comput., Vol. 59, 2 (1999), 204--228. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Mor Harchol-Balter, Alan Scheller-Wolf, and Andrew R. Young. 2009. Surprising Results on Task Assignment in Server Farms with High-variability Workloads. In Proceedings of the Eleventh International Joint Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '09). ACM, New York, NY, USA, 287--298. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Mor Harchol-Balter, Bianca Schroeder, Nikhil Bansal, and Mukesh Agrawal. 2003. Size-based Scheduling to Improve Web Performance. ACM Trans. Comput. Syst., Vol. 21, 2 (May 2003), 207--233. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Esa Hyytiä, Aleksi Penttinen, and Samuli Aalto. 2012. Size- and state-aware dispatching problem with queue-specific job sizes. European Journal of Operational Research, Vol. 217, 2 (2012), 357 -- 370.Google ScholarGoogle ScholarCross RefCross Ref
  18. Stefano Leonardi and Danny Raz. 2007. Approximating total flow time on parallel machines. J. Comput. System Sci., Vol. 73, 6 (2007), 875--891. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Quan-Lin Li, John C. S. Lui, and Yang Wang. 2011. A Matrix-Analytic Solution for Randomized Load Balancing Models with PH Service Times .Springer Berlin Heidelberg, Berlin, Heidelberg, 240--253.Google ScholarGoogle Scholar
  20. Minghong Lin, Adam Wierman, and Bert Zwart. 2011. Heavy-traffic analysis of mean response time under Shortest Remaining Processing Time. Performance Evaluation (2011).Google ScholarGoogle Scholar
  21. Zhen Liu and Rhonda Righter. 1998. Optimal Load Balancing on Distributed Homogeneous Unreliable Processors. Operations Research, Vol. 46, 4 (1998), 563--573. ://www.jstor.org/stable/168596 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Richard R. Weber. 1978. On the optimal assignment of customers to parallel servers. Journal of Applied Probability, Vol. 15, 2 (1978), 406--413.Google ScholarGoogle ScholarCross RefCross Ref
  23. Adam Wierman and Mor Harchol-Balter. 2003. Classifying scheduling policies with respect to unfairness in an M/GI/1. In ACM SIGMETRICS Performance Evaluation Review, Vol. 31. ACM, 238--249. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Adam Wierman, Mor Harchol-Balter, and Takayuki Osogami. 2005. Nearly insensitive bounds on SMART scheduling. In ACM SIGMETRICS Performance Evaluation Review, Vol. 33. ACM, 205--216. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Wayne Winston. 1977. Optimality of the shortest line discipline. Journal of Applied Probability, Vol. 14, 1 (1977), 181--189.Google ScholarGoogle ScholarCross RefCross Ref
  26. Ronald W. Wolff. 1982. Poisson arrivals see time averages. Operations Research, Vol. 30, 2 (1982), 223--231. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Xingyu Zhou, Jian Tan, and Ness Shroff. 2018. Flexible load balancing with multi-dimensional state-space collapse: Throughput and heavy-traffic delay optimality. Performance Evaluation, Vol. 127--128 (2018), 176 -- 193.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Load Balancing Guardrails: Keeping Your Heavy Traffic on the Road to Low Response Times

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!