Abstract
Load balancing systems, comprising a central dispatcher and a scheduling policy at each server, are widely used in practice, and their response time has been extensively studied in the theoretical literature. While much is known about the scenario where the scheduling at the servers is First-Come-First-Served (FCFS), to minimize mean response time we must use Shortest-Remaining-Processing-Time (SRPT) scheduling at the servers. Much less is known about dispatching polices when SRPT scheduling is used. Unfortunately, traditional dispatching policies that are used in practice in systems with FCFS servers often have poor performance in systems with SRPT servers. In this paper, we devise a simple fix that can be applied to any dispatching policy. This fix, called guardrails, ensures that the dispatching policy yields optimal mean response time under heavy traffic when used in a system with SRPT servers. Any dispatching policy, when augmented with guardrails, becomes heavy-traffic optimal. Our results yield the first analytical bounds on mean response time for load balancing systems with SRPT scheduling at the servers.
- Samuli Aalto, Urtzi Ayesta, and Rhonda Righter. 2009. On the Gittins index in the M/G/1 queue. Queueing Systems, Vol. 63, 1 (2009), 437--458. Google Scholar
Digital Library
- E. Altman, U. Ayesta, and B. J. Prabhu. 2011. Load balancing in processor sharing systems. Telecommunication Systems, Vol. 47, 1 (01 Jun 2011), 35--48. Google Scholar
Digital Library
- Nir Avrahami and Yossi Azar. 2003. Minimizing Total Flow Time and Total Completion Time with Immediate Dispatching. In Proceedings of the Fifteenth Annual ACM Symposium on Parallel Algorithms and Architectures (SPAA '03). ACM, New York, NY, USA, 11--18. Google Scholar
Digital Library
- Eitan Bachmat and Hagit Sarfati. 2008. Analysis of Size Interval Task Assignment Policies. SIGMETRICS Perform. Eval. Rev., Vol. 36, 2 (Aug. 2008), 107--109.Google Scholar
Digital Library
- T. Bonald, M. Jonckheere, and A. Proutiére. 2004. Insensitive Load Balancing. In Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '04/Performance '04). ACM, New York, NY, USA, 367--377. Google Scholar
Digital Library
- F. Bonomi. 1990. On job assignment for a parallel system of processor sharing queues. IEEE Trans. Comput., Vol. 39, 7 (July 1990), 858--869. Google Scholar
Digital Library
- Maury Bramson, Yi Lu, and Balaji Prabhakar. 2012. Asymptotic independence of queues under randomized load balancing. Queueing Systems, Vol. 71, 3 (01 Jul 2012), 247--292. Google Scholar
Digital Library
- Rodolpho G. de Siqueira and Daniel R. Figueiredo. 2017. A Control-based Load Balancing Algorithm with Flow Control for Dynamic and Heterogeneous Servers. In Anais do XXXV Simpósio Brasileiro de Redes de Computadores e Sistemas Distribuídos. SBC, Porto Alegre, RS, Brasil. http://portaldeconteudo.sbc.org.br/index.php/sbrc/article/view/2626Google Scholar
- Douglas G. Down and Rong Wu. 2006. Multi-layered round robin routing for parallel servers. Queueing Systems, Vol. 53, 4 (01 Aug 2006), 177--188. Google Scholar
Digital Library
- Hanhua Feng, Vishal Misra, and Dan Rubenstein. 2005. Optimal state-free, size-aware dispatching for heterogeneous M/G/-type systems. Performance Evaluation, Vol. 62, 1 (2005), 475 -- 492. Performance 2005. Google Scholar
Digital Library
- John C. Gittins, Kevin D. Glazebrook, and Richard Weber. 2011. Multi-armed Bandit Allocation Indices .John Wiley & Sons.Google Scholar
- Varun Gupta, Mor Harchol Balter, Karl Sigman, and Ward Whitt. 2007. Analysis of join-the-shortest-queue routing for web server farms. Performance Evaluation, Vol. 64, 9 (2007), 1062--1081. Performance 2007. Google Scholar
Digital Library
- Mor Harchol-Balter. 2013. Performance Modeling and Design of Computer Systems: Queueing Theory in Action 1st ed.). Cambridge University Press, New York, NY, USA.Google Scholar
Digital Library
- Mor Harchol-Balter, Mark E. Crovella, and Cristina D. Murta. 1999. On Choosing a Task Assignment Policy for a Distributed Server System. J. Parallel and Distrib. Comput., Vol. 59, 2 (1999), 204--228. Google Scholar
Digital Library
- Mor Harchol-Balter, Alan Scheller-Wolf, and Andrew R. Young. 2009. Surprising Results on Task Assignment in Server Farms with High-variability Workloads. In Proceedings of the Eleventh International Joint Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '09). ACM, New York, NY, USA, 287--298. Google Scholar
Digital Library
- Mor Harchol-Balter, Bianca Schroeder, Nikhil Bansal, and Mukesh Agrawal. 2003. Size-based Scheduling to Improve Web Performance. ACM Trans. Comput. Syst., Vol. 21, 2 (May 2003), 207--233. Google Scholar
Digital Library
- Esa Hyytiä, Aleksi Penttinen, and Samuli Aalto. 2012. Size- and state-aware dispatching problem with queue-specific job sizes. European Journal of Operational Research, Vol. 217, 2 (2012), 357 -- 370.Google Scholar
Cross Ref
- Stefano Leonardi and Danny Raz. 2007. Approximating total flow time on parallel machines. J. Comput. System Sci., Vol. 73, 6 (2007), 875--891. Google Scholar
Digital Library
- Quan-Lin Li, John C. S. Lui, and Yang Wang. 2011. A Matrix-Analytic Solution for Randomized Load Balancing Models with PH Service Times .Springer Berlin Heidelberg, Berlin, Heidelberg, 240--253.Google Scholar
- Minghong Lin, Adam Wierman, and Bert Zwart. 2011. Heavy-traffic analysis of mean response time under Shortest Remaining Processing Time. Performance Evaluation (2011).Google Scholar
- Zhen Liu and Rhonda Righter. 1998. Optimal Load Balancing on Distributed Homogeneous Unreliable Processors. Operations Research, Vol. 46, 4 (1998), 563--573. ://www.jstor.org/stable/168596 Google Scholar
Digital Library
- Richard R. Weber. 1978. On the optimal assignment of customers to parallel servers. Journal of Applied Probability, Vol. 15, 2 (1978), 406--413.Google Scholar
Cross Ref
- Adam Wierman and Mor Harchol-Balter. 2003. Classifying scheduling policies with respect to unfairness in an M/GI/1. In ACM SIGMETRICS Performance Evaluation Review, Vol. 31. ACM, 238--249. Google Scholar
Digital Library
- Adam Wierman, Mor Harchol-Balter, and Takayuki Osogami. 2005. Nearly insensitive bounds on SMART scheduling. In ACM SIGMETRICS Performance Evaluation Review, Vol. 33. ACM, 205--216. Google Scholar
Digital Library
- Wayne Winston. 1977. Optimality of the shortest line discipline. Journal of Applied Probability, Vol. 14, 1 (1977), 181--189.Google Scholar
Cross Ref
- Ronald W. Wolff. 1982. Poisson arrivals see time averages. Operations Research, Vol. 30, 2 (1982), 223--231. Google Scholar
Digital Library
- Xingyu Zhou, Jian Tan, and Ness Shroff. 2018. Flexible load balancing with multi-dimensional state-space collapse: Throughput and heavy-traffic delay optimality. Performance Evaluation, Vol. 127--128 (2018), 176 -- 193.Google Scholar
Cross Ref
Index Terms
Load Balancing Guardrails: Keeping Your Heavy Traffic on the Road to Low Response Times
Recommendations
The Gittins Policy is Nearly Optimal in the M/G/k under Extremely General Conditions
POMACSThe Gittins scheduling policy minimizes the mean response in the single-server M/G/1 queue in a wide variety of settings. Most famously, Gittins is optimal when preemption is allowed and service requirements are unknown but drawn from a known ...
Load Balancing Guardrails: Keeping Your Heavy Traffic on the Road to Low Response Times
SIGMETRICS '19: Abstracts of the 2019 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer SystemsLoad balancing systems, comprising a central dispatcher and a scheduling policy at each server, are widely used in practice, and their response time has been extensively studied in the theoretical literature. While much is known about the scenario where ...
Load Balancing Guardrails: Keeping Your Heavy Traffic on the Road to Low Response Times
Load balancing systems, comprising a central dispatcher and a scheduling policy at each server, are widely used in practice, and their response time has been extensively studied in the theoretical literature. While much is known about the scenario where ...






Comments