skip to main content
research-article

Mean Waiting Time in Large-Scale and Critically Loaded Power of d Load Balancing Systems

Published:04 June 2021Publication History
Skip Abstract Section

Abstract

Mean field models are a popular tool used to analyse load balancing policies. In some exceptional cases the waiting time distribution of the mean field limit has an explicit form. In other cases it can be computed as the solution of a set of differential equations. In this paper we study the limit of the mean waiting time E[Wλ] as the arrival rate λ approaches 1 for a number of load balancing policies in a large-scale system of homogeneous servers which finish work at a constant rate equal to one and exponential job sizes with mean 1 (i.e. when the system gets close to instability). As E[Wλ] diverges to infinity, we scale with -log(1-λ) and present a method to compute the limit limλ-> 1- -E[Wλ]/l(1-λ). We show that this limit has a surprisingly simple form for the load balancing algorithms considered. More specifically, we present a general result that holds for any policy for which the associated differential equation satisfies a list of assumptions. For the well-known LL(d) policy which assigns an incoming job to a server with the least work left among d randomly selected servers these assumptions are trivially verified. For this policy we prove the limit is given by 1/d-1. We further show that the LL(d,K) policy, which assigns batches of K jobs to the K least loaded servers among d randomly selected servers, satisfies the assumptions and the limit is equal to K/d-K. For a policy which applies LL(di) with probability pi, we show that the limit is given by 1/ ∑i pi di - 1. We further indicate that our main result can also be used for load balancers with redundancy or memory. In addition, we propose an alternate scaling -l(pλ) instead of -l(1-λ), where pλ is adapted to the policy at hand, such that limλ-> 1- -E[Wλ]/l(1-λ)=limλ-> 1- -E[Wλ]/l(pλ), where the limit limλ-> 0+ -E[Wλ]/l(pλ) is well defined and non-zero (contrary to limλ-> 0+ -E[Wλ]/l(1-λ)). This allows to obtain relatively flat curves for -E[Wλ]/l(pλ) for λ ∈ [0,1] which indicates that the low and high load limits can be used as an approximation when λ is close to one or zero. Our results rely on the earlier proven ansatz which asserts that for certain load balancing policies the workload distribution of any finite set of queues becomes independent of one another as the number of servers tends to infinity.

References

  1. Elene Anton, Urtzi Ayesta, Matthieu Jonckheere, and Ina Maria Verloop. 2019. On the stability of redundancy models. arXiv preprint arXiv:1903.04414 (2019).Google ScholarGoogle Scholar
  2. Michel Benaim and Jean-Yves Le Boudec. 2008. A class of mean field interaction models for computer and communication systems. Performance evaluation, Vol. 65, 11--12 (2008), 823--838.Google ScholarGoogle Scholar
  3. D. Blackwell. 1951. The range of certain vector integrals. Proc. Amer. Math. Soc., Vol. 2, 3 (1951), 390--395.Google ScholarGoogle ScholarCross RefCross Ref
  4. J. Borcea. 2007. Equilibrium points of logarithmic potentials induced by positive charge distributions. I. Generalized de Bruijn-Springer relations. Trans. Amer. Math. Soc., Vol. 359, 7 (2007), 3209--3237.Google ScholarGoogle ScholarCross RefCross Ref
  5. M. Bramson, Y. Lu, and B. Prabhakar. 2010. Randomized load balancing with general service time distributions. In ACM SIGMETRICS 2010. 275--286. https://doi.org/10.1145/1811039.1811071Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. M. Bramson, Y. Lu, and B. Prabhakar. 2013. Decay of tails at equilibrium for FIFO join the shortest queue networks. Ann. Appl. Probab., Vol. 23, 5 (10 2013), 1841--1878. https://doi.org/10.1214/12-AAP888Google ScholarGoogle Scholar
  7. G. Brightwell and M. Luczak. 2012. The supermarket model with arrival rate tending to one. arXiv preprint arXiv:1201.5523 (2012).Google ScholarGoogle Scholar
  8. Qihui Bu, Liwei Liu, Jiashan Tang, and Yiqiang Q Zhao. 2020. Approximations for a Queueing Game Model with Join-the-Shortest-Queue Strategy. arXiv preprint arXiv:2012.14955 (2020).Google ScholarGoogle Scholar
  9. P. Eschenfeldt and D. Gamarnik. 2018. Join the shortest queue with many servers. The heavy-traffic asymptotics. Mathematics of Operations Research, Vol. 43, 3 (2018), 867--886.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Kristen Gardner, Samuel Zbarsky, Mor Harchol-Balter, and Alan Scheller-Wolf. 2016. The power of d choices for redundancy. In Proceedings of the 2016 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Science. 409--410.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. T. Hellemans, T. Bodas, and B. Van Houdt. 2019. Performance Analysis of Workload Dependent Load Balancing Policies. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 3, 2 (2019), 35.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. T. Hellemans and B. Van Houdt. 2018. On the power-of-d-choices with least loaded server selection. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 2, 2 (2018), 1--22.Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. T Hellemans and B. Van Houdt. 2019. Performance of Redundancy (d) with Identical/Independent Replicas. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), Vol. 4, 2 (2019), 9.Google ScholarGoogle Scholar
  14. Tim Hellemans and Benny Van Houdt. 2020. Performance analysis of load balancing policies with memory. In Proceedings of the 13th EAI International Conference on Performance Evaluation Methodologies and Tools. 27--34.Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Rooji Jinan, Ajay Badita, Tejas Bodas, and Parimal Parag. 2020. Load balancing policies with server-side cancellation of replicas. arXiv preprint arXiv:2010.13575 (2020).Google ScholarGoogle Scholar
  16. T. Kurtz. 1981. Approximation of population processes .Society for Industrial and Applied Mathematics.Google ScholarGoogle Scholar
  17. Xin Liu, Kang Gong, and Lei Ying. 2020. Steady-State Analysis of Load Balancing with Coxian-$2 $ Distributed Service Times. arXiv preprint arXiv:2005.09815 (2020).Google ScholarGoogle Scholar
  18. Xin Liu and Lei Ying. 2020. Steady-state analysis of load-balancing algorithms in the sub-Halfin--Whitt regime. Journal of Applied Probability, Vol. 57, 2 (2020), 578--596.Google ScholarGoogle ScholarCross RefCross Ref
  19. A. W. Marshall, I. Olkin, and B. C. Arnold. 1979. Inequalities: theory of majorization and its applications. Vol. 143. Springer.Google ScholarGoogle Scholar
  20. M. Mitzenmacher. 2001. The power of two choices in randomized load balancing. IEEE Transactions on Parallel and Distributed Systems, Vol. 12, 10 (2001), 1094--1104.Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. K. Ousterhout, P. Wendell, M. Zaharia, and I. Stoica. 2013. Sparrow: distributed, low latency scheduling. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles. ACM, 69--84.Google ScholarGoogle Scholar
  22. Seva Shneer and Alexander Stolyar. 2020. Large-scale parallel server system with multi-component jobs. arXiv preprint arXiv:2006.11256 (2020).Google ScholarGoogle Scholar
  23. B. Van Houdt. 2019. Global attraction of ODE-based mean field models with hyperexponential job sizes. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 3, 2 (2019), 23.Google ScholarGoogle Scholar
  24. N.D. Vvedenskaya, R.L. Dobrushin, and F.I. Karpelevich. 1996. Queueing System with Selection of the Shortest of Two Queues: an Asymptotic Approach. Problemy Peredachi Informatsii, Vol. 32 (1996), 15--27.Google ScholarGoogle Scholar
  25. H. Wadsworth Gould. 1972. Combinatorial Identities: A standardized set of tables listing 500 binomial coefficient summations. Morgantown, W Va.Google ScholarGoogle Scholar
  26. Lei Ying, Rayadurgam Srikant, and Xiaohan Kang. 2017. The power of slightly more than one sample in randomized load balancing. Mathematics of Operations Research, Vol. 42, 3 (2017), 692--722.Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Mean Waiting Time in Large-Scale and Critically Loaded Power of d Load Balancing Systems

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!