skip to main content
research-article

Performance Analysis of Workload Dependent Load Balancing Policies

Published:19 June 2019Publication History
Skip Abstract Section

Abstract

Load balancing plays a crucial role in achieving low latency in large distributed systems. Recent load balancing strategies often rely on replication or use placeholders to further improve latency. However assessing the performance and stability of these strategies is challenging and is therefore often simulation based. In this paper we introduce a unified approach to analyze the performance and stability of a broad class of workload dependent load balancing strategies. This class includes many replication policies, such as replicate below threshold, delayed replication and replicate only small jobs, as well as strategies for fork-join systems. We consider systems with general job size distributions where jobs may experience server slowdown. We show that the equilibrium workload distribution of the cavity process satisfies a functional differential equation and conjecture that the cavity process captures the limiting behavior of the system as its size tends to infinity. We study this functional differential equation in more detail for a variety of load balancing policies and propose a numerical method to solve it. The numerical method relies on a fixed point iteration or a simple Euler iteration depending on the type of functional differential equation involved. We further show that additional simplifications can be made if certain distributions are assumed to be phase-type. Various numerical examples are included that validate the numerical method and illustrate its strength and flexibility.

References

  1. R. Aghajani, X. Li, and K. Ramanan. 2017. The PDE Method for the Analysis of Randomized Load Balancing Networks. Proc. ACM Meas. Anal. Comput. Syst., Vol. 1, 2, Article 38 (Dec. 2017), 28 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica. 2013. Effective Straggler Mitigation: Attack of the Clones. In NSDI, Vol. 13. 185--198. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. U Ayesta, T Bodas, JL Dorsman, and IM Verloop. 2019. A token-based central queue with order-independent service rates. arXiv preprint arXiv:1902.02137 (2019).Google ScholarGoogle Scholar
  4. U. Ayesta, T. Bodas, and I. M. Verloop. 2018. On a unifying product form framework for redundancy models. Performance Evaluation, Vol. 127 (2018), 93--119. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. M. Bramson, Y. Lu, and B. Prabhakar. 2010. Randomized load balancing with general service time distributions. In ACM SIGMETRICS 2010. 275--286. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. M. Bramson, Y. Lu, and B. Prabhakar. 2012. Asymptotic independence of queues under randomized load balancing. Queueing Syst., Vol. 71, 3 (2012), 247--292. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. S. Foss and A. L. Stolyar. 2017. Large-scale join-idle-queue system with general service times. Journal of Applied Probability, Vol. 54, 4 (2017), 995--1007.Google ScholarGoogle ScholarCross RefCross Ref
  8. K. Gardner, M. Harchol-Balter, A. Scheller-Wolf, and B. Van Houdt. 2017. A better model for job redundancy: Decoupling server slowdown and job size. IEEE/ACM Transactions on Networking, Vol. 25, 6 (2017), 3353--3367.Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. K. Gardner, M. Harchol-Balter, A. Scheller-Wolf, M. Velednitsky, and S. Zbarsky. 2017. Redundancy-d: The Power of d Choices for Redundancy. Operations Research, Vol. 65, 4 (2017), 1078--1094.Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. T. Hellemans and B. Van Houdt. 2018. Analysis of redundancy (d) with identical replicas. In Performance evaluation review. Vol. 46. 74--79. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. T. Hellemans and B. Van Houdt. 2018. On the Power-of-d-choices with Least Loaded Server Selection. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 2, 2 (2018), 27. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. G. Joshi, Y. Liu, and E. Soljanin. 2012. Coding for fast content download. In Communication, Control, and Computing (Allerton), 2012 50th Annual Allerton Conference on. IEEE, 326--333.Google ScholarGoogle Scholar
  13. G. Joshi, E. Soljanin, and G. Wornell. 2017. Efficient redundancy techniques for latency reduction in cloud systems. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), Vol. 2, 2 (2017), 12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. J. Kriege and P. Buchholz. 2014. PH and MAP Fitting with Aggregated Traffic Traces .Springer International Publishing, Cham, 1--15. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Y. Lu, Q. Xie, G. Kliot, A. Geller, J. R. Larus, and A. Greenberg. 2011. Join-Idle-Queue: A novel load balancing algorithm for dynamically scalable web services. Perform. Eval., Vol. 68 (2011), 1056--1071. Issue 11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. M. Mitzenmacher. 2001. The Power of Two Choices in Randomized Load Balancing. IEEE Trans. Parallel Distrib. Syst., Vol. 12 (October 2001), 1094--1104. Issue 10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. K. Ousterhout, P. Wendell, M. Zaharia, and I. Stoica. 2013. Sparrow: Distributed, Low Latency Scheduling. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP '13). ACM, New York, NY, USA, 69--84.Google ScholarGoogle Scholar
  18. A. Panchenko and A. Thümmler. 2007. Efficient Phase-type Fitting with Aggregated Traffic Traces. Perform. Eval., Vol. 64, 7--8 (Aug. 2007), 629--645. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. N. B Shah, K. Lee, and K. Ramchandran. 2016. When do redundant requests reduce latency? IEEE Transactions on Communications, Vol. 64, 2 (2016), 715--722.Google ScholarGoogle ScholarCross RefCross Ref
  20. V. Shah, A. Bouillard, and F. Baccelli. 2017. Delay comparison of delivery and coding policies in data clusters. In Communication, Control, and Computing (Allerton), 2017 55th Annual Allerton Conference on. IEEE, 397--404.Google ScholarGoogle Scholar
  21. A. L. Stolyar. 2015. Pull-based load distribution in large-scale heterogeneous service systems. Queueing Systems, Vol. 80, 4 (2015), 341--361. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. N.D. Vvedenskaya, R.L. Dobrushin, and F.I. Karpelevich. 1996. Queueing System with Selection of the Shortest of Two Queues: an Asymptotic Approach. Problemy Peredachi Informatsii, Vol. 32 (1996), 15--27.Google ScholarGoogle Scholar
  23. W. Wang, M. Harchol-Balter, H. Jiang, A. Scheller-Wolf, and R. Srikant. 2017. Delay Asymptotics and Bounds for Multi-Task Parallel Jobs. (2017). Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Performance Analysis of Workload Dependent Load Balancing Policies

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in

      Full Access

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader
      About Cookies On This Site

      We use cookies to ensure that we give you the best experience on our website.

      Learn more

      Got it!