Abstract
Load balancing plays a crucial role in achieving low latency in large distributed systems. Recent load balancing strategies often rely on replication or use placeholders to further improve latency. However assessing the performance and stability of these strategies is challenging and is therefore often simulation based. In this paper we introduce a unified approach to analyze the performance and stability of a broad class of workload dependent load balancing strategies. This class includes many replication policies, such as replicate below threshold, delayed replication and replicate only small jobs, as well as strategies for fork-join systems. We consider systems with general job size distributions where jobs may experience server slowdown. We show that the equilibrium workload distribution of the cavity process satisfies a functional differential equation and conjecture that the cavity process captures the limiting behavior of the system as its size tends to infinity. We study this functional differential equation in more detail for a variety of load balancing policies and propose a numerical method to solve it. The numerical method relies on a fixed point iteration or a simple Euler iteration depending on the type of functional differential equation involved. We further show that additional simplifications can be made if certain distributions are assumed to be phase-type. Various numerical examples are included that validate the numerical method and illustrate its strength and flexibility.
- R. Aghajani, X. Li, and K. Ramanan. 2017. The PDE Method for the Analysis of Randomized Load Balancing Networks. Proc. ACM Meas. Anal. Comput. Syst., Vol. 1, 2, Article 38 (Dec. 2017), 28 pages. Google Scholar
Digital Library
- G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica. 2013. Effective Straggler Mitigation: Attack of the Clones. In NSDI, Vol. 13. 185--198. Google Scholar
Digital Library
- U Ayesta, T Bodas, JL Dorsman, and IM Verloop. 2019. A token-based central queue with order-independent service rates. arXiv preprint arXiv:1902.02137 (2019).Google Scholar
- U. Ayesta, T. Bodas, and I. M. Verloop. 2018. On a unifying product form framework for redundancy models. Performance Evaluation, Vol. 127 (2018), 93--119. Google Scholar
Digital Library
- M. Bramson, Y. Lu, and B. Prabhakar. 2010. Randomized load balancing with general service time distributions. In ACM SIGMETRICS 2010. 275--286. Google Scholar
Digital Library
- M. Bramson, Y. Lu, and B. Prabhakar. 2012. Asymptotic independence of queues under randomized load balancing. Queueing Syst., Vol. 71, 3 (2012), 247--292. Google Scholar
Digital Library
- S. Foss and A. L. Stolyar. 2017. Large-scale join-idle-queue system with general service times. Journal of Applied Probability, Vol. 54, 4 (2017), 995--1007.Google Scholar
Cross Ref
- K. Gardner, M. Harchol-Balter, A. Scheller-Wolf, and B. Van Houdt. 2017. A better model for job redundancy: Decoupling server slowdown and job size. IEEE/ACM Transactions on Networking, Vol. 25, 6 (2017), 3353--3367.Google Scholar
Digital Library
- K. Gardner, M. Harchol-Balter, A. Scheller-Wolf, M. Velednitsky, and S. Zbarsky. 2017. Redundancy-d: The Power of d Choices for Redundancy. Operations Research, Vol. 65, 4 (2017), 1078--1094.Google Scholar
Digital Library
- T. Hellemans and B. Van Houdt. 2018. Analysis of redundancy (d) with identical replicas. In Performance evaluation review. Vol. 46. 74--79. Google Scholar
Digital Library
- T. Hellemans and B. Van Houdt. 2018. On the Power-of-d-choices with Least Loaded Server Selection. Proceedings of the ACM on Measurement and Analysis of Computing Systems, Vol. 2, 2 (2018), 27. Google Scholar
Digital Library
- G. Joshi, Y. Liu, and E. Soljanin. 2012. Coding for fast content download. In Communication, Control, and Computing (Allerton), 2012 50th Annual Allerton Conference on. IEEE, 326--333.Google Scholar
- G. Joshi, E. Soljanin, and G. Wornell. 2017. Efficient redundancy techniques for latency reduction in cloud systems. ACM Transactions on Modeling and Performance Evaluation of Computing Systems (TOMPECS), Vol. 2, 2 (2017), 12. Google Scholar
Digital Library
- J. Kriege and P. Buchholz. 2014. PH and MAP Fitting with Aggregated Traffic Traces .Springer International Publishing, Cham, 1--15. Google Scholar
Digital Library
- Y. Lu, Q. Xie, G. Kliot, A. Geller, J. R. Larus, and A. Greenberg. 2011. Join-Idle-Queue: A novel load balancing algorithm for dynamically scalable web services. Perform. Eval., Vol. 68 (2011), 1056--1071. Issue 11. Google Scholar
Digital Library
- M. Mitzenmacher. 2001. The Power of Two Choices in Randomized Load Balancing. IEEE Trans. Parallel Distrib. Syst., Vol. 12 (October 2001), 1094--1104. Issue 10. Google Scholar
Digital Library
- K. Ousterhout, P. Wendell, M. Zaharia, and I. Stoica. 2013. Sparrow: Distributed, Low Latency Scheduling. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP '13). ACM, New York, NY, USA, 69--84.Google Scholar
- A. Panchenko and A. Thümmler. 2007. Efficient Phase-type Fitting with Aggregated Traffic Traces. Perform. Eval., Vol. 64, 7--8 (Aug. 2007), 629--645. Google Scholar
Digital Library
- N. B Shah, K. Lee, and K. Ramchandran. 2016. When do redundant requests reduce latency? IEEE Transactions on Communications, Vol. 64, 2 (2016), 715--722.Google Scholar
Cross Ref
- V. Shah, A. Bouillard, and F. Baccelli. 2017. Delay comparison of delivery and coding policies in data clusters. In Communication, Control, and Computing (Allerton), 2017 55th Annual Allerton Conference on. IEEE, 397--404.Google Scholar
- A. L. Stolyar. 2015. Pull-based load distribution in large-scale heterogeneous service systems. Queueing Systems, Vol. 80, 4 (2015), 341--361. Google Scholar
Digital Library
- N.D. Vvedenskaya, R.L. Dobrushin, and F.I. Karpelevich. 1996. Queueing System with Selection of the Shortest of Two Queues: an Asymptotic Approach. Problemy Peredachi Informatsii, Vol. 32 (1996), 15--27.Google Scholar
- W. Wang, M. Harchol-Balter, H. Jiang, A. Scheller-Wolf, and R. Srikant. 2017. Delay Asymptotics and Bounds for Multi-Task Parallel Jobs. (2017). Google Scholar
Digital Library
Index Terms
Performance Analysis of Workload Dependent Load Balancing Policies
Recommendations
Performance Analysis of Workload Dependent Load Balancing Policies
SIGMETRICS '19: Abstracts of the 2019 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer SystemsLoad balancing plays a crucial role in achieving low latency in large distributed systems. Recent load balancing strategies often rely on replication or use placeholders to further improve latency. However assessing the performance and stability of ...
Performance Analysis of Workload Dependent Load Balancing Policies
Load balancing plays a crucial role in achieving low latency in large distributed systems. Recent load balancing strategies often rely on replication or use placeholders to further improve latency. However assessing the performance and stability of ...
Variable threshold-based hierarchical load balancing technique in Grid
Load balancing is an important aspect of Grid resource scheduling. This paper attempts to address the issue of load balancing in a Grid, while maintaining the resource utilization and response time for dynamic and decentralized Grid environment. Here, ...






Comments