skip to main content
research-article
Public Access

Heavy-traffic Delay Optimality in Pull-based Load Balancing Systems: Necessary and Sufficient Conditions

Published:21 December 2018Publication History
Skip Abstract Section

Abstract

In this paper, we consider a load balancing system under a general pull-based policy. In particular, each arrival is randomly dispatched to one of the servers with queue length below a threshold; if none exists, this arrival is randomly dispatched to one of the entire set of servers. We are interested in the fundamental relationship between the threshold and the delay performance of the system in heavy traffic. To this end, we first establish the following necessary condition to guarantee heavy-traffic delay optimality: the threshold will grow to infinity as the exogenous arrival rate approaches the boundary of the capacity region (i.e., the load intensity approaches one) but the growth rate should be slower than a polynomial function of the mean number of tasks in the system. As a special case of this result, we directly show that the delay performance of the popular pull-based policy Join-Idle-Queue (JIQ) lies strictly between that of any heavy-traffic delay optimal policy and that of random routing. We further show that a sufficient condition for heavy-traffic delay optimality is that the threshold grows logarithmically with the mean number of tasks in the system. This result directly resolves a generalized version of the conjecture by Kelly and Laws.

References

  1. Mor Armony. 2005. Dynamic routing in large-scale service systems with heterogeneous servers. Queueing Systems 51, 3--4 (2005), 287--329. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Steven L Bell and Ruth J Williams. 2001. Dynamic scheduling of a system with two parallel servers in heavy traffic with resource pooling: asymptotic optimality of a threshold policy. Annals of Applied Probability (2001), 608--649.Google ScholarGoogle Scholar
  3. Maury Bramson. 1998. State space collapse with application to heavy traffic limits for multiclass queueing networks. Queueing Systems 30, 1--2 (1998), 89--140. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Amarjit Budhiraja and Chihoon Lee. 2009. Stationary distribution convergence for generalized Jackson networks in heavy traffic. Mathematics of Operations Research 34, 1 (2009), 45--56. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Hong Chen and Heng-Qing Ye. 2012. Asymptotic optimality of balanced routing. Operations research 60, 1 (2012), 163--179. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. JG Dai and Tolga Tezcan. 2011. State space collapse in many-server diffusion limits of parallel server systems. Mathematics of Operations Research 36, 2 (2011), 271--320. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Atilla Eryilmaz and R Srikant. 2012. Asymptotically tight steady-state queue length bounds implied by drift conditions. Queueing Systems 72, 3--4 (2012), 311--359. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. G Foschini and J. Salz. 1978. A basic dynamic routing problem and diffusion. IEEE Transactions on Communications 26, 3 (1978), 320--327.Google ScholarGoogle ScholarCross RefCross Ref
  9. David Gamarnik and Assaf Zeevi. 2006. Validity of heavy traffic steady-state approximations in generalized Jackson networks. The Annals of Applied Probability 16, 1 (2006), 56--90.Google ScholarGoogle ScholarCross RefCross Ref
  10. Itay Gurvich and Ward Whitt. 2009. Queue-and-idleness-ratio controls in many-server service systems. Mathematics of Operations Research 34, 2 (2009), 363--396. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Bruce Hajek. 1982. Hitting-time and occupation-time bounds implied by drift analysis with applications. Advances in Applied probability (1982), 502--525.Google ScholarGoogle Scholar
  12. Shlomo Halfin and Ward Whitt. 1981. Heavy-traffic limits for queues with many exponential servers. Operations research 29, 3 (1981), 567--588. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Zhang Hanqin and Wang Rongxin. 1989. Heavy traffic limit theorems for a queueing system in which customers join the shortest line. Advances in Applied Probability 21, 2 (1989), 451--469.Google ScholarGoogle ScholarCross RefCross Ref
  14. J Michael Harrison. 1998. Heavy traffic analysis of a system with parallel servers: asymptotic optimality of discretereview policies. Annals of applied probability (1998), 822--848.Google ScholarGoogle Scholar
  15. FP Kelly and CN Laws. 1993. Dynamic routing in open queueing networks: Brownian models, cut constraints and resource pooling. Queueing systems 13, 1--3 (1993), 47--86.Google ScholarGoogle Scholar
  16. Yi Lu, Qiaomin Xie, Gabriel Kliot, Alan Geller, James R Larus, and Albert Greenberg. 2011. Join-Idle-Queue: A novel load balancing algorithm for dynamically scalable web services. Performance Evaluation 68, 11 (2011), 1056--1071. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Siva Theja Maguluri, Sai Kiran Burle, and R Srikant. 2018. Optimal heavy-traffic queue length scaling in an incompletely saturated switch. Queueing Systems 88, 3--4 (2018), 279--309. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Siva Theja Maguluri and R. Srikant. 2016. Heavy traffic queue length behavior in a switch under the MaxWeight algorithm. Stochastic Systems 6, 1 (2016), 211--250.Google ScholarGoogle ScholarCross RefCross Ref
  19. Siva Theja Maguluri, R Srikant, and Lei Ying. 2014. Heavy traffic optimal resource allocation algorithms for cloud computing clusters. Performance Evaluation 81 (2014), 20--39. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Debankur Mukherjee, Sem C Borst, Johan SH Van Leeuwaarden, and Philip A Whiting. 2016. Universality of load balancing schemes on the diffusion scale. Journal of Applied Probability 53, 4 (2016), 1111--1124.Google ScholarGoogle ScholarCross RefCross Ref
  21. Martin I Reiman. 1984. Some diffusion approximations with state space collapse. In Modelling and performance evaluation methodology. Springer, 207--240.Google ScholarGoogle Scholar
  22. Alexander L Stolyar. 2015. Pull-based load distribution in large-scale heterogeneous service systems. Queueing Systems 80, 4 (2015), 341--361. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Yih-Choung Teh and Amy R Ward. 2002. Critical thresholds for dynamic routing in queueing networks. Queueing Systems 42, 3 (2002), 297--316. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Weina Wang, Siva Theja Maguluri, R Srikant, and Lei Ying. 2018. Heavy-traffic delay insensitivity in connection-level models of data transfer with proportionally fair bandwidth sharing. ACM SIGMETRICS Performance Evaluation Review 45, 2 (2018), 232--245. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Weina Wang, Kai Zhu, Lei Ying, Jian Tan, and Li Zhang. 2016. MapTask scheduling in MapReduce with data locality: Throughput and heavy-traffic optimality. IEEE/ACM Transactions on Networking 24, 1 (2016), 190--203. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Ruth J Williams. 1998. Diffusion approximations for open multiclass queueing networks: sufficient conditions involving state space collapse. Queueing systems 30, 1 (1998), 27--88. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Qiaomin Xie and Yi Lu. 2015. Priority algorithm for near-data scheduling: Throughput and heavy-traffic optimality. In Proceedings of IEEE International Conference on Computer Communications (INFOCOM). 963--972.Google ScholarGoogle ScholarCross RefCross Ref
  28. Qiaomin Xie, Ali Yekkehkhany, and Yi Lu. 2016. Scheduling with multi-level data locality: Throughput and heavy-traffic optimality. In Proceedings of IEEE International Conference on Computer Communications (INFOCOM). 1--9.Google ScholarGoogle ScholarCross RefCross Ref
  29. Xingyu Zhou, Jian Tan, and Ness Shroff. 2018. Flexible Load Balancing with Multi-dimensional State-space Collapse: Throughput and Heavy-traffic Delay Optimality. arXiv preprint arXiv:1806.02939 (2018).Google ScholarGoogle Scholar
  30. Xingyu Zhou, Fei Wu, Jian Tan, Yin Sun, and Ness Shroff. 2017. Designing Low-Complexity Heavy-Traffic Delay- Optimal Load Balancing Schemes: Theory to Algorithms. Proceedings of the ACM on Measurement and Analysis of Computing Systems 1, 2 (2017), 39. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Heavy-traffic Delay Optimality in Pull-based Load Balancing Systems: Necessary and Sufficient Conditions

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader
        About Cookies On This Site

        We use cookies to ensure that we give you the best experience on our website.

        Learn more

        Got it!