skip to main content
research-article

A Distributed Algorithm to Calculate Max-Min Fair Rates Without Per-Flow State

Published:19 June 2019Publication History
Skip Abstract Section

Abstract

Most congestion control algorithms, like TCP, rely on a reactive control system that detects congestion, then marches carefully towards a desired operating point (e.g. by modifying the window size or adjusting a rate). In an effort to balance stability and convergence speed, they often take hundreds of RTTs to converge; an increasing problem as networks get faster, with less time to react.

This paper is about an alternative class of congestion control algorithms based on proactive-scheduling: switches and NICs "pro-actively" exchange control messages to run a \em distributed algorithm to pick "explicit rates for each flow. We call these Proactive Explicit Rate Control (PERC) algorithms. They take as input the routing matrix and link speeds, but not a congestion signal. By exploiting information such as the number of flows at a link, they can converge an order of magnitude faster than reactive algorithms.

Our main contributions are (1) s-PERC ("stateless" PERC), a new practical distributed PERC algorithm without per-flow state at the switches, and (2) a proof that s-PERC computes exact max-min fair rates in a known bounded time, the first such algorithm to do so without per-flow state. To analyze s-PERC, we introduce a parallel variant of standard waterfilling, 2-Waterfilling. We prove that s-PERC converges to max-min fair in 6N rounds, where N is the number of iterations 2-Waterfilling takes for the same routing matrix.

We describe how to make s-PERC practical and robust to deploy in real networks. We confirm using realistic simulations and an FPGA hardware testbed that s-PERC converges 10-100x faster than reactive algorithms like TCP, DCTCP and RCP in data-center networks and 1.3--6x faster in wide-area networks (WANs). Long flows complete in close to the ideal time, while short-lived flows are prioritized, making it appropriate for data-centers and WANs.

References

  1. Yehuda Afek, Yishay Mansour, and Zvi Ostfeld. 1996. Phantom: A Simple and Effective Flow Control Scheme. In Conference Proceedings on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM '96). ACM, New York, NY, USA, 169--182. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Yehuda Afek, Yishay Mansour, and Zvi Ostfeld. 1999. Convergence Complexity of Optimistic Rate-Based Flow-Control Algorithms. Journal of Algorithms, Vol. 30, 1 (Jan. 1999), 106--143. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Mohammad Alizadeh, Albert Greenberg, David A. Maltz, Jitendra Padhye, Parveen Patel, Balaji Prabhakar, Sudipta Sengupta, and Murari Sridharan. 2010. Data Center TCP (DCTCP). In Proceedings of the SIGCOMM 2010 Conference (SIGCOMM '10). ACM, New York, NY, USA, 63--74. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Mohammad Alizadeh, Shuang Yang, Milad Sharif, Sachin Katti, Nick McKeown, Balaji Prabhakar, and Scott Shenker. 2013. pFabric: Minimal Near-optimal Datacenter Transport. In Proceedings of the ACM SIGCOMM 2013 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM '13). ACM, New York, NY, USA, 435--446. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Noga Alon, Yossi Matias, and Mario Szegedy. 1999. The Space Complexity of Approximating the Frequency Moments. J. Comput. System Sci., Vol. 58, 1 (Feb. 1999), 137--147. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Baruch Awerbuch and Rohit Khandekar. 2007. Greedy Distributed Optimization of Multi-commodity Flows. In Proceedings of the Twenty-sixth Annual ACM Symposium on Principles of Distributed Computing (PODC '07). ACM, New York, NY, USA, 274--283. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. B. Awerbuch and Y. Shavitt. 1998. Converging to approximated max-min flow fairness in logarithmic time. In Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98, Vol. 3. 1350--1357 vol.3.Google ScholarGoogle Scholar
  8. Wei Bai, Li Chen, Kai Chen, Dongsu Han, Chen Tian, and Hao Wang. 2017. PIAS: Practical Information-Agnostic Flow Scheduling for Commodity Data Centers. IEEE/ACM Transactions on Networking, Vol. 25, 4 (Aug. 2017), 1954--1967. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Amotz Bar-Noy, Magnús M Halldórsson, Guy Kortsarz, Ravit Salman, and Hadas Shachnai. 2000. Sum multicoloring of graphs. Journal of Algorithms, Vol. 37, 2 (2000), 422--450. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Yair Bartal, Martin Farach-Colton, Shibu Yooseph, and Lisa Zhang. 2002. Fast, fair and frugal bandwidth allocation in atm networks. Algorithmica, Vol. 33, 3 (2002), 272--286.Google ScholarGoogle ScholarCross RefCross Ref
  11. Dimitri Bertsekas and Robert Gallager. 1987. Data Networks .Prentice-Hall, Inc., Upper Saddle River, NJ, USA.Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Anna Charny, David D Clark, and Raj Jain. 1995. Congestion control with explicit rate indication. Communications, 1995. ICC'95 Seattle,'Gateway to Globalization', 1995 IEEE International Conference on, Vol. 3. IEEE, 1954--1963.Google ScholarGoogle ScholarCross RefCross Ref
  13. Anna Charny, KK Ramakrishnan, and Anthony Lauck. 1996. Time scale analysis scalability issues for explicit rate allocation in ATM networks. IEEE/ACM Transactions on Networking, Vol. 4, 4 (1996), 569--581. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Inho Cho, Keon Jang, and Dongsu Han. 2017. Credit-Scheduled Delay-Bounded Congestion Control for Datacenters. In Proceedings of the 2017 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '17). ACM, New York, NY, USA, 239--252. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Jorge A. Cobb and Mohamed G. Gouda. 2011. Stabilization of Max-min Fair Networks Without Per-flow State. Theoretical Computer Science, Vol. 412, 40 (Sept. 2011), 5562--5579.Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. A. Demers, S. Keshav, and S. Shenker. 1989. Analysis and Simulation of a Fair Queueing Algorithm. In Symposium Proceedings on Communications Architectures &Amp; Protocols (SIGCOMM '89). ACM, New York, NY, USA, 1--12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Nandita Dukkipati. 2008. Rate Control Protocol (Rcp): Congestion Control to Make Flows Complete Quickly. Ph.D. Dissertation. Stanford University, Stanford, CA, USA. Advisor(s) Mckeown, Nick. AAI3292347. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Paul Emmerich, Sebastian Gallenmüller, Daniel Raumer, Florian Wohlfart, and Georg Carle. 2015. Moongen: A scriptable high-speed packet generator. Proceedings of the 2015 ACM Conference on Internet Measurement Conference. ACM, 275--287. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Eli Gafni and Dimitri Bertsekas. 1984. Dynamic control of session input rates in communication networks. IEEE Trans. Automat. Control, Vol. 29, 11 (1984), 1009--1016.Google ScholarGoogle ScholarCross RefCross Ref
  20. Dimitris Giannopoulos, Nikos Chrysos, Evangelos Mageiropoulos, Giannis Vardas, Leandros Tzanakis, and Manolis Katevenis. 2018. Accurate Congestion Control for RDMA Transfers. In 2018 Twelfth IEEE/ACM International Symposium on Networks-on-Chip (NOCS). IEEE, 1--8. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Mark Handley, Costin Raiciu, Alexandru Agache, Andrei Voinescu, Andrew W Moore, Gianni Antichi, and Marcin Wójcik. 2017. Re-architecting datacenter networks and stacks for low latency and high performance. In Proceedings of the 2017 ACM Conference on Special Interest Group on Data Communication. ACM, 29--42. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Howard P Hayden. 1981. Voice flow control in integrated packet networks. Ph.D. Dissertation. Massachusetts Institute of Technology.Google ScholarGoogle Scholar
  23. Chi-Yao Hong, Matthew Caesar, and P. Brighten Godfrey. 2012. Finishing Flows Quickly with Preemptive Scheduling. In Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM '12). ACM, New York, NY, USA, 127--138. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Y. Thomas Hou, Shivendra S. Panwar, and Henry H. Y. Tzeng. 2004. On Generalized Max-Min Rate Allocation and Distributed Convergence Algorithm for Packet Networks. IEEE Trans. Parallel Distrib. Syst., Vol. 15, 5 (May 2004), 401--416.Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Jeffrey Jaffe. 1981. Bottleneck flow control. IEEE Transactions on Communications, Vol. 29, 7 (1981), 954--962.Google ScholarGoogle ScholarCross RefCross Ref
  26. Lavanya Jose, Lisa Yan, Mohammad Alizadeh, George Varghese, Nick McKeown, and Sachin Katti. 2015. High Speed Networks Need Proactive Congestion Control. In Proceedings of the 14th ACM Workshop on Hot Topics in Networks (HotNets-XIV). ACM, New York, NY, USA, Article 14, bibinfonumpages7 pages. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Shivkumar Kalyanaraman, Raj Jain, Sonia Fahmy, Rohit Goyal, and Bobby Vandalore. 2000. The ERICA switch algorithm for ABR traffic management in ATM networks. IEEE/ACM Transactions on Networking, Vol. 8, 1 (2000), 87--98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Dina Katabi, Mark Handley, and Charlie Rohrs. 2002. Congestion Control for High Bandwidth-delay Product Networks. In Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM '02). ACM, New York, NY, USA, 89--102. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Yuseok Kim, Wei Kang Tsai, Mahadevan Iyer, and Jordi Ros-Giralt. 1999. Minimum rate guarantee without per-flow information. In Network Protocols, 1999.(ICNP'99) Proceedings. Seventh International Conference on. IEEE, 155--162. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Alok Kumar, Sushant Jain, Uday Naik, Anand Raghuraman, Nikhil Kasinadhuni, Enrique Cauich Zermeno, C. Stephen Gunn, Jing Ai, Björn Carlin, Mihai Amarandei-Stavila, Mathieu Robin, Aspi Siganporia, Stephen Stuart, and Amin Vahdat. 2015. BwE: Flexible, Hierarchical Bandwidth Allocation for WAN Distributed Computing. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '15). ACM, New York, NY, USA, 1--14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Jean-Yves Le Boudec. 2000. Rate adaptation, congestion control and fairness: A tutorial. (2000).Google ScholarGoogle Scholar
  32. Tae-Jin Lee and G. De Veciana. 1998. A decentralized framework to achieve max-min fair bandwidth allocation for ATM networks. In IEEE GLOBECOM 1998 (Cat. NO. 98CH36250), Vol. 3. 1515--1520 vol.3.Google ScholarGoogle Scholar
  33. Jelena Marasevic, Cliff Stein, and Gil Zussman. 2015. A Fast Distributed Stateless Algorithm for alpha-Fair Packing Problems. arXiv preprint arXiv:1502.03372 (2015).Google ScholarGoogle Scholar
  34. Alain Mayer, Yoram Ofek, and Moti Yung. 1996. Approximating max-min fair rates via distributed local scheduling with partial information. In Proceedings of the Fifteenth annual joint conference of the IEEE computer and communications societies conference on The conference on computer communications (INFOCOMM'98), Vol. 2. IEEE, 928--936.Google ScholarGoogle ScholarCross RefCross Ref
  35. Radhika Mittal, Vinh The Lam, Nandita Dukkipati, Emily Blem, Hassan Wassel, Monia Ghobadi, Amin Vahdat, Yaogong Wang, David Wetherall, and David Zats. 2015. TIMELY: RTT-based Congestion Control for the Datacenter. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '15). ACM, New York, NY, USA, 537--550. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Jeannine Mosely. 1984. Asynchronous distributed flow control algorithms. Ph.D. Dissertation. Massachusetts Institute of Technology.Google ScholarGoogle Scholar
  37. Alberto Mozo, Jose Luis López-Presa, and Antonio Fernandez Anta. 2012. SLBN: A Scalable Max-min Fair Algorithm for Rate-Based Explicit Congestion Control. In Network Computing and Applications (NCA), 2012 11th IEEE International Symposium on. IEEE, 212--219. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Alberto Mozo, José Luis López-Presa, and Antonio Fernández Anta. 2018. A distributed and quiescent max-min fair algorithm for network congestion control. Expert Systems with Applications, Vol. 91 (2018), 492 -- 512. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Ali Munir, Ghufran Baig, Syed M. Irteza, Ihsan A. Qazi, Alex X. Liu, and Fahad R. Dogar. 2014. Friends, Not Foes: Synthesizing Existing Transport Strategies for Data Center Networks. In Proceedings of the 2014 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '14). ACM, New York, NY, USA, 491--502.Google ScholarGoogle Scholar
  40. Abhay K Parekh and Robert G Gallager. 1993. A generalized processor sharing approach to flow control in integrated services networks: the single-node case. IEEE/ACM Transactions on Networking, Vol. 1, 3 (1993), 344--357. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Jonathan Perry, Hari Balakrishnan, and Devavrat Shah. 2017. Flowtune: Flowlet Control for Datacenter Networks. Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation (NSDI'17). USENIX Association, Berkeley, CA, USA, 421--435. http://dl.acm.org/citation.cfm?id=3154630.3154665 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Jordi Ros-Giralt. 2003. A Theory of Lexicographic Optimization for Computer Networks. Ph.D. Dissertation. University of California, Irvine.Google ScholarGoogle Scholar
  43. Jordi Ros-Giralt and Wei Kang Tsai. 2001. A theory of convergence order of maxmin rate allocation and an optimal protocol. In Proceedings IEEE INFOCOM 2001 Conference on Computer Communications. IEEE, 717--726.Google ScholarGoogle Scholar
  44. Jordi Ros-Giralt and Wei K Tsai. 2010. A lexicographic optimization framework to the flow control problem. IEEE Transactions on Information Theory, Vol. 56, 6 (2010), 2875--2886.Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Naveen Kr. Sharma, Antoine Kaufmann, Thomas Anderson, Changhoon Kim, Arvind Krishnamurthy, Jacob Nelson, and Simon Peter. 2017. Evaluating the Power of Flexible Packet Processing for Network Resource Allocation. In Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation (NSDI'17). USENIX Association, Berkeley, CA, USA, 67--82. http://dl.acm.org/citation.cfm?id=3154630.3154637 Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Fabian Skivée and Guy Leduc. 2004. A distributed algorithm for weighted max-min fairness in MPLS networks. In International Conference on Telecommunications. Springer, 644--653. build on hou to reduce rm cells and use faster update architecture to improve convergence time by a factor of 2 or 3 depending on precision, asynchronous, needs per-flow state, proactive.Google ScholarGoogle ScholarCross RefCross Ref
  47. T. Voice and G. Raina. 2009. Stability Analysis of a Max-Min Fair Rate Control Protocol (RCP) in a Small Buffer Regime. IEEE Trans. Automat. Control, Vol. 54, 8 (Aug 2009), 1908--1913.Google ScholarGoogle ScholarCross RefCross Ref
  48. Christo Wilson, Hitesh Ballani, Thomas Karagiannis, and Ant Rowtron. 2011. Better Never Than Late: Meeting Deadlines in Datacenter Networks. In Proceedings of the ACM SIGCOMM 2011 Conference (SIGCOMM '11). ACM, New York, NY, USA, 50--61. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Noa Zilberman, Yury Audzevich, G Adam Covington, and Andrew W Moore. 2014. NetFPGA SUME: Toward 100 Gbps as research commodity. IEEE Micro, Vol. 34, 5 (2014), 32--41.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. A Distributed Algorithm to Calculate Max-Min Fair Rates Without Per-Flow State

            Recommendations

            Comments

            Login options

            Check if you have access through your login credentials or your institution to get full access on this article.

            Sign in

            Full Access

            PDF Format

            View or Download as a PDF file.

            PDF

            eReader

            View online with eReader.

            eReader
            About Cookies On This Site

            We use cookies to ensure that we give you the best experience on our website.

            Learn more

            Got it!