Abstract
Most congestion control algorithms, like TCP, rely on a reactive control system that detects congestion, then marches carefully towards a desired operating point (e.g. by modifying the window size or adjusting a rate). In an effort to balance stability and convergence speed, they often take hundreds of RTTs to converge; an increasing problem as networks get faster, with less time to react.
This paper is about an alternative class of congestion control algorithms based on proactive-scheduling: switches and NICs "pro-actively" exchange control messages to run a \em distributed algorithm to pick "explicit rates for each flow. We call these Proactive Explicit Rate Control (PERC) algorithms. They take as input the routing matrix and link speeds, but not a congestion signal. By exploiting information such as the number of flows at a link, they can converge an order of magnitude faster than reactive algorithms.
Our main contributions are (1) s-PERC ("stateless" PERC), a new practical distributed PERC algorithm without per-flow state at the switches, and (2) a proof that s-PERC computes exact max-min fair rates in a known bounded time, the first such algorithm to do so without per-flow state. To analyze s-PERC, we introduce a parallel variant of standard waterfilling, 2-Waterfilling. We prove that s-PERC converges to max-min fair in 6N rounds, where N is the number of iterations 2-Waterfilling takes for the same routing matrix.
We describe how to make s-PERC practical and robust to deploy in real networks. We confirm using realistic simulations and an FPGA hardware testbed that s-PERC converges 10-100x faster than reactive algorithms like TCP, DCTCP and RCP in data-center networks and 1.3--6x faster in wide-area networks (WANs). Long flows complete in close to the ideal time, while short-lived flows are prioritized, making it appropriate for data-centers and WANs.
- Yehuda Afek, Yishay Mansour, and Zvi Ostfeld. 1996. Phantom: A Simple and Effective Flow Control Scheme. In Conference Proceedings on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM '96). ACM, New York, NY, USA, 169--182. Google Scholar
Digital Library
- Yehuda Afek, Yishay Mansour, and Zvi Ostfeld. 1999. Convergence Complexity of Optimistic Rate-Based Flow-Control Algorithms. Journal of Algorithms, Vol. 30, 1 (Jan. 1999), 106--143. Google Scholar
Digital Library
- Mohammad Alizadeh, Albert Greenberg, David A. Maltz, Jitendra Padhye, Parveen Patel, Balaji Prabhakar, Sudipta Sengupta, and Murari Sridharan. 2010. Data Center TCP (DCTCP). In Proceedings of the SIGCOMM 2010 Conference (SIGCOMM '10). ACM, New York, NY, USA, 63--74. Google Scholar
Digital Library
- Mohammad Alizadeh, Shuang Yang, Milad Sharif, Sachin Katti, Nick McKeown, Balaji Prabhakar, and Scott Shenker. 2013. pFabric: Minimal Near-optimal Datacenter Transport. In Proceedings of the ACM SIGCOMM 2013 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM '13). ACM, New York, NY, USA, 435--446. Google Scholar
Digital Library
- Noga Alon, Yossi Matias, and Mario Szegedy. 1999. The Space Complexity of Approximating the Frequency Moments. J. Comput. System Sci., Vol. 58, 1 (Feb. 1999), 137--147. Google Scholar
Digital Library
- Baruch Awerbuch and Rohit Khandekar. 2007. Greedy Distributed Optimization of Multi-commodity Flows. In Proceedings of the Twenty-sixth Annual ACM Symposium on Principles of Distributed Computing (PODC '07). ACM, New York, NY, USA, 274--283. Google Scholar
Digital Library
- B. Awerbuch and Y. Shavitt. 1998. Converging to approximated max-min flow fairness in logarithmic time. In Proceedings. IEEE INFOCOM '98, the Conference on Computer Communications. Seventeenth Annual Joint Conference of the IEEE Computer and Communications Societies. Gateway to the 21st Century (Cat. No.98, Vol. 3. 1350--1357 vol.3.Google Scholar
- Wei Bai, Li Chen, Kai Chen, Dongsu Han, Chen Tian, and Hao Wang. 2017. PIAS: Practical Information-Agnostic Flow Scheduling for Commodity Data Centers. IEEE/ACM Transactions on Networking, Vol. 25, 4 (Aug. 2017), 1954--1967. Google Scholar
Digital Library
- Amotz Bar-Noy, Magnús M Halldórsson, Guy Kortsarz, Ravit Salman, and Hadas Shachnai. 2000. Sum multicoloring of graphs. Journal of Algorithms, Vol. 37, 2 (2000), 422--450. Google Scholar
Digital Library
- Yair Bartal, Martin Farach-Colton, Shibu Yooseph, and Lisa Zhang. 2002. Fast, fair and frugal bandwidth allocation in atm networks. Algorithmica, Vol. 33, 3 (2002), 272--286.Google Scholar
Cross Ref
- Dimitri Bertsekas and Robert Gallager. 1987. Data Networks .Prentice-Hall, Inc., Upper Saddle River, NJ, USA.Google Scholar
Digital Library
- Anna Charny, David D Clark, and Raj Jain. 1995. Congestion control with explicit rate indication. Communications, 1995. ICC'95 Seattle,'Gateway to Globalization', 1995 IEEE International Conference on, Vol. 3. IEEE, 1954--1963.Google Scholar
Cross Ref
- Anna Charny, KK Ramakrishnan, and Anthony Lauck. 1996. Time scale analysis scalability issues for explicit rate allocation in ATM networks. IEEE/ACM Transactions on Networking, Vol. 4, 4 (1996), 569--581. Google Scholar
Digital Library
- Inho Cho, Keon Jang, and Dongsu Han. 2017. Credit-Scheduled Delay-Bounded Congestion Control for Datacenters. In Proceedings of the 2017 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '17). ACM, New York, NY, USA, 239--252. Google Scholar
Digital Library
- Jorge A. Cobb and Mohamed G. Gouda. 2011. Stabilization of Max-min Fair Networks Without Per-flow State. Theoretical Computer Science, Vol. 412, 40 (Sept. 2011), 5562--5579.Google Scholar
Digital Library
- A. Demers, S. Keshav, and S. Shenker. 1989. Analysis and Simulation of a Fair Queueing Algorithm. In Symposium Proceedings on Communications Architectures &Amp; Protocols (SIGCOMM '89). ACM, New York, NY, USA, 1--12. Google Scholar
Digital Library
- Nandita Dukkipati. 2008. Rate Control Protocol (Rcp): Congestion Control to Make Flows Complete Quickly. Ph.D. Dissertation. Stanford University, Stanford, CA, USA. Advisor(s) Mckeown, Nick. AAI3292347. Google Scholar
Digital Library
- Paul Emmerich, Sebastian Gallenmüller, Daniel Raumer, Florian Wohlfart, and Georg Carle. 2015. Moongen: A scriptable high-speed packet generator. Proceedings of the 2015 ACM Conference on Internet Measurement Conference. ACM, 275--287. Google Scholar
Digital Library
- Eli Gafni and Dimitri Bertsekas. 1984. Dynamic control of session input rates in communication networks. IEEE Trans. Automat. Control, Vol. 29, 11 (1984), 1009--1016.Google Scholar
Cross Ref
- Dimitris Giannopoulos, Nikos Chrysos, Evangelos Mageiropoulos, Giannis Vardas, Leandros Tzanakis, and Manolis Katevenis. 2018. Accurate Congestion Control for RDMA Transfers. In 2018 Twelfth IEEE/ACM International Symposium on Networks-on-Chip (NOCS). IEEE, 1--8. Google Scholar
Digital Library
- Mark Handley, Costin Raiciu, Alexandru Agache, Andrei Voinescu, Andrew W Moore, Gianni Antichi, and Marcin Wójcik. 2017. Re-architecting datacenter networks and stacks for low latency and high performance. In Proceedings of the 2017 ACM Conference on Special Interest Group on Data Communication. ACM, 29--42. Google Scholar
Digital Library
- Howard P Hayden. 1981. Voice flow control in integrated packet networks. Ph.D. Dissertation. Massachusetts Institute of Technology.Google Scholar
- Chi-Yao Hong, Matthew Caesar, and P. Brighten Godfrey. 2012. Finishing Flows Quickly with Preemptive Scheduling. In Proceedings of the ACM SIGCOMM 2012 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM '12). ACM, New York, NY, USA, 127--138. Google Scholar
Digital Library
- Y. Thomas Hou, Shivendra S. Panwar, and Henry H. Y. Tzeng. 2004. On Generalized Max-Min Rate Allocation and Distributed Convergence Algorithm for Packet Networks. IEEE Trans. Parallel Distrib. Syst., Vol. 15, 5 (May 2004), 401--416.Google Scholar
Digital Library
- Jeffrey Jaffe. 1981. Bottleneck flow control. IEEE Transactions on Communications, Vol. 29, 7 (1981), 954--962.Google Scholar
Cross Ref
- Lavanya Jose, Lisa Yan, Mohammad Alizadeh, George Varghese, Nick McKeown, and Sachin Katti. 2015. High Speed Networks Need Proactive Congestion Control. In Proceedings of the 14th ACM Workshop on Hot Topics in Networks (HotNets-XIV). ACM, New York, NY, USA, Article 14, bibinfonumpages7 pages. Google Scholar
Digital Library
- Shivkumar Kalyanaraman, Raj Jain, Sonia Fahmy, Rohit Goyal, and Bobby Vandalore. 2000. The ERICA switch algorithm for ABR traffic management in ATM networks. IEEE/ACM Transactions on Networking, Vol. 8, 1 (2000), 87--98. Google Scholar
Digital Library
- Dina Katabi, Mark Handley, and Charlie Rohrs. 2002. Congestion Control for High Bandwidth-delay Product Networks. In Proceedings of the 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications (SIGCOMM '02). ACM, New York, NY, USA, 89--102. Google Scholar
Digital Library
- Yuseok Kim, Wei Kang Tsai, Mahadevan Iyer, and Jordi Ros-Giralt. 1999. Minimum rate guarantee without per-flow information. In Network Protocols, 1999.(ICNP'99) Proceedings. Seventh International Conference on. IEEE, 155--162. Google Scholar
Digital Library
- Alok Kumar, Sushant Jain, Uday Naik, Anand Raghuraman, Nikhil Kasinadhuni, Enrique Cauich Zermeno, C. Stephen Gunn, Jing Ai, Björn Carlin, Mihai Amarandei-Stavila, Mathieu Robin, Aspi Siganporia, Stephen Stuart, and Amin Vahdat. 2015. BwE: Flexible, Hierarchical Bandwidth Allocation for WAN Distributed Computing. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '15). ACM, New York, NY, USA, 1--14. Google Scholar
Digital Library
- Jean-Yves Le Boudec. 2000. Rate adaptation, congestion control and fairness: A tutorial. (2000).Google Scholar
- Tae-Jin Lee and G. De Veciana. 1998. A decentralized framework to achieve max-min fair bandwidth allocation for ATM networks. In IEEE GLOBECOM 1998 (Cat. NO. 98CH36250), Vol. 3. 1515--1520 vol.3.Google Scholar
- Jelena Marasevic, Cliff Stein, and Gil Zussman. 2015. A Fast Distributed Stateless Algorithm for alpha-Fair Packing Problems. arXiv preprint arXiv:1502.03372 (2015).Google Scholar
- Alain Mayer, Yoram Ofek, and Moti Yung. 1996. Approximating max-min fair rates via distributed local scheduling with partial information. In Proceedings of the Fifteenth annual joint conference of the IEEE computer and communications societies conference on The conference on computer communications (INFOCOMM'98), Vol. 2. IEEE, 928--936.Google Scholar
Cross Ref
- Radhika Mittal, Vinh The Lam, Nandita Dukkipati, Emily Blem, Hassan Wassel, Monia Ghobadi, Amin Vahdat, Yaogong Wang, David Wetherall, and David Zats. 2015. TIMELY: RTT-based Congestion Control for the Datacenter. In Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '15). ACM, New York, NY, USA, 537--550. Google Scholar
Digital Library
- Jeannine Mosely. 1984. Asynchronous distributed flow control algorithms. Ph.D. Dissertation. Massachusetts Institute of Technology.Google Scholar
- Alberto Mozo, Jose Luis López-Presa, and Antonio Fernandez Anta. 2012. SLBN: A Scalable Max-min Fair Algorithm for Rate-Based Explicit Congestion Control. In Network Computing and Applications (NCA), 2012 11th IEEE International Symposium on. IEEE, 212--219. Google Scholar
Digital Library
- Alberto Mozo, José Luis López-Presa, and Antonio Fernández Anta. 2018. A distributed and quiescent max-min fair algorithm for network congestion control. Expert Systems with Applications, Vol. 91 (2018), 492 -- 512. Google Scholar
Digital Library
- Ali Munir, Ghufran Baig, Syed M. Irteza, Ihsan A. Qazi, Alex X. Liu, and Fahad R. Dogar. 2014. Friends, Not Foes: Synthesizing Existing Transport Strategies for Data Center Networks. In Proceedings of the 2014 ACM Conference on Special Interest Group on Data Communication (SIGCOMM '14). ACM, New York, NY, USA, 491--502.Google Scholar
- Abhay K Parekh and Robert G Gallager. 1993. A generalized processor sharing approach to flow control in integrated services networks: the single-node case. IEEE/ACM Transactions on Networking, Vol. 1, 3 (1993), 344--357. Google Scholar
Digital Library
- Jonathan Perry, Hari Balakrishnan, and Devavrat Shah. 2017. Flowtune: Flowlet Control for Datacenter Networks. Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation (NSDI'17). USENIX Association, Berkeley, CA, USA, 421--435. http://dl.acm.org/citation.cfm?id=3154630.3154665 Google Scholar
Digital Library
- Jordi Ros-Giralt. 2003. A Theory of Lexicographic Optimization for Computer Networks. Ph.D. Dissertation. University of California, Irvine.Google Scholar
- Jordi Ros-Giralt and Wei Kang Tsai. 2001. A theory of convergence order of maxmin rate allocation and an optimal protocol. In Proceedings IEEE INFOCOM 2001 Conference on Computer Communications. IEEE, 717--726.Google Scholar
- Jordi Ros-Giralt and Wei K Tsai. 2010. A lexicographic optimization framework to the flow control problem. IEEE Transactions on Information Theory, Vol. 56, 6 (2010), 2875--2886.Google Scholar
Digital Library
- Naveen Kr. Sharma, Antoine Kaufmann, Thomas Anderson, Changhoon Kim, Arvind Krishnamurthy, Jacob Nelson, and Simon Peter. 2017. Evaluating the Power of Flexible Packet Processing for Network Resource Allocation. In Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation (NSDI'17). USENIX Association, Berkeley, CA, USA, 67--82. http://dl.acm.org/citation.cfm?id=3154630.3154637 Google Scholar
Digital Library
- Fabian Skivée and Guy Leduc. 2004. A distributed algorithm for weighted max-min fairness in MPLS networks. In International Conference on Telecommunications. Springer, 644--653. build on hou to reduce rm cells and use faster update architecture to improve convergence time by a factor of 2 or 3 depending on precision, asynchronous, needs per-flow state, proactive.Google Scholar
Cross Ref
- T. Voice and G. Raina. 2009. Stability Analysis of a Max-Min Fair Rate Control Protocol (RCP) in a Small Buffer Regime. IEEE Trans. Automat. Control, Vol. 54, 8 (Aug 2009), 1908--1913.Google Scholar
Cross Ref
- Christo Wilson, Hitesh Ballani, Thomas Karagiannis, and Ant Rowtron. 2011. Better Never Than Late: Meeting Deadlines in Datacenter Networks. In Proceedings of the ACM SIGCOMM 2011 Conference (SIGCOMM '11). ACM, New York, NY, USA, 50--61. Google Scholar
Digital Library
- Noa Zilberman, Yury Audzevich, G Adam Covington, and Andrew W Moore. 2014. NetFPGA SUME: Toward 100 Gbps as research commodity. IEEE Micro, Vol. 34, 5 (2014), 32--41.Google Scholar
Cross Ref
Index Terms
A Distributed Algorithm to Calculate Max-Min Fair Rates Without Per-Flow State
Recommendations
A Distributed Algorithm to Calculate Max-Min Fair Rates Without Per-Flow State
SIGMETRICS '19: Abstracts of the 2019 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer SystemsMost congestion control algorithms, like TCP, rely on a reactive control system that detects congestion, then marches carefully towards a desired operating point (e.g. by modifying the window size or adjusting a rate). In an effort to balance stability ...
A Distributed Algorithm to Calculate Max-Min Fair Rates Without Per-Flow State
Most congestion control algorithms, like TCP, rely on a reactive control system that detects congestion, then marches carefully towards a desired operating point (e.g. by modifying the window size or adjusting a rate). In an effort to balance stability ...
Stabilization of Max-Min Fair Networks without Per-flow State
SSS '08: Proceedings of the 10th International Symposium on Stabilization, Safety, and Security of Distributed SystemsLet a <em>flow</em> be a sequence of packets sent from a source computer to a destination computer. Routers at the core of the Internet do not maintain any information about the flows that traverse them. This has allowed for great speeds at the routers, ...






Comments