research-article

Open access

On the Bottleneck Structure of Congestion-Controlled Networks

Authors:

Jordi Ros-Giralt,

Sruthi Yellamraju,

M. Harper Langston,

Richard Lethin,

Leandros Tassiulas,

Malathi VeeraraghavanAuthors Info & Claims

Proceedings of the ACM on Measurement and Analysis of Computing Systems, Volume 3, Issue 3

Article No.: 59, Pages 1 - 31

https://doi.org/10.1145/3366707

Published: 17 December 2019 Publication History

Abstract

In this paper, we introduce theTheory of Bottleneck Ordering, a mathematical framework that reveals the bottleneck structure of data networks. This theoretical framework provides insights into the inherent topological properties of a network in at least three areas: (1) It identifies the regions of influence of each bottleneck; (2) it reveals the order in which bottlenecks (and flows traversing them) converge to their steady state transmission rates in distributed congestion control algorithms; and (3) it provides key insights into the design of optimized traffic engineering policies. We demonstrate the efficacy of the proposed theory in TCP congestion-controlled networks for two broad classes of algorithms: Congestion-based algorithms (TCP BBR) and loss-based additive-increase/multiplicative-decrease algorithms (TCP Cubic and Reno). Among other results, our network experiments show that: (1) Qualitatively, both classes of congestion control algorithms behave as predicted by the bottleneck structure of the network; (2) flows compete for bandwidth only with other flows operating at the same bottleneck level; (3) BBR flows achieve higher performance and fairness than Cubic and Reno flows due to their ability to operate at the right bottleneck level; (4) the bottleneck structure of a network is continuously changing and its levels can be folded due to variations in the flows' round trip times; and (5) against conventional wisdom, low-hitter flows can have a large impact to the overall performance of a network.

References

[1]

Dimitri P. Bertsekas and Robert G. Gallager. 1992. Data Networks. Vol. 2. Prentice-Hall Inc., Englewood Cliffs, New Jersey 07632.

[2]

Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Soheil Hassas Yeganeh, and Van Jacobson. 2016. BBR: Congestion- Based Congestion Control. ACM Queue 14, 5, Article 50 (October 2016), 34 pages. https://doi.org/10.1145/3012426.3022184

[3]

Dah-Ming W. Chiu and Raj Jain. 1989. Analysis of the Increase and Decrease Algorithms for Congestion Avoidance in Computer Networks. Computer Networks and ISDN systems 17, 1 (June 1989), 1--14. https://doi.org/10.1016/0169--7552(89)90019--6

[4]

Benoit Claise, Ganesh Sadasivan, Vamsi Valluri, and Martin Djernaes. 2004. NetFlow Specifications, Cisco Systems. (2004). Retrieved October 24, 2019 from https://www.ietf.org/rfc/rfc3954.txt

[5]

COSMOS Lab. 2019. The Cosmos Testbed. (2019). Retrieved October 24, 2019 from https://cosmos-lab.org

[6]

ESnet. 2019. ESnet Energy Sciences Network. (2019). Retrieved October 24, 2019 from http://es.net/network-r-and-d/ experimental-network-testbeds/test-circuit-service/

[7]

Kevin Fall and Sally Floyd. 1996. Simulation-based Comparisons of Tahoe, Reno and SACK TCP. SIGCOMM Computer Communication Review 26, 3 (July 1996), 5--21. https://doi.org/10.1145/235160.235162

Digital Library

[8]

Tiago Fioreze, Lisandro Granville, Ramin Sadre, and Aiko Pras. 2009. A Statistical Analysis of Network Parameters for the Self-Management of Lambda-Connections. In Scalability of Networks and Services. Springer Berlin Heidelberg, Berlin, Heidelberg, 15--27. https://doi.org/10.1007/978--3--642-02627-0_2

[9]

Hannes Gredler, Jan Medved, Stefano Previdi, Adrian Farrel, and Saikat Ray. 2016. BGP-LS Protocol Specification. (2016). Retrieved October 24, 2019 from https://tools.ietf.org/html/rfc7752

[10]

Sangtae Ha, Injong Rhee, and Lisong Xu. 2008. CUBIC: A New TCP-friendly High-speed TCP Variant. SIGOPS operating systems review 42, 5 (July 2008), 64--74. https://doi.org/10.1145/1400097.1400105

Digital Library

[11]

Iperf.fr. 2019. iPerf - The ultimate speed test tool for TCP, UDP and SCTP. (2019). Retrieved October 24, 2019 from https://iperf.fr/

[12]

M. Schoffstall J. Davin J. Case, M. Fedor. 1990. A Simple Network Management Protocol (SNMP). (1990). Retrieved October 24, 2019 from https://tools.ietf.org/html/rfc1157

[13]

Van Jacobson. 1988. Congestion Avoidance and Control. SIGCOMM computer communication review 18, 4 (August 1988), 314--329. https://doi.org/10.1145/52325.52356

[14]

Raj Jain, Dah-Ming W. Chiu, and William R. Hawe. 1998. A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems. CoRR cs.NI/9809099 (1998), 38. http://arxiv.org/abs/cs.NI/9809099

[15]

Sushant Jain, Alok Kumar, Subhasree Mandal, Joon Ong, Leon Poutievski, Arjun Singh, Subbaiah Venkata, Jim Wanderer, Junlan Zhou, Min Zhu, Jon Zolla, Urs Hölzle, Stephen Stuart, and Amin Vahdat. 2013. B4: Experience with a Globally-Deployed Software Defined WAN. SIGCOMM Computer Communication Review 43, 4 (August 2013), 3--14. https://doi.org/10.1145/2534169.2486019

Digital Library

[16]

Lavanya Jose, Lisa Yan, Mohammad Alizadeh, George Varghese, Nick McKeown, and Sachin Katti. 2015. High Speed Networks Need Proactive Congestion Control. In Proceedings of the 14th ACM Workshop on Hot Topics in Networks (HotNets-XIV). ACM, New York, NY, USA, Article 14, 7 pages. https://doi.org/10.1145/2834050.2834096

Digital Library

[17]

Frank P. Kelly, Aman K. Maulloo, and David K. H. Tan. 1998. Rate Control for Communication Networks: Shadow Prices, Proportional Fairness and Stability. Journal of the Operational Research society 49, 3 (01 March 1998), 237--252. https://doi.org/10.1057/palgrave.jors.2600523

[18]

Reservoir Labs. 2019. G2-Mininet Sandbox: Mininet extensions to support the analysis of the bottleneck structure of networks. (2019). Retrieved October 24, 2019 from https://github.com/reservoirlabs/g2-mininet

[19]

Kun-Chan Lan and John Heidemann. 2006. A Measurement Study of Correlations of Internet Flow Characteristics. Computer Networks 50, 1 (2006), 46--62. https://doi.org/10.1016/j.comnet.2005.02.008

Digital Library

[20]

Yi Lu, Mei Wang, Balaji Prabhakar, and Flavio Bonomi. 2007. ElephantTrap: A Low Cost Device for Identifying Large Flows. In 15th Annual IEEE Symposium on High-Performance Interconnects (HOTI 2007). IEEE, 99--108. https://doi.org/10.1109/HOTI.2007.13

[21]

Mininet. 2019. Mininet: An Instant Virtual Network on your Laptop (or other PC). (2019). Retrieved October 24, 2019 from http://mininet.org/

[22]

Peter Phaal, Sonia Panchen, and Neil McKee. 2001. sFlow Specifications, InMon Corporation. IETF RFC 3176 (2001).

[23]

NOX Repo POX. 2019. The POX Network Software Platform. (2019). Retrieved October 24, 2019 from https://noxrepo.github.io/pox-doc/html/

[24]

Konstantinos Psounis, Arpita Ghosh, Balaji Prabhakar, and Gang Wang. 2005. SIFT: A Simple Algorithm for Tracking Elephant Flows, and Taking Advantage of Power Laws. In 43rd Allerton Conference on Communication, Control and Computing.

[25]

Jordi Ros-Giralt. 2003. A Theory of Lexicographic Optimization for Computer Networks. University of California, Irvine, Irvine, California. https://doi.org/10.13140/RG.2.1.2188.1368 AAI3101616.

[26]

Jordi Ros-Giralt and Wei K. Tsai. 2001. A Theory of Convergence Order of Maxmin Rate Allocation and an Optimal Protocol. In Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213), Vol. 2. IEEE, 717--726 vol.2. https://doi.org/10.1109/INFCOM.2001.916260

[27]

Jordi Ros-Giralt and Wei K. Tsai. 2010. A Lexicographic Optimization Framework to the Flow Control Problem. IEEE Transactions on Information Theory 56, 6 (June 2010), 2875--2886. https://doi.org/10.1109/TIT.2010.2046227

[28]

Jerome H. Saltzer, David P. Reed, and David D. Clark. 1984. End-to-end Arguments in System Design. ACM Transactions on Computer Systems (TOCS) 2, 4 (November 1984), 277--288. https://doi.org/10.1145/357401.357402

Digital Library

[29]

Shriram Sarvotham, Rudolf Riedi, and Richard Baraniuk. 2001. Connection-level Analysis and Modeling of Network Traffic. In Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement (IMW '01). ACM, New York, NY, USA, 99--103. https://doi.org/10.1145/505202.505215

Digital Library

[30]

SCinet. 2019. The SCinet Network at Supercomputing. (2019). Retrieved October 24, 2019 from https://sc19. supercomputing.org/scinet/all-about-scinet/

[31]

Suricata. 2019. Suricata: Open Source IDS / IPS / NSM engine. (2019). Retrieved October 24, 2019 from https://suricata-ids.org/

[32]

Zeek. 2019. The Zeek Network Security Monitor. (2019). Retrieved October 24, 2019 from https://www.zeek.org/

[33]

Yu Zhang, Binxing Fang, and Yongzheng Zhang. 2010. Identifying High-Rate Flows based on Bayesian Single Sampling. In 2010 2nd International Conference on Computer Engineering and Technology, Vol. 1. IEEE, V1--370--V1--374. https://doi.org/10.1109/ICCET.2010.5486097

Cited By

Kinnear RMazumdar RMarbach P(2024)Convexity in Real-time Bidding and Related ProblemsACM Transactions on Economics and Computation10.1145/3656552Online publication date: 15-Apr-2024
https://doi.org/10.1145/3656552
Li XZhou HMa LXin JHuang S(2024)Cost and Latency Customized SFC Deployment in Hybrid VNF and PNF EnvironmentIEEE Transactions on Network and Service Management10.1109/TNSM.2024.341517421:4(4312-4331)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3415174
He YZeng A(2024)Expanding bottlenecks reveals hidden bottlenecks and leads to more congested city centersPhysica A: Statistical Mechanics and its Applications10.1016/j.physa.2024.129707640(129707)Online publication date: Apr-2024
https://doi.org/10.1016/j.physa.2024.129707
Show More Cited By

Index Terms

On the Bottleneck Structure of Congestion-Controlled Networks
1. Networks
  1. Network performance evaluation
    1. Network performance analysis

Recommendations

On the Bottleneck Structure of Congestion-Controlled Networks
SIGMETRICS '20: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems

In this paper, we introduce the Theory of Bottleneck Ordering, a mathematical framework that reveals the bottleneck structure of data networks. This theoretical framework provides insights into the inherent topological properties of a network in at ...
On the Bottleneck Structure of Congestion-Controlled Networks

In this paper,we introduce the Theory of Bottleneck Ordering, a mathematical framework that reveals the bottleneck structure of data networks. This theoretical framework provides insights into the inherent topological properties of a network in at least ...
Unreliable transport protocol using congestion control for high-speed networks

Currently there is no control for the real-time traffic of multimedia applications using UDP (User Datagram Protocol) in high-speed networks. Therefore, although a number of high-speed TCP (Transmission Control Protocol) protocols have been developed ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Measurement and Analysis of Computing Systems

Proceedings of the ACM on Measurement and Analysis of Computing Systems Volume 3, Issue 3

SIGMETRICS

December 2019

525 pages

EISSN:2476-1249

DOI:10.1145/3376928

Editors:
Augustin Chaintreau
Columbia University
,
Leana Golubchik
University of Southern California
,
Zhi-Li Zhang
University of Minnesota

Issue’s Table of Contents

Copyright © 2019 Owner/Author.

This work is licensed under a Creative Commons Attribution-ShareAlike International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 December 2019

Published in POMACS Volume 3, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

DOE Advanced Scientific Computing Research

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
2,170
Total Downloads

Downloads (Last 12 months)222
Downloads (Last 6 weeks)24

Reflects downloads up to 28 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kinnear RMazumdar RMarbach P(2024)Convexity in Real-time Bidding and Related ProblemsACM Transactions on Economics and Computation10.1145/3656552Online publication date: 15-Apr-2024
https://doi.org/10.1145/3656552
Li XZhou HMa LXin JHuang S(2024)Cost and Latency Customized SFC Deployment in Hybrid VNF and PNF EnvironmentIEEE Transactions on Network and Service Management10.1109/TNSM.2024.341517421:4(4312-4331)Online publication date: 17-Jun-2024
https://dl.acm.org/doi/10.1109/TNSM.2024.3415174
He YZeng A(2024)Expanding bottlenecks reveals hidden bottlenecks and leads to more congested city centersPhysica A: Statistical Mechanics and its Applications10.1016/j.physa.2024.129707640(129707)Online publication date: Apr-2024
https://doi.org/10.1016/j.physa.2024.129707
Wang TWang X(2023)Boosting Capacity for 6G Terahertz Mesh Networks Based on Bottleneck StructuresGLOBECOM 2023 - 2023 IEEE Global Communications Conference10.1109/GLOBECOM54140.2023.10436964(4589-4594)Online publication date: 4-Dec-2023
https://doi.org/10.1109/GLOBECOM54140.2023.10436964
Kinnear RMazumdar RMarbach P(2022)Real-time Bidding for Time Constrained Impression Contracts in First and Second Price Auctions - Theory and AlgorithmsACM SIGMETRICS Performance Evaluation Review10.1145/3547353.352263450:1(93-94)Online publication date: 7-Jul-2022
https://dl.acm.org/doi/10.1145/3547353.3522634
Kinnear RMazumdar RMarbach PManjunath DNair JCarlsson NCohen ERobert P(2022)Real-time Bidding for Time Constrained Impression Contracts in First and Second Price Auctions - Theory and AlgorithmsAbstract Proceedings of the 2022 ACM SIGMETRICS/IFIP PERFORMANCE Joint International Conference on Measurement and Modeling of Computer Systems10.1145/3489048.3522634(93-94)Online publication date: 6-Jun-2022
https://dl.acm.org/doi/10.1145/3489048.3522634
Gombos GKis DTothmeresz LKiraly TNadas SLaki S(2022)Flow Fairness With Core-Stateless Resource Sharing in Arbitrary TopologyIEEE Access10.1109/ACCESS.2022.322206210(120312-120328)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3222062
Anvari HLu P(2021)Machine-Learned Recognition of Network Traffic for Optimization through Protocol SelectionComputers10.3390/computers1006007610:6(76)Online publication date: 11-Jun-2021
https://doi.org/10.3390/computers10060076
Kinnear RMazumdar RMarbach P(2021)Real-time Bidding for Time Constrained Impression Contracts in First and Second Price Auctions - Theory and AlgorithmsProceedings of the ACM on Measurement and Analysis of Computing Systems10.1145/34910495:3(1-37)Online publication date: 15-Dec-2021
https://dl.acm.org/doi/10.1145/3491049
Ros-Giralt JAmsel NYellamraju SEzick JLethin RJiang YFeng ATassiulas LWu ZTeh MBergman KKuipers FCaesar M(2021)Designing data center networks using bottleneck structuresProceedings of the 2021 ACM SIGCOMM 2021 Conference10.1145/3452296.3472898(319-348)Online publication date: 9-Aug-2021
https://dl.acm.org/doi/10.1145/3452296.3472898
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents