research-article

Public Access

The ties that un-bind: decoupling IP from web services and sockets for robust addressing agility at CDN-scale

Authors:

Vasileios Giotsas,

Marek Majkowski,

Pavel Odintsov,

Jakub Sitnicki,

Taejoong Chung,

Christopher A. Wood, and

Nick SullivanAuthors Info & Claims

SIGCOMM '21: Proceedings of the 2021 ACM SIGCOMM 2021 Conference

August 2021

Pages 433 - 446

https://doi.org/10.1145/3452296.3472922

Published: 09 August 2021 Publication History

Abstract

The couplings between IP addresses, names of content or services, and socket interfaces, are too tight. This impedes system manageability, growth, and overall provisioning. In turn, large-scale content providers are forced to use staggering numbers of addresses, ultimately leading to address exhaustion (IPv4) and inefficiency (IPv6).

In this paper, we revisit IP bindings, entirely. We attempt to evolve addressing conventions by decoupling IP in DNS and from network sockets. Alongside technologies such as SNI and ECMP, a new architecture emerges that ``unbinds'' IP from services and servers, thereby returning IP's role to merely that of reachability. The architecture is under evaluation at a major CDN in multiple datacenters. We show that addresses can be generated randomly \emph{per-query}, for 20M+ domains and services, from as few as ~4K addresses, 256 addresses, and even \emph{one} IP address. We explain why this approach is transparent to routing, L4/L7 load-balancers, distributed caching, and all surrounding systems -- and is \emph{highly desirable}. Our experience suggests that many network-oriented systems and services (e.g., route leak mitigation, denial of service, measurement) could be improved, and new ones designed, if built with addressing agility.

Supplementary Material

feldmann-public-review (428-public-review.pdf)

The Ties that un-Bind: Decoupling IP from web services and sockets for robust addressing agility at CDN-scale: Public Review

Download
66.92 KB

MP4 File (video-presentation.mp4)

Conference Presentation Video

Download
77.94 MB

References

[1]

Akamai IP Ranges. https://bgp.he.net/search?search%5Bsearch%5D=Akamai&commit=Search.

[2]

AWS: Multivalue Answer Routing. https://aws.amazon.com/premiumsupport/knowledge-center/multivalue-versus-simple-policies/. Last access: 2020/06.

[3]

Amazon. AWS IP address ranges. https://docs.aws.amazon.com/general/latest/gr/aws-ip-ranges.html.

[4]

Apple, Inc. Network Framework. https://developer.apple.com/documentation/network/, June 2020.

[5]

J. T. Araujo. Addressing IPv6: A CDN Perspective. https://ripe74.ripe.net/presentations/presentation-archive/, May 2017.

[6]

H. Balakrishnan, K. Lakshminarayanan, S. Ratnasamy, S. Shenker, I. Stoica, and M. Walfish. A Layered Naming Architecture for the Internet. In ACM SIGCOMM, 2004.

Digital Library

[7]

M. Belshe, R. Peon, and M. Thomson. Hypertext Transfer Protocol Version 2 (HTTP/2). RFC 7540, May 2015.

[8]

M. Bishop. Hypertext Transfer Protocol Version 3 (HTTP/3). Internet-Draft draft-ietf-quic-http-29, Internet Engineering Task Force, June 2020. Work in Progress.

[9]

B. Briscoe. RFC1794: DNS Support for Load Balancing. https://tools.ietf.org/html/rfc1794, Apr 1995.

[10]

M. Caesar, T. Condie, J. Kannan, K. Lakshminarayanan, I. Stoica, and S. Shenker. ROFL: Routing on Flat Labels. In ACM SIGCOMM, 2006.

Digital Library

[11]

D. Clark, R. Braden, A. Falk, and V. Pingali. FARA: Reorganizing the Addressing Architecture. In ACM SIGCOMM Workshop on Future Directions in Network Architectures (FDNA), 2003.

Digital Library

[12]

Cloudflare. IP Ranges. https://www.cloudflare.com/ips/.

[13]

Cloudflare Blog. What is round-robin DNS? https://www.cloudflare.com/learning/dns/glossary/round-robin-dns/. Last access: 2020/06.

[14]

Cloudflare Blog. Fixing reachability to 1.1.1.1, globally. https://blog.cloudflare.com/fixing-reachability-to-1-1-1-1-globally/, 2018.

[15]

Cloudflare Blog. The Technical Challenges of Building Cloudflare WARP. https://blog.cloudflare.com/warp-technical-challenges/, October 2019.

[16]

Cloudflare Technical Documentation. Network Ports Compatible with Cloudflare. https://support.cloudflare.com/hc/en-us/articles/200169156-Identifying-network-ports-compatible-with-Cloudflare-s-proxy.

[17]

Cloudflare Technical Documentation. What Is A Reverse Proxy? Proxy Servers Explained. https://www.cloudflare.com/en-gb/learning/cdn/glossary/reverse-proxy/.

[18]

M. D'Ambrosio, C. Dannewitz, H. Karl, and V. Vercellone. MDHT: A Hierarchical Name Resolution Service for Information-centric Networks. In ACM SIGCOMM Workshop on Information-Centric Networking (ICN), 2011.

Digital Library

[19]

M. D'Ambrosio, P. Fasano, M. Marchisio, V. Vercellone, and M. Ullio. Providing Data Dissemination Services in the Future Internet. In IEEE Global Communications Conference (GLOBECOM), 2008.

[20]

C. Dannewitz, D. Kutscher, B. Ohlman, S. Farrell, B. Ahlgren, and H. Karl. Network of information (netinf)--an information-centric networking architecture. Computer Communications, 36(7):721--735, 2013.

Digital Library

[21]

E. Dumazet and D. S. Miller. inet: add IPBINDADDRESSNOPORT to overcome bind(0) limitations. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=90c337da1524863838658078ec34241f45d8394d, June 2015.

[22]

D. E. Eisenbud, C. Yi, C. Contavalli, C. Smith, R. Kononov, E. Mann-Hielscher, A. Cilingiroglu, B. Cheyney, W. Shang, and J. D. Hosein. Maglev: A Fast and Reliable Software Network Load Balancer. In Symposium on Networked Systems Design and Implementation (NSDI), 2016.

[23]

P. Francis and R. Gummadi. IPNL: A NAT-Extended Internet Architecture. In ACM SIGCOMM, 2001.

Digital Library

[24]

A. Ghodsi, T. Koponen, B. Raghavan, S. Shenker, A. Singla, and J. Wilcox. Information-Centric Networking: Seeing the Forest for the Trees. In Workshop on Hot Topics in Networks (HotNets), 2011.

Digital Library

[25]

A. Ghodsi, T. Koponen, J. Rajahalme, P. Sarolahti, and S. Shenker. Naming in Content-Oriented Architectures. In ACM SIGCOMM Workshop on Information-Centric Networking (ICN), 2011.

[26]

V. Giotsas, I. Livadariu, and P. Gigis. A First Look at the Misuse and Abuse of the IPv4 Transfer Market. In Passive and Active Measurement (PAM). Springer International Publishing, 2020.

[27]

G. Gonthier. Formal proof--the four-color theorem. Notices of the AMS, 55(11):1382--1393, 2008.

[28]

B. Gregg. Performance Superpowers with Enhanced BPF. Santa Clara, CA, July 2017. USENIX Association.

[29]

M. Gritter and D. R. Cheriton. An Architecture for Content Routing Support in the Internet. In Proceedings of the USENIX Symposium on Internet Technologies and Systems, USITS'01, USA, 2001. USENIX Association.

[30]

D. Han, A. Anand, F. Dogar, B. Li, H. Lim, M. Machado, A. Mukundan, W. Wu, A. Akella, D. G. Andersen, et al. {XIA}: Efficient Support for Evolvable Internetworking. In Presented as part of the 9th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 12), 2012.

[31]

N. P. Hoang, A. A. Niaki, M. Polychronakis, and P. Gill. The Web is Still Small after More than a Decade. ACM SIGCOMM Computer Communication Review (CCR), 50(2):24--31, May 2020.

[32]

R. Holz, J. Hiller, J. Amann, A. Razaghpanah, T. Jost, N. Vallina-Rodriguez, and O. Hohlfeld. Tracking the Deployment of TLS 1.3 on the Web: A Story of Experimentation and Centralization. SIGCOMM Comput. Commun. Rev., 50(3):3--15, July 2020.

[33]

G. Huston. The Architecture of the Internet or Waist Watching in IP. APNIC Labs Presentation Archive, https://labs.apnic.net/presentations/store/2004-05-04-waistwatching.pdf, May 2004.

[34]

J. Iyengar and M. Thomson. QUIC: A UDP-Based Multiplexed and Secure Transport. https://tools.ietf.org/html/draft-ietf-quic-transport-28#section-9.6, May 2020.

[35]

V. Jacobson, D. K. Smetters, J. D. Thornton, M. F. Plass, N. H. Briggs, and R. L. Braynard. Networking Named Content. In Proceedings of the ACM Conference on Emerging Networking Experiments and Technologies, CoNEXT '09, New York, NY, USA, 2009. Association for Computing Machinery.

Digital Library

[36]

P. Jokela, P. Nikander, J. Melen, J. Ylitalo, and J. Wall. Host Identity Protocol - Extended Abstract. In Wireless World Research Forum, 2004.

[37]

A. Jonsson, M. Folke, and B. Ahlgren. The Split Naming/Forwarding Network Architecture. In Swedish National Computer Networking Workshop, 2003.

[38]

Juniper Networks. What's New in Release 19.2R2: Routing Protocols. https://www.juniper.net/documentation/en_US/junos/information-products/topic-collections/release-notes/19.2/topic-147567.html#rn-junos-qfx-new-and-changed-features, May 2020.

[39]

C. Kim, M. Caesar, and J. Rexford. Floodless in SEATTLE: A Scalable Ethernet Architecture for Large Enterprises. In ACM SIGCOMM, 2008.

Digital Library

[40]

T. Koponen, M. Chawla, B.-G. Chun, A. Ermolinskiy, K. H. Kim, S. Shenker, and I. Stoica. A Data-Oriented (and Beyond) Network Architecture. In ACM SIGCOMM, 2007.

Digital Library

[41]

J. Larisch, D. Choffnes, D. Levin, B. M. Maggs, A. Mislove, and C. Wilson. CRLite: A Scalable System for Pushing All TLS Revocations to All Browsers. In IEEE Symposium on Security and Privacy, 2017.

[42]

Linux Foundation. ip(7) --- Linux manual page, see entry for IPBINDADDRESSNOPORT. Linux Programmer's Manual, 2021.

[43]

BPF sklookup merge commit. Linux kernel 5.9, https://github.com/torvalds/linux/commit/e57892f50a07953053dcb1e0c9431197e569c258, 2020.

[44]

Support for running bpf programs on socket lookups. Linux kernel 5.9, https://kernelnewbies.org/Linux_5.9\#Support_for_running_BPF_programs_on_socket_lookups, December 2020.

[45]

R. Miao, H. Zeng, C. Kim, J. Lee, and M. Yu. SilkRoad: Making Stateful Layer-4 Load Balancing Fast and Cheap Using Switching ASICs. In ACM SIGCOMM, 2017.

Digital Library

[46]

G. Moura. DNS TTL Violations in the Wild - Measured with RIPE Atlas. https://labs.ripe.net/Members/giovane_moura/dns-ttl-violations-in-the-wild-with-ripe-atlas-2, Dec 2017.

[47]

G. C. M. Moura, J. Heidemann, R. d. O. Schmidt, and W. Hardaker. Cache Me If You Can: Effects of DNS Time-to-Live. In ACM Internet Measurement Conference (IMC), 2019.

[48]

U. Naseer, L. Niccolini, U. Pant, A. Frindell, R. Dasineni, and T. A. Benson. Zero Downtime Release: Disruption-Free Load Balancing of a Multi-Billion User Website. In ACM SIGGCOMM, 2020.

Digital Library

[49]

E. Nygren. Reaching Toward Universal TLS SNI. https://blogs.akamai.com/2017/03/reaching-toward-universal-tls-sni.html, 2017.

[50]

T. Pauly, B. Trammell, A. Brunstrom, G. Fairhurst, C. Perkins, P. S. Tiesel, and C. A. Wood. An Architecture for Transport Services. Internet-Draft draft-ietf-taps-arch-10, Internet Engineering Task Force, Apr. 2021. Work in Progress.

[51]

S. Ren, D. Yu, G. Li, S. Hu, Y. Tian, X. Gong, and R. Moskowitz. Routing and Addressing with Length Variable IP Address. In Proceedings of the ACM SIGCOMM 2019 Workshop on Networking for Emerging Applications and Technologies, NEAT'19, pages 43--48, New York, NY, USA, 2019. Association for Computing Machinery.

Digital Library

[52]

RFC 791: Internet Protocol. https://datatracker.ietf.org/doc/html/rfc791, Sept. 1981.

[53]

P. Richter, G. Smaragdakis, D. Plonka, and A. Berger. Beyond Counting: New Perspectives on the Active IPv4 Address Space. In ACM Internet Measurement Conference (IMC), 2016.

Digital Library

[54]

J. H. Saltzer, D. P. Reed, and D. D. Clark. End-to-end arguments in system design. ACM Trans. Comput. Syst., 2(4):277--288, Nov. 1984.

Digital Library

[55]

F. W. Scholz and M. A. Stephens. K-sample Anderson--Darling tests. Journal of the American Statistical Association, 82(399):918--924, 1987.

[56]

K. Schomp and R. Al-Dalky. Partitioning the Internet Using Anycast Catchments. SIGCOMM Comput. Commun. Rev., 50(4):3--9, Oct. 2020.

Digital Library

[57]

K. Schomp, O. Bhardwaj, E. Kurdoglu, M. Muhaimen, and R. K. Sitaraman. Akamai DNS: Providing Authoritative Answers to the World's Queries. In ACM SIGCOMM, 2020.

Digital Library

[58]

S. Sevilla and J. J. Garcia-Luna-Aceves. Freeing the IP Internet Architecture from Fixed IP Addresses. In IEEE International Conference on Network Protocols (ICNP), 2015.

[59]

J. F. Shoch. A note on Inter-Network Naming, Addressing, and Routing. IEN 19, 1978.

[60]

M. Silverlock and G. Redner. Bringing Modern Transport Security to Google Cloud with TLS 1.3. https://cloud.google.com/blog/products/networking/tls-1-3-is-now-on-by-default-for-google-cloud-services, June 2020.

[61]

R. P. Singh, T. Brecht, and S. Keshav. IP Address Multiplexing for VEEs. SIGCOMM Comput. Commun. Rev., 44(2):36--43, Apr. 2014.

Digital Library

[62]

J. Sitnicki. BPF sklookup - TCP SYN and UDP 0-len flood benchmarks. https://lore.kernel.org/bpf/[email protected]/, Aug 2020.

[63]

J. Sitnicki. Run a BPF program on socket lookup. https://lore.kernel.org/bpf/[email protected]/, May 2020.

[64]

I. Stoica, D. Adkins, S. Zhuang, S. Shenker, and S. Surana. Internet Indirection Infrastructure. In ACM SIGCOMM, 2002.

Digital Library

[65]

S. Sundaresan, N. Feamster, R. Teixeira, and N. Magharei. Measuring and Mitigating Web Performance Bottlenecks in Broadband Access Networks. In ACM Internet Measurement Conference (IMC), 2013.

[66]

B. Trammell, C. Perkins, T. Pauly, M. Kühlewind, and C. A. Wood. Post Sockets, An Abstract Programming Interface for the Transport Layer. Internet-Draft draft-trammell-taps-post-sockets-03, Internet Engineering Task Force, Oct. 2017. Work in Progress.

[67]

W3Techs Web Technology Surveys. Usage statistics and market share of cloudflare. https://w3techs.com/technologies/details/cn-cloudflare last accessed 06/2021.

[68]

M. Walfish, H. Balakrishnan, and S. Shenker. Untangling the Web from DNS. In Symposium on Networked Systems Design and Implementation (NSDI), 2004.

[69]

M. Walfish, J. Stribling, M. Krohn, H. Balakrishnan, R. Morris, and S. Shenker. Middleboxes No Longer Considered Harmful. In Symposium on Operating Systems Design and Implementation (OSDI), 2004.

[70]

J. M. Winett. Definition of a socket. RFC 147, https://rfc-editor.org/rfc/rfc147.txt, May 1971.

[71]

L. Zhang, A. Afanasyev, J. Burke, V. Jacobson, k. claffy, P. Crowley, C. Papadopoulos, L. Wang, and B. Zhang. Named Data Networking. ACM SIGCOMM Computer Communication Review (CCR), 44(3):66--73, July 2014.

[72]

J. Žorž, S. Steffann, P. Dražumerič, M. Townsley, A. Alston, G. Doering, J. Palet, J. Linkova, L. Balbinot, K. Meynell, and L. Howard. Best Current Operational Practice for Operators: IPv6 prefix assignment for end-users - persistent vs non-persistent, and what size to choose. https://www.ripe.net/publications/docs/ripe-690, October 2017.

Cited By

Sattler PZirngibl JJonker MGasser OCarle GHolz R(2023)Packed to the Brim: Investigating the Impact of Highly Responsive Prefixes on Internet-wide Measurement CampaignsProceedings of the ACM on Networking10.1145/36291461:CoNEXT3(1-21)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3629146
Liu HDiallo APatras P(2023)Amoeba: Circumventing ML-supported Network Censorship via Adversarial Reinforcement LearningProceedings of the ACM on Networking10.1145/36291311:CoNEXT3(1-25)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3629131
Tusa FGriffin DRio M(2023)Homomorphic RoutingProceedings of the 2nd ACM SIGCOMM Workshop on Future of Internet Routing & Addressing10.1145/3607504.3609287(1-7)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3607504.3609287
Show More Cited By

Index Terms

The ties that un-bind: decoupling IP from web services and sockets for robust addressing agility at CDN-scale
1. Networks

Recommendations

RFC 8539: Softwire Provisioning Using DHCPv4 over DHCPv6
Read More
Datacast: a scalable and efficient reliable group data delivery service for data centers
CoNEXT '12: Proceedings of the 8th international conference on Emerging networking experiments and technologies

Reliable Group Data Delivery (RGDD) is a pervasive traffic pattern in data centers. In an RGDD group, a sender needs to reliably deliver a copy of data to all the receivers. Existing solutions either do not scale due to the large number of RGDD groups (...
Read More
A new location layer for the TCP/IP protocol stack

The IPv4 address space is quickly getting exhausted, putting a tremendous pressure on the adoption of even more NAT levels or IPv6. On the other hand, many authors propose the adoption of new Internet addressing capabilities, namely content-based ...
Read More

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCOMM '21: Proceedings of the 2021 ACM SIGCOMM 2021 Conference

August 2021

868 pages

ISBN:9781450383837

DOI:10.1145/3452296

General Chairs:
Fernando Kuipers
Delft University of Technology
,
Matthew Caesar
University of Illinois at Urbana-Champaign

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCOMM: ACM Special Interest Group on Data Communication

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 August 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF (National Science Foundation)

Conference

SIGCOMM '21

Sponsor:

SIGCOMM

SIGCOMM '21: ACM SIGCOMM 2021 Conference

August 23 - 27, 2021

Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 554 of 3,547 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
6,570
Total Downloads

Downloads (Last 12 months)402
Downloads (Last 6 weeks)38

Other Metrics

View Author Metrics

Citations

Cited By

Sattler PZirngibl JJonker MGasser OCarle GHolz R(2023)Packed to the Brim: Investigating the Impact of Highly Responsive Prefixes on Internet-wide Measurement CampaignsProceedings of the ACM on Networking10.1145/36291461:CoNEXT3(1-21)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3629146
Liu HDiallo APatras P(2023)Amoeba: Circumventing ML-supported Network Censorship via Adversarial Reinforcement LearningProceedings of the ACM on Networking10.1145/36291311:CoNEXT3(1-25)Online publication date: 28-Nov-2023
https://dl.acm.org/doi/10.1145/3629131
Tusa FGriffin DRio M(2023)Homomorphic RoutingProceedings of the 2nd ACM SIGCOMM Workshop on Future of Internet Routing & Addressing10.1145/3607504.3609287(1-7)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3607504.3609287
Singanamalla SParacha MAhmad SHoyland JValenta LSafronov YWu PGalloni AHeimerl KSullivan NWood CFayed MBarakat CPelsser CBenson TChoffnes D(2022)Respect the ORIGIN!Proceedings of the 22nd ACM Internet Measurement Conference10.1145/3517745.3561453(664-678)Online publication date: 25-Oct-2022
https://dl.acm.org/doi/10.1145/3517745.3561453
Luo ZLi TZhang QZhu RSong TSchulzrinne HWählisch MZhang L(2022)A global name mapping system for ICN-IP coexistenceProceedings of the 9th ACM Conference on Information-Centric Networking10.1145/3517212.3559492(189-191)Online publication date: 6-Sep-2022
https://dl.acm.org/doi/10.1145/3517212.3559492
Babakian AMonclus PBraun RLipman J(2022)A Retrospective on Workload Identifiers: From Data Center to Cloud-Native NetworksIEEE Access10.1109/ACCESS.2022.321129310(105518-105527)Online publication date: 2022
https://doi.org/10.1109/ACCESS.2022.3211293
Devraj AWang LRexford J(2021)REDACTACM SIGCOMM Computer Communication Review10.1145/3503954.350395751:4(15-22)Online publication date: 3-Dec-2021
https://dl.acm.org/doi/10.1145/3503954.3503957
Gigis PCalder MManassakis LNomikos GKotronis VDimitropoulos XKatz-Bassett ESmaragdakis GKuipers FCaesar M(2021)Seven years in the life of Hypergiants' off-netsProceedings of the 2021 ACM SIGCOMM 2021 Conference10.1145/3452296.3472928(516-533)Online publication date: 9-Aug-2021
https://dl.acm.org/doi/10.1145/3452296.3472928
Koch TKatz-Bassett EHeidemann JCalder MArdi CLi KKuipers FCaesar M(2021)Anycast In contextProceedings of the 2021 ACM SIGCOMM 2021 Conference10.1145/3452296.3472891(398-417)Online publication date: 9-Aug-2021
https://dl.acm.org/doi/10.1145/3452296.3472891

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents