Abstract

The main premise of this work is that since large cloud providers can and do manipulate probe packets that traverse their privately owned and operated backbones, standard traceroute-based measurement techniques are no longer a reliable means for assessing network connectivity in large cloud provider infrastructures. In response to these developments, we present a new empirical approach for elucidating private connectivity in today's Internet. Our approach relies on using only "light-weight" ( i.e., simple, easily-interpretable, and readily available) measurements, but requires applying a "heavy-weight" or advanced mathematical analysis. In particular, we describe a new method for assessing the characteristics of network path connectivity that is based on concepts from Riemannian geometry ( i.e., Ricci curvature) and also relies on an array of carefully crafted visualizations ( e.g., a novel manifold view of a network's delay space). We demonstrate our method by utilizing latency measurements from RIPE Atlas anchors and virtual machines running in data centers of three large cloud providers to (i) study different aspects of connectivity in their private backbones and (ii) show how our manifold-based view enables us to expose and visualize critical aspects of this connectivity over different geographic scales.
- Emile Aben. 2021. Latency Into Your Network - As Seen From RIPE Atlas. (2021). https://labs.ripe.net/author/emileaben/latency-into-your-network-as-seen-from-ripe-atlas/Google Scholar
- Bruno Abrahao and Robert Kleinberg. 2008. On the internet delay space dimensionality. In Proceedings of the 8th ACM SIGCOMM conference on Internet measurement. 157--168.Google Scholar
Digital Library
- Najam Ahmad. 2016. Building one of the highest-capacity subsea cables in the Pacific . https://engineering.fb.com/connectivity/building-one-of-the-highest-capacity-subsea-cables-in-the-pacific/. (2016).Google Scholar
- Marder Alexander, Matthew Luckie, Amogh Dhamdhere, Bradley Huffaker, Claffy KC, and Smith M. Jonathan. 2018. Pushing the Boundaries with bdrmapIT: Mapping Router Ownership at Internet Scale. In Proceedings of the ACM Internet Measurement Conference.Google Scholar
- Anima Anandkumar, Avinatan Hassidim, and J. Kelner. 2011. Topology discovery of sparse random graphs with few participants. ArXiv , Vol. abs/1102.5063 (2011).Google Scholar
- M. Weber andd J. Jost and E. Saucan. 2018. Detecting the coarse geometry of networks. In Proceedings of NeurIPS 2018 Workshop.Google Scholar
- David Andersen, Hari Balakrishnan, Frans Kaashoek, and Robert Morris. 2001. Resilient Overlay Networks. In Proceedings of ACM Symposium on Operating System Principles.Google Scholar
Digital Library
- T. Arnold, E. Gurmericliler, G. Essig, A. Gupta, M. Calder, V. Giotsas, and E. Katz-Bassett. 2020 a. (How Much) Does a Private WAN Improve Cloud Performance?. In Proceedings of IEEE INFOCOM .Google Scholar
- T. Arnold, J. He, W. Jiang, M. Calder, Í. Cunha, V. Giotsas, and E. Katz-Bassett. 2020 b. Cloud Provider Connectivity in the Flat Internet. Proc. ACM IMC'20 .Google Scholar
- AT&T. 2020. Global IP Network Performance Measurements . https://ipnetwork.bgtmo.ip.att.net/pws/network_delay.html . (2020).Google Scholar
- B. Augustin, X. Cuvellier, B. Orgogozo, F. Viger, T. Friedman, M. Latapy, C. Magnien, and R. Teixeira. 2006. Avoiding traceroute anomalies with Paris traceroute. In Proceedings of Internet Measurement Conference.Google Scholar
- Brice Augustin, Balachander Krishnamurthy, and Walter Willinger. 2009. IXPs: Mapped?. In Proceedings of the ACM Internet Measurement Conference.Google Scholar
Digital Library
- AWS. 2020. AWS Global Cloud Infrstructure . https://www.infrastructure.aws . (2020).Google Scholar
- R. Beverly. 2016. Yarrp'ing the Internet: Randomized High-Speed Active Topology Discovery. In Proceedings of Internet Measurement Conference.Google Scholar
Digital Library
- D. Bhattacherjee, S. Abdu Jyothi, I. Nadi Bozkurt, M. Tirmazi, W. Aqeel, A. Aguirre, B. Chandrasekaran, B. Godfrey, G. Laughlin, B. Maggs, and A. Singla. 2018. cISP: A Speed-of-Light Internet Service Provider . arXiv 1809.10897 (2018).Google Scholar
- Béla Bollobás, Svante Janson, and Oliver Riordan. 2010. The cut metric, random graphs, and branching processes. Journal of statistical physics , Vol. 140, 2 (2010), 289--335.Google Scholar
Cross Ref
- Ilker Bozkurt, Waqar Aqeel, Debopam Bhattacherjee, Balakrishnan Chandrasekaran, Brighten Godfrey, Gregory Laughlin, Bruce Maggs, and Ankit Singlar. 2018. Dissecting Latency in the Internet's Fiber Infrastructure . arXiv 1811.10737 (2018).Google Scholar
- CAIDA. 2020. Archipelago Measurement Infrastructure . http://www.caida.org . (2020).Google Scholar
- Fan Chung, Mark Garrett, Ronald Graham, and David Shallcross. 2000. Distance Realization Problems with Applications to Internet Tomography. J. Comput. System Sci. , Vol. 63 (12 2000), 432--448. https://doi.org/10.1006/jcss.2001.1785Google Scholar
Digital Library
- Keenan Crane, Clarisse Weischedel, and Max Wardetzky. 2013. Geodesics in Heat: A New Approach to Computing Distance Based on Heat Flow. ACM Transactions on Graphics , Vol. 32 (09 2013). https://doi.org/10.1145/2516971.2516977Google Scholar
Digital Library
- Alfredo Cuzzocrea, Alexis Papadimitriou, Dimitrios Katsaros, and Yannis Manolopoulos. 2012. Edge betweenness centrality: A novel algorithm for QoS-based topology control over wireless sensor networks. Journal of Network and Computer Applications , Vol. 35, 4 (2012), 1210--1217.Google Scholar
Digital Library
- G. Detal, B. Hesmans, O. Bonaventure, Y. Vanaubel, and B. Donnet. 2013. Revealing Middlebox Interference with Tracebox. In Proc. ACM IMC'13 .Google Scholar
- R. Durairajan, S. Ghosh, X. Tang, P. Barford, and B. Eriksson. 2013. Internet Atlas: A Geographic Database of the Internet. In Proceedings of the ACM HotPlanet Workshop.Google Scholar
- Ramakrishnan Durairajan, Joel Sommers, and Paul Barford. 2014. Layer 1-informed internet topology measurement. In Proceedings of the 2014 Conference on Internet Measurement Conference. 381--394.Google Scholar
Digital Library
- R. Durairajan, J. Sommers, W. Willinger, and P. Barford. 2015. InterTubes: A Study of the US Long-haul Fiber-optic Infrastructure. In Proceedings of ACM SIGCOMM.Google Scholar
- E. Katz-Bassett and J. P. John and A. Krishnamurthy and D. Wetherall and T. Anderson and Y. Chawathe. 2006. Towards IP Geolocation Using Delay and Topology Measurements. In Proceedings of the ACM Internet Measurement Conference.Google Scholar
- B. Eriksson, P. Barford, B. Maggs, and R. Nowak. 2012. Posit: A Lightweight Approach for IP Geolocation . ACM SIGMETRICS Performance Evaluation Review (2012).Google Scholar
Digital Library
- ESRI. 2020. ArcGIS: The Mapping and Analytics Platform . https://www.esri.com/en-us/arcgis/about-arcgis/overview . (2020).Google Scholar
- F. Dabek and R. Cox and F. Kaashoek and R. Morris. 2004. Vivaldi: A Decentralized Network Coordinate System. In Proceedings of ACM SIGCOMM.Google Scholar
- Charles Fefferman, Sergei Ivanov, Matti Lassas, and Hariharan Narayanan. 2020. Reconstruction of a Riemannian manifold from noisy intrinsic distances. SIAM Journal on Mathematics of Data Science , Vol. 2, 3 (2020), 770--808.Google Scholar
Cross Ref
- W. Felter, A. Ferreira, R. Rajamony, and J. Rubio. 2015. An Updated Performance Comparison of Virtual Machines and Linux Containers. In 2015 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) .Google Scholar
- Paul Francis, Sugih Jamin, Cheng Jin, Yixin Jin, Danny Raz, Yuval Shavitt, and Lixia Zhang. 2001. IDMaps: A Global Internet Host Distance Estimation Service . IEEE/ACM Transactions on Networking , Vol. 9, 5 (2001).Google Scholar
Digital Library
- P. Gill, Y. Ganjali, B. Wong, and D. Lie. 2010. Dude, Where's that IP?: Circumventing Measurement-based IP Geolocation. In Proceedings of the USENIX Security Symposium.Google Scholar
- Google. 2020. Curie subsea cable set to transmit to Chile, with a pit stop to Panama . https://cloud.google.com/blog/products/infrastructure/curie-subsea-cable-set-to-transmit-to-chile-with-a-pit-stop-to-panama . (2020).Google Scholar
- Google. 2020 (last updated). Virtual Private Cloud (VPC) Documentation: Traceroute to external IP addresses. https://cloud.google.com/vpc/docs/vpctraceroute . (2020 (last updated)).Google Scholar
- B. Gueye, A. Ziviani, M. Crovella, and S. Fdida. 2006. Constraint-Based Geolocation of Internet Hosts . IEEE/ACM Transactions on Networking , Vol. 14, 6 (2006).Google Scholar
Digital Library
- K. Gummadi, S. Saroiu, and S. Gribble. 2002. King: Estimating latency between arbitrary internet end hosts. In Proceedings of the ACM Internet Measurement Workshop.Google Scholar
- C. Guo, L. Yuan, D. Xiang, Y. Dang, R. Huang, D. Maltz, Z. Liu, V. Wang, B. Pang, H. Chen, Z.-W. Lin, and V. Kurien. 2015. Pingmesh: A Large-Scale System for Data Center Network Latency Measurement and Analysis. In Proceedings of the ACM SIGCOMM Conference.Google Scholar
- O. Haq, M. Raja, and F. Dogar. 2017. Measuring and Improving the Reliability of Wide-Area Cloud Paths. In Proceedings of WWW'17.Google Scholar
- C.-Y. Hong, S. Kandula nd R. Mahajan, M. Zhang, V. Gill, M. Nanduri, and R. Wattenhofer. 2013. Achieving High Utilization with Software-Driven WAN. In Proceedings of the ACM SIGCOMM Conference.Google Scholar
- Peter J Huber. 2004. Robust statistics . Vol. 523. John Wiley & Sons.Google Scholar
- V. Jacobson. 1989. traceroute. ftp://ftp.ee.lbl.gov/traceroute.tar.gz . (1989).Google Scholar
- S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh, S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla, U. Hoelzle, S. Stuart, and A. Vahdat. 2013. B4: Experience with a Globally-Deployed Software Defined WAN. In Proceedings of the ACM SIGCOMM Conference.Google Scholar
- M. Jimenez and H. Kwok. 2017. Building Express Backbone: Facebook's new long-haul Network . https://engineering.fb.com/data-center-engineering/building-express-backbone-facebook-s-new-long-haul-network/. (2017).Google Scholar
- R. Jones and D. Fitzgerald. 2020. Google Plans Fiber-Optic Network to Connect Via Saudi Arabia and Israel for First Time . https://www.wsj.com/articles/google-plans-fiber-optic-network-to-connect-via-saudi-arabia-and-israel-for-first-time-11606143590 . (2020).Google Scholar
- C. Kaufmann. 2018. ICN - Akamai's Backbone . https://www.linx.net/ wp-content/uploads/LINX101-Akamai-ICN-ChristianKaufmann.pdf . (2018).Google Scholar
- A. Kesavan. 2019. Comparing the Network Performance of AWS, Azure and GCP . https://pc.nanog.org/static/published/meetings/NANOG75/1909/20190218_Kesavan_Comparing_The_Network_v1.pdf . (2019).Google Scholar
- Dmitri Krioukov, Fragkiskos Papadopoulos, Maksim Kitsak, Amin Vahdat, and Marián Boguná. 2010. Hyperbolic geometry of complex networks. Physical Review E , Vol. 82, 3 (2010), 036106.Google Scholar
Cross Ref
- A. Li, X. Yang, S. Kandula, and M. Zhang. 2010. CloudCmp: Comparing Public Cloud Providers. In Proceedings of the ACM Internet Measurement Conference.Google Scholar
- L. Li, D. Alderson, W. Willinger, and J.C. Doyle. 2004. A First-principles Approach to Understanding the Internet's Router-level Topology. In Proc. ACM SIGCOMM'04.Google Scholar
- Harsha V. Madhyastha, Tomas Isdal, Michael Piatek, Colin Dixon, Thomas Anderson, Arvind Krishnamurthy, and Arun Venkataramani. 2006. iPlane: An Information Plane for Distributed Services. In Proceedings of the USENIX Symposium on Network Systems Design and Implementation.Google Scholar
- Zhuoqing Morley Mao, Jennifer Rexford, Jia Wang, and Randy H Katz. 2003. Towards an Accurate AS-Level Traceroute Tool. In Proceedings of ACM SIGCOMM.Google Scholar
Digital Library
- Microsoft. 2015. Microsoft Invests in Subsea Cables to Connect Datacenters Globally . https://www.esri.com/en-us/arcgis/about-arcgis/overview . (2015).Google Scholar
- Microsoft. 2020. Azure Microsoft Global Network Map . https://docs.microsoft.com/en-us/azure/networking/microsoft-global-network . (2020).Google Scholar
- Microsoft. 2021. Azure Virtual Network frequently asked questions (FAQ) . https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-faq. (2021).Google Scholar
- R. Motamedi, R. Rejaie, and W. Willinger. 2014. A survey of techniques for internet topology discovery. In IEEE Communications Survey & Tutorials.Google Scholar
- Hariharan Narayanan and Sanjoy Mitter. 2010. Sample complexity of testing the manifold hypothesis. In Proceedings of the 23rd International Conference on Neural Information Processing Systems-Volume 2 . 1786--1794.Google Scholar
- E. Ng and H. Zhang. 2002. Predicting Internet Network Distance with Coordinates-based Approaches. In Proceedings of IEEE INFOCOM.Google Scholar
- C. Ni, Y. Lin, F. Luo, and J. Gao. 2019. Community Detection on Networks with Ricci Flow . Scientific Reports , Vol. 9, 9984 (2019).Google Scholar
- Chien-Chun Ni, Yu-Yao Lin, Jie Gao, Xianfeng David Gu, and Emil Saucan. 2015. Ricci Curvature of the Internet Topology . arXiv 1501.04138 (2015).Google Scholar
- Yann Ollivier. 2009. Ricci curvature of Markov chains on metric spaces . Journal of Functional Analysis , Vol. 256, 3 (2009), 810--864.Google Scholar
Cross Ref
- Y. Ollivier. 2011. A Visual Introduction to Riemannian Curvatures and Some Discrete Generalizations . Analysis and Geometry of Metric Measure Spaces: Lecture Notes of the 50th Seminaire de Mathematiques Superieures (SMS). (2011).Google Scholar
- N. Patwari, A. Hero, and A. Pacholski. 2008. Manifold learning visualization of network traffic data. In Proceedings of the ACM Internet Measurement Conference.Google Scholar
- Marcelo Pias, Jon Crowcroft, Steve Wilbur, Tim Harris, and Saleem Bhatti. 2003. Lighthouses for Scalable Distributed Location . Proceedings of the USENIX International Workshop on Peer to Peer Systems .Google Scholar
Cross Ref
- John W Pinney and David R Westhead. 2006. Betweenness-based decomposition methods for social and biological networks. Interdisciplinary statistics and bioinformatics , Vol. 25 (2006), 87--90.Google Scholar
- K. Putzier. 2020. Property Investors See Fiber-Optic Cables as 'Railroads of the Future' . (2020).Google Scholar
- R. Mahajan and M. Zhang and L. Poole and V. Pai. 2008. Uncovering Performance Differences in Backbone ISPs with Netdiff. In Proceedings of the USENIX Symposium on Network Systems Design and Implementation.Google Scholar
- Corinne Reichert. 2018. Hawaiki subsea cable goes live . https://www.zdnet.com/article/hawaiki-subsea-cable-goes-live/. (2018).Google Scholar
- RIPE. 2020. RIPE Atlas . https://atlas.ripe.net . (2020).Google Scholar
- B. Schlinker, H. Kim, T. Cui, E. Katz-Bassett, H. V. Madhyastha, I. Cunha, J. Quinn, S. Hasan, P. Lapukhov, and H. Zeng. 2017. Engineering Egress with Edge Fabric: Steering Oceans of Content to the World. In Proceedings of the ACM SIGCOMM Conference.Google Scholar
- Y. Shavitt and T. Tankel. 2004. On the Curvature of the Internet and its Usage for Overlay Construction and Distance Estimation. In Proceedings of IEEE INFOCOM .Google Scholar
- R. Shea, F. Wang, H. Wang, and J. Liu. 2014. A Deep Investigation Into Network Performance in Virtual Machine Based Cloud Environments. In Proceedings of IEEE INFOCOM .Google Scholar
- Rob Sherwood, Adam Bender, and Neil Spring. 2008. Discarte: A Disjunctive Internet Cartographer . Proceedings of ACM SIGCOMM.Google Scholar
Digital Library
- Ankit Singla, Balakrishnan Chandrasekaran, P Brighten Godfrey, and Bruce Maggs. 2014. The Internet at the Speed of Light. In Proceedings of the ACM ACM Workshop on Hot Topics in Networks.Google Scholar
Digital Library
- J. Sommers, P. Barford, and B. Eriksson. 2011. On the prevalence and characteristics of MPLS deployments in the open Internet. In Proc. of the ACM Internet Measurement Conference (IMC'11).Google Scholar
- Neil Spring, Ratul Mahajan, and David Wetherall. 2002. Measuring ISP Topologies with Rocketfuel. In Proceedings of ACM SIGCOMM.Google Scholar
Digital Library
- Liying Tang and Mark Crovella. 2003. Virtual landmarks for the Internet. In Proceedings of the ACM Internet Measurement Conference.Google Scholar
Digital Library
- V. Padmanabhan and L. Subramanian. 2001. An Investigation of Geographic Mapping Techniques for Internet Hosts. In Proceedings of ACM SIGCOMM.Google Scholar
- A. Vahdat, D. Clark, and J. Rexford. 2015. A Purpose-built Global Network: Google's Move to SDN . ACMQueue , Vol. 13, 8 (2015).Google Scholar
- A. Vogel, D. Griebler, C. Schepke, and L. Fernandes. 2017. An Intra-Cloud Networking Performance Evaluation on CloudStack Environment. In IEEE 25th Euromicro International Conference on Parallel, Distributed and Network-Based Processing .Google Scholar
- G. Wang and T. Ng. 2010. The Impact of Virtualization on Network Performance of Amazon EC2 Data Center. In Proceedings of IEEE INFOCOM .Google Scholar
- W. Willinger, D. Alderson, and J. C. Doyle. 2009. Mathematics and the Internet: A Source of Enormous Confusion and Great Potential. In Notices of the AMS.Google Scholar
- F. Wohlfart, N. Chatzis, C. Dabanoglu, G. Carle, and W. Willinger. 2018. Leveraging Interconnections for Performance: The Serving Infrastructure of a Large CDN. In Proceedings of the ACM SIGCOMM Conference.Google Scholar
- B. Wong, A. Slivkins, and E. Sirer. 2005. Meridian: A Lightweight Network Location Service Without Virtual Coordinates. In Proceedings of ACM SIGCOMM.Google Scholar
- B. Wong, I. Stoyanov, and E. Sirer. 2007. Octant: A Comprehensive Framework for the Geolocation of Internet Hosts. In Proceedings of the USENIX Symposium on Network Systems Design and Implementation.Google Scholar
- M. Xavier, M. Neves, F. Rossi, T. Ferreto, T. Lange, and C. De Rose. 2013. Performance Evaluation of Container-based Virtualization for High Performance Computing Environments. In IEEE 21st Euromicro International Conference on Parallel, Distributed, and Network-Based Processing .Google Scholar
- K.-K. Yap, M. Motiwala, J. Rahe, S. Padgett, M. Holliman, G. Baldus, M. Hines, T. Kim, A. Narayanan, A. Jain, V. Lin, C. Rice, B. Rogan, A. Singh, B. Tanaka, M. Verma, M. Tariq, M. Tierney, D. Trumic, V. Valancius, P. Sood, C. Ying, M. Kallahalla, B. Koley, and A. Vahdat. 2017. Taking the Edge off with Espresso: Scale, Reliability and Programmability for Global Internet Peering. In Proceedings of the ACM SIGCOMM Conference.Google Scholar
- B. Yeganeh, R. Durairajan, R. Rejaie, and W. Willinger. 2019. How Cloud Traffic Goes Hiding: A Study of Amazon's Peering Fabric. In Proceedings of the Internet Measurement Conference (IMC).Google Scholar
- B. Yeganeh, R. Durairajan, R. Rejaie, and W. Willinger. 2020. A First Comparative Characterization of Multi-cloud Connectivity in Today's Internet. In Proceedings of PAM'20.Google Scholar
- Beichuan Zhang, Raymond Liu, Daniel Massey, and Lixia Zhang. 2005. Collecting the Internet AS-Level Topology . ACM SIGCOMM Computer Communications Review , Vol. 35, 1 (2005).Google Scholar
Digital Library
- Yin Zhang and Nick Duffield. 2001. On the Constancy of Internet Path Properties. In Proceedings of the ACM Internet Measurement Workshop .Google Scholar
Digital Library
Index Terms
Curvature-based Analysis of Network Connectivity in Private Backbone Infrastructures
Recommendations
Curvature-based Analysis of Network Connectivity in Private Backbone Infrastructures
SIGMETRICS/PERFORMANCE '22: Abstract Proceedings of the 2022 ACM SIGMETRICS/IFIP PERFORMANCE Joint International Conference on Measurement and Modeling of Computer SystemsThe main premise of this work is that since large cloud providers can and do manipulate probe packets that traverse their privately owned and operated backbones, standard traceroute-based measurement techniques are no longer a reliable means for ...
Curvature-based Analysis of Network Connectivity in Private Backbone Infrastructures
SIGMETRICS '22The main premise of this work is that since large cloud providers can and do manipulate probe packets that traverse their privately owned and operated backbones, standard traceroute-based measurement techniques are no longer a reliable means for ...
Constrained large Margin Local Projection algorithms and extensions for multimodal dimensionality reduction
A Constrained large Margin Local Projection (CMLP) technique for multimodal dimensionality reduction is proposed. We elaborate the criterion of CMLP from a pairwise constrained marginal perspective. Four effective CMLP solution schemes are presented and ...






Comments