skip to main content
article

Lord of the links: a framework for discovering missing links in the internet topology

Published: 01 April 2009 Publication History

Abstract

The topology of the Internet at the Autonomous System (AS) level is not yet fully discovered despite significant research activity. The community still does not know how many links are missing, where these links are and finally, whether the missing links will change our conceptual model of the Internet topology. An accurate and complete model of the topology would be important for protocol design, performance evaluation and analyses. The goal of our work is to develop methodologies and tools to identify and validate such missing links between ASes. In this work, we develop several methods and identify a significant number of missing links, particularly of the peer-to-peer type. Interestingly, most of the missing AS links that we find exist as peer-to-peer links at the Internet Exchange Points (IXPs). First, in more detail, we provide a large-scale comprehensive synthesis of the available sources of information. We cross-validate and compare BGP routing tables, Internet Routing Registries, and traceroute data, while we extract significant new information from the less-studied Internet Exchange Points (IXPs). We identify 40% more edges and approximately 300% more peer-to-peer edges compared to commonly used data sets. All of these edges have been verified by either BGP tables or traceroute. Second, we identify properties of the new edges and quantify their effects on important topological properties. Given the new peer-to-peer edges, we find that for some ASes more than 50% of their paths stop going through their ISPs assuming policy-aware routing. A surprising observation is that the degree of an AS may be a poor indicator of which ASes it will peer with.

References

[1]
S. Floyd and V. Paxson, "Difficulties in simulating the Internet," IEEE/ACM Trans. Networking, vol. 9, no. 4, pp. 392-403, Aug. 2001.
[2]
H. Chang, R. Govindan, S. Jamin, S. Shenker, and W. Willinger, "Towards capturing representative AS-level Internet topologies," Computer Networks, vol. 44, no. 6, pp. 737-755, 2004.
[3]
B. Zhang, R. A. Liu, D. Massey, and L. Zhang, "Collecting the Internet as-level topology," ACM SIGCOMM Comput. Commun. Rev. (CCR), vol. 35, no. 1, pp. 53-61, Jan. 2005.
[4]
Y. Shavitt and E. Shir, "DIMES: let the Internet measure itself," ACM SIGCOMM Comput. Commun. Rev. (CCR), vol. 35, no. 5, pp. 71-74, Oct. 2005.
[5]
X. Dimitropoulos, D. Krioukov, and G. Riley, "Revisiting Internet AS-level topology discovery," in Proc. Passive and Active Measurement (PAM) Workshop, Boston, MA, 2005.
[6]
P. Mahadevan, D. Krioukov, M. Fomenkov, B. Huffaker, X. Dimitropoulos, K. claffy, and A. Vahdat, "The Internet AS-level topology: Three data sources and one definitive metric," ACM SIGCOMM Comput. Commun. Rev. (CCR), vol. 36, no. 1, pp. 17-26, Jan. 2006.
[7]
H. Chang, S. Jamin, and W. Willinger, "To peer or not to peer: Modeling the evolution of the Internet's AS-level topology," in Proc. IEEE INFOCOM, 2006, 12 pp.
[8]
L. Colitti, G. DiBattista, M. Patrignani, M. Pizzonia, and M. Rimondini, "Investigating prefix propagation through active BGP probing," Microprocessors Microsyst., vol. 31, no. 7, pp. 460-474, 2007.
[9]
R. Cohen and D. Raz, "The Internet dark matter--on the missing links in the AS connectivity map," in Proc. IEEE INFOCOM, 2006, 12 pp.
[10]
Internet Routing Registry. {Online}. Available: http://www.irr.net
[11]
K. Xu, Z. Duan, Z. Zhang, and J. Chandrashekar, "On properties of Internet exchange points and their impact on AS tolology and relationship," in Networking, 2004, pp. 284-295.
[12]
G. Siganos and M. Faloutsos, "Analyzing BGP policies: Methodology and tool," in Proc. IEEE INFOCOM, 2004, vol. 3, pp. 1640-1651.
[13]
Y. He, G. Siganos, M. Faloutsos, and S. Krishnamurthy, "A systematic framework for unearthing the missing links: Measurements and impact," in USENIX NSDI, Cambridge, MA, Apr. 2007.
[14]
Oregon Routeview Project. {Online}. Available: http://www.route-views.org
[15]
Ripe Route Information Service. {Online}. Available: http://www.ripe. net/ris
[16]
A. Lakhina, J. W. Byers, M. Crovella, and P. Xie, "Sampling biases in IP topology measurements," in Proc. IEEE INFOCOM, 2003, vol. 1, pp. 332-341.
[17]
D. Achlioptas, A. Clauset, D. Kempe, and C. Moore, "On the bias of traceroute sampling, or power-law degree distributions in regular graphs," in Proc. STOC'05, Baltimore, MD, May 2005, pp. 694-703.
[18]
R. V. Oliveira, B. Zhang, and L. Zhang, "Observing the evolution of Internet AS topology," in Proc. ACM SIGCOMM, 2007, pp. 313-324.
[19]
Z. Mao, J. Rexford, J. Wang, and R. Katz, "Towards an accurate AS-level traceroute tool," in Proc. ACM SIGCOMM, 2003, pp. 365-378.
[20]
Z. M. Mao, D. Johnson, J. Rexford, J. Wang, and R. Katz, "Scalable and accurate identification of AS-level forwarding paths," in Proc. IEEE INFOCOM, 2004, vol. 3, pp. 1605-1615.
[21]
B. Augustin, X. Cuvellier, B. Orgogozo, F. Viger, T. Friedman, M. Latapy, C. Magnien, and R. Teixeira, "Avoiding traceroute anomalies with Paris traceroute," in Proc. ACM IMC'06, Rio de Janeiro, Brazil, Oct. 2006, pp. 153-158.
[22]
E. Shir, Dec. 2005, Personal communication via e-mail.
[23]
B. Donnet, P. Raoult, T. Friedman, and M. Crovella, "Efficient algorithms for large-scale topology discovery," in Proc. ACM SIGMETRICS , Jun. 2005, pp. 327-338.
[24]
N. Spring, R. Mahajan, D. Wetherall, and T. Anderson, "Measuring ISP topologies with Rocketfuel," IEEE/ACM Trans. Netw., vol. 12, no. 1, pp. 2-16, Feb. 2004.
[25]
S. Jaiswal, A. Rosenberg, and D. Towsley, "Comparing the structure of power law graphs and the Internet AS graph," in Proc. 12th IEEE Int. Conf. Network Protocols (ICNP'04), 2004, pp. 294-303.
[26]
S. Jin and A. Bestavros, "An empirical study of inherent routing bias in variable-degree networks," Boston Univ., Boston, MA, Tech. Rep., 2003.
[27]
J. Xia and L. Gao, "On the evaluation of AS relationship inferences," in Proc. IEEE Globecom, 2004, vol. 3, pp. 1373-1377.
[28]
Traceroute. {Online}. Available: http://www.traceroute.org
[29]
M. Faloutsos, P. Faloutsos, and C. Faloutsos, "On power-law relationships of the Internet topology," in Proc. ACM SIGCOMM, 1999, pp. 251-262.
[30]
A. Medina, I. Matta, and J. Byers, "On the origin of powerlaws in Internet topologies," ACM SIGCOMM Comput. Commun. Rev. (CCR), vol. 30, no. 2, pp. 18-34, Apr. 2000.
[31]
Q. Chen, H. Chang, R. Govindan, S. Jamin, S. J. Shenker, and W. Willinger, "The origin of power laws in Internet topologies revisited," in Proc. IEEE INFOCOM, 2002, pp. 608-617.
[32]
L. Gao and F. Wang, "The extent of AS path inflation by routing policies," in Proc. IEEE Globecom, 2002, vol. 3, pp. 2180-2184.
[33]
N. T. Spring, R. Mahajan, and T. E. Anderson, "The causes of path inflation," in Proc. ACM SIGCOMM, 2003, p. 113-124.
[34]
M. Caesar and J. Rexford, "BGP routing policies in ISP networks," IEEE Network, vol. 19, no. 6, pp. 5-11, Nov./Dec. 2005.
[35]
Packete Cleaning House. {Online}. Available: http://www.pch.net
[36]
L. Amimi, A. Shaikh, and H. Schulzrinne, "Issues with inferring Internet topological attributes," in Proc. SPIE ITCom, 2002, vol. 4685, pp. 80-90.
[37]
Y. Hyun, A. Broido, and K. Claffy, Traceroute and BGP AS path incongruities. {Online}. Available: www.caida.org/outreach/papers/ 2003/ASP/
[38]
European Internet Exchange Assoc. {Online}. Available: http://www. euro-ix.net

Cited By

View all
  • (2024)metAScritic: Reframing AS-Level Topology Discovery as a Recommendation SystemProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688429(337-364)Online publication date: 4-Nov-2024
  • (2022)Peering Only? Analyzing the Reachability Benefits of Joining Large IXPs TodayPassive and Active Measurement10.1007/978-3-030-98785-5_15(338-366)Online publication date: 28-Mar-2022
  • (2020)TopoScopeProceedings of the ACM Internet Measurement Conference10.1145/3419394.3423627(266-280)Online publication date: 27-Oct-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image IEEE/ACM Transactions on Networking
IEEE/ACM Transactions on Networking  Volume 17, Issue 2
April 2009
319 pages

Publisher

IEEE Press

Publication History

Published: 01 April 2009
Revised: 13 January 2008
Received: 22 June 2007
Published in TON Volume 17, Issue 2

Author Tags

  1. BGP
  2. inter-domain
  3. internet
  4. measurement
  5. missing links
  6. routing
  7. topology

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)metAScritic: Reframing AS-Level Topology Discovery as a Recommendation SystemProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688429(337-364)Online publication date: 4-Nov-2024
  • (2022)Peering Only? Analyzing the Reachability Benefits of Joining Large IXPs TodayPassive and Active Measurement10.1007/978-3-030-98785-5_15(338-366)Online publication date: 28-Mar-2022
  • (2020)TopoScopeProceedings of the ACM Internet Measurement Conference10.1145/3419394.3423627(266-280)Online publication date: 27-Oct-2020
  • (2019)Stable and practical AS relationship inference with problinkProceedings of the 16th USENIX Conference on Networked Systems Design and Implementation10.5555/3323234.3323282(581-597)Online publication date: 26-Feb-2019
  • (2017)AS-Level Topology DiscoveryComputer Communications10.1016/j.comcom.2017.08.015112:C(47-57)Online publication date: 1-Nov-2017
  • (2017)The Impact of 16-bit and 32-bit ASNs Coexistence on the Accuracy of Internet AS GraphJournal of Network and Systems Management10.1007/s10922-016-9389-525:2(253-268)Online publication date: 1-Apr-2017
  • (2016)Detecting Unusually-Routed ASesProceedings of the 2016 Internet Measurement Conference10.1145/2987443.2987478(445-459)Online publication date: 14-Nov-2016
  • (2016)Hidden Internet topologies infoProceedings of the 2016 workshop on Fostering Latin-American Research in Data Communication Networks10.1145/2940116.2940118(4-6)Online publication date: 22-Aug-2016
  • (2015)Measuring the Internet Topology with SmartphonesProceedings of the 2015 ACM SIGCOMM Workshop on Crowdsourcing and Crowdsharing of Big (Internet) Data10.1145/2787394.2787398(45-50)Online publication date: 17-Aug-2015
  • (2015)A novel methodology to address the internet AS-level data incompletenessIEEE/ACM Transactions on Networking10.1109/TNET.2014.232312823:4(1314-1327)Online publication date: 1-Aug-2015
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media