Scanning the Internet for Liveness

Abstract

Internet-wide scanning depends on a notion of liveness: does a target IP address respond to a probe packet? However, the interpretation of such responses, or lack of them, is nuanced and depends on multiple factors, including: how we probed, how different protocols in the network stack interact, the presence of filtering policies near the target, and temporal churn in IP responsiveness. Although often neglected, these factors can significantly affect the results of active measurement studies. We develop a taxonomy of liveness which we employ to develop a method to perform concurrent IPv4 scans using ICMP, five TCP-based, and two UDP-based protocols, comprehensively capturing all responses to our probes, including negative and cross-layer responses. Leveraging our methodology, we present a systematic analysis of liveness and how it manifests in active scanning campaigns, yielding practical insights and methodological improvements for the design and the execution of active Internet measurement studies.

References

1. Lance Alt, Robert Beverly, and Alberto Dainotti. Uncovering Network Tarpits with Degreaser. In Proceedings of the 30th Annual Computer Security Applications Conference, ACSAC '14, New Orleans, Louisiana, USA, 2014.
2. Genevieve Bartlett, John Heidemann, and Christos Papadopoulos. Understanding Passive and Active Service Discovery. In Proceedings of ACM IMC 2007, San Diego, California, USA, 2007.
3. John Blackford and Mike Digdon. CPE WAN Management Protocol. Technical Report TR-069, Broadband Forum, November 2013. Issue 1 Amendment 5. CWMP v1.4.
4. Randy Bush, Olaf Maennel, Matthew Roughan, and Steve Uhlig. Internet Optometry: Assessing the Broken Glasses in Internet Reachability. In Proceedings of ACM IMC 2009, Chicago, Illinois, USA, 2009.
5. Xue Cai and John Heidemann. Understanding Block-level Address Usage in the Visible Internet. In Proceedings of ACM SIGCOMM 2010, New Delhi, India, 2010.
6. k. claffy, Y. Hyun, K. Keys, M. Fomenkov, and D. Krioukov. Internet Mapping: from Art to Science. In IEEE DHS Cybersecurity Applications and Technologies Conference for Homeland Security (CATCH), pages 205–211, Waltham, MA, Mar 2009.
7. TR-069 CPE WAN Management Protocol. https://www.broadband-forum.org/technical/download/TR-069_Amendment-5.pdf.
8. A. Dainotti, K. Benson, A. King, k. claffy, M. Kallitsis, E. Glatz, and X. Dimitropoulos. Estimating Internet address space usage through passive measurements. ACM CCR, 44(1):42–49, Jan 2014.
9. A. Dainotti, K. Benson, A. King, B. Huffaker, E. Glatz, X. Dimitropoulos, P. Richter, A. Finamore, and A. Snoeren. Lost in Space: Improving Inference of IPv4 Address Space Utilization. IEEE Journal on Selected Areas in Communications (JSAC), 34(6):1862–1876, Jun 2016.
10. Zakir Durumeric, David Adrian, Ariana Mirian, Michael Bailey, and J. Alex Halderman. A Search Engine Backed by Internet-Wide Scanning. In Proceedings of the 22nd ACM Conference on Computer and Communications Security, October 2015.
11. Zakir Durumeric, James Kasten, Michael Bailey, and J. Alex Halderman. Analysis of the HTTPS Certificate Ecosystem. In Proceedings of ACM IMC 2013, Barcelona, Spain, 2013. ACM.
12. Zakir Durumeric, Eric Wustrow, and J. Alex Halderman. ZMap: Fast Internet-wide Scanning and Its Security Applications. In Proceedings of the 22Nd USENIX Conference on Security, SEC'13, pages 605–620, Berkeley, CA, USA, 2013. USENIX Association.
13. Xun Fan and John Heidemann. Selecting Representative IP Addresses for Internet Topology Studies. In Proceedings of ACM IMC 2010, Melbourne, Australia, 2010.
14. Ramesh Govindan and Hongsuda Tangmunarunkit. Heuristics for Internet map discovery. In Proceedings of INFOCOM 2000, Tel Aviv, Israel, 2000.
15. M. H. Gunes and K. Saracc. Analyzing router responsiveness to active measurement probes. In Proceedings of PAM 2009, 2009.
16. John Heidemann, Yuri Pradkin, Ramesh Govindan, Christos Papadopoulos, and Joseph Bannister. Exploring Visible Internet Hosts through Census and Survey. Technical Report ISI-TR-2007-640, USC/Information Sciences Institute, May 2007.
17. Nadia Heninger, Zakir Durumeric, Eric Wustrow, and J. Alex Halderman. Mining Your Ps and Qs: Detection of Widespread Weak Keys in Network Devices. In Proceedings of the 21st USENIX Conference on Security Symposium, Security'12, Berkeley, CA, USA, 2012.
18. B. Huffaker, M. Fomenkov, D. Moore, and k. claffy. Macroscopic analyses of the infrastructure: measurement and visualization of Internet connectivity and performance. In PAM 2001, Amsterdam, Netherlands, 2001.
19. Sheharbano Khattak, David Fifield, Sadia Afroz, Mobin Javed, Srikanth Sundaresan, Vern Paxson, Steven J. Murdoch, and Damon McCoy. Do You See What I See?: Differential Treatment of Anonymous Users. In Proceedings of NDSS 2016, San Diego, CA, United States, 2016.
20. Marc Kührer, Thomas Hupperich, Jonas Bushart, Christian Rossow, and Thorsten Holz. Going Wild: Large-Scale Classification of Open DNS Resolvers. In Proceedings of ACM IMC 2015, Tokyo, Japan, 2015.
21. Derek Leonard and Dmitri Loguinov. Demystifying Service Discovery: Implementing an Internet-wide Scanner. In Proceedings of ACM IMC 2010, Melbourne, Australia, 2010.
22. M. Luckie, Y. Hyun, and B. Huffaker. Traceroute Probe Method and Forward IP Path Inference. In Proceedings of ACM IMC 2008, Vouliagmeni, Greece, 2008.
23. Antonio Nappa, Zhaoyan Xu, Juan Caballero, and Guofei Gu. CyberProbe: Towards Internet-Scale Active Detection of Malicious Servers. In Proceedings of NDSS 2014, San Diego, CA, USA, 2014.
24. Ramakrishna Padmanabhan, Amogh Dhamdhere, Emile Aben, kc claffy, and Neil Spring. Reasons Dynamic Addresses Change. In Proceedings of ACM IMC 2016, Santa Monica, California, USA, 2016.
25. Jeffrey Pang, James Hendricks, Aditya Akella, Roberto De Prisco, Bruce Maggs, and Srinivasan Seshan. Availability, Usage, and Deployment Characteristics of the Domain Name System. In Proceedings of ACM IMC 2004, Taormina, Sicily, Italy, 2004.
26. Jean-Jacques Pansiot and Dominique Grad. On Routes and Multicast Trees in the Internet. ACM CCR, 28(1):41–50, January 1998.
27. J. Postel. Internet Control Message Protocol. RFC 792, September 1981. https://tools.ietf.org/html/rfc792.
28. J. Postel. Transmission Control Protocol. RFC 793, September 1981. https://tools.ietf.org/html/rfc793.
29. N. Provos and P. Honeyman. ScanSSH - Scanning the Internet for SSH Servers. In 16th USENIX Systems Administration Conference (LISA), New York, NY, USA, 2001.
30. Lin Quan and John Heidemann. Detecting Internet Outages with Active Probing (extended). Technical Report ISI-TR-2011-672, USC/Information Sciences Institute, May 2010.
31. Lin Quan, John Heidemann, and Yuri Pradkin. When the Internet Sleeps: Correlating Diurnal Networks With External Factors (extended). Technical Report ISI-TR-2014-691b, USC/Information Sciences Institute, May 2014. (updated August 2014).
32. Philipp Richter, Georgios Smaragdakis, David Plonka, and Arthur Berger. Beyond Counting: New Perspectives on the Active IPv4 Address Space. In Proceedings of ACM IMC 2016, Santa Monica, California, USA, 2016.
33. Philipp Richter, Florian Wohlfart, Narseo Vallina-Rodriguez, Mark Allman, Randy Bush, Anja Feldmann, Christian Kreibich, Nicholas Weaver, and Vern Paxson. A Multi-perspective Analysis of Carrier-Grade NAT Deployment. In Proceedings of ACM IMC 2016, Santa Monica, California, USA, 2016.
34. Matthew Roughan, Walter Willinger, Olaf Maennel, Debbie Perouli, and Randy Bush. 10 Lessons from 10 Years of Measuring and Modeling the Internet's Autonomous Systems. IEEE Journal on Selected Areas in Communications, 29(9):1810–1821, 2011.
35. Yuval Shavitt and Eran Shir. DIMES: Let the Internet Measure Itself. ACM CCR, 35(5):71–74, October 2005.
36. Neil Spring, Ratul Mahajan, and David Wetherall. Measuring ISP Topologies with Rocketfuel. In Proceedings of ACM SIGCOMM 2002, New York, NY, USA, 2002.
37. P. Srisuresh, B. Ford, S. Sivakumar, and S. Guha. NAT Behavioral Requirements for ICMP. RFC 5508 (Best Current Practice), April 2009. Updated by RFC 7857.
38. Mark Thomas, Leigh Metcalf, Jonathan M. Spring, Paul Krystosek, and Katherine Prevost. SiLK: A tool suite for unsampled network flow analysis at scale. In IEEE BigData Congress, pages 184–191, Anchorage, Jul 2014.
39. Feng Wang, Zhuoqing Morley Mao, Jia Wang, Lixin Gao, and Randy Bush. A Measurement Study on the Impact of Routing Events on End-to-end Internet Path Performance. In Proceedings of ACM SIGCOMM 2006, Pisa, Italy, 2006.
40. ZMap. https://github.com/zmap/zmap/.

Index Terms

• Networks

  • Network properties

    • Network dynamics

  • Network protocols

    • Application layer protocols

    • Cross-layer protocols

    • Network layer protocols

      • Signaling protocols

    • Transport protocols