research-article

No WAN's Land: Mapping U.S. Broadband Coverage with Millions of Address Queries to ISPs

Authors:

Jonathan MayerAuthors Info & Claims

IMC '20: Proceedings of the ACM Internet Measurement Conference

Pages 393 - 419

https://doi.org/10.1145/3419394.3423652

Published: 27 October 2020 Publication History

Abstract

Accurate broadband coverage data is essential for public policy planning and government support programs. In the United States, the Federal Communications Commission is responsible for maintaining national broadband coverage data. Observers have panned the FCC's broadband maps for overstating availability, due to coarsegrained data collection and a low coverage threshold.

We demonstrate a new approach to building broadband coverage maps: automated large-scale queries to the public availability checking tools offered by major internet service providers. We reverse engineer the coverage tools for nine major ISPs in the U.S., test over 19 million residential street addresses across nine states for service, and compare the results to the FCC's maps.

Our results demonstrate that the FCC's coverage data significantly overstates the availability of each ISP's service, access to any broadband, connection speeds available to consumers, and competition in broadband markets. We also find that the FCC's data disproportionately overstates coverage in rural and minority communities. Our results highlight a promising direction for developing more accurate broadband maps and validating coverage reports.

Supplementary Material

MP4 File (imc2020-328-long.mp4)

Presentation Video - No WAN's Land: Mapping U.S. Broadband Coverage with Millions of Address Queries to ISPs

Download
53.33 MB

References

[1]

2012. The Impact of Broadband on the Economy: Research to Date and Policy Issues. International Telecommunication Union, (April 2012), 136.

[2]

B. C. Bauerly et al. 2019. Broadband Access as a Public Health Issue: The Role of Law in Expanding Broadband Access and Connecting Underserved Communities for Better Health Outcomes. J Law Med Ethics, 47, 2suppl, (June 2019), 39--42.

[3]

2020. Universal service. https://www.fcc.gov/general/universal-service.

[4]

2020. Policies on Broadband Europe. https://ec.europa.eu/digital-single-market/en/policies/75980/3608.

[5]

2019. 2019 Broadband Deployment Report. GN Docket No.18--238. (May 8, 2019). https://docs.fcc.gov/public/attachments/FCC-19-44A1.pdf.

[6]

U.S. House. 104th Congress. 1996. (February 1996). https://www.congress.gov/104/plaws/publ104/PLAW-104publ104.pdf.

[7]

2000. In the Matter of Local Competition and Broadband Reporting. Report and Order CC Docket No.99--301. Federal Communications Commission, (March 24, 2000). https://transition.fcc.gov/Bureaus/Common_Carrier/Orders/2000/fcc00114.pdf.

[8]

2000. In the Matter of Local Competition and Broadband Reporting. (March 3, 2000). https://transition.fcc.gov/Bureaus/Common%5C_Carrier/Orders/2000/fcc00114.pdf.

[9]

2020. FCC Fixed Broadband Deployment Map. en. Retrieved 09/23/2020 from https://broadbandmap.fcc.gov/.

[10]

2019. FCC Form 477 Local Telephone Competition and Broadband Reporting Instructions. (December 23, 2019). https://us-fcc.box.com/v/Form477Instructions.

[11]

2011. What are census blocks? https://www.census.gov/newsroom/blogs/random-samplings/2011/07/what-are-census-blocks.html.

[12]

2019. FCC form 477 local telephone competition and broadband reporting instructions for filings as of December 31, 2019. https://us-fcc.app.box.com/v/Form477Instructions.

[13]

FCC. 2020. 2020 Broadband Deployment Report. FCC20--50. Federal Communications Commission, (April 20, 2020).

[14]

2020. FCC Launches $20 Billion Rural Digital Opportunity Fund To Expand Rural Broadband Deployment. FCC-20-5. Federal Communications Commission, (February 7, 2020).

[15]

2017. White Paper on Improving the Nation's Digital Infrastructure. Federal Communications Commission, (January 19, 2017).

[16]

Kelsey L. Conley and Brian E. Whitacre. 2020. Home is where the internet is? high-speed internet's impact on rural housing values. International Regional Science Review, 43, 5, 501--530. eprint: https://doi. org/10.1177/0160017620918652. https://doi.org/10.1177/0160017620918652.

[17]

Bento J. Lobo, Md Rafayet Alam, and Brian E. Whitacre. 2020. Broadband speed and unemployment rates: data and measurement issues. Telecommunications Policy, 44, 1, 101829. issn:0308--5961. http://www.sciencedirect.com/science/article/pii/S0308596118303823.

[18]

Elizabeth A. Mack. 2014. Businesses and the need for speed: the impact of broadband speed on business presence. Telematics and Informatics, 31, 4, 617--627. issn:0736--5853. http://www.sciencedirect.com/science/article/pii/S0736585313000890.

[19]

Federal Communications Commission. 2020. Mobility fund phase ii coverage maps investigation staff report. EN-US. (2020). https://docs.fcc.gov/public/attachments/DOC-361165A1.pdf.

[20]

2019. Mobility Fund Phase II Coverage Maps Investigation Staff Report. GN Docket No.19--367. Federal Communications Commission, (December 4, 2019). https://docs.fcc.gov/public/attachments/DOC-361165A1.pdf.

[21]

2020. Re: Rural Digital Opportunity Fund, WC Docket No.19--126. (April 10, 2020). https://ecfsapi.fcc.gov/file/104101431124557RDOF%5C%20Challenge%5C%20Process%5C%20WC%5C%20Docket%5C%20No.%5C%2019-126.pdf.

[22]

2020. In the Matter of Barrier Communications Corporation d/b/a BarrierFree. (September 2, 2020). https://docs.fcc.gov/public/attachments/FCC-20-123A1.pdf.

[23]

2019. Re: GN Docket No.18--238. (March 5, 2019). https://ecfsapi.fcc.gov/file/10306056687881/Free%5C%20Press% 5C%20706%5C%20Report%5C%20Form%5C%20477%5C%20Erroneous%5C%20Data%5C%20ex%5C%20parte.pdf.

[24]

2020. FCC under fire from senators over rural broadband funding limitations. Retrieved 03/10/2020 from https://www.theverge.com/2020/3/10/21173534/fcc-broadband-rural-funding-limits-ajit-pai-chuck-schumer-gillibrand.

[25]

2020. FCC 'Coloring Outside the Lines' on Broadband Mapping, Say Critics at Next Century Cities Event. (January 24, 2020). http://broadbandbreakfast.com/2020/01/fcc-coloring-outside-the-lines-on-broadband-mapping-say-critics-at-next-century-cities-event/.

[26]

2020. Sen. Manchin Pledges to be FCC's Broadband Speed Pen Pal. (January 24, 2020). https://www.nexttv.com/news/sen-manchin-pledges-to-be-fccs-broadband-speed-penpal.

[27]

2018. How skeptics called Big Telecom's bluff on broadband coverage maps. (December 18, 2018). https://statescoop.com/broadband-mobility-fund-phase-challenges-fcc-ajit-pai/.

[28]

2019. Establishing the Digital Opportunity Data Collection; Modernizing the FCC Form 477 Data Program. FCC-19-79. Federal Communications Commission, (August 6, 2019). https://docs.fcc.gov/public/attachments/FCC-19-79A1.pdf.

[29]

Senate - Commerce Science and Transportation - 104th Congress. 2019. (June 2019). https://www.congress.gov/bill/116th-congress/senate-bill/1822/text.

[30]

2019. In the Matter of Establishing the Digital Opportunity Data Collection, Modernizing the FCC Form 477 Data Program. (July 16, 2019). https://docs.fcc.gov/public/attachments/FCC-20-94A1.pdf.

[31]

2020. US Broadband Usage Percentages. original-date: 2020-04-16T02:40:42Z. (May 2020). Retrieved 05/31/2020 from https://github.com/microsoft/USBroadbandUsagePercentages.

[32]

John Kahan. 2019. It's time for a new approach for mapping broadband data to better serve americans. Microsoft on the Issues. (April 8, 2019). Retrieved 02/10/2020 from https://blogs.microsoft.com/on-the-issues/2019/04/08/its-time-for-a-new-approach-for-mapping-broadband-data-to-better-serve-americans/.

[33]

John Busby and Julia Tanberk. 2020. FCC Underestimates Americans Unserved by Broadband Internet by 50%. Broad-bandNow Research, (February 2020). Retrieved 02/26/2020 from https://broadbandnow.com/research/fcc-underestimates-unserved-by-50-percent.

[34]

2019. Broadband Mapping Initiative: FAQ. https://www.costquest.com/uploads/pdf/broadbandmappinginitiative-summary.pdf.

[35]

2019. Broadband Mapping Initiative: Proof of Concept. https://www.ustelecom.org/wp-content/uploads/2019/03/Mapping-FAQ.pdf.

[36]

National Telecommunications and Information Administration. 2019. National broadband availability map | broadband usa. (June 4, 2019). https://broadbandusa.ntia.doc.gov/map.

[37]

2019. Guidelines for broadband data submission. https://www.cpuc.ca.gov/General.aspx?id=2541.

[38]

Igor Canadi, Paul Barford, and Joel Sommers. 2012. Revisiting broadband performance. In Proceedings of the 2012 Internet Measurement Conference (IMC '12). Association for Computing Machinery, Boston, Massachusetts, USA, 273--286. isbn: 9781450317054.

Digital Library

[39]

Srikanth Sundaresan et al. 2011. Broadband internet performance: a view from the gateway. In Proceedings of the ACM SIGCOMM 2011 Conference (SIGCOMM '11). Association for Computing Machinery, Toronto, Ontario, Canada, 134--145. isbn: 9781450307970. https://doi.org/10.1145/2018436.2018452.

[40]

Srikanth Sundaresan et al. 2014. Bismark: a testbed for deploying measurements and applications in broadband access networks. In 2014 USENIX Annual Technical Conference (USENIX ATC 14). USENIX Association, Philadelphia, PA, (June 2014), 383--394. isbn: 978-1-931971-10-2. https://www.usenix.org/conference/atc14/technical-sessions/presentation/sundaresan.

[41]

Marcel Dischinger et al. 2007. Characterizing residential broadband networks. In Proceedings of the 7th ACM SIGCOMM Conference on Internet Measurement (IMC '07). Association for Computing Machinery, San Diego, California, USA, 43--56. isbn: 9781595939081. https://doi.org/10.1145/1298306.1298313.

[42]

Bischof, Zachary S. and Otto, John S. and Sánchez, Mario A. and Rula, John P. and Choffnes, David R. and Bustamante, Fabian E. 2011. Crowdsourcing isp characterization to the network edge. In (Proceedings of the 1st ACM SIGCOMM Workshop on Measurements Up the Stack, W-MUST′11). (September 2011), 61--66. isbn: 9781450308007.

[43]

Zachary S. Bischof, Fabian E. Bustamante, and Nick Feamster. 2017. Characterizing and improving the reliability of broadband internet access. CoRR, abs/1709.09349. arXiv: 1709. 09349. http://arxiv.org/abs/1709.09349.

[44]

Aruna Dunna, Zachary Bischof, and Romain Fontugne. 2020. Sanitizing a view of consumer broadband in the united states. In 2020 Network Traffic Measurement and Analysis Conference (TMA), 1--9.

[45]

Ramakrishna Padmanabhan et al. 2019. Residential links under the weather. In Proceedings of the ACM Special Interest Group on Data Communication (SIGCOMM '19). Association for Computing Machinery, Beijing, China, 145--158. isbn: 9781450359566. https://doi. org/10.1145/3341302.3342084.

[46]

Hattie Joanne Elkins and Rachelle Lee Lanfair. 2017. ECFS filing detail. (September 25, 2017). Retrieved 05/31/2020 from https://www.fcc.gov/ecfs/filing/1092355565047.

[47]

The Daily Yonder. 2018. Broadband speed: fcc map vs. experience on the ground. Retrieved 06/01/2020 from https://www.dailyyonder. com/broadband-speed-fcc-map-vs-experience-ground/2018/07/25/.

[48]

Institute for Local Self-Reliance. 2018. Broadband competition in the rochester region: reality vs federal statistics. EN-US. https://muninetworks.org/sites/www.muninetworks.org/files/rochester-mn-policy-brief-2018.pdf.

[49]

Ramakrishnan Durairajan and Paul Barford. 2016. A technoeconomic framework for broadband deployment in under-served areas. In Proceedings of the 2016 Workshop on Global Access to the Internet for All (GAIA '16). Association for Computing Machinery, Florianopolis, Brazil, 7--12. isbn: 9781450344234. https://doi.org/10.1145/2940157.2940159.

Digital Library

[50]

Mariya Zheleva et al. 2013. The increased bandwidth fallacy: performance and usage in rural zambia. In Proceedings of the 4th Annual Symposium on Computing for Development (ACM DEV-4 '13) Article 2. Association for Computing Machinery, Cape Town, South Africa, 10 pages. isbn: 9781450325585. https://doi.org/10.1145/2537052.2537060.

[51]

2019. American Community Survey (ACS). https://www.census.gov/programs-surveys/acs.

[52]

2020. AT&T Internet Plans - Home Internet - AT&T®® Official Site. en. Retrieved 09/23/2020 from https://www.att.com/internet/.

[53]

2020. CenturyLink | Authentication. Retrieved 09/23/2020 from https://shop.centurylink.com/MasterWebPortal/freeRange/login/shop/addressAuthentication?cx=true.

[54]

2020. Charter Spectrum® Official - Internet Cable TV and Phone Service. Retrieved 09/23/2020 from https://buy.spectrum.com/buyflow/buyflow-localization?expiration=session.

[55]

2020. Order Internet Service | Consolidated Communications. Retrieved 09/23/2020 from https://www.consolidated.com/residential/internet/order-internet.

[56]

2020. Order Cox Services | Cox Communications. Retrieved 09/23/2020 from https://www.cox.com/residential-shop/order-cox-services.cox.

[57]

2020. Order Online. Retrieved 09/23/2020 from https://frontier.com/order-online/address.

[58]

2020. In Home Product Qualification. Retrieved 09/23/2020 from https://www.verizon.com/inhome/qualification?message=visitidempty.

[59]

2020. High Speed Internet Service Provider | Kinetic by Wind-stream. Retrieved 09/23/2020 from https://www.windstream. com/high-speed-internet#/.

[60]

2020. Xfinity Offers In My Area | Xfinity. en. Retrieved 09/23/2020 from https://www.xfinity.com/locations/in-my-area.

[61]

2016. Staff block estimates. Federal Communications Commission. Library Catalog: www.fcc.gov. (June 29, 2016). Retrieved 04/20/2020 from https://www.fcc.gov/reports-research/data/staff-block-estimates.

[62]

2020. National address database | US department of transportation. Retrieved 02/11/2020 from https://www.transportat.ion.gov/gis/national-address-database/national-address-database-0.

[63]

2015. National Address Database Summit Final Report. Department of Transportation, (June 2015), 49. https://www.transportation.gov/sites/dot.gov/files/docs/1%5C%20NAD%5C_Summit%5C_Report%5C_0.pdf.

[64]

2018. National address database schema. (April 2018). https://www.transportation.gov/sites/dot.gov/files/docs/mission/gis/national-address-database/308816/nad-schema-v2.pdf.

[65]

2020. Postal addressing standards | postal explorer. Retrieved 04/17/2020 from https://pe.usps.com/text/pub28/welcome. htm.

[66]

SmartyStreets. 2020. US Street Address API documentation -SmartyStreets. Library Catalog: smartystreets.com. Retrieved 07/14/2020 from https://smartystreets.com/docs/cloud/us-street-api.

[67]

2020. DPV | PostalPro. https://postalpro.usps.com/address-quality/dpv.

[68]

2020. USPS - Residential Delivery Indicator. https://www.usps.com/nationalpremieraccounts/rdi.htm.

[69]

2010. FCC area API. Retrieved 02/23/2020 from https://geo. fcc.gov/api/census/#!/area/get%5C_area.

[70]

2010. 2010 TIGER Line Shapefiles. Retrieved 08/20/2020 from https://www.census.gov/cgi-bin/geo/shapefiles/index.php? year=2010&layergroup=Blocks.

[71]

US Census Bureau. 2016. New Census Data Show Differences Between Urban and Rural Populations. EN-US. Library Catalog: www.census.gov Section: Government. (2016). Retrieved 05/24/2020 from https://www.census.gov/newsroom/press-releases/2016/cb16-210.html.

[72]

Federal Communications Commission. 2019. Establishing new digital opportunity data collection. EN-US. (2019). https://www.fcc.gov/document/establishing-new-digital-opportunity-data-collection.

[73]

2020. Patsy - Describing statistical models in Python using symbolic formulas. https://github.com/pydata/patsy.

Cited By

Manda HSrinivasavaradhan VKoduru LZhang KZhou XPaul UBelding EGupta ANarechania TSekar VYu MSeneviratne AVeitch D(2024)The Efficacy of the Connect America Fund in Addressing US Internet Access InequitiesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672272(484-505)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672272
Whitacre B(2024)The fibre broadband housing premium across three US StatesRegional Studies, Regional Science10.1080/21681376.2024.230595111:1(38-62)Online publication date: 8-Feb-2024
https://doi.org/10.1080/21681376.2024.2305951
Hollman AObermier TAndrews J(2024)Rural broadband usage: analysing satisfaction and internet speedRural Society10.1080/10371656.2024.234498233:1(19-32)Online publication date: 25-Apr-2024
https://doi.org/10.1080/10371656.2024.2344982
Show More Cited By

Index Terms

No WAN's Land: Mapping U.S. Broadband Coverage with Millions of Address Queries to ISPs
1. Networks
  1. Network types
    1. Public Internet
2. Social and professional topics
  1. Computing / technology policy
    1. Commerce policy
      1. Governmental regulations
    2. Network access control
      1. Broadband access

Recommendations

Land mobile radio systems in Poland

The current situation of land mobile radio (LMR) systems in Poland is described. This includes paging, dispatch, trunking, and cellular systems. Frequency plan and organization of LMR communication in Poland are presented. Finally, the plan for the ...
Public and Private Land Mobile Radio: Technologies, Services and Economics
Spectral analysis of civil conflict-induced forced migration on land-use/land-cover change: the case of a primate and lower-ranked cities in Sierra Leone

Involuntary migration triggered by war has the capacity to generate substantial socioeconomic and environmental changes in cities of developing countries, resulting in aberrant alterations to land use and land cover LULC. This scenario has the potential ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IMC '20: Proceedings of the ACM Internet Measurement Conference

October 2020

751 pages

ISBN:9781450381383

DOI:10.1145/3419394

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

IMC '20

Sponsor:

IMC '20: ACM Internet Measurement Conference

October 27 - 29, 2020

Virtual Event, USA

Acceptance Rates

IMC '20 Paper Acceptance Rate 53 of 216 submissions, 25%;

Overall Acceptance Rate 277 of 1,083 submissions, 26%

Upcoming Conference

IMC '24

Sponsor:
sigcomm
sigcomm

ACM Internet Measurement Conference

November 4 - 6, 2024

Madrid , AA , Spain

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
521
Total Downloads

Downloads (Last 12 months)72
Downloads (Last 6 weeks)5

Reflects downloads up to 26 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Manda HSrinivasavaradhan VKoduru LZhang KZhou XPaul UBelding EGupta ANarechania TSekar VYu MSeneviratne AVeitch D(2024)The Efficacy of the Connect America Fund in Addressing US Internet Access InequitiesProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672272(484-505)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672272
Whitacre B(2024)The fibre broadband housing premium across three US StatesRegional Studies, Regional Science10.1080/21681376.2024.230595111:1(38-62)Online publication date: 8-Feb-2024
https://doi.org/10.1080/21681376.2024.2305951
Hollman AObermier TAndrews J(2024)Rural broadband usage: analysing satisfaction and internet speedRural Society10.1080/10371656.2024.234498233:1(19-32)Online publication date: 25-Apr-2024
https://doi.org/10.1080/10371656.2024.2344982
Hollman AObermier TAndrews J(2024)A hard look at quality and policy from the lens of the rural internet consumerTelecommunications Policy10.1016/j.telpol.2024.10276248:6Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1016/j.telpol.2024.102762
Paul UGunasekaran VLiu JNarechania TGupta ABelding ESchulzrinne HKohler EMaltz DMisra V(2023)Decoding the Divide: Analyzing Disparities in Broadband Plans Offered by Major US ISPsProceedings of the ACM SIGCOMM 2023 Conference10.1145/3603269.3604831(578-591)Online publication date: 10-Sep-2023
https://dl.acm.org/doi/10.1145/3603269.3604831
Paul ULiu JFarias-llerenas DAdarsh VGupta ABelding E(2022)Characterizing Internet Access and Quality Inequities in California M-Lab MeasurementsProceedings of the 5th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies10.1145/3530190.3534813(257-265)Online publication date: 29-Jun-2022
https://dl.acm.org/doi/10.1145/3530190.3534813
Paul ULiu JGu MGupta ABelding EBarakat CPelsser CBenson TChoffnes D(2022)The importance of contextualization of crowdsourced active speed test measurementsProceedings of the 22nd ACM Internet Measurement Conference10.1145/3517745.3561441(274-289)Online publication date: 25-Oct-2022
https://dl.acm.org/doi/10.1145/3517745.3561441
Mangla TShowalter EAdarsh VJones KVigil-Hayes MBelding EZegura E(2022)A tale of three datasetsCommunications of the ACM10.1145/346267265:3(67-74)Online publication date: 23-Feb-2022
https://dl.acm.org/doi/10.1145/3462672
Biedny CWhitacre BGallardo R(2022)Do ‘dig once’ and permitting policies improve fiber availability?Telecommunications Policy10.1016/j.telpol.2021.10229446:5Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1016/j.telpol.2021.102294
Whitacre BBiedny C(2022)A preview of the broadband fabricTelecommunications Policy10.1016/j.telpol.2021.10228146:3Online publication date: 1-Apr-2022
https://dl.acm.org/doi/10.1016/j.telpol.2021.102281
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents