[Paper Review] Measuring the Consolidation of DNS and Web Hosting Providers

Synthia Wang, Kyle MacMillan+3 more|arXiv (Cornell University)|Oct 28, 2021

Caching and Content Delivery10 references4 citations

TL;DR

This paper measures the concentration of DNS and web hosting providers for the top 10,000 websites using the Tranco list, revealing that Cloudflare and Amazon alone host authoritative name servers for 37.1% of domains, while five providers—Cloudflare, Amazon, Akamai, Fastly, and Google—host 60% of index pages and most external resources, indicating significant global consolidation in critical Internet infrastructure components.

ABSTRACT

Despite the Internet's continued growth, it increasingly depends on a small set of service providers to support Domain Name System (DNS) and web content hosting. This trend poses many potential threats including susceptibility to outages, failures, and potential censorship by providers. This paper aims to quantify consolidation in terms of popular domains' reliance on a small set of organizations for both DNS and web hosting. We highlight the extent to which a set of relatively few platforms host the authoritative name servers and web content for the top million websites. Our results show that both DNS and web hosting are concentrated, with Cloudflare and Amazon hosting over $30\%$ of the domains for both services. With the addition of Akamai, Fastly, and Google, these five organizations host $60\%$ of index pages in the Tranco top 10K, as well as the majority of external page resources. These trends are consistent across six different global vantage points, indicating that consolidation is happening globally and popular organizations can influence users' online experience across the world.

Motivation & Objective

• To quantify the extent of consolidation in authoritative DNS name server hosting and web content hosting for popular websites.
• To assess the implications of this consolidation for Internet resilience, security, and content moderation.
• To measure the degree to which a small number of providers control critical infrastructure components for the most widely accessed domains.
• To provide a baseline for ongoing monitoring of infrastructure dependency trends using publicly available data.

Proposed method

• Collected DNS and web hosting data for the Tranco top 10,000 domains using active measurement from six global vantage points.
• Mapped authoritative name server IP addresses to hosting providers via reverse DNS and IP geolocation.
• Identified web hosting providers by analyzing the IP addresses of domain index pages and external resources.
• Aggregated results to compute the share of domains and resources hosted by each provider across multiple metrics.
• Used AS-level analysis to determine the number of autonomous systems used per domain for name server hosting.
• Released measurement data and framework publicly to enable ongoing monitoring and reproducibility.

Experimental results

Research questions

• RQ1 What percentage of the top 10,000 websites rely exclusively on Cloudflare or Amazon for authoritative DNS name server hosting?
• RQ2 How many organizations host the majority of index pages and external resources for the most popular websites?
• RQ3 To what extent are domains centralized in terms of the number of unique hosting providers or ASes used for their name servers?
• RQ4 How consistent are consolidation trends across different geographic vantage points?
• RQ5 What are the implications of this consolidation for Internet resilience, surveillance, and content moderation?

Key findings

• Cloudflare and Amazon together host the authoritative name servers for 37.1% of domains in the Tranco top 10,000, indicating a high degree of concentration in DNS infrastructure.
• Over 89% of the top 10,000 domains use name servers under the same primary domain, suggesting widespread reliance on shared hosting platforms.
• More than 75% of domains use only a single organization to host all their authoritative name servers, increasing systemic risk.
• Five providers—Cloudflare, Amazon, Akamai, Fastly, and Google—host approximately 60% of index pages in the Tranco top 10,000 and the majority of external page resources.
• These consolidation trends are consistent across six global vantage points, indicating a globally pervasive dependency on a small number of providers.
• The findings highlight significant risks related to single points of failure, surveillance, and content moderation power due to infrastructure centralization.