Measure The Web

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== IP Classification ======

You finish a crawl and you have a list of IP addresses: the servers your browser connected to, the resolvers that answered your DNS queries, the clients in a server log someone gave you. Now a reviewer wants to know what they //are// — which company, which country, whether that request left the EEA, whether the address that keeps appearing is one user or ten thousand.

This page is about turning an observed address into a claim you can defend. Its companion, [[Design:Crawling location]], is about the one address you control: your own.

The single most useful thing to internalise is that **"classify an IP" is not one measurement.** It is five, they use different data, and they are not equally reliable:

^ Question ^ Best source ^ How good it is ^ Free? ^
| Which network //announces// it? | BGP: Team Cymru, RouteViews, RIPE RIS | Essentially exact — it is an observation of the routing system, not an estimate | Yes |
| Which //organisation// runs that network? | CAIDA AS2Org, ASdb, PeeringDB, RIR WHOIS | Good, but one company holds many ASNs and one ASN can be resold | Yes |
| Is it a datacenter, VPN, proxy, mobile? | The operator's own published prefix list; then commercial feeds | Excellent where the operator publishes; noisy elsewhere | Partly |
| Where //is// it? | Geolocation databases; active measurement | Country usually right — 77.5–89.4% measured on router interfaces, 99.8% claimed by the vendor. **City is a guess.** | Partly |
| Is it the //same host// as last week? | Nothing reliable | Bad. Assume no unless you can show otherwise | — |

The rows are in descending order of defensibility, and papers reach for the bottom rows far more often than the top ones. Of the 295 papers in our corpus that classify an IP address, 114 name a geolocation source and only 68 name a routing or ASN source (see [[#Use in Publications]]) — even though the routing answer is the one that is nearly always correct.

<WRAP important>
**If your research question can be answered at the ASN or prefix level, answer it there.** "This tracker's requests terminated in AS16509 (Amazon)" is a fact about the routing table. "This tracker's requests terminated in Frankfurt" is an inference from a commercial database that you have not validated, and that four vendors will answer three different ways.
</WRAP>

If you read one thing outside this page before starting, make it {[zilberman2025_survey]} (//IEEE Communications Surveys & Tutorials//, 2025) — the current survey of Internet geolocation, covering the techniques, the evasion literature, and the cloud case. It is outside the seven venues this site's corpus covers, which is itself the point: **most of the methods literature you need here is published at PAM, TMA, CoNEXT, ANRW and ACM CCR, not at the security venues you are probably submitting to.**

===== First: Where the Addresses Come From, and What They Turn Out to Be =====

If you are coming from web measurement, the server address is not in the place you would look. It is not in the URL, not in the response headers, and not in anything the page can see — it is a property of the connection. Three places give it to you, and they agree:

  * **Playwright**: ''await response.serverAddr()'' → ''{ipAddress, port}''.
  * **Chrome DevTools Protocol** (so also Puppeteer, and OpenWPM through its instrumentation): ''Network.responseReceived'' → ''e.response.remoteIPAddress'' and ''remotePort''.
  * **A HAR file**: the per-entry ''serverIPAddress'' field, which HAR 1.2 defines and Chrome, Firefox and Playwright all populate. (More on working with HAR: [[Programming:Traffic files]].)

Measured on one page load of ''https://www.bbc.com/'' on 6 August 2026: all 130 responses carried an address via Playwright and via CDP; the HAR was missing one, a beacon still in flight when the context closed. So take it from the live event stream if you can, and treat a HAR as very slightly lossy.

Now the part that reframes the rest of this page. **Those 130 responses came from 13 distinct IP addresses.** Taking one per distinct network and running them through the script in [[#A Script]]:

^ IP ^ ASN ^ Organisation ^ Matched by the script's prefix lists((The script loads only AWS, Google Cloud and Cloudflare. Fastly and Google's non-GCP space also publish lists — see [[#Ask the operator first]] — they are simply not in this script's three. Akamai publishes no open prefix list, so "—" means different things in different rows, which is exactly why the ASN column is the reliable one.)) ^
| 23.212.192.147 | 16625 | Akamai | — |
| 143.204.55.25 | 16509 | Amazon (CloudFront) | AWS 143.204.0.0/16 |
| 3.165.198.163 | 16509 | Amazon (CloudFront) | AWS 3.165.0.0/16 |
| 104.18.65.57 | 13335 | Cloudflare | Cloudflare 104.16.0.0/13 |
| 146.75.116.81 | 54113 | Fastly | — |
| 34.49.241.189 | 396982 | Google Cloud | GCP 34.49.0.0/16 |
| 172.217.208.154 | 15169 | Google | — |

<WRAP important>
**Not one of them is the BBC's own network.** Every address a modern page load touches belongs to a CDN or a cloud. Two consequences that determine what this page is for:

  - **Geolocating the server addresses of a web crawl tells you where CDN edge nodes are.** It does not tell you where a company is, where its servers are, or where anyone's personal data came to rest. Any claim of the form "//n// % of trackers are hosted in country X" built on unfiltered crawl IPs is measuring CDN topology.
  - **The 130-to-13 collapse cuts the other way too.** Distinct addresses badly under-count distinct parties, and a single address serves many unrelated organisations. Attribute by **name** — hostname, CNAME chain, certificate — and use the address for what it is good at: which network carried it, and whether that network is a cloud.

This is why the ASN and prefix layer is the useful one for web measurement, and why the geolocation layer needs so many caveats.
</WRAP>

===== Whose Network Is It: ASN, Prefix and Organisation =====

==== The routing layer ====

Every routable address is inside a prefix that some autonomous system announces into BGP. That mapping is public, is not a vendor's opinion, and is free:

  * **[[https://team-cymru.com/community-services/ip-asn-mapping/|Team Cymru IP-to-ASN mapping]]** — the practical default. A ''whois'' bulk interface on port 43 takes thousands of addresses per connection and returns ASN, covering prefix, the country and RIR on the allocation record, the allocation date and the AS name; there is also a DNS interface. Free for research use, no key. The script in [[#A Script]] uses it.
  * **[[https://www.routeviews.org/|RouteViews]]** and **[[https://ris.ripe.net/docs/|RIPE RIS]]** — the raw BGP collector archives everything else is derived from. Use these if you need a mapping //as of a historical date//, which no live API gives you.
  * **[[https://stat.ripe.net/docs/02.data-api/|RIPEstat Data API]]** — HTTP, no key, one address at a time; convenient for a handful of lookups. No hard rate limit for casual use; registration is asked for above 1,000 requests a day.
  * **[[https://www.caida.org/catalog/datasets/routeviews-prefix2as/|CAIDA Routeviews Prefix2AS]]** — daily prefix-to-ASN files since 2005, which is how you get a mapping //as of// the day of a historical crawl.
  * **[[https://github.com/hadiasghari/pyasn|pyasn]]** — offline prefix-to-ASN lookups from an MRT snapshot, which is what you want inside a post-processing loop rather than a network call per row. Note the maintenance state before you depend on it: last commit September 2024, last tag 1.6.2 (2023).
  * **[[https://bgp.tools/kb/api|bgp.tools]]** — a free whois and bulk-table interface with a friendlier output format; requires an identifying User-Agent.

==== From ASN to organisation ====

An ASN is not a company. Large operators hold dozens; acquisitions leave the old ASN in place for years; a hosting reseller announces space that belongs to someone else. This is the **AS-sibling problem** and there are two free datasets for it:

  * **[[https://www.caida.org/catalog/datasets/as-organizations/|CAIDA AS2Org]]** clusters ASNs into organisations from quarterly RIR WHOIS dumps. This is the canonical answer to "are these two ASNs the same company?", and it is current (files through July 2026 at the time of writing).
  * **[[https://asdb.stanford.edu/|ASdb]]** {[ziv2021_asdb]} classifies ASNs into 17 industry categories and 95 sub-categories (hosting, ISP, education, government…), reporting 96% coverage of ASes at 93% accuracy on the top level and 75% on sub-categories. Snapshots run through March 2026. Used, for example, by {[darwich2023_replication]} to describe what kind of networks their geolocation targets sat in.

<WRAP todo>**CAIDA's separate //AS Classification// dataset is discontinued** — CAIDA's own page states it is no longer supported and download access has been removed. Papers from 2015–2022 cite it routinely; ASdb is the live replacement. Check before you cite a dataset you found in a related-work section.</WRAP>

The current state of the art on the sibling problem is {[selmo2025_borges]} (IMC 2025), which few-shot prompts GPT-4o-mini at temperature 0 over PeeringDB free-text fields plus website and domain evidence, and reports a 7% improvement in sibling-ASN identification over AS2Org-style methods. That is, as of 2026, **the one place in the AS-and-organisation half of this page where an LLM method has cleared peer review** — see [[#Open Questions]]. It is a //different// paper from the single ''llm'' firing in this page's corpus population, which is LLMCloudHunter {[schwartz2025_llmcloudhunter]} extracting IP indicators from threat-intelligence prose; do not conflate the two, and note that Borges's classification tuples are filed under ''other'' rather than ''ip-address'', so it does not appear in the method table in [[#How they classify]] at all.

Two traps specific to web measurement. First, **the AS that announces a server's address is very often not the party you care about**: a tracker served through Cloudflare terminates in AS13335, and attributing the tracking to Cloudflare is wrong. Resolve the //name// (the request URL, the certificate, the CNAME chain) before or instead of the address. Second, **ASNs churn**: {[nemmi2021_parallel]} tracked the administrative and operational lives of ASNs separately and found 22,729 administrative lifetimes (17.9%) with no observed BGP activity at all, plus 1,667 ASNs announcing in BGP without an overlapping allocation. An ASN-to-organisation table from 2019 is not a table about 2026.

===== Where Is It: Geolocation =====

==== A twenty-year argument, dated ====

Read anything on this topic and you land in the middle of an argument between databases and active measurement. Knowing where each position comes from stops you citing a 2011 finding as if it described 2026.

^ Period ^ Position ^ Where it comes from ^
| 2004–2009 | **Measure it yourself.** Convert latency from landmarks into distance constraints and multilaterate. | Constraint-Based Geolocation {[gueye2004_cbg]} and its successors (Octant, Spotter). Still the reference technique. |
| 2011–2016 | **Databases are unreliable.** | {[poese2011_unreliable]} ground-truthed several databases against a large European ISP and found entries that do not reflect actual BGP allocations, concentrated in a few popular countries, and — the counter-intuitive part — that "overly fine granularity of database entries makes their accuracy worse, not better". {[shavitt2011_geodb]} found none of the databases close to its claimed country-level accuracy, with MaxMind disproportionately defaulting ambiguous addresses to the US. |
| 2017–2020 | **Some databases are much worse than others, and routers are the hard case.** | {[gharaibeh2017_look]}, below. |
| 2023–2024 | **The gap between vendors now exceeds the gap between vendors and research techniques.** Also: the databases quietly stopped being pure guesswork and started ingesting operator-published data. | {[darwich2023_replication]}, below; {[livadariu2024_geofeeds]} on geofeeds. |
| 2026 | **Aggregate accuracy hides who it fails for.** | A 2026 preprint {[nabi2026_lostprefix]}, below. |

The "databases are unreliable" line is the one that got repeated most and is the one most often cited without a date. It was accurate about the products of 2011. Do not paste it into a 2026 related-work section without checking.

==== Country is reliable; city is not ====

This is the one finding to carry away, and the corpus states it precisely. {[gharaibeh2017_look]} (IMC 2017) built a 16,586-address ground truth for router interfaces and compared four databases:

> NetAcuity outperforms the other databases at only 89.4% accuracy while IP2Location-Lite and MaxMind databases are comparable with 77.5% to 78.6% accuracy.

That is **country-level** accuracy, and 89.4% is the //best// of the four. The cross-database picture is worse than the headline: overall country-level agreement between all four databases was about 95.8%, but more than 29% of city-level results differed by over 40 km between vendors, and coverage itself was uneven — IP2Location-Lite and NetAcuity had near-perfect city coverage where MaxMind's free and paid products covered "only 43% and 61.6% ... at the city-level respectively".

Router interfaces are a hard case (they sit in transit networks that geolocation is bad at), so read those numbers as a lower bound for ordinary web servers. The corresponding figure for end hosts comes from {[darwich2023_replication]} (IMC 2023), which used RIPE Atlas anchors as ground truth and measured the share of targets located to within 40 km — "correct at city level":

^ Method ^ Targets within 40 km ^
| IPinfo (free API) | 89% |
| Constraint-Based Geolocation from all RIPE Atlas vantage points | 73% |
| MaxMind (free version) | 55% |

<WRAP important>
Note the direction of that result and its date. In 2023, a **commercial database beat a full active-measurement campaign from a global probe network**, and beat the other commercial database by 34 points. Conclusions of the form "geolocation databases are unreliable, use active measurement" were the field's consensus for a decade and the 2023 replication explicitly declines to reproduce it:

> Our work shows a more nuanced story: At least one geolocation database, IPinfo, outperformed CBG with all the RIPE Atlas vantage points on our dataset, and we know now that IPinfo mostly uses standard geolocation techniques.

Two consequences. **Do not treat "MaxMind" and "a geolocation database" as synonyms** — the gap between vendors is larger than the gap between vendors and research techniques. And **do not assume today's ranking holds**: it is a claim about one dataset in 2023.
</WRAP>

One more corpus data point, because it shows the databases disagreeing on a population you might actually study rather than on a ground-truth set. {[khan2018_commercial]} took 626 commercial-VPN vantage points and asked three databases whether each one was where the provider //claimed// it was. Note the denominators, which differ because the services answered for different subsets: Google agreed for **377 of 541 (70%)**, IP2Location Lite for **552 of 612 (90%)**, MaxMind GeoLite2 for **583 of 612 (95%)**. This measures agreement with a marketing claim rather than accuracy against ground truth — but a 25-point spread between three databases on one address set is the point. Their observation about the residue matters more than the ranking:

> For all databases, about one third of the inconsistencies were the database claiming a vantage point was hosted in the US when the claimed location was elsewhere.

That is the same US-default bias {[shavitt2011_geodb]} identified in 2011, still visible in 2018. If your result is "//n// addresses are in the United States", check how many of them are simply the database's fallback.

==== Aggregate accuracy hides who it fails for ====

Both figures above are medians over a target set dominated by fixed-line hosts in well-instrumented regions. A 2026 preprint, {[nabi2026_lostprefix]}, evaluated MaxMind GeoLite2, IPinfo, IP2Location and DB-IP against RIPE Atlas and UNICEF Giga ground truth in 175 countries, and reports that the aggregate hides two large systematic gaps:

  * **Mobile networks**: median error 179–207 km, against 3–16 km for fixed networks — more than ten times worse, across //all four// providers.
  * **Geography**: failure rates of 53–61% in Asia and 66–72% in Africa, against 9–20% in Europe.
  * Both traced to the same cause: the provider's prefix granularity is coarser than the actual BGP announcement, and about 70% of mobile prefixes span more than 100 km.

This is a preprint and has not been peer reviewed — we cite it because it is the most recent multi-provider evaluation we could find and because the direction of the result matters more than the exact figures. Its practical implication is concrete: **if your population is mobile clients, or is outside Europe and North America, a city- or even region-level geolocation claim is not supportable**, and you should say so rather than reporting the same confidence you would for a German fixed-line dataset. This also compounds with sampling: see [[Design:Website selection]] on lists that over-represent the same regions.

==== How the databases actually get their answers ====

Worth knowing, because it tells you where they will be wrong. When {[darwich2023_replication]} asked IPinfo why they were doing so well, the answer was that they use "hints extracted from DNS, WHOIS, geofeeds [...] or other proprietary data" on top of latency measurements. So:

  * **Self-published geofeeds.** [[https://www.rfc-editor.org/rfc/rfc8805.html|RFC 8805]] (2020) defines a CSV format in which a network operator publishes the location of its own prefixes; [[https://www.rfc-editor.org/rfc/rfc9632.html|RFC 9632]] (August 2024) defines how to find that file from RPSL ''remarks:'' in the RIR databases and authenticate it with RPKI — **it obsoletes RFC 9092**, which is what most papers and blog posts still cite. Where an operator publishes one, it is as close to ground truth as this field gets. **Adoption is small and you should check rather than assume**: two years after standardisation, {[livadariu2024_geofeeds]} measured geofeed coverage at **1.50% of allocated IPv4 prefixes and 0.70% of IPv6**, with erroneous entries for 0.9% of client, 4.0% of router and 8.5% of server addresses. Their own conclusion is that geofeeds are "at least a geolocation hint", not a ground-truth dataset.
  * **Reverse DNS naming** — router and server hostnames often encode a city or airport code (''ae-1.fra3.example.net''). This is the ''undns''/DDec line of work, systematised in **Hoiho** {[luckie2021_hoiho]}, which learns per-operator regexes and validates them against RTT constraints; CAIDA's topology data kit tags each router location with which method produced it. It is why {[gamerogarrido2022_using]} could identify institutional networks by resolving 616,705 addresses to 440,233 hostnames.
  * **WHOIS and RIR allocation records** give the //registrant's// country, which is often not the host's. The script below shows the canonical example: 1.1.1.1 is allocated under APNIC with country ''AU'', is operated by a US company, and is anycast on every continent.
  * **Latency** — constraint-based geolocation {[gueye2004_cbg]} from a probe network. Accurate when a probe is close, useless when there is not one. {[darwich2023_replication]} quantified the ceiling: removing vantage points closer than 40 km moved the median error from 8 km to 120 km. Accuracy is not the binding constraint, though — throughput is. Replicating the finer-grained //street-level// technique, they measured a median of **1,238 seconds (about 20 minutes) to geolocate one address** on a 32-core machine with 62 GB of RAM, against the 1–2 seconds its authors had projected. Budget for that before you plan to geolocate a crawl's worth of servers actively. If you want this without building it, [[https://ipmap.ripe.net/|RIPE IPmap]] {[du2020_ipmap]} runs a latency engine and a reverse-DNS engine over RIPE Atlas and exposes the result through a public API — for infrastructure addresses, not arbitrary hosts.

==== Anycast: addresses with no location ====

An anycast address is announced from many places at once, so "where is it" has no answer. Every geolocation service will nonetheless return one. This is not a corner case for web measurement — it covers the large CDNs and the public resolvers, which is to say a large share of the requests a crawl records.

{[kumar2024_choices]} (IMC 2024) handled it properly and is a good template: of 4,286 government server addresses, MAnycast2 identified 433 (10.10%) as anycast, and those were then verified separately with five RIPE Atlas probes per country rather than trusted to the database. 72 addresses that could not be confirmed were **excluded from the analysis** rather than reported with a guessed country.

Detect anycast before you geolocate. **MAnycast2** {[sommese2020_manycast2]} uses an anycast platform to probe candidate anycast destinations and publishes periodic census snapshots — that is the low-effort route, and the one {[kumar2024_choices]} took. **iGreedy** (INFOCOM 2015) detects anycast from speed-of-light violations across vantage points and enumerates replicas; **Verfploeter** (IMC 2017) maps a single anycast deployment's catchment by pinging a full hitlist from inside it. Several commercial APIs also return an ''anycast'' boolean directly, which the script below prints.

==== Which geolocation source to use in 2026 ====

Checked against each vendor's own documentation on **6 August 2026**. Dated deliberately, because this is the part of the page that goes stale first.

^ Source ^ What you get free ^ Status ^ Notes ^
| **MaxMind GeoLite2** | Country, City (incl. ''accuracy_radius'') and ASN databases, under the GeoLite2 EULA plus CC BY-SA 4.0 for redistribution | Current, but **since 30 December 2019 you need an account and a licence key** to download | The field's default by inertia. MaxMind's own stated reason for the 2019 change was that GDPR and CCPA restricted their "ability to continue distributing our GeoLite databases on a public page". Measured at 55% within 40 km in 2023 {[darwich2023_replication]}. No committed release cadence — their docs say to check for a new release periodically through the day — so "we used MaxMind" without a date is not reproducible. |
| **MaxMind GeoIP2 / Insights** | — (paid: GeoIP City listed at $148/month or $1,628/year on 2026-08-06, Country at $38/month) | Current | Same ''accuracy_radius'' field. **Use it**: it is the vendor telling you not to trust the point. Note the listed price covers internal use only; a commercial licence is separate. |
| **IPinfo** | The free "Lite" database is **country and ASN only**, unlimited, CC BY-SA with attribution. City-level and privacy detection start at the paid tier. | Current | Best performer in the only recent head-to-head in the corpus {[darwich2023_replication]} — but note that measurement used IPinfo's //free API// as it stood in 2023, and the free tier has been restructured since. Check what you are actually getting. |
| **IP2Location LITE** | Country/city/ASN under CC BY-SA 4.0 with mandatory attribution | Current | Its Lite tier tied with NetAcuity for //coverage// in {[gharaibeh2017_look]} — "near perfect", against MaxMind's 43% and 61.6% at city level. Coverage is not accuracy; on accuracy it sat with MaxMind at 77.5–78.6%. |
| **DB-IP Lite** | Country, City and ASN databases, monthly, CC Attribution | Current | DB-IP describes the Lite tiers as "subsets of the commercial database with reduced coverage and accuracy" — so it is a third opinion, not a third measurement. Least-used of the four in the corpus. |
| **NetAcuity (Digital Element)** | — (paid, enterprise, sales-gated) | Current | Best accuracy in {[gharaibeh2017_look]} (2017). We found no academic-access programme; 5 papers in the corpus. |
| **RIPE IPmap** | Latency and reverse-DNS engines over RIPE Atlas, open API {[du2020_ipmap]} | Current | Infrastructure addresses (routers, IXPs) rather than arbitrary hosts. |
| **Geofeeds (RFC 8805, RFC 9632)** | The operator's own published claim | Current but **thinly adopted** — 1.50% of IPv4 prefixes {[livadariu2024_geofeeds]} | Check for one; do not build a pipeline that assumes one exists. |
| **Quova, Neustar IP Intelligence, HostIP.info, freegeoip.net, IPInfoDB** | — | **Historical.** Present in the corpus; superseded or gone | If a paper's method rests on one of these, that paper is telling you about 2012. |

Vendor accuracy claims are worth reading precisely because they are more modest than the reputation. MaxMind's own support documentation puts GeoIP2 at about **99.8% country-level**, but for the United States only about **80% at state level and 66% within 50 km of the correct city**. Nobody who reads that page would then publish an unqualified city-level claim.

===== What Kind of Host Is It =====

==== Ask the operator first ====

The most accurate datacenter detection is free, authoritative and almost unused in this literature: **the cloud and CDN operators publish their own prefix lists.** Grepping the full text of all 5,859 papers in the corpus for the URLs below finds them in **seven**.((Full-text grep of `paper.cols.txt` for `ip-ranges.amazonaws`, `gstatic.com/ipranges`, `cloudflare.com/ips`, `public_ip_ranges.json` or `ServiceTags_Public`, re-run 2026-08-12 against the extended corpus. Seven papers, all network-measurement rather than web-privacy; the seventh, added with the 2025 venue-year, is //Secure IP Address Allocation at Cloud Scale// (NDSS 2025). This undercounts papers that used a list without citing its URL, so read it as a floor.))

All of the following were fetched and confirmed serving on 6 August 2026:

  * **AWS**: ''https://ip-ranges.amazonaws.com/ip-ranges.json'' — carries ''service'' //and// ''region'' per prefix, so it also geolocates an AWS address better than any database does.
  * **Google Cloud**: ''https://www.gstatic.com/ipranges/cloud.json'' (GCP), ''.../goog.json'' (all Google).
  * **Cloudflare**: ''https://www.cloudflare.com/ips-v4'', ''ips-v6'', or JSON at ''https://api.cloudflare.com/client/v4/ips''.
  * **Fastly**: ''https://api.fastly.com/public-ip-list''.
  * **Oracle Cloud**: ''https://docs.oracle.com/en-us/iaas/tools/public_ip_ranges.json''.
  * **Azure**: there is **no stable JSON URL** — the filename rotates weekly (''ServiceTags_Public_YYYYMMDD.json''). Link the [[https://www.microsoft.com/en-us/download/details.aspx?id=56519|download landing page]] in your paper, or use the Service Tag Discovery REST API, which lags by up to about four weeks. Do not hard-code a dated filename.
  * **DigitalOcean** has no documented ranges page; a first-party CSV exists but 404s intermittently. Treat it as unreliable.
  * **Tor**: the Tor Project publishes the [[https://check.torproject.org/torbulkexitlist|bulk exit list]], which is definitive for current Tor exits in a way that no commercial ''is_tor'' flag is. For historical questions ("was this address a Tor exit on 12 March 2024?") use [[https://metrics.torproject.org/exonerator.html|ExoneraTor]] — the older TorDNSEL service was retired in April 2020.

If an address is in AWS's own file, it is AWS. No vendor score improves on that. Note the limit, which the script output below demonstrates: these files cover what the operator wants you to route to. Google's ''cloud.json'' is GCP customer space and does **not** contain 8.8.8.8; Cloudflare's ''ips-v4'' is their proxy range and does **not** contain 1.1.1.1.

There is a gap here worth stating carefully. We found **no peer-reviewed general classifier for hosting and datacenter address space at prefix granularity** beyond these published lists and the commercial products below. Adjacent work exists and is worth reading before you conclude nobody has tried: AS-level classification ({[ziv2021_asdb]}, {[selmo2025_borges]}) labels a whole AS rather than a prefix, and the //off-net// literature — Gigis et al., "Seven Years in the Life of Hypergiants' Off-Nets" (SIGCOMM 2021), and He et al., "Next Stop, the Cloud" (IMC 2013) — identifies specific serving infrastructure by other means. None of them is the thing you want, which is a maintained prefix-to-network-type map for the whole address space.

==== Commercial VPN, proxy and abuse feeds ====

Beyond the published ranges you are buying an opinion. Checked against vendor documentation on 6 August 2026:

  * **MaxMind GeoIP2 Anonymous IP** — booleans named exactly ''is_anonymous'', ''is_anonymous_vpn'', ''is_hosting_provider'', ''is_public_proxy'', ''is_residential_proxy'', ''is_tor_exit_node''. Paid. Note that MaxMind's separate //Connection Type// database is not this: it gives one categorical field (Cable/DSL, Cellular, Corporate, Satellite), which is useful and is often confused with hosting detection.
  * **IPinfo Privacy Detection** — ''hosting'', ''proxy'', ''tor'', ''relay'', ''vpn'', ''service''. Paid (not in the free Lite tier); residential-proxy identification is higher still.
  * **IP2Proxy** — including a free LITE tier; distinguishes VPN, Tor, public proxy, web proxy, hosting/datacenter, search-engine bot, residential proxy, consumer privacy network.
  * **ipapi.is** — returns ''is_datacenter''/''is_vpn''/''is_proxy''/''is_tor''/''is_abuser'' on a **free, keyless** query (1,000/day), which makes it the easiest thing to cross-check against. **ip-api.com** ships ''proxy''/''hosting''/''mobile'' free too, at 45 requests/minute and **non-commercial use only**.
  * **Spur**, **IPQualityScore**, **Udger** (which publishes a free browsable datacenter list), **IPHub**, **Scamalytics** — commercial, mostly sales-gated.
  * **GreyNoise** covers Internet-wide scanners specifically, and — unusually — runs a free [[https://docs.greynoise.io/docs/vip-program|Research Community programme]] giving students and independent researchers non-commercial access to the full enterprise API. If you need this class of data for a paper, ask them before you pay anyone.

Only 17 of the 295 IP-classifying papers in the corpus name a feed of this kind, and **all but one are from 2019 or later** (the exception is 2018) — this is the newest of the five questions and the least settled.

<WRAP important>
**These feeds have false positives you can see with one command.** In the run below, ''ipapi.is'' flags **8.8.8.8** — Google Public DNS — as ''is_vpn'' and ''is_abuser'', and **1.1.1.1** as ''is_vpn'' and ''is_abuser''. Both are public resolvers. If you filter your dataset on such a flag, say which vendor's flag, on what date, and spot-check the exclusions by hand. A vendor's risk score is tuned for fraud prevention, where a false positive costs a declined transaction; it is not tuned for your denominator.
</WRAP>

==== Residential proxies ====

If your measurement encounters traffic from residential address space that is not a residential user, this is why. The corpus has the definitive characterisations:

  * {[mi2019_resident]} captured 6,183,876 distinct residential proxy IPs and classified 5.9M (95.22%) as genuinely residential; of the hosts they could fingerprint, 237,029 (43%) were IoT devices — cameras, DVRs, printers — and 67 potentially unwanted programs were found enrolling hosts.
  * {[mi2021_your]} found 963 Android apps embedding proxy SDKs, and 624,989 proxy exit addresses that IPinfo classified as cellular — a number that is itself the output of the kind of commercial classification this page is about, which is worth noticing.
  * {[yang2022_extensive]} infiltrated Chinese residential-proxy services and captured 8,176,522 backconnect proxy IPs — which, despite the services targeting the China market, were spread across 227 countries. They also measured lifetime: around 91% of the directly-exposed ones lasted **less than 10 days**.

For the ethics of //using// one, see [[Design:Crawling location]] and [[Practices:Ethics]]. For //detecting// them in data you already have, the current technique is a **cross-layer latency discrepancy**: an application-layer RTT that is much larger than the network-layer RTT to the same address means something is terminating the connection in front of the real host. {[ramesh2024_calculatency]} (USENIX Security 2024) is the reference implementation — at a 50 ms threshold it reports 2.9% false negatives and 0.95% false positives, and 94.2% of the VPN addresses it tested answered ICMP at all, which is what makes the measurement possible. Two caveats before you adopt it:

  * It is an **active** technique. You are probing third-party hosts; that belongs in your ethics section.
  * The threshold may be fragile in exactly the conditions a real client has. {[chiapponi2023_poster]} probed the RTT delta from one laptop and one handset in France, 20 connections per configuration, and found the handset on 4G at **51.5 ± 14.5 ms** — over a 50 ms threshold with no proxy present — while a browser rather than a script, or a Wi-Fi hotspot, each added delay without individually crossing it. It is a two-page poster on a single carrier (SFR), and the authors themselves attribute the 4G result to that carrier running a transparent proxy, so read it as a warning to calibrate on your own population rather than as a general false-positive rate.

Earlier and complementary: {[weinberg2018_catch]} used active geolocation to show that 638 of 2,269 advertised proxy locations were false, 401 of them not even on the right continent. The line continues into the corpus's newest venue-year — Ali et al., [[https://www.ndss-symposium.org/ndss-paper/beyond-rtt-an-adversarially-robust-two-tiered-approach-for-residential-proxy-detection/|"Beyond RTT: An Adversarially Robust Two-Tiered Approach for Residential Proxy Detection"]] (NDSS 2026), is explicitly a response to the fragility above — so check for something newer still before you build your own.

===== IP as an Identifier: Four Ways It Breaks =====

This is where measurements go wrong quietly, because nothing errors — you just get a denominator that means something other than what you wrote.

**Address sharing.** Carrier-grade NAT puts many subscribers behind one address. {[richter2016_multi]} measured the deployment: CGN penetration of **17–18% of all eyeball ASes**, and "more than 90% of all cellular ASes deploying CGNs". {[gegenhuber2023_mobileatlas]} tested 14 European mobile operators in 2023 and reports that **all 14 used carrier-grade NAT for IPv4**, with only two offering IPv6 at all. So a "unique visitors" count from IP addresses on mobile traffic is not a visitor count, and the effect is not a rounding error.

[[https://www.rfc-editor.org/rfc/rfc6269.html|RFC 6269]] (//Issues with IP Address Sharing//) is the catalogue of everything this breaks, and worth ten minutes if you are about to count users. [[https://www.rfc-editor.org/rfc/rfc6598.html|RFC 6598]] reserves 100.64.0.0/10 for shared address space — but **do not use that prefix as your CGNAT test.** In the same 2023 study only three of the 14 operators actually used 100.64.0.0/10; two used it alongside 10.0.0.0/8, and the rest used something else. A positive is informative; a negative tells you nothing.

**Churn.** Residential addresses are reassigned; prefixes are reallocated between organisations; geolocation database entries change under you. **How fast depends enormously on the network, and you cannot assume.** {[padmanabhan2020_dynamips]} measured assignment durations directly, from six years of RIPE Atlas dual-stack probe data plus 32.7 billion address associations observed by a CDN, and found practice differs between ISPs by orders of magnitude. In IPv4 (non-dual-stack probes) they saw consistent periodic renumbering on 35 networks, with modes at 1 day (Deutsche Telekom), 1.5 days (Proximus), a week (Orange) and two weeks (BT); in IPv6, fixed cycles as short as 12 hours (ANTEL, Uruguay), 24 hours (Deutsche Telekom and four other German ISPs) and 48 hours (Global Village, Brazil), while elsewhere IPv6 /64 prefixes //"tend to be stable for months and years"//. Measure it for your own population, or state the assumption. This is also why the database snapshot date matters more than the vendor: between the run below and the one on [[Design:Crawling location]] the previous day, the //same// Swiss address was placed in Zurich, Samnaun, Lausanne and Biel/Bienne by different services on different days.

**Anycast and load balancing.** One address, many hosts, many countries — see above. In the reverse direction, one host answers on many addresses, so counting distinct server addresses over-counts servers.

**IPv6 is different, and in the direction people do not expect.** The intuition is that IPv6 clients rotate temporary addresses ([[https://www.rfc-editor.org/rfc/rfc8981.html|RFC 8981]]) and are therefore harder to track. That is true of the interface identifier and false of the assignment: {[padmanabhan2020_dynamips]} found that **//"IPv6 assignments have longer durations than IPv4 assignments—often remaining stable for months—thereby allowing the possibility of long-term fingerprinting of IPv6 subscribers"//**, year after year across their six-year window, and that the delegated prefix identifies the subscriber //"even if subscribers' devices are using privacy addresses"//. Long-term tracking of an IPv6 subscriber is therefore //easier//, not harder. Aggregate on the /64 or the delegated prefix, not the full address, and do not assume dual-stack clients are equally identifiable on both stacks. Where hardware still derives the interface identifier from a MAC address, the address is //more// identifying than IPv4 ever was: {[rye2023_ipvseeyou]} extracted EUI-64-derived MACs from over 12M routers in 146 countries and geolocated them by correlating with wardriving data, reporting a **median error of 39 metres**. For the same devices, MaxMind's locations sat a median **26 km** from those wardriving positions — three orders of magnitude apart, from the same input address.

<WRAP todo>Before you use an IP as a user identifier, write down which of these four you have ruled out and how. If the answer is "none", use it as a network identifier (prefix or ASN) instead, where all four are far weaker.</WRAP>

===== A Script =====

Three layers, most authoritative first: routing from Team Cymru, ownership from the operators' own published prefixes, and only then the geolocation estimates — printed side by side so disagreement is visible rather than averaged away. Standard library only, no API keys.

<file python classify_ips.py>
#!/usr/bin/env python3
"""Classify a list of observed IP addresses, from the most authoritative source down.

    python3 classify_ips.py 8.8.8.8 1.1.1.1 104.16.132.229 13.32.99.63
    cut -d, -f2 server_ips.csv | python3 classify_ips.py -

Three layers, deliberately in this order:

  1. ROUTING (Team Cymru bulk whois, free, no key). Which ASN announces the
     covering BGP prefix, which RIR allocated it, and when. This is ground truth
     about the routing system, not an estimate.
  2. OPERATOR-PUBLISHED PREFIXES (AWS, Google Cloud, Cloudflare). If the operator
     says the prefix is theirs, it is theirs. Free, authoritative, and better
     than any commercial "is_datacenter" flag for the networks it covers.
  3. GEOLOCATION ESTIMATES (four free services). These are inferences. The script
     prints them side by side and flags disagreement rather than picking one.

Stdlib only. Free tiers are rate-limited (ip-api.com: 45 requests/minute from one
address), so this is for tens of addresses, not for a crawl's worth. For bulk
work take layer 1 and 2 offline: Team Cymru's bulk whois accepts thousands of
addresses per connection, and the cloud prefix files are a few megabytes.
"""
import argparse
import ipaddress
import json
import socket
import sys
import urllib.error
import urllib.request
from collections import Counter

TIMEOUT, UA = 15, "measuretheweb-ip-classify/1.0"

# Operator-published prefix lists. Authoritative by construction: these are the
# operators telling you which address space is theirs.
PREFIX_SOURCES = {
    "AWS": ("https://ip-ranges.amazonaws.com/ip-ranges.json",
            lambda d: [(p["ip_prefix"], f'{p["service"]}/{p["region"]}') for p in d["prefixes"]]),
    "Google Cloud": ("https://www.gstatic.com/ipranges/cloud.json",
                     lambda d: [(p["ipv4Prefix"], p.get("scope", "")) for p in d["prefixes"]
                                if "ipv4Prefix" in p]),
}
CLOUDFLARE_V4 = "https://www.cloudflare.com/ips-v4"

# Country/city estimates. Each entry maps the response onto (country, city, network).
GEO_SERVICES = {
    "ipinfo.io":  ("https://ipinfo.io/{ip}/json",   lambda d: (d["country"], d.get("city"), d.get("org"))),
    "ip-api.com": ("http://ip-api.com/json/{ip}",   lambda d: (d["countryCode"], d.get("city"), d.get("as"))),
    "ipwho.is":   ("https://ipwho.is/{ip}",         lambda d: (d["country_code"], d.get("city"),
                                                               d.get("connection", {}).get("isp"))),
    # ipapi.is returns a REDUCED object (cc, flags, asn_org, no city) for keyless
    # queries about a third-party address, and the full object with a key. Handle
    # both rather than crashing on the free tier.
    "ipapi.is":   ("https://api.ipapi.is?q={ip}",   lambda d: (d.get("location", {}).get("country_code") or d["cc"],
                                                               d.get("location", {}).get("city"),
                                                               d.get("asn", {}).get("org") or d.get("asn_org"))),
}
FLAGS = ["is_datacenter", "is_vpn", "is_proxy", "is_tor", "is_abuser", "is_mobile"]


def fetch_json(url):
    req = urllib.request.Request(url, headers={"User-Agent": UA, "Accept": "application/json"})
    with urllib.request.urlopen(req, timeout=TIMEOUT) as resp:
        return json.loads(resp.read().decode("utf-8"))


def cymru_bulk(ips):
    """Team Cymru IP-to-ASN, bulk mode over whois/43. One connection, all addresses.

    Returns {ip: {asn, prefix, cc, registry, allocated, as_name}}.
    Two separate fields that are easy to conflate, so both are printed:
      cc       -- the country on the ALLOCATION record. Where the resource
                  holder is registered, NOT where the host is. 1.1.1.1 is
                  registered AU, run by a US company, and answered from every
                  continent.
      registry -- which RIR made the allocation (arin/ripencc/apnic/...).
    """
    query = "begin\nverbose\n" + "\n".join(ips) + "\nend\n"
    sock = socket.create_connection(("whois.cymru.com", 43), timeout=TIMEOUT)
    sock.sendall(query.encode())
    chunks = []
    while True:
        data = sock.recv(8192)
        if not data:
            break
        chunks.append(data)
    sock.close()

    out = {}
    for line in b"".join(chunks).decode("utf-8", "replace").splitlines():
        parts = [f.strip() for f in line.split("|")]
        if len(parts) != 7 or parts[0] == "AS":
            continue                      # banner line or the column header
        asn, ip, prefix, cc, registry, allocated, as_name = parts
        out[ip] = {"asn": asn, "prefix": prefix, "cc": cc, "registry": registry,
                   "allocated": allocated, "as_name": as_name}
    return out


def load_operator_prefixes():
    """Download the operators' own prefix lists and build a flat lookup table."""
    table = []                            # [(ip_network, owner, detail)]
    for owner, (url, extract) in PREFIX_SOURCES.items():
        for prefix, detail in extract(fetch_json(url)):
            table.append((ipaddress.ip_network(prefix), owner, detail))
    req = urllib.request.Request(CLOUDFLARE_V4, headers={"User-Agent": UA})
    with urllib.request.urlopen(req, timeout=TIMEOUT) as resp:
        for prefix in resp.read().decode().split():
            table.append((ipaddress.ip_network(prefix), "Cloudflare", ""))
    return table


def owner_of(addr, table):
    """Most specific operator prefix covering addr, or None."""
    best = None
    for net, owner, detail in table:
        if addr in net and (best is None or net.prefixlen > best[0].prefixlen):
            best = (net, owner, detail)
    return best


def geolocate(ip):
    """Query every free service; return {service: (cc, city, net)} plus raw ipapi.is."""
    results, raw, errors = {}, {}, {}
    for name, (url, extract) in GEO_SERVICES.items():
        try:
            data = fetch_json(url.format(ip=ip))
        except (urllib.error.URLError, urllib.error.HTTPError, TimeoutError) as exc:
            errors[name] = f"{type(exc).__name__}: {exc}"
            continue
        if data.get("success") is False or data.get("error"):
            errors[name] = "service error"
            continue
        raw[name] = data
        results[name] = extract(data)
    return results, raw, errors


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("ips", nargs="+",
                    help="IP addresses, or - to read from stdin. IPv6 works for "
                         "layers 1 and 3; layer 2 loads IPv4 prefix lists only.")
    ap.add_argument("--no-geo", action="store_true",
                    help="routing and operator prefixes only; no per-IP API calls")
    args = ap.parse_args()

    ips = []
    for item in args.ips:
        ips.extend(sys.stdin.read().split() if item == "-" else [item])
    ips = [str(ipaddress.ip_address(i)) for i in dict.fromkeys(ips)]

    routing = cymru_bulk(ips)
    print(f"layer 1  routing (Team Cymru bulk whois, {len(routing)}/{len(ips)} answered)\n")
    print(f"  {'IP':<16} {'ASN':<8} {'BGP prefix':<20} {'reg-CC':<7} {'RIR':<9} "
          f"{'allocated':<11} AS name")
    for ip in ips:
        r = routing.get(ip)
        if r is None:
            print(f"  {ip:<16} -- not announced in the global routing table --")
            continue
        print(f"  {ip:<16} {r['asn']:<8} {r['prefix']:<20} {r['cc']:<7} {r['registry']:<9} "
              f"{r['allocated']:<11} {r['as_name']}")

    print("\nlayer 2  operator-published prefixes (AWS, Google Cloud, Cloudflare)\n")
    table = load_operator_prefixes()
    print(f"  {len(table)} prefixes loaded")
    for ip in ips:
        addr = ipaddress.ip_address(ip)
        if addr.version == 6:
            # PREFIX_SOURCES and CLOUDFLARE_V4 are the v4 files. An IPv6 address
            # is never "in" an IPv4 network, so without this the answer would
            # read as "not a cloud address" when it means "not checked".
            print(f"  {ip:<16} NOT CHECKED -- only IPv4 prefix lists are loaded")
            continue
        hit = owner_of(addr, table)
        if hit:
            net, owner, detail = hit
            print(f"  {ip:<16} {owner} {net}  {detail}")
        else:
            print(f"  {ip:<16} not in any loaded operator list")

    if args.no_geo:
        return 0

    print("\nlayer 3  geolocation estimates -- these are inferences, not facts\n")
    disputed = 0
    for ip in ips:
        results, raw, errors = geolocate(ip)
        if not results:
            print(f"  {ip}: no service answered ({errors})")
            continue
        countries = Counter(c for c, _, _ in results.values())
        cities = Counter(city for _, city, _ in results.values() if city)
        agree = len(countries) == 1
        disputed += 0 if agree else 1
        print(f"  {ip}   country {'/'.join(f'{c} x{n}' for c, n in countries.most_common())}"
              f"   {'agreed' if agree else 'DISPUTED'}")
        print(f"  {'':<16} city    {'/'.join(f'{c} x{n}' for c, n in cities.most_common()) or '(none given)'}"
              f"{'' if len(cities) <= 1 else '   <-- services disagree'}")
        for name, (cc, city, net) in sorted(results.items()):
            print(f"  {'':<16}   {name:<12} {cc}  {str(city):<16} {net}")
        if "ipapi.is" in raw:
            on = [f for f in FLAGS if raw["ipapi.is"].get(f)]
            print(f"  {'':<16}   ipapi.is flags {', '.join(on) if on else 'none set'}")
        if raw.get("ipinfo.io", {}).get("anycast"):
            print(f"  {'':<16}   ipinfo.io says ANYCAST: this address has no single location")
        for name, err in errors.items():
            print(f"  {'':<16}   {name:<12} unavailable: {err}")
        print()

    print(f"summary: {disputed} of {len(ips)} addresses had a cross-service country disagreement")
    return 0


if __name__ == "__main__":
    sys.exit(main())
</file>

Real output, run on 2026-08-06 against five addresses chosen to break something each: Google Public DNS, Cloudflare's resolver, a Cloudflare CDN edge, an AWS CloudFront edge, and the Swiss residential address used on [[Design:Crawling location]].

<code>
layer 1  routing (Team Cymru bulk whois, 5/5 answered)

  IP               ASN      BGP prefix           reg-CC  RIR       allocated   AS name
  8.8.8.8          15169    8.8.8.0/24           US      arin      2023-12-28  GOOGLE - Google LLC, US
  1.1.1.1          13335    1.1.1.0/24           AU      apnic     2011-08-11  CLOUDFLARENET - Cloudflare, Inc., US
  104.16.132.229   13335    104.16.128.0/20      US      arin      2014-03-28  CLOUDFLARENET - Cloudflare, Inc., US
  13.32.99.63      16509    13.32.98.0/23        US      arin      2020-08-05  AMAZON-02 - Amazon.com, Inc., US
  82.220.84.43     9044     82.220.0.0/16        CH      ripencc   2003-12-02  SOLNET - BSE Software GmbH, CH

layer 2  operator-published prefixes (AWS, Google Cloud, Cloudflare)

  11628 prefixes loaded
  8.8.8.8          not in any loaded operator list
  1.1.1.1          not in any loaded operator list
  104.16.132.229   Cloudflare 104.16.0.0/13  
  13.32.99.63      AWS 13.32.0.0/15  AMAZON/GLOBAL
  82.220.84.43     not in any loaded operator list

layer 3  geolocation estimates -- these are inferences, not facts

  8.8.8.8   country US x4   agreed
                   city    Mountain View x1/Ashburn x1/San Jose x1   <-- services disagree
                     ip-api.com   US  Ashburn          AS15169 Google LLC
                     ipapi.is     US  None             Google LLC
                     ipinfo.io    US  Mountain View    AS15169 Google LLC
                     ipwho.is     US  San Jose         Google LLC
                     ipapi.is flags is_datacenter, is_vpn, is_abuser
                     ipinfo.io says ANYCAST: this address has no single location

  1.1.1.1   country AU x4   agreed
                   city    Brisbane x2/South Brisbane x1   <-- services disagree
                     ip-api.com   AU  South Brisbane   AS13335 Cloudflare, Inc.
                     ipapi.is     AU  None             Cloudflare, Inc.
                     ipinfo.io    AU  Brisbane         AS13335 Cloudflare, Inc.
                     ipwho.is     AU  Brisbane         Cloudflare, Inc.
                     ipapi.is flags is_vpn, is_abuser
                     ipinfo.io says ANYCAST: this address has no single location

  104.16.132.229   country US x3/CA x1   DISPUTED
                   city    San Francisco x2/Toronto x1   <-- services disagree
                     ip-api.com   CA  Toronto          AS13335 Cloudflare, Inc.
                     ipapi.is     US  None             Cloudflare, Inc.
                     ipinfo.io    US  San Francisco    AS13335 Cloudflare, Inc.
                     ipwho.is     US  San Francisco    Cloudflare, Inc.
                     ipapi.is flags is_datacenter, is_abuser
                     ipinfo.io says ANYCAST: this address has no single location

  13.32.99.63   country DE x3/US x1   DISPUTED
                   city    Frankfurt am Main x2/New York x1   <-- services disagree
                     ip-api.com   US  New York         AS16509 Amazon.com, Inc.
                     ipapi.is     DE  None             Amazon.com, Inc.
                     ipinfo.io    DE  Frankfurt am Main AS16509 Amazon.com, Inc.
                     ipwho.is     DE  Frankfurt am Main Amazon.com, Inc.
                     ipapi.is flags is_datacenter

  82.220.84.43   country CH x4   agreed
                   city    Biel/Bienne x1/Samnaun x1/Zurich x1   <-- services disagree
                     ip-api.com   CH  Samnaun          AS9044 BSE Software GmbH
                     ipapi.is     CH  None             BSE Software GmbH
                     ipinfo.io    CH  Biel/Bienne      AS9044 BSE Software GmbH
                     ipwho.is     CH  Zurich           BSE Software GmbH
                     ipapi.is flags none set

summary: 2 of 5 addresses had a cross-service country disagreement
</code>

Everything this page argues is visible in that one run:

  - **Layer 1 answered for all five, with no ambiguity to reconcile.** An ASN, a covering prefix, the allocation record and its date, read out of the routing system. There is no confidence interval because nothing is being estimated — which is why nothing below layer 1 gets the same treatment on this page. Note that ''reg-CC'' and ''RIR'' are two different things and neither is a location: ''1.1.1.1'' is ''AU'' under ''apnic'' and is served from everywhere.
  - **The RIR country is not the host's country.** ''1.1.1.1'' is registered ''AU'' and all four geolocation services dutifully say Australia — for an anycast resolver that answers you from the nearest of hundreds of sites.
  - **Country disagreement is not rare.** Two of five addresses, and one of them (the AWS CloudFront edge, DE vs US) is exactly the kind of address a web crawl records by the thousand. If your paper says "//n// % of third-party requests left the EEA", that number moves depending on which vendor you asked.
  - **City is noise.** Five addresses, five sets of mutually inconsistent cities. The Swiss address is the one [[Design:Crawling location]]'s ''verify_vantage.py'' queried a day earlier, which got Zurich, Zürich, Samnaun and Lausanne from the same services; today one of them says Biel/Bienne. Same address, different day, different city.
  - **Risk flags misfire on the most benign addresses in the world.** Both public resolvers are ''is_vpn'' and ''is_abuser''.
  - **Layer 2 is authoritative but partial.** The CDN edges are matched exactly; the resolvers are not in the published lists at all, because those lists describe proxy and customer space.

===== Use in Publications =====

The figures below come from a structured extraction over **5,859 full-text papers** from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026 — of which the 2025 and 2026 venue-years are provisional, since CCS and IMC 2026 have not been held and IEEE S&P and WWW 2026 abstracts are not yet in the selection source. Populations are named per table; sentinel values (''not-stated'') are reported as themselves and never counted as answers; and papers, not tuples, are counted. Free-text resource names are folded through an explicit family map with the unmatched residue printed. Method and limitations are at the end of the section. The audit script is ''scripts/report_ip_classification.mjs''.

==== This is an IMC topic ====

**295 papers (5.0% of 5,859) classify an IP address** — that is, carry at least one classification record whose target is an IP address.

^ Venue ^ Papers classifying an IP ^ Papers in corpus ^ Share of venue ^
| IMC | 124 | 638 | 19.4% |
| USENIX Security | 55 | 1,410 | 3.9% |
| NDSS | 38 | 701 | 5.4% |
| CCS | 24 | 990 | 2.4% |
| TheWebConf | 24 | 843 | 2.8% |
| IEEE S&P | 21 | 767 | 2.7% |
| PETS | 9 | 510 | 1.8% |

One paper in five at IMC classifies an IP address; about one in thirty across the other six venues (171 of 5,221). If you are coming from the web-privacy side, **the methods literature you need is not in the venue you publish in** — that is the practical reason this page exists.

The absolute count grows steadily (31 papers in 2010–2013, 61 in 2014–2017, 68 in 2018–2021, 84 in 2022–2024, and 51 in the provisional 2025–2026 window). The //share// of the corpus peaked in the middle and has since flattened: 6.1%, 7.9%, 4.7%, 4.3%, 4.3%. That is the corpus growing around the topic rather than the topic shrinking.

==== How they classify ====

^ Method ^ Papers ^ Share of 295 ^
| Third-party service | 124 | 42.0% |
| Curated database | 109 | 36.9% |
| Heuristic rules | 88 | 29.8% |
| Blocklist | 16 | 5.4% |
| Regex or signature | 12 | 4.1% |
| Other | 9 | 3.1% |
| Manual labelling | 8 | 2.7% |
| Supervised ML | 7 | 2.4% |
| Graph analysis | 5 | 1.7% |
| Dynamic analysis | 4 | 1.4% |
| Static analysis | 2 | 0.7% |
| LLM | 1 | 0.3% |

Shares exceed 100% because a paper can use several. The distribution is the opposite of most classification tasks on this site: IP classification is overwhelmingly a **look-it-up** problem, not a machine-learning one. Supervised ML has not moved at all in absolute terms (7 papers on the old 4,322-paper corpus, 7 on this one), and the ''llm'' method fires exactly **once** — LLMCloudHunter {[schwartz2025_llmcloudhunter]} (TheWebConf 2025), GPT-4o, in the 2025–2026 window, and it is extracting IP indicators from threat-intelligence text rather than classifying addresses. Whatever LLM classification is doing elsewhere on this site, it has not arrived here: **1 of the 295 corpus papers that classify an ''ip-address'' target (0.3%)**, against 177 papers corpus-wide that classify //something// with an LLM. That contrast is the point — the method is mainstream and this target is untouched. The per-target ranking is on [[design:website_classification#Where LLMs actually appear]], where ''ip-address'' at 0.3% is the **lowest non-zero row** — below it are only the five targets no LLM paper touches at all, ''javascript'', ''fingerprinting-script'', ''malware'', ''sdk-or-library'' and ''website-popularity''.

==== Which resources, folded ====

Folded into families, because MaxMind alone appears under **50 distinct spellings** across the two fields we searched ("MaxMind", "Maxmind geoip", "MAXMIND GeoIP2", "Max-Mind GeoIP2 Precision Insights", "GeoIP: MaxMind", …). Folded, **134 papers in the corpus** name it; the commonest single spelling covers 28 of them. **Counting exact strings would understate the field's most-used resource by 79%** — which is why nothing on this page is aggregated by raw string.

Within the 295 IP-classifying papers specifically, MaxMind is named by 65; the other 69 name it only when describing their own vantage point, which is [[Design:Crawling location]]'s subject rather than this page's.

Grouped by the question the resource answers, families named by two or more papers:

^ Question ^ Family ^ Papers ^ Share of 295 ^
| Home-grown | Custom heuristic or classifier | 104 | 35.3% |
| Where is it? | MaxMind | 65 | 22.0% |
| Where is it? | IPinfo | 26 | 8.8% |
| Where is it? | IP2Location | 14 | 4.7% |
| Where is it? | Free geo-lookup APIs (freegeoip, ipstack, HostIP, IPInfoDB…) | 6 | 2.0% |
| Where is it? | NetAcuity (Digital Element) | 5 | 1.7% |
| Where is it? | RIPE IPmap | 5 | 1.7% |
| Where is it? | Unnamed commercial geo database | 4 | 1.4% |
| Where is it? | Chinese geo databases (Chunzhen/QQWry, AIWEN, QAX…) | 3 | 1.0% |
| Where is it? | Quova / Neustar //(historical)// | 2 | 0.7% |
| Location reference data //(not IP-based)// | GeoNames, Google Geocoding, Skyhook, WiGLE, Nominatim, Apple WPS | 4 | 1.4% |
| Whose network? | Team Cymru IP-to-ASN | 16 | 5.4% |
| Whose network? | RouteViews | 14 | 4.7% |
| Whose network? | WHOIS / IRR / RIR delegation files (incl. the IANA special-purpose registry) | 13 | 4.4% |
| Whose network? | CAIDA (prefix2as, AS2Org, ITDK) | 11 | 3.7% |
| Whose network? | Raw BGP feeds and IX data | 7 | 2.4% |
| Whose network? | RIPE RIS / RIPEstat / RIPE Atlas | 7 | 2.4% |
| Whose network? | PeeringDB | 6 | 2.0% |
| Whose network? | ASdb | 5 | 1.7% |
| Whose network? | pyasn / iptoasn.com | 2 | 0.7% |
| Known-bad? | Other IP blocklists (DShield, FireHOL, CBL, AbuseIPDB, Honey Pot, SpamCop, SinkDB…) | 33 | 11.2% |
| Known-bad? | VirusTotal / Google Safe Browsing | 14 | 4.7% |
| Known-bad? | Spamhaus | 12 | 4.1% |
| Known-bad? | Chainalysis | 2 | 0.7% |
| Router, and which? | bdrmapIT / MAP-IT / MIDAR / Hoiho | 16 | 5.4% |
| What runs on it? | Censys / Shodan / Nmap / Snort / Suricata / Criminal IP | 15 | 5.1% |
| What kind of host? | Fraud-score APIs (IPQualityScore, IPHub, Scamalytics, VPNAPI, Udger) | 5 | 1.7% |
| What kind of host? | GreyNoise | 5 | 1.7% |
| What kind of host? | MaxMind Anonymous IP / minFraud | 3 | 1.0% |
| What kind of host? | IP2Proxy | 2 | 0.7% |
| What kind of host? | Spur | 2 | 0.7% |

Nine more families were named by exactly one paper each (ip-api.com, Akamai EdgeScape, a CDN's internal geo, bgp.tools, DB-IP, a cloud provider's own published address ranges, SPF records, phone-number reference services, and one use of GPT-4o). **The unfolded residue is zero of 443 distinct strings** — every name in the corpus is in one of these families.

Read the top row carefully. **The single most common thing a paper does with an IP address is apply a rule it invented itself** (104 of 295), and that family is genuinely heterogeneous: reverse-DNS keyword matching, TTL clustering, hand-built allowlists, custom CGN criteria. That is not a criticism — for many questions no product exists — but it does mean the field has very little shared, comparable IP-classification method, which is the opposite of the situation for, say, [[Design:Website classification]].

==== Almost nobody cross-checks ====

Of the 114 papers that name a geolocation source, **only 14 (12.3%) name more than one.** Given a measured 89.4%-best country accuracy {[gharaibeh2017_look]} and a 34-point spread between two free databases {[darwich2023_replication]}, one source is one opinion.

But cross-checking is a floor, not a ceiling. {[gharaibeh2017_look]}'s own summary of what agreement buys you:

> the databases generally agree on the country-level (95.8% of the time), but the databases — from different vendors — show more discrepancy at city-level with more than 29% pairwise disagreements. However, we show that **agreement among the databases does not imply correctness.**

Two databases that both read the same RIR record agree with each other and are both wrong together. Use cross-checking to //find// the addresses you cannot trust, not to certify the ones where the vendors happen to concur.

==== Almost nobody validates ====

^ Validation of the IP classification ^ Papers ^ Share of 295 ^
| Not applicable | 121 | 41.0% |
| **None reported** | 99 | 33.6% |
| Manual validation | 54 | 18.3% |
| Comparison to another method | 49 | 16.6% |
| Held-out test set | 3 | 1.0% |
| Cross-validation | 1 | 0.3% |

**196 of 295 papers (66.4%) report no validation at all** on any of their IP classification records — against **29.9% across all 4,439 papers that classify anything** ([[Statistics:Annotation]], which owns the corpus-wide validation figures and the per-target comparison this row sits in) — that is, every such record is ''none-reported'' or ''not-applicable''. Separately and with a different meaning, 127 papers (43.1%) name a ground-truth source. The high //not-applicable// share is partly legitimate — looking up an ASN is not a classifier that needs a test set — but it is also where "we used MaxMind, therefore it is true" hides.

==== Papers do not say which snapshot they used ====

A geolocation database is reissued continuously and MaxMind commits to no fixed cadence — their own documentation tells you to check for a new release periodically through the day. "We used MaxMind" is therefore not a reproducible statement. Of the **134 papers in the whole corpus whose extraction names MaxMind** (in either an IP classification or a vantage-point record), all 134 mention it in their full text, and at most **36 (26.9%)** have any date, month or version token in the same sentence. That is an upper bound twice over: it counts crawl dates and page furniture as if they were database dates, and the unfiltered count of 52 shrinks to 36 only once bibliography entries are dropped. The script is ''scripts/maxmind_version.mjs''.

The standard to copy is the minority that names the file it read. One sentence is enough — this is from Benson et al., //Leveraging Internet Background Radiation for Opportunistic Network Analysis// (IMC 2015):

> Since MaxMind updates the database regularly (to reflect changes in the address space), we use the databases produced on August 1, 2012 and August 16, 2013 for the 2012 census and 2013 census periods, respectively.

==== Crawling papers rarely name a geolocation service ====

This table counts something narrower than its heading might suggest: naming a geolocation service **for the paper's own vantage point**.

^ Population ^ N ^ Names a geolocation service for its own vantage point ^ Share ^
| Ran an automated web crawl | 1,120 | 45 | 4.0% |
| Took a measurement from a vantage point | 3,908 | 194 | 5.0% |

Crawling papers that classify //observed// addresses are a partly different set: 54 of the 1,120. The union — a crawling paper that geolocates anything at all, its own position or someone else's — is **76 of 1,120 (6.8%)**.

Of the 194 papers naming a service for their vantage point, MaxMind takes 108 (55.7%), IPinfo 17 (8.8%), IP2Location 8 (4.1%), ip-api.com 7 (3.6%), RIPE IPmap 7 (3.6%), NetAcuity 5 (2.6%). This is a near-monoculture: **one vendor is the field's de facto ground truth for location.** We cannot tell from the extraction how many of those 108 used the free GeoLite2 rather than paid GeoIP2 — the folding does not separate the products, and most papers do not say (see above). Where the free tier //is// what was used, note that it is the one that scored 55% at city level in the only recent head-to-head {[darwich2023_replication]}.

==== Where it is used well in web measurement ====

Three examples from the corpus of IP classification carrying a web-privacy result, all of which report what they used:

  * {[urban2020beyond]} passively logged DNS responses and used MaxMind to test whether third-party addresses sat in countries without a GDPR adequacy decision, finding "93 % of the analyzed websites embedded third parties that are located in regions that might not be in line with the current legal framework". The claim is a country-level one, which is the level geolocation supports. It also carries a lesson the authors could not have avoided: they define adequate as "part of the Privacy Shield ... and countries part of the European Economic Area", and **Privacy Shield was invalidated by //Schrems II// three months after the paper appeared**. Your geolocation may be right and your classification still expire. Publish the country codes, not only the adequate/non-adequate verdict, so the analysis can be re-run against a later list.
  * {[iordanou2018_tracing]} traced cross-border web tracking and is one of the fourteen papers that used more than one geolocation source.
  * {[kumar2024_choices]} geolocated 4,286 government server addresses with IPinfo, then detected anycast with MAnycast2 and //verified// the country with RIPE Atlas probes, excluding the 72 anycast addresses it could not confirm. 87% of government URLs were served domestically and 98.3% of EU government URLs from inside the EU — figures that only mean something because of the verification step.

==== Methodology and limitations of these figures ====

  * **How they were produced.** One structured record per paper, extracted from full text; every tuple carries a verbatim evidence quote and its section, so any figure traces to a sentence. We re-read the quotes behind the accuracy figures against ''paper.cols.txt'' before publishing them.
  * **What "classifies an IP address" means.** The paper has at least one classification record with target ''ip-address''. A paper that geolocates addresses only to describe its own vantage point is counted in the //vantage// tables instead, not here.
  * **Folding.** ''scripts/ip_fold.mjs'' maps 443 distinct free-text names onto ordered regex families, specific products before the generic term they contain, and returns anything unmatched. Residue is zero and is printed on every run — but it was **18 strings** when the corpus grew to 2026 and had to be extended; see [[provenance:design:ip_classification]] for what was added and where each new string went. The families are ours, not the papers': ''Other IP blocklists'' merges a dozen services that are not interchangeable.
  * **Field stability.** The enums (''method'', ''validation'', ''target'') are reproducible on a repeat extraction to within a few points. Free-text resource names are ~20% stable by exact string, which is why they appear here only as folded rankings.
  * **Venue coverage.** Seven venues only — the scope and the selection funnel are on [[literature:corpus]]. For //this// topic the venues where much of the work actually appears are all absent: **PAM, TMA, ANRW, SIGCOMM and ACM CCR are not in the corpus**, and for IP geolocation specifically that is a serious gap.
  * **The corpus reaches 2026, but its last two years are provisional** (see [[literature:corpus]]), so a per-period row ending in 2025–2026 rests on fewer papers than a complete window would give. The "current in 2026" judgements on this page are ours, checked against vendor and standards documentation, not derived from the corpus.
  * **The LLM figures come from a shared script, not from this page's own query.** ''scripts/report_llm_currency.mjs'' computes ''classification.method == "llm"'' once, per target, for all three classification pages ([[design:website_classification]], [[privacy:javascript]] and this one), and prints each page's own sentence next to what the corpus says so a drift between them fails visibly. Its output is on [[provenance:design:ip_classification]].
  * **Every query behind this section, the report script and its unedited output** are on [[provenance:design:ip_classification]]; corpus-level caveats are on [[literature:corpus]].

===== What to Report =====

For an IP-derived claim to be checkable, a methods section needs:

  - **Which question you answered** — routing, organisation, network type, location. Do not write "we geolocated the IPs" when what you needed was the ASN.
  - **The source, by product name**, not by company: ''GeoLite2-City'', not "MaxMind".
  - **The snapshot date** of the database, or the date range over which the API was queried. This is the single most-omitted item.
  - **Whether you cross-checked**, against what, and how often they disagreed. A one-line "the two sources agreed on country for //n// % of addresses" is worth more than a paragraph of caveats.
  - **What you did with anycast addresses** — detected and excluded, verified separately, or ignored.
  - **What you did with unresolved addresses.** Dropping them silently changes your denominator.
  - **The granularity you claim.** If your source gives an accuracy radius, report it. If you are claiming a city, justify it.
  - **The raw classification, not only the derived verdict.** Publish the country codes and ASNs alongside "adequate / non-adequate" or "EU / non-EU", so the analysis survives the list changing under it — as it did for {[urban2020beyond]} when //Schrems II// invalidated Privacy Shield.
  - **For anything derived from a risk or proxy flag**, the vendor, the flag name, the date, and a hand-checked sample of what it excluded.

===== Legal and Ethical Notes =====

**An IP address is personal data.** In the EU the reference point is the CJEU's //Breyer// judgment (Case C-582/14, ECLI:EU:C:2016:779, 19 October 2016), which held that a dynamic IP address is personal data in the hands of an online media services provider where that provider has legal means reasonably likely to be used to identify the subscriber, e.g. via the ISP. Note the shape of the test: identifiability is assessed **relative to who holds the data and what means are available to them**, not in the abstract. The Court restated that framing in //EDPS v SRB// (Case C-413/23 P, ECLI:EU:C:2025:645, 4 September 2025), holding that the existence of additional information enabling identification "does not, in itself, mean that pseudonymised data must be regarded as constituting, in all cases and for every person, personal data" — while setting the General Court's judgment aside and referring the case back, so it is not the last word.((Read the judgment before relying on it; this is a methods page, not legal advice, and the case was remitted rather than finally decided.)) Practical consequences for a measurement:

  * Client IP addresses in a log you were given are personal data; treat the dataset accordingly and say so in the ethics section. See [[Practices:Ethics]].
  * Truncating to a /24 (or /48 for IPv6) is the common mitigation and is **pseudonymisation, not anonymisation** — it reduces but does not eliminate identifiability, and CGNAT means the residual population behind a /24 varies enormously by network.
  * Server-side addresses — the third parties a crawl connects to — are a different case: they identify an organisation, not a person, and publishing them is normal practice.

**Geolocation is often the evidence in a compliance claim**, which raises the bar on it. If the paper's contribution is "//n// % of EU websites send data to non-adequate countries", the country classification //is// the result, and a single unvalidated free-tier lookup will not survive review. Cross-check, report the disagreement rate, and where a specific site matters, verify it actively as {[kumar2024_choices]} did.

===== Open Questions =====

<WRAP todo>
  * **No head-to-head of the geolocation databases on a web-measurement population.** {[gharaibeh2017_look]} used router interfaces (2017), {[darwich2023_replication]} RIPE Atlas anchors (2023), {[nabi2026_lostprefix]} Atlas and Giga (2026). None of them is "the servers a Tranco top-10k crawl connects to" — a population dominated by CDN and cloud edges, which is exactly the population the anycast and datacenter caveats bite hardest on.
  * **How much does the choice of database move a published cross-border-transfer figure?** Re-running one compliance paper's analysis with four databases would be a small, cheap, useful replication, and the answer is not obviously small: the run above disagreed on country for the AWS edge.
  * **Geofeed adoption is not measured for the web.** {[livadariu2024_geofeeds]} gives 1.50% of allocated IPv4 prefixes overall; nobody has asked what share of the //address space a crawl actually touches// has a geofeed, which — given how concentrated that space is on a few large operators — could be much higher or much lower.
  * **No systematic evaluation of the commercial datacenter/VPN/proxy flags against ground truth.** Every paper that uses one takes it on faith, and the run above shows two public DNS resolvers flagged ''is_vpn'' and ''is_abuser''. A labelled benchmark here would be immediately useful and is well within a single student's reach.
  * **No peer-reviewed method for identifying hosting/datacenter address space** beyond the operators' own published lists — which means everything outside the big five clouds is guesswork.
  * **LLMs have reached AS-to-organisation mapping {[selmo2025_borges]} but not IP classification.** We found nothing peer-reviewed applying an LLM to geolocation, host typing or residential/VPN/datacenter labelling as of 2026-09-03. This page previously said that was "unlike cookie and policy classification, where LLM methods are now routine"; that was wrong and is corrected here. Measured per target across the 5,859-paper corpus, the LLM share of the papers classifying that target at all is **11.8% for ''privacy-policy'' (12 of 102)** — the highest of any target and the only one that could support the word "routine" — but only **1.9% for ''cookie'' (1 of 53)**, a single TheWebConf 2025 paper. Against those, ''ip-address'' at 1 of 295 is low but not the outlier the old sentence implied; ''javascript'' and ''fingerprinting-script'' are at zero. See [[design:website_classification#Where LLMs actually appear]] for the whole table. Whether the gap here is because the task has no useful text to read, or because nobody has tried, is still an open question.
  * **No strong successor to {[richter2016_multi]} on CGNAT prevalence.** The best numbers on how much of the client Internet sits behind shared addresses are a decade old, and IPv4 exhaustion has only got worse since.
</WRAP>

===== Related Pages =====

  * [[Statistics:Annotation|Annotation and Validation]] — validating a label set whatever it labels; IP addresses are the second-worst-validated target on this wiki.
  * [[Design:Crawling location]] — the mirror image: classifying and verifying //your own// vantage point before a crawl.
  * [[Design:Website classification]] — classifying sites by topic; the same "which service, which taxonomy, which validation" questions with entirely different answers.
  * [[Privacy:Requests]] — where the addresses on this page come from in a web crawl.
  * [[Programming:Traffic files]] — extracting server addresses from a HAR or a packet capture.
  * [[Practices:Ethics]] — client IP addresses are personal data.
  * [[Design:Sampling]] //(not yet written)// — why an IP is a bad sampling unit.

====== References ======

<bibtex bibliography></bibtex>


/* This enables discussion under this article. */
~~DISCUSSION~~