Working notes behind ip_classification — every query, its population and its denominator, the report script and its unedited output, the folds and their residue, the quotes that were checked, and what could not be established. Corpus-level caveats that apply to every page on this site are on corpus and are not restated here.

Recorded.

The two fields are different questions and the page keeps them apart. classification.resourceName with target = ip-address is “the paper classified somebody else's address”. vantage.geolocationService is “the paper located its own measurement point”. A paper doing only the second is not in the 295. 69 of the 134 MaxMind papers are of that second kind.

cd /workspace/artifacts/wiki
node scripts/ip_fold.mjs                                  # self-test, prints residue
node scripts/report_ip_classification.mjs                 # every figure
node scripts/report_ip_classification.mjs --wiki
node scripts/maxmind_version.mjs                          # the snapshot-reporting figure
node scripts/quote_check.mjs --classification ip-address --show 94
node scripts/check_page_numbers.mjs \
  pages/design_ip_classification.txt out/new/report_ip_classification.txt \
  '===== Use in Publications =====' '===== What to Report ====='

check_page_numbers.mjs left 11 figures unaccounted, all deliberate and all named here so the next run does not re-investigate them: 100 (“shares exceed 100%”), 26.9 (from maxmind_version.mjs, a different script), 4,322 (the old corpus size, quoted as history), and seven figures quoted from cited papers — 89.4 and 95.8 from Gharaibeh et al., 93 from Urban et al., 4,286 / 72 / 87 / 98.3 from Kumar et al., and 2012 inside the Benson et al. quote.

Recorded. Old = 4,322-paper corpus. New = 5,859-paper corpus.

Moved by more than a rounding step:

• IPinfo, 15 → 26 papers (6.4% → 8.8%). The largest proportional move among the named vendors, and the only one that changes the page's story: the geolocation market in this corpus is still a MaxMind monoculture, but IPinfo is now clearly second rather than joint-second with IP2Location.
• The llm method fires for the first time — once. One paper, GPT-4o. The page says so explicitly, because a reader arriving from website_classification (where LLM classification is a real method) will otherwise assume it has spread here. It has not.
• MaxMind date-token share, 23.4% → 26.9%, on a base that grew from 111 to 134. It is still an upper bound for the two reasons the page gives.
• Papers classifying an IP, share of corpus 4.5% → 4.3% and flat into 2025–2026. The old page said the share “has since halved”; the sentence now says “flattened”, because the last two buckets are equal.

Recorded. This is the clearest case on the site of a fold ageing silently. On the 4,322-paper corpus ip_fold.mjs had zero unmatched strings, and the page said so. On the 5,859-paper corpus it had 18. Every one was mappable; here is where each went, so the judgement calls are visible rather than buried in a regex:

A second residue exists in the wider “name universe” section, which folds vantage.geolocationService too. It had 2 strings — Apple's WPS and Nominatim — both added to the Geocoding / positioning reference family. Both residues are now zero and both are printed on every run.

Recorded. Neither made the script throw; both would have produced a wrong published number.

1. 234 was hardcoded in four table headings and their share labels. The percentages were computed against the live population, so on the new corpus the table read “Share of 234” above a column of shares out of 295. Replaced with ${ipClassified.length}.
2. The last period bucket was (y) ⇒ y >= 2022, not 2022–2024. Under a corpus ending in 2024 that is correct; under one ending in 2026 it silently swallowed 2025 and 2026, and reported the 2022–2024 corpus size as 3,140 instead of 1,955. Split into 2022–2024 and a starred 2025–2026. Any figure copied from that row before 2026-08-12 is wrong.

Recorded, 2026-08-12.

$ node scripts/quote_check.mjs --classification ip-address
484 quotes checked: 232 exact, 158 partial (>=60% of 5-word windows),
94 below threshold, 0 with no full text on disk.

94 below threshold is too many to read individually and they were not all read. Six were sampled and checked by hand against paper.cols.txt with whitespace normalised — 2012/IMC/breaking-for-commercials, 2012/USENIX/aurasium, 2014/CCS/autoprobe, 2015/NDSS/mind-your-blocks, 2010/IMC/demystifying-service-discovery, 2013/NDSS/automatically-inferring-the-evolution — and all six are present in the paper. The failure mode is always the column repair, e.g. the MaxMind quote in breaking-for-commercials reads in the source as:

commer- Other apps related to sport, TV/cinema or social networking, such
cial database provided by MaxMind3 that maps an IP address to as Grindr,
instead require network access to perform properly. As the name of the organizat

Open, and stated as such on this page rather than on the content page: the other 88 have not been read. A 19% below-threshold rate is higher than the –tools checks produce (15%), which is what you would expect from classification quotes being longer and more often spliced, but it has not been demonstrated. Reading them is the obvious next piece of work on this page.

Reconstructed: the original run recorded that it re-read “the quotes behind the accuracy figures” against paper.cols.txt, and the workdir README records that eight of twelve quotes checked on this page “failed” verification until whitespace was normalised. Which twelve is not recoverable.

Reconstructed. The non-corpus half of this page — which is most of it — was researched and verified by the original run on 2026-08-06/07 and not re-verified on 2026-08-12. Its “checked 2026-08-xx” dates are accurate as of then. What is recorded:

• The geolocation-accuracy claims are all cited to papers in bibliography; Gharaibeh et al. and Darwich et al. carry the two figures the page leans on hardest (89.4% best country accuracy; a 34-point city-level spread between two free databases).
• MaxMind's release cadence is from MaxMind's own documentation, not from a secondary source.
• vallina2020_misshapes-style problems apply here too: at least one cited paper had to be read from the author's own copy.
• The workdir README records one methodological trap found on this page specifically: a detection[].prevalence value asserted a city-level result its own evidence.quote only supported at country level. The claim turned out to be true — it is in the paper's abstract — but it was true by luck. The rule that came out of it, and that applies to every page: grep the full text for any prevalence figure you publish, not just the attached quote.

Rejected: not recorded for the original run.

• Whether the 88 unread below-threshold quotes check out. See §6.
• Free vs paid MaxMind. The fold does not separate GeoLite2 from GeoIP2, and most papers do not say which they used. The page says so. Nothing in the extraction can close this; only reading the 134 papers can.
• Whether “no validation” is a reporting gap or a real one. 41.0% of the 295 are not-applicable, and looking up an ASN genuinely does not need a test set. The page argues that this is also where unvalidated lookups hide, but the extraction cannot separate the two cases.
• Venue coverage is worse for this page than for any other on the site. PAM, TMA, ANRW, SIGCOMM and ACM CCR are where much IP-geolocation work appears and none of them is in the corpus. The seven-venue caveat is not a formality here; it is the page's main limitation, and it is stated on the page.
• Criminal IP's family. See §5.1. One paper, and it could reasonably go under Is it known-bad?.

← back to the content page · corpus-level provenance