Differences

This shows you the differences between two versions of the page.

--- design:ip_classification [2026/08/12 15:45] – Methodology section: replaced the generic corpus caveats (absent venues, provisional 2025-2026) with a pointer to the new literature:corpus page, keeping the page-specific consequence. Authored by Claude. karel.kubicek.claude
+++ design:ip_classification [2026/09/11 02:31] (current) – Point at the new statistics:annotation page from the validation section and Related Pages. Authored by Claude karel.kubicek.claude
@@ Line 28: / Line 28: @@
   * **Playwright**: ''await response.serverAddr()'' → ''{ipAddress, port}''.
   * **Chrome DevTools Protocol** (so also Puppeteer, and OpenWPM through its instrumentation): ''Network.responseReceived'' → ''e.response.remoteIPAddress'' and ''remotePort''.
-  * **A HAR file**: the per-entry ''serverIPAddress'' field, which HAR 1.2 defines and Chrome, Firefox and Playwright all populate. (More on working with HAR: [[Programming:Traffic files]], not yet written.)
+  * **A HAR file**: the per-entry ''serverIPAddress'' field, which HAR 1.2 defines and Chrome, Firefox and Playwright all populate. (More on working with HAR: [[Programming:Traffic files]].)
 Measured on one page load of ''https://www.bbc.com/'' on 6 August 2026: all 130 responses carried an address via Playwright and via CDP; the HAR was missing one, a beacon still in flight when the context closed. So take it from the live event stream if you can, and treat a HAR as very slightly lossy.
@@ Line 72: / Line 72: @@
   * **[[https://asdb.stanford.edu/|ASdb]]** {[ziv2021_asdb]} classifies ASNs into 17 industry categories and 95 sub-categories (hosting, ISP, education, government…), reporting 96% coverage of ASes at 93% accuracy on the top level and 75% on sub-categories. Snapshots run through March 2026. Used, for example, by {[darwich2023_replication]} to describe what kind of networks their geolocation targets sat in.
-<wrap todo>**CAIDA's separate //AS Classification// dataset is discontinued** — CAIDA's own page states it is no longer supported and download access has been removed. Papers from 2015–2022 cite it routinely; ASdb is the live replacement. Check before you cite a dataset you found in a related-work section.</wrap>
+<WRAP todo>**CAIDA's separate //AS Classification// dataset is discontinued** — CAIDA's own page states it is no longer supported and download access has been removed. Papers from 2015–2022 cite it routinely; ASdb is the live replacement. Check before you cite a dataset you found in a related-work section.</WRAP>
-The current state of the art on the sibling problem is {[selmo2025_borges]} (IMC 2025), which few-shot prompts an LLM over PeeringDB free-text fields plus website and domain evidence, and reports a 7% improvement in sibling-ASN identification over AS2Org-style methods. That is, as of 2026, **the one place in IP classification where an LLM method has cleared peer review** — see [[#Open Questions]].
+The current state of the art on the sibling problem is {[selmo2025_borges]} (IMC 2025), which few-shot prompts GPT-4o-mini at temperature 0 over PeeringDB free-text fields plus website and domain evidence, and reports a 7% improvement in sibling-ASN identification over AS2Org-style methods. That is, as of 2026, **the one place in the AS-and-organisation half of this page where an LLM method has cleared peer review** — see [[#Open Questions]]. It is a //different// paper from the single ''llm'' firing in this page's corpus population, which is LLMCloudHunter {[schwartz2025_llmcloudhunter]} extracting IP indicators from threat-intelligence prose; do not conflate the two, and note that Borges's classification tuples are filed under ''other'' rather than ''ip-address'', so it does not appear in the method table in [[#How they classify]] at all.
 Two traps specific to web measurement. First, **the AS that announces a server's address is very often not the party you care about**: a tracker served through Cloudflare terminates in AS13335, and attributing the tracking to Cloudflare is wrong. Resolve the //name// (the request URL, the certificate, the CNAME chain) before or instead of the address. Second, **ASNs churn**: {[nemmi2021_parallel]} tracked the administrative and operational lives of ASNs separately and found 22,729 administrative lifetimes (17.9%) with no observed BGP activity at all, plus 1,667 ASNs announcing in BGP without an overlapping allocation. An ASN-to-organisation table from 2019 is not a table about 2026.
@@ Line 227: / Line 227: @@
 [[https://www.rfc-editor.org/rfc/rfc6269.html|RFC 6269]] (//Issues with IP Address Sharing//) is the catalogue of everything this breaks, and worth ten minutes if you are about to count users. [[https://www.rfc-editor.org/rfc/rfc6598.html|RFC 6598]] reserves 100.64.0.0/10 for shared address space — but **do not use that prefix as your CGNAT test.** In the same 2023 study only three of the 14 operators actually used 100.64.0.0/10; two used it alongside 10.0.0.0/8, and the rest used something else. A positive is informative; a negative tells you nothing.
-**Churn.** Residential addresses are reassigned; prefixes are reallocated between organisations; geolocation database entries change under you. **How fast depends enormously on the network, and you cannot assume.** {[padmanabhan2020_dynamips]} measured assignment durations directly, from RIPE Atlas dual-stack probes plus 32.7 billion address associations observed by a CDN, and found the distribution spans orders of magnitude between ISPs — some reassign on a fixed daily cycle, others leave an address in place for months. Measure it for your own population, or state the assumption. This is also why the database snapshot date matters more than the vendor: between the run below and the one on [[Design:Crawling location]] the previous day, the //same// Swiss address was placed in Zurich, Samnaun, Lausanne and Biel/Bienne by different services on different days.
+**Churn.** Residential addresses are reassigned; prefixes are reallocated between organisations; geolocation database entries change under you. **How fast depends enormously on the network, and you cannot assume.** {[padmanabhan2020_dynamips]} measured assignment durations directly, from six years of RIPE Atlas dual-stack probe data plus 32.7 billion address associations observed by a CDN, and found practice differs between ISPs by orders of magnitude. In IPv4 (non-dual-stack probes) they saw consistent periodic renumbering on 35 networks, with modes at 1 day (Deutsche Telekom), 1.5 days (Proximus), a week (Orange) and two weeks (BT); in IPv6, fixed cycles as short as 12 hours (ANTEL, Uruguay), 24 hours (Deutsche Telekom and four other German ISPs) and 48 hours (Global Village, Brazil), while elsewhere IPv6 /64 prefixes //"tend to be stable for months and years"//. Measure it for your own population, or state the assumption. This is also why the database snapshot date matters more than the vendor: between the run below and the one on [[Design:Crawling location]] the previous day, the //same// Swiss address was placed in Zurich, Samnaun, Lausanne and Biel/Bienne by different services on different days.
 **Anycast and load balancing.** One address, many hosts, many countries — see above. In the reverse direction, one host answers on many addresses, so counting distinct server addresses over-counts servers.
-**IPv6 is different, and in the direction people do not expect.** The intuition is that IPv6 clients rotate temporary addresses ([[https://www.rfc-editor.org/rfc/rfc8981.html|RFC 8981]]) and are therefore harder to track. That is true of the interface identifier and false of the assignment: {[padmanabhan2020_dynamips]} found that **IPv6 assignments last //longer// than IPv4 ones, often remaining stable for months** — which makes long-term tracking of an IPv6 subscriber //easier//, not harder.((Paraphrased rather than quoted: we verified the paper, venue and topic against Crossref and the authors' own listing, but could not reach a copy to quote the sentence verbatim. Check it before citing the wording.)) Aggregate on the /64 or the delegated prefix, not the full address, and do not assume dual-stack clients are equally identifiable on both stacks. Where hardware still derives the interface identifier from a MAC address, the address is //more// identifying than IPv4 ever was: {[rye2023_ipvseeyou]} extracted EUI-64-derived MACs from over 12M routers in 146 countries and geolocated them by correlating with wardriving data, reporting a **median error of 39 metres**. For the same devices, MaxMind's locations sat a median **26 km** from those wardriving positions — three orders of magnitude apart, from the same input address.
+**IPv6 is different, and in the direction people do not expect.** The intuition is that IPv6 clients rotate temporary addresses ([[https://www.rfc-editor.org/rfc/rfc8981.html|RFC 8981]]) and are therefore harder to track. That is true of the interface identifier and false of the assignment: {[padmanabhan2020_dynamips]} found that **//"IPv6 assignments have longer durations than IPv4 assignments—often remaining stable for months—thereby allowing the possibility of long-term fingerprinting of IPv6 subscribers"//**, year after year across their six-year window, and that the delegated prefix identifies the subscriber //"even if subscribers' devices are using privacy addresses"//. Long-term tracking of an IPv6 subscriber is therefore //easier//, not harder. Aggregate on the /64 or the delegated prefix, not the full address, and do not assume dual-stack clients are equally identifiable on both stacks. Where hardware still derives the interface identifier from a MAC address, the address is //more// identifying than IPv4 ever was: {[rye2023_ipvseeyou]} extracted EUI-64-derived MACs from over 12M routers in 146 countries and geolocated them by correlating with wardriving data, reporting a **median error of 39 metres**. For the same devices, MaxMind's locations sat a median **26 km** from those wardriving positions — three orders of magnitude apart, from the same input address.
-<wrap todo>Before you use an IP as a user identifier, write down which of these four you have ruled out and how. If the answer is "none", use it as a network identifier (prefix or ASN) instead, where all four are far weaker.</wrap>
+<WRAP todo>Before you use an IP as a user identifier, write down which of these four you have ruled out and how. If the answer is "none", use it as a network identifier (prefix or ASN) instead, where all four are far weaker.</WRAP>
 ===== A Script =====
@@ Line 574: / Line 574: @@
 | LLM | 1 | 0.3% |
-Shares exceed 100% because a paper can use several. The distribution is the opposite of most classification tasks on this site: IP classification is overwhelmingly a **look-it-up** problem, not a machine-learning one. Supervised ML has not moved at all in absolute terms (7 papers on the old 4,322-paper corpus, 7 on this one), and the ''llm'' method fires exactly **once** — one paper, GPT-4o, in the 2025–2026 window. Whatever LLM classification is doing elsewhere on this site, it has not arrived here.
+Shares exceed 100% because a paper can use several. The distribution is the opposite of most classification tasks on this site: IP classification is overwhelmingly a **look-it-up** problem, not a machine-learning one. Supervised ML has not moved at all in absolute terms (7 papers on the old 4,322-paper corpus, 7 on this one), and the ''llm'' method fires exactly **once** — LLMCloudHunter {[schwartz2025_llmcloudhunter]} (TheWebConf 2025), GPT-4o, in the 2025–2026 window, and it is extracting IP indicators from threat-intelligence text rather than classifying addresses. Whatever LLM classification is doing elsewhere on this site, it has not arrived here: **1 of the 295 corpus papers that classify an ''ip-address'' target (0.3%)**, against 177 papers corpus-wide that classify //something// with an LLM. That contrast is the point — the method is mainstream and this target is untouched. The per-target ranking is on [[design:website_classification#Where LLMs actually appear]], where ''ip-address'' at 0.3% is the **lowest non-zero row** — below it are only the five targets no LLM paper touches at all, ''javascript'', ''fingerprinting-script'', ''malware'', ''sdk-or-library'' and ''website-popularity''.
 ==== Which resources, folded ====
@@ Line 641: / Line 641: @@
 | Cross-validation | 1 | 0.3% |
-**196 of 295 papers (66.4%) report no validation at all** on any of their IP classification records — that is, every such record is ''none-reported'' or ''not-applicable''. Separately and with a different meaning, 127 papers (43.1%) name a ground-truth source. The high //not-applicable// share is partly legitimate — looking up an ASN is not a classifier that needs a test set — but it is also where "we used MaxMind, therefore it is true" hides.
+**196 of 295 papers (66.4%) report no validation at all** on any of their IP classification records — against **29.9% across all 4,439 papers that classify anything** ([[Statistics:Annotation]], which owns the corpus-wide validation figures and the per-target comparison this row sits in) — that is, every such record is ''none-reported'' or ''not-applicable''. Separately and with a different meaning, 127 papers (43.1%) name a ground-truth source. The high //not-applicable// share is partly legitimate — looking up an ASN is not a classifier that needs a test set — but it is also where "we used MaxMind, therefore it is true" hides.
 ==== Papers do not say which snapshot they used ====
@@ Line 679: / Line 679: @@
   * **Venue coverage.** Seven venues only — the scope and the selection funnel are on [[literature:corpus]]. For //this// topic the venues where much of the work actually appears are all absent: **PAM, TMA, ANRW, SIGCOMM and ACM CCR are not in the corpus**, and for IP geolocation specifically that is a serious gap.
   * **The corpus reaches 2026, but its last two years are provisional** (see [[literature:corpus]]), so a per-period row ending in 2025–2026 rests on fewer papers than a complete window would give. The "current in 2026" judgements on this page are ours, checked against vendor and standards documentation, not derived from the corpus.
+  * **The LLM figures come from a shared script, not from this page's own query.** ''scripts/report_llm_currency.mjs'' computes ''classification.method == "llm"'' once, per target, for all three classification pages ([[design:website_classification]], [[privacy:javascript]] and this one), and prints each page's own sentence next to what the corpus says so a drift between them fails visibly. Its output is on [[provenance:design:ip_classification]].
   * **Every query behind this section, the report script and its unedited output** are on [[provenance:design:ip_classification]]; corpus-level caveats are on [[literature:corpus]].
@@ Line 707: / Line 708: @@
 ===== Open Questions =====
-  * <wrap todo>**No head-to-head of the geolocation databases on a web-measurement population.** {[gharaibeh2017_look]} used router interfaces (2017), {[darwich2023_replication]} RIPE Atlas anchors (2023), {[nabi2026_lostprefix]} Atlas and Giga (2026). None of them is "the servers a Tranco top-10k crawl connects to" — a population dominated by CDN and cloud edges, which is exactly the population the anycast and datacenter caveats bite hardest on.</wrap>
+<WRAP todo>
-  * <wrap todo>**How much does the choice of database move a published cross-border-transfer figure?** Re-running one compliance paper's analysis with four databases would be a small, cheap, useful replication, and the answer is not obviously small: the run above disagreed on country for the AWS edge.</wrap>
+  * **No head-to-head of the geolocation databases on a web-measurement population.** {[gharaibeh2017_look]} used router interfaces (2017), {[darwich2023_replication]} RIPE Atlas anchors (2023), {[nabi2026_lostprefix]} Atlas and Giga (2026). None of them is "the servers a Tranco top-10k crawl connects to" — a population dominated by CDN and cloud edges, which is exactly the population the anycast and datacenter caveats bite hardest on.
-  * <wrap todo>**Geofeed adoption is not measured for the web.** {[livadariu2024_geofeeds]} gives 1.50% of allocated IPv4 prefixes overall; nobody has asked what share of the //address space a crawl actually touches// has a geofeed, which — given how concentrated that space is on a few large operators — could be much higher or much lower.</wrap>
+  * **How much does the choice of database move a published cross-border-transfer figure?** Re-running one compliance paper's analysis with four databases would be a small, cheap, useful replication, and the answer is not obviously small: the run above disagreed on country for the AWS edge.
-  * <wrap todo>**No systematic evaluation of the commercial datacenter/VPN/proxy flags against ground truth.** Every paper that uses one takes it on faith, and the run above shows two public DNS resolvers flagged ''is_vpn'' and ''is_abuser''. A labelled benchmark here would be immediately useful and is well within a single student's reach.</wrap>
+  * **Geofeed adoption is not measured for the web.** {[livadariu2024_geofeeds]} gives 1.50% of allocated IPv4 prefixes overall; nobody has asked what share of the //address space a crawl actually touches// has a geofeed, which — given how concentrated that space is on a few large operators — could be much higher or much lower.
-  * <wrap todo>**No peer-reviewed method for identifying hosting/datacenter address space** beyond the operators' own published lists — which means everything outside the big five clouds is guesswork.</wrap>
+  * **No systematic evaluation of the commercial datacenter/VPN/proxy flags against ground truth.** Every paper that uses one takes it on faith, and the run above shows two public DNS resolvers flagged ''is_vpn'' and ''is_abuser''. A labelled benchmark here would be immediately useful and is well within a single student's reach.
-  * <wrap todo>**LLMs have reached AS-to-organisation mapping {[selmo2025_borges]} but not IP classification.** We found nothing peer-reviewed applying an LLM to geolocation, host typing or residential/VPN/datacenter labelling as of August 2026 — unlike cookie and policy classification, where LLM methods are now routine. Whether that is because the task has no useful text to read, or because nobody has tried, is an open question.</wrap>
+  * **No peer-reviewed method for identifying hosting/datacenter address space** beyond the operators' own published lists — which means everything outside the big five clouds is guesswork.
-  * <wrap todo>**No strong successor to {[richter2016_multi]} on CGNAT prevalence.** The best numbers on how much of the client Internet sits behind shared addresses are a decade old, and IPv4 exhaustion has only got worse since.</wrap>
+  * **LLMs have reached AS-to-organisation mapping {[selmo2025_borges]} but not IP classification.** We found nothing peer-reviewed applying an LLM to geolocation, host typing or residential/VPN/datacenter labelling as of 2026-09-03. This page previously said that was "unlike cookie and policy classification, where LLM methods are now routine"; that was wrong and is corrected here. Measured per target across the 5,859-paper corpus, the LLM share of the papers classifying that target at all is **11.8% for ''privacy-policy'' (12 of 102)** — the highest of any target and the only one that could support the word "routine" — but only **1.9% for ''cookie'' (1 of 53)**, a single TheWebConf 2025 paper. Against those, ''ip-address'' at 1 of 295 is low but not the outlier the old sentence implied; ''javascript'' and ''fingerprinting-script'' are at zero. See [[design:website_classification#Where LLMs actually appear]] for the whole table. Whether the gap here is because the task has no useful text to read, or because nobody has tried, is still an open question.
+  * **No strong successor to {[richter2016_multi]} on CGNAT prevalence.** The best numbers on how much of the client Internet sits behind shared addresses are a decade old, and IPv4 exhaustion has only got worse since.
+</WRAP>
 ===== Related Pages =====
+  * [[Statistics:Annotation|Annotation and Validation]] — validating a label set whatever it labels; IP addresses are the second-worst-validated target on this wiki.
   * [[Design:Crawling location]] — the mirror image: classifying and verifying //your own// vantage point before a crawl.
   * [[Design:Website classification]] — classifying sites by topic; the same "which service, which taxonomy, which validation" questions with entirely different answers.