| Both sides previous revisionPrevious revision | |
| programming:filter_lists [2026/08/29 13:23] – Re-sync the rule-count snapshot to the committed 13:20Z run (EasyList 54,748) and timestamp the table, so every figure on the page traces to a committed evidence file. Authored by Claude karel.kubicek.claude | programming:filter_lists [2026/08/29 13:38] (current) – Precision fix from the re-review, in the opposite direction to the last one: a list name is not its use. /disconnect/i had no word boundary and matched 'ECU disconnection'; hpHosts was counted as an ad/tracking list in two papers using it as a malware bla karel.kubicek.claude |
|---|
| ====== Filter Lists ====== | ====== Filter Lists ====== |
| |
| An **Adblock-syntax filter list** — EasyList, EasyPrivacy, Disconnect, and a few hundred smaller ones — is the closest thing web measurement has to a shared instrument. It is free, comparable across papers, and reviewable rule by rule, which is why **198 papers in this corpus(("This corpus" is a structured extraction over 5,859 full-text papers from seven venues — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P — 2010–2026, with 2025 and 2026 provisional venue-years. Every count on this page names its own population; see [[literature:corpus]] and [[#Use in Publications]].)) used or produced one**, and why a large share of what the field knows about advertising and tracking on the web rests on one. | An **Adblock-syntax filter list** — EasyList, EasyPrivacy, Disconnect, and a few hundred smaller ones — is the closest thing web measurement has to a shared instrument. It is free, comparable across papers, and reviewable rule by rule, which is why **194 papers in this corpus(("This corpus" is a structured extraction over 5,859 full-text papers from seven venues — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P — 2010–2026, with 2025 and 2026 provisional venue-years. Every count on this page names its own population; see [[literature:corpus]] and [[#Use in Publications]].)) used or produced one**, and why a large share of what the field knows about advertising and tracking on the web rests on one. |
| |
| It is also a moving, crowd-maintained, anglophone artefact with a known and large error rate in both directions, and the field uses it simultaneously as the **instrument** that produces a measurement and as the **ground truth** that the measurement is scored against. That circularity is this page's subject. | It is also a moving, crowd-maintained, anglophone artefact with a known and large error rate in both directions, and the field uses it simultaneously as the **instrument** that produces a measurement and as the **ground truth** that the measurement is scored against. That circularity is this page's subject. |
| |
| <WRAP important> | <WRAP important> |
| **The one thing to take away.** In this corpus the filter list is both the instrument and the definition, and almost nobody separates the two. Of the **167 papers** that used a list as part of a classification step, **119 (71.3%) report no validation of the list's labels at all** — every validation value they carry is ''not-applicable'' or ''none-reported''. The list //is// the label. Calzavara et al. {[calzavara2026_syntactic]} put a number on what that costs: comparing syntactic list matching against dynamic taint tracking request by request over the same crawl, they estimate **16%–19% of the list's own matches are false positives**, rising to **27%–30%** among requests only the list flagged, and find **7,021 requests (≈17% of their 40,605)** that only taint tracking saw. Treat a filter-list hit as a noisy label with error bars in the high teens. **Say so in your paper.** | **The one thing to take away.** In this corpus the filter list is both the instrument and the definition, and almost nobody separates the two. Of the **163 papers** that used a list as part of a classification step, **117 (71.8%) report no validation of the list's labels at all** — every validation value they carry is ''not-applicable'' or ''none-reported''. The list //is// the label. Calzavara et al. {[calzavara2026_syntactic]} put a number on what that costs: comparing syntactic list matching against dynamic taint tracking request by request over the same crawl, they estimate **16%–19% of the list's own matches are false positives**, rising to **27%–30%** among requests only the list flagged, and find **7,021 requests (≈17% of their 40,605)** that only taint tracking saw. Treat a filter-list hit as a noisy label with error bars in the high teens. **Say so in your paper.** |
| </WRAP> | </WRAP> |
| |
| ===== Which Lists Exist ===== | ===== Which Lists Exist ===== |
| |
| **If you want the short answer: load EasyList and EasyPrivacy, add the regional lists your population needs, and add AdGuard's CNAME list if — and only if — your crawl recorded DNS resolution.** That is what the field does: 56.6% of the list-using papers below name EasyList and 35.9% name EasyPrivacy. Everything else in this table is a reference for when that default is wrong for your question — a domain-level comparison point, a cookie-notice ruleset, an entity map, or a list that is dead and should not appear in a 2026 paper. | **If you want the short answer: load EasyList and EasyPrivacy, add the regional lists your population needs, and add AdGuard's CNAME list if — and only if — your crawl recorded DNS resolution.** That is what the field does: 57.7% of the list-using papers below name EasyList and 36.6% name EasyPrivacy. Everything else in this table is a reference for when that default is wrong for your question — a domain-level comparison point, a cookie-notice ruleset, an entity map, or a list that is dead and should not appear in a 2026 paper. |
| |
| ==== Who maintains this, and what that implies ==== | ==== Who maintains this, and what that implies ==== |
| | [[https://justdomains.github.io/blocklists/|justdomains]] | Not a list: a domain-only re-encoding of EasyList, EasyPrivacy, AdGuard DNS and others | **Active as a conversion layer** (HTTP 200). Cite the upstream list, not this. Three papers in this corpus label cookies with it | | | [[https://justdomains.github.io/blocklists/|justdomains]] | Not a list: a domain-only re-encoding of EasyList, EasyPrivacy, AdGuard DNS and others | **Active as a conversion layer** (HTTP 200). Cite the upstream list, not this. Three papers in this corpus label cookies with it | |
| | **EasyList Cookie List** | CSS selectors for cookie notices. Lives in the main easylist repo under ''easylist_cookie/'' | **Active** (''Last modified: 2026-08-29 12:50 UTC''). **No ''Version:'' field**, only ''Last modified:'' — so record the date and the hash | | | **EasyList Cookie List** | CSS selectors for cookie notices. Lives in the main easylist repo under ''easylist_cookie/'' | **Active** (''Last modified: 2026-08-29 12:50 UTC''). **No ''Version:'' field**, only ''Last modified:'' — so record the date and the hash | |
| | **hpHosts / Malwarebytes ATS** | Historic hosts-file ad/tracking list | **Dead.** ''hosts-file.net'' still has a DNS record but HTTPS to it timed out on 2026-08-29; Malwarebytes stopped updating around 2020. It appears in older papers. **27 papers in this corpus used a hosts-file list of some kind**, but that family also contains live ones (Pi-hole, NoTrack), so the 27 is not 27 papers using a dead list — read the hosts-file row in //Which lists// below | | | **hpHosts / Malwarebytes ATS** | Historic hosts-file ad/tracking list | **Dead.** ''hosts-file.net'' still has a DNS record but HTTPS to it timed out on 2026-08-29; Malwarebytes stopped updating around 2020. It appears in older papers. **24 papers in this corpus used a hosts-file list of some kind**, but that family also contains live ones (Pi-hole, NoTrack), so the 27 is not 27 papers using a dead list — read the hosts-file row in //Which lists// below | |
| | DNS-blocking lists generally | Domain-level, no path granularity | See [[Design:Website Classification|Website classification]], and the smart-TV row in [[#Coverage Holes, Measured]] | | | DNS-blocking lists generally | Domain-level, no path granularity | See [[Design:Website Classification|Website classification]], and the smart-TV row in [[#Coverage Holes, Measured]] | |
| |
| |
| <WRAP important> | <WRAP important> |
| **Of the 178 papers in this corpus that name a filter list as a tool they used, 53 (29.8%) attach any version or date, and only 4 (2.2%) give something that identifies the actual rules** — a list ''Version:'' string or a commit hash. The rest is "EasyList", or a bare year. Some of the 53 do better than that — ''EasyList :: 2024-08-17'' and ''EasyList :: 2021-07-05'' come close to identifying a daily build — but a date is not the ''Version:'' the list itself publishes. Given that the list publishes its own commit and changes several times an hour, this is the single largest reproducibility gap in this literature. | **Of the 177 papers in this corpus that name a filter list as a tool they used, 53 (29.9%) attach any version or date, and only 4 (2.3%) give something that identifies the actual rules** — a list ''Version:'' string or a commit hash. The rest is "EasyList", or a bare year. Some of the 53 do better than that — ''EasyList :: 2024-08-17'' and ''EasyList :: 2021-07-05'' come close to identifying a daily build — but a date is not the ''Version:'' the list itself publishes. Given that the list publishes its own commit and changes several times an hour, this is the single largest reproducibility gap in this literature. |
| </WRAP> | </WRAP> |
| |
| </WRAP> | </WRAP> |
| |
| The practice is not confined to request classifiers. **96 papers in this corpus name a filter list as the ''groundTruthSource'' for a classification step, across twelve different kinds of object** — requests, domains, cookies, scripts, IP addresses, website categories, privacy policies, consent notices, network traffic, even email messages. The list has become the field's general-purpose definition of "advertising or tracking", far outside what it was written to describe. | The practice is not confined to request classifiers. **92 papers in this corpus name a filter list as the ''groundTruthSource'' for a classification step, across eleven different kinds of object** — requests, domains, scripts, cookies, website categories, fingerprinting scripts, privacy policies, consent notices, network traffic, even email messages. The list has become the field's general-purpose definition of "advertising or tracking", far outside what it was written to describe. |
| |
| The clearest recent examples, each verified against the paper's own text: | The clearest recent examples, each verified against the paper's own text: |
| |
| * **CookieGraph** {[shaoor2023cookiegraph]}, CCS 2023 labels a **cookie** by the provenance of the script that set it: "if a script that sets a cookie is not marked by any of the filter lists, we label these cookies as Non-ATS. Conservatively, if any one of the filter lists marks the cookie's setter as ATS, we label the cookie as Unknown".. It then adds **Cookiepedia** as a second, independent label source, and treats its self-declared purposes conservatively. That combination — filter-list provenance plus a CMP-maintained label database — is the strongest published approach to cookie labelling, and a real improvement on scoring the list against itself. It is one paper, and it rests on Cookiepedia, which nobody in this corpus has audited. | * **CookieGraph** {[shaoor2023cookiegraph]}, CCS 2023 labels a **cookie** by the provenance of the script that set it: "if a script that sets a cookie is not marked by any of the filter lists, we label these cookies as Non-ATS. Conservatively, if any one of the filter lists marks the cookie's setter as ATS, we label the cookie as Unknown". It then adds **Cookiepedia** as a second, independent label source, and treats its self-declared purposes conservatively. That combination — filter-list provenance plus a CMP-maintained label database — is the strongest published approach to cookie labelling, and a real improvement on scoring the list against itself. It is one paper, and it rests on Cookiepedia, which nobody in this corpus has audited. |
| * **NoT.js** {[amjad2024_notjs]}, CCS 2024 labels a **JavaScript function** by whether the network requests it participates in match EasyList/EasyPrivacy, calling the result "a conservative ground truth" in which mixed-behaviour functions (3.9% of it) are labelled non-tracking. | * **NoT.js** {[amjad2024_notjs]}, CCS 2024 labels a **JavaScript function** by whether the network requests it participates in match EasyList/EasyPrivacy, calling the result "a conservative ground truth" in which mixed-behaviour functions (3.9% of it) are labelled non-tracking. |
| * **SugarCoat** {[smith2021_sugarcoat]}, CCS 2021 takes both the tracking labels //and// the breakage signal from list artefacts — rules and exception rules. | * **SugarCoat** {[smith2021_sugarcoat]}, CCS 2021 takes both the tracking labels //and// the breakage signal from list artefacts — rules and exception rules. |
| * **Read Between the Lines** {[ghasemisharif2023_bytecode]}, CCS 2023 classifies tracking JavaScript from bytecode, with EasyList, EasyPrivacy and Fanboy's Enhanced Tracking as the labels. | * **Read Between the Lines** {[ghasemisharif2023_bytecode]}, CCS 2023 classifies tracking JavaScript from bytecode, with EasyList, EasyPrivacy and Fanboy's Enhanced Tracking as the labels. |
| |
| The learned-classifier roster whose stated ground truth is a filter list runs to **sixteen papers**, and **fourteen of them are a who's-who of the reference baselines**: AdGraph, WebGraph, WTAGRAPH, Khaleesi, AdFlush, Duumviri, Beyond the Request, CookieGraph, NoT.js, Net-track, PURL, NoMoAds, the Ad Wars, and Tramèr et al.'s attack on perceptual ad blocking. (The other two are a 2015 website-category classifier and a 2018 host-population classifier whose labels mix hpHosts with PhishTank — in the count because hpHosts is an ad/tracking list, but not tracker detectors.) The roster is printed in full, with each paper's own ground-truth string, on the [[provenance:programming:filter_lists|provenance page]]. | The learned-classifier roster whose stated ground truth is a filter list runs to **fifteen papers**, and **fourteen of them are a who's-who of the reference baselines**: AdGraph, WebGraph, WTAGRAPH, Khaleesi, AdFlush, Duumviri, Beyond the Request, CookieGraph, NoT.js, Net-track, PURL, NoMoAds, the Ad Wars, and Tramèr et al.'s attack on perceptual ad blocking. (The fifteenth is a 2015 website-category classifier.) The roster is printed in full, with each paper's own ground-truth string, on the [[provenance:programming:filter_lists|provenance page]]. |
| |
| ==== What to do instead, and what it costs ==== | ==== What to do instead, and what it costs ==== |
| Nobody has escaped the list entirely, but four routes exist, in rising order of effort: | Nobody has escaped the list entirely, but four routes exist, in rising order of effort: |
| |
| - **Hand-label a sample and report precision against the list.** A few hundred requests is an afternoon. **34 of the 167 list-using classification papers (20.4%) did this**, and 119 (71.3%) report no validation at all; it is the single cheapest improvement available and reviewers notice its absence. | - **Hand-label a sample and report precision against the list.** A few hundred requests is an afternoon. **34 of the 163 list-using classification papers (20.9%) did this**, and 117 (71.8%) report no validation at all; it is the single cheapest improvement available and reviewers notice its absence. |
| - **Add an independent label source.** Cookiepedia for cookies {[shaoor2023cookiegraph]}, a CMP declaration, an entity map. Disagreements between two imperfect sources are informative in a way agreement with one is not. | - **Add an independent label source.** Cookiepedia for cookies {[shaoor2023cookiegraph]}, a CMP declaration, an entity map. Disagreements between two imperfect sources are informative in a way agreement with one is not. |
| - **Use a behavioural or differential detector as the reference**, not as the thing being scored: invisible pixels {[fouad2020_missed]}, breakage-differential testing {[shuang2025_duumviri]}, dynamic taint tracking {[calzavara2026_syntactic]}. Each has been done roughly once. | - **Use a behavioural or differential detector as the reference**, not as the thing being scored: invisible pixels {[fouad2020_missed]}, breakage-differential testing {[shuang2025_duumviri]}, dynamic taint tracking {[calzavara2026_syntactic]}. Each has been done roughly once. |
| ^ Signal ^ Papers ^ | ^ Signal ^ Papers ^ |
| | corpus | 5,859 | | | corpus | 5,859 | |
| | **used or produced a filter list — the population** | **198** (3.4% of the corpus) | | | **used or produced a filter list — the population** | **194** (3.3% of the corpus) | |
| | … named it in ''tools[]'' | 178 | | | … named it in ''tools[]'' | 177 | |
| | … named it in a ''classification[]'' step | 167 | | | … named it in a ''classification[]'' step | 163 | |
| | … both | 147 | | | … both | 146 | |
| | … only in ''tools[]'' — an instrument, never a labelling step | 31 | | | … only in ''tools[]'' — an instrument, never a labelling step | 31 | |
| | named a list but only //mentioned// or //compared against// it — excluded | 18 | | | named a list but only //mentioned// or //compared against// it — excluded | 19 | |
| | | named a list but used it for something **off-topic** — malware, phishing or piracy labelling — excluded on the evidence quote, not the name | 7 tuples across 5 papers | |
| |
| **"Mentioned" is not "used".** The 18 excluded papers are 8.4% of what a raw name search would score as adoption, and the //compared// ones are the dangerous rows: a paper that beats EasyList did not adopt it. | **"Mentioned" is not "used", and the name is not the use.** The 19 //mentioned//-only papers are 8.9% of what a raw name search would score as adoption, and the //compared// ones are the dangerous rows: a paper that beats EasyList did not adopt it. A further **five papers name a real list and use it for something else** — hpHosts as a malware blacklist alongside DNS-BH and Google Safe Browsing, NextDNS's piracy list to find torrent sites — and are excluded on the evidence quote rather than the name. All seven tuples are printed on the [[provenance:programming:filter_lists|provenance page]]. |
| |
| ==== Which lists ==== | ==== Which lists ==== |
| |
| Of the **198** papers that used or produced a filter list. A paper naming several is counted under each, so shares do not sum to 100%. Names are free text and were folded into families; //spellings// is how many distinct strings the corpus uses, and //unfolded// is what the single most common exact string alone would have found. | Of the **194** papers that used or produced a filter list. A paper naming several is counted under each, so shares do not sum to 100%. Names are free text and were folded into families; //spellings// is how many distinct strings the corpus uses, and //unfolded// is what the single most common exact string alone would have found. |
| |
| ^ List family ^ Papers ^ Share of 198 ^ Spellings ^ Unfolded ^ | ^ List family ^ Papers ^ Share of 194 ^ Spellings ^ Unfolded ^ |
| | EasyList | 112 | 56.6% | 28 | 92 | | | EasyList | 112 | 57.7% | 28 | 92 | |
| | EasyPrivacy | 71 | 35.9% | 22 | 58 | | | EasyPrivacy | 71 | 36.6% | 22 | 58 | |
| | Disconnect | 48 | 24.2% | 23 | 26 | | | Disconnect | 48 | 24.7% | 23 | 26 | |
| | Ghostery / WhoTracks.me | 34 | 17.2% | 9 | 20 | | | Ghostery / WhoTracks.me | 34 | 17.5% | 9 | 20 | |
| | hosts-file lists (hpHosts, AdAway, MoaAB, Pi-hole, NoTrack, …) | 27 | 13.6% | 33 | 3 | | | Adblock Plus (the lists shipped with it) | 26 | 13.4% | 8 | 20 | |
| | Adblock Plus (the lists shipped with it) | 26 | 13.1% | 8 | 20 | | | hosts-file lists (hpHosts, AdAway, MoaAB, Pi-hole, NoTrack, …) | 24 | 12.4% | 32 | 3 | |
| | uBlock Origin lists | 17 | 8.6% | 11 | 13 | | | uBlock Origin lists | 17 | 8.8% | 11 | 13 | |
| | DuckDuckGo Tracker Radar | 15 | 7.6% | 9 | 8 | | | DuckDuckGo Tracker Radar | 15 | 7.7% | 9 | 8 | |
| | **unnamed or merely counted** ("nine crowd-sourced filter lists") | 10 | 5.1% | 10 | 1 | | | **unnamed or merely counted** ("nine crowd-sourced filter lists") | 9 | 4.6% | 9 | 1 | |
| | AdGuard | 8 | 4.0% | 13 | 4 | | | AdGuard | 8 | 4.1% | 13 | 4 | |
| | EasyList annoyance / anti-adblock variants | 7 | 3.5% | 8 | 2 | | | EasyList annoyance / anti-adblock variants | 7 | 3.6% | 8 | 2 | |
| | anti-adblock scripts and services | 3 | 1.5% | 4 | 1 | | | anti-adblock scripts and services | 3 | 1.5% | 4 | 1 | |
| | cryptomining lists (NoCoin, CoinBlockerLists, MinerBlock) | 3 | 1.5% | 4 | 3 | | | cryptomining lists (NoCoin, CoinBlockerLists, MinerBlock) | 3 | 1.5% | 4 | 3 | |
| | Acceptable Ads exception list | 1 | 0.5% | 1 | 1 | | | Acceptable Ads exception list | 1 | 0.5% | 1 | 1 | |
| |
| **Folding is not cosmetic.** Counting exact strings would find 92 EasyList papers instead of 112 (−17.9%), 26 Disconnect papers instead of 48 (**−45.8%**, because Disconnect appears as //Disconnect list//, //Disconnect.me//, //Disconnect blacklist//, //Disconnect Entity List// and twenty more), and 3 hosts-file papers instead of 27. Any adoption table built on exact strings is wrong by tens of percent. | **Folding is not cosmetic.** Counting exact strings would find 92 EasyList papers instead of 112 (−17.9%), 26 Disconnect papers instead of 48 (**−45.8%**, because Disconnect appears as //Disconnect list//, //Disconnect.me//, //Disconnect blacklist//, //Disconnect Entity List// and twenty more), and 3 hosts-file papers instead of 24. Any adoption table built on exact strings is wrong by tens of percent. |
| |
| ==== What the list is used to label ==== | ==== What the list is used to label ==== |
| |
| Of the **167** papers with at least one list-backed classification step. This is the finding that motivated this page: **the filter list is not a request classifier, it is the field's general-purpose "is this advertising or tracking" oracle**, and it is applied to objects it was never written to describe. | Of the **163** papers with at least one list-backed classification step. This is the finding that motivated this page: **the filter list is not a request classifier, it is the field's general-purpose "is this advertising or tracking" oracle**, and it is applied to objects it was never written to describe. |
| |
| ^ ''classification.target'' ^ Papers ^ Share of 167 ^ | ^ ''classification.target'' ^ Papers ^ Share of 163 ^ |
| | ''web-request'' | 101 | 60.5% | | | ''web-request'' | 101 | 62.0% | |
| | ''domain'' | 53 | 31.7% | | | ''domain'' | 51 | 31.3% | |
| | ''cookie'' | 8 | 4.8% | | | ''cookie'' | 8 | 4.9% | |
| | ''website-category'' | 7 | 4.2% | | | ''website-category'' | 5 | 3.1% | |
| | ''javascript'' | 5 | 3.0% | | | ''javascript'' | 5 | 3.1% | |
| | ''other'' | 3 | 1.8% | | | ''other'' | 3 | 1.8% | |
| | ''fingerprinting-script'' | 3 | 1.8% | | | ''fingerprinting-script'' | 3 | 1.8% | |
| Grouped by what that does to the instrument: | Grouped by what that does to the instrument: |
| |
| ^ Granularity ^ Papers ^ Share of 167 ^ | ^ Granularity ^ Papers ^ Share of 163 ^ |
| | **The list's own unit** — a URL pattern evaluated with request context | 101 | 60.5% | | | **The list's own unit** — a URL pattern evaluated with request context | 101 | 62.0% | |
| | **Coarser than the list** — a hostname or eTLD+1, so path, resource-type and ''$third-party'' options are discarded | 53 | 31.7% | | | **Coarser than the list** — a hostname or eTLD+1, so path, resource-type and ''$third-party'' options are discarded | 51 | 31.3% | |
| | **Objects a filter list was not written to label** — the list supplies provenance, not a rule match | 30 | 18.0% | | | **Objects a filter list was not written to label** — the list supplies provenance, not a rule match | 28 | 17.2% | |
| |
| **These are not a partition and the shares exceed 100%**: 17 of the 167 papers apply a list at more than one granularity — typically requests //and// domains in the same study. | **These are not a partition and the shares exceed 100%**: 17 of the 163 papers apply a list at more than one granularity — typically requests //and// domains in the same study. |
| |
| **Nearly a third of these 167 papers throw away the list's granularity before using it.** That is often deliberate and sometimes unavoidable — a DNS blocklist or a passive traffic capture gives you nothing finer than a hostname — but it changes what the label means, and it is rarely stated. The per-paper roster behind every non-request row is on the [[provenance:programming:filter_lists|provenance page]] so you can check the claim. | **Nearly a third of these 163 papers throw away the list's granularity before using it.** That is often deliberate and sometimes unavoidable — a DNS blocklist or a passive traffic capture gives you nothing finer than a hostname — but it changes what the label means, and it is rarely stated. The per-paper roster behind every non-request row is on the [[provenance:programming:filter_lists|provenance page]] so you can check the claim. |
| |
| ==== Ground truth ==== | ==== Ground truth ==== |
| |
| ^ Question ^ Papers ^ | ^ Question ^ Papers ^ |
| | papers whose ''classification.groundTruthSource'' names a filter list (whole corpus) | 96 | | | papers whose ''classification.groundTruthSource'' names a filter list (whole corpus) | 92 | |
| | … of the 198 list-using papers | 88 (44.4%) | | | … of the 194 list-using papers | 86 (44.3%) | |
| | distinct kinds of object those tuples label | 12 | | | distinct kinds of object those tuples label | 11 | |
| | … of which the list is named **generically** ("eight crowdsourced filter lists") rather than by name | 13 | | | … of which the list is named **generically** ("eight crowdsourced filter lists") rather than by name | 13 | |
| | papers with a **learned** classifier (''supervised-ml'', ''unsupervised-ml'', ''llm'') whose stated ground truth is a filter list | 16 | | | papers with a **learned** classifier (''supervised-ml'', ''unsupervised-ml'', ''llm'') whose stated ground truth is a filter list | 15 | |
| | … as a share of the 62 list-using papers that ran any learned classifier | 15 (24.2%)((The 16 and the 15 differ by one paper, which uses a filter list as ground truth but is not itself in the 198-paper list-using population — it names no list as a tool or classification resource, only as its label source.)) | | | … as a share of the 60 list-using papers that ran any learned classifier | 15 (25.0%) | |
| |
| Validation of the list-backed classification steps, over the **167** papers. Sentinels are printed as themselves and never counted as an answer: | Validation of the list-backed classification steps, over the **163** papers. Sentinels are printed as themselves and never counted as an answer: |
| |
| ^ ''classification.validation'' ^ Papers ^ Share of 167 ^ | ^ ''classification.validation'' ^ Papers ^ Share of 163 ^ |
| | ''not-applicable'' //(sentinel)// | 69 | 41.3% | | | ''not-applicable'' //(sentinel)// | 68 | 41.7% | |
| | ''none-reported'' //(sentinel)// | 61 | 36.5% | | | ''none-reported'' //(sentinel)// | 60 | 36.8% | |
| | manual validation | 34 | 20.4% | | | manual validation | 34 | 20.9% | |
| | comparison to another method | 14 | 8.4% | | | comparison to another method | 12 | 7.4% | |
| |
| **The rows overlap** — 11 papers carry both a sentinel and a real value, so the two sentinel rows must not be added. The paper-level figure is: **119 of 167 (71.3%) report nothing but sentinels**, and 48 (28.7%) report at least one real validation. | **The rows overlap** — 11 papers carry both a sentinel and a real value, so the two sentinel rows must not be added. The paper-level figure is: **117 of 163 (71.8%) report nothing but sentinels**, and 46 (28.2%) report at least one real validation. |
| |
| The ''not-applicable'' row is mostly papers that applied a list as-is and reasonably consider the list itself the definition. That is exactly the assumption Calzavara et al. measured at 16%–19% false positives — on one crawl, with one independent detector, so treat it as the best available estimate rather than a constant. | The ''not-applicable'' row is mostly papers that applied a list as-is and reasonably consider the list itself the definition. That is exactly the assumption Calzavara et al. measured at 16%–19% false positives — on one crawl, with one independent detector, so treat it as the best available estimate rather than a constant. |
| |
| ^ Question ^ Papers ^ Share ^ | ^ Question ^ Papers ^ Share ^ |
| | named a filter list in ''tools[]'' | 178 | — | | | named a filter list in ''tools[]'' | 177 | — | |
| | attached **any** version or date | 53 | 29.8% | | | attached **any** version or date | 53 | 29.9% | |
| | attached something that identifies **the rules** — a list ''Version:'' or a commit | 4 | **2.2%** | | | attached something that identifies **the rules** — a list ''Version:'' or a commit | 4 | **2.3%** | |
| | silent | 125 | 70.2% | | | silent | 124 | 70.1% | |
| |
| Read the 29.8% as an order of magnitude in both directions: the extraction fills the version field when the paper puts a version next to the name, so a paper that dates its lists in a crawl paragraph may not be credited — and several of the 53 give an //extension// version (''Ghostery 5.4.1'', ''Adblock Plus 3.7'') rather than a list version, which does not identify the rules at all. The **2.2%** is the figure that matters and it is not an artefact: a list ''Version:'' string is unmistakable, and only four papers have one. | Read the 29.9% as an order of magnitude in both directions: the extraction fills the version field when the paper puts a version next to the name, so a paper that dates its lists in a crawl paragraph may not be credited — and several of the 53 give an //extension// version (''Ghostery 5.4.1'', ''Adblock Plus 3.7'') rather than a list version, which does not identify the rules at all. The **2.3%** is the figure that matters and it is not an artefact: a list ''Version:'' string is unmistakable, and only four papers have one. |
| |
| ==== Where and when ==== | ==== Where and when ==== |
| ^ Venue ^ Corpus papers ^ Population ^ Share of venue ^ | ^ Venue ^ Corpus papers ^ Population ^ Share of venue ^ |
| | PETS | 510 | 55 | 10.8% | | | PETS | 510 | 55 | 10.8% | |
| | IMC | 638 | 36 | 5.6% | | | IMC | 638 | 35 | 5.5% | |
| | TheWebConf | 843 | 31 | 3.7% | | | TheWebConf | 843 | 30 | 3.6% | |
| | IEEE S&P | 767 | 19 | 2.5% | | | IEEE S&P | 767 | 19 | 2.5% | |
| | CCS | 990 | 23 | 2.3% | | | CCS | 990 | 23 | 2.3% | |
| | USENIX Security | 1,410 | 24 | 1.7% | | | USENIX Security | 1,410 | 22 | 1.6% | |
| | NDSS | 701 | 10 | 1.4% | | | NDSS | 701 | 10 | 1.4% | |
| |
| ^ Period ^ Corpus papers ^ Population ^ Per 1,000 ^ | ^ Period ^ Corpus papers ^ Population ^ Per 1,000 ^ |
| | 2010–2013 | 511 | 6 | 11.7 | | | 2010–2013 | 511 | 6 | 11.7 | |
| | 2014–2017 | 769 | 25 | 32.5 | | | 2014–2017 | 769 | 24 | 31.2 | |
| | 2018–2021 | 1,439 | 58 | 40.3 | | | 2018–2021 | 1,439 | 57 | 39.6 | |
| | 2022–2024 | 1,955 | 70 | 35.8 | | | 2022–2024 | 1,955 | 69 | 35.3 | |
| | 2025–2026 //(provisional)// | 1,185 | 39 | 32.9 | | | 2025–2026 //(provisional)// | 1,185 | 38 | 32.1 | |
| |
| **Filter-list use peaked in 2018–2021 and has been flat since, at roughly 3–4% of these seven venues.** Nothing in the corpus supports "the field has moved on from filter lists": machine learning was added alongside, not instead. Among the population, papers carrying **any** LLM-based classification step appear only in 2024 (2), 2025 (4) and 2026 (1) — real, new, and far too thin to call current practice on the strength of the corpus's most provisional years. | **Filter-list use peaked in 2018–2021 and has been flat since, at roughly 3–4% of these seven venues.** Nothing in the corpus supports "the field has moved on from filter lists": machine learning was added alongside, not instead. Among the population, papers carrying **any** LLM-based classification step appear only in 2024 (2), 2025 (4) and 2026 (1) — real, new, and far too thin to call current practice on the strength of the corpus's most provisional years. |
| * **A paper counts once**, never once per tuple, and shares do not sum to 100% because the fields are multi-valued. | * **A paper counts once**, never once per tuple, and shares do not sum to 100% because the fields are multi-valued. |
| * **Sentinels are counted as what they are.** ''not-applicable'' and ''none-reported'' are never folded into a stated value; where they are the largest rows, that is the finding. | * **Sentinels are counted as what they are.** ''not-applicable'' and ''none-reported'' are never folded into a stated value; where they are the largest rows, that is the finding. |
| | * **A list's name is not its use.** For the ambiguous families — hosts-file lists and unnamed aggregations — the tuple's own taxonomy and verbatim evidence quote decide whether it was an advertising/tracking list or a security blacklist. Seven tuples across five papers were excluded on that test and are printed in full on the provenance page. |
| * **Free-text names were folded, and the residue is printed** rather than dropped: 34 distinct strings matched the candidate regex but no family, and almost all are generic off-topic phrases (''12 IP reputation blacklists'', ''combined public blacklists''). Off-topic blocklist families were excluded by name and counted: malware/phishing 10 papers, spam 8, IP reputation 7, content-category 6, censorship 3, certificate revocation 2. | * **Free-text names were folded, and the residue is printed** rather than dropped: 34 distinct strings matched the candidate regex but no family, and almost all are generic off-topic phrases (''12 IP reputation blacklists'', ''combined public blacklists''). Off-topic blocklist families were excluded by name and counted: malware/phishing 10 papers, spam 8, IP reputation 7, content-category 6, censorship 3, certificate revocation 2. |
| * **Free-text fields are ~20% stable run-to-run.** List names are given as folded families and rankings; the enum fields (''classification.target'', ''classification.validation'') are considerably more stable and are given as percentages. | * **Free-text fields are ~20% stable run-to-run.** List names are given as folded families and rankings; the enum fields (''classification.target'', ''classification.validation'') are considerably more stable and are given as percentages. |
| * **Venue coverage.** Seven venues only. EuroS&P, ACSAC, RAID, AsiaCCS, WPES, CHI and SOUPS are absent, and several works this page relies on were published outside them — Snyder et al. at SIGMETRICS in particular. Any count here is a lower bound. | * **Venue coverage.** Seven venues only. EuroS&P, ACSAC, RAID, AsiaCCS, WPES, CHI and SOUPS are absent, and several works this page relies on were published outside them — Snyder et al. at SIGMETRICS in particular. Any count here is a lower bound. |
| * **Overlap with [[Privacy:Requests]].** That page computes a partly overlapping population (its "S1" signal) and reports 197 papers where this page reports 198, with per-list counts differing by one or two. The two scripts define membership slightly differently — this page counts a list named anywhere in ''tools[]'' or ''classification[]'', that page additionally intersects with its request-classification task fold. Neither is wrong; the difference is a live example of why a population definition belongs in the provenance, not in a sentence. | * **Overlap with [[Privacy:Requests]].** That page computes a partly overlapping population (its "S1" signal) and reports **197** papers where this page reports **194**. The folds are shared, so the difference is the membership rule: that page intersects with its own request-classification task fold, and this page additionally excludes a paper whose list name is right but whose //use// is a malware or piracy blacklist. Neither is wrong; the difference is a live example of why a population definition belongs in the provenance rather than in a sentence, and it is set out there in full. |
| |
| ===== What to Report ===== | ===== What to Report ===== |
| * **The 16%–19% false-positive estimate is one study.** {[calzavara2026_syntactic]} used one taint-tracking browser on 7,614 sites. A very large amount of this page leans on it. It should be replicated with a different independent detector before it becomes folklore. | * **The 16%–19% false-positive estimate is one study.** {[calzavara2026_syntactic]} used one taint-tracking browser on 7,614 sites. A very large amount of this page leans on it. It should be replicated with a different independent detector before it becomes folklore. |
| * **Regional lists have two papers in seven venues across seventeen years.** {[sjosten2020_filter]} and {[bottger2025_regional]}. Nobody has measured how much a per-country tracking comparison changes when the matching regional list is loaded — which means a large body of comparative geographic tracking work rests on an instrument of unmeasured, varying sensitivity. | * **Regional lists have two papers in seven venues across seventeen years.** {[sjosten2020_filter]} and {[bottger2025_regional]}. Nobody has measured how much a per-country tracking comparison changes when the matching regional list is loaded — which means a large body of comparative geographic tracking work rests on an instrument of unmeasured, varying sensitivity. |
| * **No learned tracker detector in this corpus escapes the list.** Of the 16 papers whose stated ground truth is a filter list, every one that is a tracker detector — AdGraph, WebGraph, WTAGRAPH, Khaleesi, AdFlush, Duumviri, CookieGraph, NoT.js, Net-track, Beyond the Request — takes its labels from one. **This is a statement about the 16, not about all learned classifiers in the corpus**: 47 of the 62 list-using papers that ran a learned classifier state some other ground truth, and nobody has audited what those 47 are classifying or how. The two nearest misses to an independent label source — differential breakage testing {[shuang2025_duumviri]} and taint tracking {[calzavara2026_syntactic]} — have each been done once. | * **No learned tracker detector in this corpus escapes the list.** Of the 15 papers whose stated ground truth is a filter list, every one that is a tracker detector — AdGraph, WebGraph, WTAGRAPH, Khaleesi, AdFlush, Duumviri, CookieGraph, NoT.js, Net-track, Beyond the Request — takes its labels from one. **This is a statement about the 15, not about all learned classifiers in the corpus**: 45 of the 60 list-using papers that ran a learned classifier state some other ground truth, and nobody has audited what those 45 are classifying or how. The two nearest misses to an independent label source — differential breakage testing {[shuang2025_duumviri]} and taint tracking {[calzavara2026_syntactic]} — have each been done once. |
| * **Python still has no maintained filter-list engine.** Someone should revive ''python-adblock'' against ''adblock'' 0.13.x, or the field should state loudly that Python pipelines must shell out. | * **Python still has no maintained filter-list engine.** Someone should revive ''python-adblock'' against ''adblock'' 0.13.x, or the field should state loudly that Python pipelines must shell out. |
| * **Nobody has published which rules the shipped MV3 blockers drop** to fit the 30,000-rule static cap, or what that omits from a measurement made with one. | * **Nobody has published which rules the shipped MV3 blockers drop** to fit the 30,000-rule static cap, or what that omits from a measurement made with one. |
