provenance:privacy:cookies
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
| provenance:privacy:cookies [2026/09/10 16:25] – Create: query log, denominators, folds and residues, quote checks, external verification with controls, judgement calls, and the two report scripts with their real output, behind privacy:cookies. Authored by Claude karel.kubicek.claude | provenance:privacy:cookies [2026/09/10 16:44] (current) – Replace the literal citekey span count with the invariant it was checking; a snapshot count is stale after the next edit. Authored by Claude karel.kubicek.claude | ||
|---|---|---|---|
| Line 81: | Line 81: | ||
| ===== 4. The fold, and why it throws instead of leaving a residue ===== | ===== 4. The fold, and why it throws instead of leaving a residue ===== | ||
| - | '' | + | '' |
| The fold is a **hand-keyed map** in '' | The fold is a **hand-keyed map** in '' | ||
| Line 169: | Line 169: | ||
| ^ Claim on the page ^ Primary source, and how it was checked ^ | ^ Claim on the page ^ Primary source, and how it was checked ^ | ||
| | Cookiepedia holds **42, | | Cookiepedia holds **42, | ||
| - | | Cookiepedia rejects plain '' | + | | Cookiepedia rejects plain '' |
| | The CookieGraph Cookiepedia name table: **45,785** names, **39.3%** categorised, | | The CookieGraph Cookiepedia name table: **45,785** names, **39.3%** categorised, | ||
| | The CookieGraph '' | | The CookieGraph '' | ||
| Line 329: | Line 329: | ||
| ^ Family ^ Status ^ What it rests on ^ | ^ Family ^ Status ^ What it rests on ^ | ||
| | Purpose database lookup | current, the default | 8 of the 10 papers in 2023–2024 and 5 of 12 in 2025–2026 | | | Purpose database lookup | current, the default | 8 of the 10 papers in 2023–2024 and 5 of 12 in 2025–2026 | | ||
| - | | Own hand-written rule | current, largest family, mostly unreusable | 20 of 53; 31 of 61 distinct '' | + | | Own hand-written rule | current, largest family, mostly unreusable | 20 of 53; 27 of 61 distinct '' |
| | Identifier heuristic | current, **but the predicate changed** | Family present 2014→2026 including 3 papers in 2025–2026; | | Identifier heuristic | current, **but the predicate changed** | Family present 2014→2026 including 3 papers in 2025–2026; | ||
| | Filter list provenance | current and growing | 8 papers, **3 of them 2025**. Small numbers, said so on the page | | | Filter list provenance | current and growing | 8 papers, **3 of them 2025**. Small numbers, said so on the page | | ||
| Line 366: | Line 366: | ||
| Checked before saving: the key does not exist in '' | Checked before saving: the key does not exist in '' | ||
| - | Every other citekey on the page already existed. | + | Every other citekey on the page already existed. |
| + | |||
| + | **The new key did not render until the bibliography' | ||
| ===== 11. The report script and its output ===== | ===== 11. The report script and its output ===== | ||
| Line 488: | Line 490: | ||
| the script throws instead of silently dropping one). The listing in section 2.2 is what to | the script throws instead of silently dropping one). The listing in section 2.2 is what to | ||
| check: a zero residue means the map is complete, not that every judgement in it is right. | check: a zero residue means the map is complete, not that every judgement in it is right. | ||
| + | |||
| + | Of the 61 distinct strings, 27 BEGIN with the word " | ||
| + | That is the reusability finding: these are one-off predicates written for one paper. | ||
| ============================================================================== | ============================================================================== | ||
| Line 517: | Line 522: | ||
| Security role: authentication or session cookie, attribute correctness | Security role: authentication or session cookie, attribute correctness | ||
| Something else (named in the residue listing) | Something else (named in the residue listing) | ||
| + | |||
| + | ============================================================================== | ||
| + | 2.1c NAMED LABEL SOURCES, PAPER-COUNTED — the figures the prose quotes | ||
| + | ============================================================================== | ||
| + | resourceName (exact string) | ||
| + | --------------------------------------- | ||
| + | Cookiepedia | ||
| + | CookieBlock | ||
| + | Cookie-Script | ||
| + | justdomains blocklist | ||
| + | Cookie Database | ||
| + | Cookie Script | ||
| + | Cookiedatabase.org | ||
| + | CookieGraph | ||
| + | Disconnect.me | ||
| + | EasyList | ||
| + | EasyList/ | ||
| + | Ghostery | ||
| + | Ghostery and Disconnect intersection | ||
| + | GPT-3.5 | ||
| + | GPT-3.5-turbo (custom fine-tuned model) | ||
| + | justdomains | ||
| + | Public Suffix List | ||
| + | WebGraph | ||
| + | zxcvbn | ||
| + | |||
| + | Merged across spellings (papers, not tuples): | ||
| + | Cookiepedia | ||
| + | Cookie-Script / " | ||
| + | cookiedatabase.org | ||
| + | CookieBlock | ||
| + | any purpose database | ||
| + | justdomains (both spellings) | ||
| ============================================================================== | ============================================================================== | ||
| Line 782: | Line 820: | ||
| 7. LLM CLASSIFICATION OF COOKIES — the currency check | 7. LLM CLASSIFICATION OF COOKIES — the currency check | ||
| ============================================================================== | ============================================================================== | ||
| - | Classification target | + | Classification target |
| - | --------------------- | + | --------------------- |
| - | other 2594 | + | privacy-policy |
| - | user-generated-text | + | dark-pattern |
| - | vulnerability | + | user-generated-text |
| - | privacy-policy | + | consent-notice |
| - | mobile-app | + | other 2592 |
| - | website-category | + | mobile-app |
| - | domain | + | email-message |
| - | network-traffic | + | cookie |
| - | consent-notice | + | vulnerability |
| - | ip-address | + | website-category |
| - | web-request | + | domain |
| - | email-message | + | network-traffic |
| - | cookie | + | web-request |
| - | dark-pattern | + | ip-address |
| - | malware | + | malware |
| - | sdk-or-library | + | sdk-or-library |
| - | javascript | + | javascript |
| - | fingerprinting-script | + | fingerprinting-script |
| - | website-popularity | + | website-popularity |
| - | The cookie row is the one privacy: | + | The PUBLISHED column is columns 2-4 and matches report_llm_currency.mjs row for row. |
| - | LLM + cookie: WWW/ | + | Rows where the two definitions disagree: |
| + | user-generated-text: | ||
| + | other: used/used 115/2592 = 4.4% | ||
| + | |||
| + | | ||
| + | LLM + cookie: WWW/ | ||
| ============================================================================== | ============================================================================== | ||
| Line 1657: | Line 1700: | ||
| } | } | ||
| const distinct = new Set(allTuples.map((t) => t.c.resourceName)); | const distinct = new Set(allTuples.map((t) => t.c.resourceName)); | ||
| + | // Computed, not counted by hand: the page states this figure and a hand count | ||
| + | // of it was wrong by four on the first attempt. | ||
| + | const customPrefixed = [...distinct].filter((n) => / | ||
| + | const customAnywhere = [...distinct].filter((n) => / | ||
| console.log( | console.log( | ||
| `\n${allTuples.length} cookie classification tuples across ${CLASSIFY.length} papers, ` + | `\n${allTuples.length} cookie classification tuples across ${CLASSIFY.length} papers, ` + | ||
| `${distinct.size} distinct resourceName strings, all mapped (residue 0 by construction —\n` + | `${distinct.size} distinct resourceName strings, all mapped (residue 0 by construction —\n` + | ||
| `the script throws instead of silently dropping one). The listing in section 2.2 is what to\n` + | `the script throws instead of silently dropping one). The listing in section 2.2 is what to\n` + | ||
| - | `check: a zero residue means the map is complete, not that every judgement in it is right.` | + | `check: a zero residue means the map is complete, not that every judgement in it is right.\n\n` + |
| + | `Of the ${distinct.size} distinct strings, ${customPrefixed.length} BEGIN with the word " | ||
| + | `${customAnywhere.length} contain it anywhere.\nThat is the reusability finding: these are one-off ` + | ||
| + | `predicates written for one paper.` | ||
| ); | ); | ||
| Line 1692: | Line 1742: | ||
| ]) | ]) | ||
| ); | ); | ||
| + | |||
| + | h(' | ||
| + | // Every named (non-" | ||
| + | // sentence on the page has to hand-count one. Two hand counts were wrong on the | ||
| + | // first attempt: Cookiepedia (12, really 14) and Cookie-Script (4, really 3 — | ||
| + | // one paper contributes two tuples). | ||
| + | const NAMED = / | ||
| + | const namedRows = new Map(); | ||
| + | for (const t of allTuples) { | ||
| + | if (!NAMED.test(t.c.resourceName)) continue; | ||
| + | if (!namedRows.has(t.c.resourceName)) namedRows.set(t.c.resourceName, | ||
| + | namedRows.get(t.c.resourceName).add(key(t.p)); | ||
| + | } | ||
| + | T( | ||
| + | [' | ||
| + | [...namedRows.entries()] | ||
| + | .map(([n, set]) => { | ||
| + | const ts = allTuples.filter((t) => t.c.resourceName === n); | ||
| + | return [n, set.size, ts.length, ts[0].family, | ||
| + | }) | ||
| + | .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0])) | ||
| + | ); | ||
| + | // The two spellings of Cookie-Script and of justdomains are one product each. | ||
| + | const merge = (re) => new Set(allTuples.filter((t) => re.test(t.c.resourceName)).map((t) => key(t.p))).size; | ||
| + | console.log(` | ||
| + | Merged across spellings (papers, not tuples): | ||
| + | Cookiepedia | ||
| + | Cookie-Script / " | ||
| + | cookiedatabase.org | ||
| + | CookieBlock | ||
| + | any purpose database | ||
| + | justdomains (both spellings) | ||
| h('2.2 EVERY TUPLE, GROUPED BY FAMILY — the listing to argue with' | h('2.2 EVERY TUPLE, GROUPED BY FAMILY — the listing to argue with' | ||
| Line 1852: | Line 1934: | ||
| // --------------------------------------------------------------------------- | // --------------------------------------------------------------------------- | ||
| // Cross-checks scripts/ | // Cross-checks scripts/ | ||
| - | // for every target. If these two disagree, one of them is wrong. | + | // for every target |
| + | // and design: | ||
| + | // two published pages contradict each other. | ||
| + | // | ||
| + | // That script' | ||
| + | // produced LLM labels at that target, denominator = papers that used or | ||
| + | // produced ANY classifier at that target. `compared` and `mentioned` tuples are | ||
| + | // a baseline the authors argued against; counting them as adoption overstates | ||
| + | // it, and counting them in the denominator but not the numerator understates | ||
| + | // the share. This script reproduces that definition EXACTLY so the pages cannot | ||
| + | // drift, and prints the naive any/any variant beside it so the difference is | ||
| + | // visible rather than hidden. The rows where they disagree are listed by the | ||
| + | // script itself under the table rather than being asserted in this comment, | ||
| + | // because a comment does not get re-derived when the corpus moves. The cookie | ||
| + | // row is 1/53 under both definitions. | ||
| + | const USED_OM = new Set([' | ||
| + | const setAdd = (m, k, v) => { | ||
| + | if (!m.has(k)) m.set(k, new Set()); | ||
| + | m.get(k).add(v); | ||
| + | }; | ||
| + | const llmUsedByTarget = new Map(); | ||
| const llmAnyByTarget = new Map(); | const llmAnyByTarget = new Map(); | ||
| + | const usedByTarget = new Map(); | ||
| const anyByTarget = new Map(); | const anyByTarget = new Map(); | ||
| for (const p of CLASSIFIED) { | for (const p of CLASSIFIED) { | ||
| for (const c of p.classification) { | for (const c of p.classification) { | ||
| - | | + | if (isSentinel(c.target)) continue; |
| - | | + | |
| - | anyByTarget.get(t).add(key(p)); | + | if (c.method === ' |
| - | if (c.method === ' | + | if (USED_OM.has(c.usedOrMentioned)) { |
| - | if (!llmAnyByTarget.has(t)) llmAnyByTarget.set(t, new Set()); | + | setAdd(usedByTarget, c.target, key(p)); |
| - | | + | |
| } | } | ||
| } | } | ||
| } | } | ||
| + | const sz = (m, t) => (m.get(t) ?? new Set()).size; | ||
| T( | T( | ||
| - | [' | + | [ |
| - | [...anyByTarget.entries()] | + | |
| - | .map(([t, s]) => [t, s.size, (llmAnyByTarget.get(t) ?? new Set()).size, pct((llmAnyByTarget.get(t) ?? new Set()).size, s.size)]) | + | |
| - | .sort((a, b) => b[2] - a[2] || b[1] - a[1]) | + | |
| + | | ||
| + | ' | ||
| + | ' | ||
| + | ' | ||
| + | | ||
| + | [...anyByTarget.keys()] | ||
| + | .map((t) => [ | ||
| + | | ||
| + | sz(usedByTarget, t), | ||
| + | sz(llmUsedByTarget, | ||
| + | pct(sz(llmUsedByTarget, | ||
| + | sz(llmAnyByTarget, | ||
| + | sz(anyByTarget, | ||
| + | | ||
| + | | ||
| + | .sort((a, b) => parseFloat(b[3]) - parseFloat(a[3]) || b[1] - a[1]) | ||
| ); | ); | ||
| - | console.log(' | + | console.log(` |
| - | for (const k of llmAnyByTarget.get(' | + | The PUBLISHED column is columns 2-4 and matches report_llm_currency.mjs row for row. |
| + | Rows where the two definitions disagree: | ||
| + | for (const t of anyByTarget.keys()) { | ||
| + | const a = pct(sz(llmUsedByTarget, | ||
| + | const b = pct(sz(llmAnyByTarget, | ||
| + | if (a !== b) console.log(` | ||
| + | } | ||
| + | console.log(' | ||
| + | for (const k of llmAnyByTarget.get(' | ||
| // --------------------------------------------------------------------------- | // --------------------------------------------------------------------------- | ||
| Line 2056: | Line 2184: | ||
| ===== 12. Review log ===== | ===== 12. Review log ===== | ||
| - | _(review pass not yet run)_ | + | Four passes were planned: three focused |
| + | three focused passes returned and every finding is logged below with whether it | ||
| + | was accepted or rejected. The generic pass was not run** — see the last | ||
| + | subsection, which is the largest remaining gap in this page' | ||
| + | |||
| + | Between the self-audit and the three passes, **21 published figures, citations | ||
| + | or claims were corrected**. Most are the same underlying mistake in two | ||
| + | flavours: a number that the report script did not produce, hand-carried into | ||
| + | the prose; and a number read out of a paper' | ||
| + | header. Every figure of the first kind has since been moved into the script. | ||
| + | |||
| + | ==== Self-audit, before any reviewer returned ==== | ||
| + | |||
| + | Found by re-deriving figures the prose had hand-counted. All were live on the | ||
| + | page for roughly one hour and are now fixed. Every one is a case of the same | ||
| + | mistake — **a number in the prose that the report script did not produce** — so | ||
| + | the script was extended to compute each of them (§2.1c and §2 of its output). | ||
| + | |||
| + | ^ Figure ^ Published ^ Correct ^ How it was caught ^ | ||
| + | | Distinct '' | ||
| + | | Papers naming Cookiepedia | 12 | **14** | Paper-counted with '' | ||
| + | | Papers naming Cookie-Script | 4 | **3** | Same; one paper contributes two tuples | | ||
| + | | Papers reporting more than one consent state | "at most 9 of 49" | **7 of 49** state accept-and-reject; | ||
| + | | Cookie papers vs field on crawl reporting | "two to three times" | 2.0x on consent action, 3.5x on statefulness | Divided the two columns | | ||
| + | | PoPETs vs IEEE S&P publication rate | "eight times" | more than ten times, on n=2 | Divided the two shares | | ||
| + | | LLM share for '' | ||
| + | | " | ||
| + | |||
| + | **The LLM one is the interesting failure.** The page's per-target table is | ||
| + | supposed to be the same table as [[Design: | ||
| + | " | ||
| + | independently and got 32/419 = 7.6% for '' | ||
| + | published cell says 7.2%. Neither was stale: '' | ||
| + | computes the share as **used/ | ||
| + | to '' | ||
| + | any/any. The two definitions differ for exactly two targets | ||
| + | ('' | ||
| + | agree everywhere else, including the cookie row, which is why it nearly went | ||
| + | unnoticed. '' | ||
| + | exactly, prints the any/any variant beside it, and prints the rows where the two | ||
| + | disagree. **Sharing a script is a claim about provenance, not about definition.** | ||
| + | |||
| + | **Also caught: the new citekey silently did not render.** See §10. | ||
| + | |||
| + | ==== Pass 1 — figures against the script (Sonnet) ==== | ||
| + | |||
| + | Given the page text, both scripts, both committed outputs, the provenance draft | ||
| + | and the dataset; asked to re-run everything and hunt for stale or mis-scoped | ||
| + | numbers. | ||
| + | |||
| + | * **Re-ran both scripts and diffed against the committed outputs: byte-identical.** Also ran the '' | ||
| + | * **Accepted — "31 of 61 begin with custom" | ||
| + | * **Accepted — the span arithmetic in §10 was 128/64 and the live page has 130 spans.** Its diagnosis is better than the self-audit' | ||
| + | * **Rejected — "at most 9 of 49 is correct, being 7 accept-and-reject plus 2 reject-all" | ||
| + | * **Noted, no change — the CookieGraph 89.86%.** It flagged that the figure is over a 20% sample of the top million rather than the whole top million, then concluded the page's wording matches the paper' | ||
| + | * Everything else it checked — every cell of the methods, questions, categories, population, venue, year, coverage, quiet, crawl-config, | ||
| + | |||
| + | ==== Pass 2 — citations and quotes (Sonnet) ==== | ||
| + | |||
| + | Given both pages, the bibliography, | ||
| + | check that every key resolves, every attributed claim is supported, and every | ||
| + | quotation is verbatim. **Six findings accepted, four of them material.** Each | ||
| + | was re-verified against the source before the page was changed. | ||
| + | |||
| + | * **Accepted, and the worst error on the page — {[lin2024_browsing]}' | ||
| + | * **Accepted — {[jiwani2024crumbling]}' | ||
| + | * **Accepted after independent verification — the opening sentence' | ||
| + | * **Accepted — the {[bollinger2022automating]} discrepancy footnote was itself wrong twice.** The 83.4% is in **§4.1 " | ||
| + | * **Accepted — {[sanchezrola2021_journey]}' | ||
| + | * **Accepted, minor — the 7.2% label-noise figure** is a lower bound on **third-party** cookie labels; the page had dropped the scope. | ||
| + | * **Noted, no change — README quote casing.** The page quotes "its outputs should not be used with the CookieBlock extension directly"; | ||
| + | * **Noted, no change — the Chrome Web Store string.** This reviewer could not reach the store page (its fetch hit Google' | ||
| + | * Everything else reproduced: sixteen papers' | ||
| + | |||
| + | **What this pass says about the run.** Three of its findings are the same | ||
| + | mistake: **a number read out of a table without reading the column header or | ||
| + | the caption.** The automated quote check in §5 passed all three, because the | ||
| + | quote really is in the paper — it is the // | ||
| + | was wrong, and no substring match can catch that. That is the argument for this | ||
| + | reviewer slot existing, and it is recorded here rather than smoothed over. | ||
| + | |||
| + | ==== Pass 3 — external currency (Sonnet) ==== | ||
| + | |||
| + | Asked to fetch, | ||
| + | CookieBlock removal singled out for controls. | ||
| + | |||
| + | * **Accepted — "12 purposes by IAB" is stale.** Verified independently against the live policy: it now defines **Purposes 1–11, Special Purposes 1–3, Features 1–3 and Special Features 1–2**. The archive.org snapshot the page linked is from 2021 and has 10 + 2 = 12. TCF v2.2 added Purpose 11. The page now gives the current counts, links the live canonical, keeps the snapshot as the version most of the literature actually used, and says to cite the version rather than the number. | ||
| + | * **Accepted — the Cookiepedia '' | ||
| + | * **Accepted — '' | ||
| + | * **Noted, no change here — the 2025-10-17 Privacy Sandbox retirements.** Real (Topics, Attribution Reporting, Protected Audience, Related Website Sets and six more retired; CHIPS, FedCM and Private State Tokens kept) and correctly out of scope for this page, which defers all of it to [[Privacy: | ||
| + | * **Noted, no change — "There will be soon a new release based on December 2024 crawl." | ||
| + | * Everything else it fetched reproduced exactly, including all three CookieBlock availability checks with their controls, the Chrome MV2 quote, the AMO and Edge listings, all four label-source sites, the Cookie-Script 404, every CookieGraph artifact count, the Classifier README quotes and the Zenodo file path. It also found the correct live URL for Mozilla' | ||
| + | |||
| + | ==== Pass 4 — generic (Fable) — not run ==== | ||
| + | |||
| + | The generic pass was **not run**: the three focused passes returned late in the | ||
| + | session and the run closed after acting on them. This page has therefore had no | ||
| + | reader without a checklist — the pass that historically catches overstated | ||
| + | claims, structural problems and a page that does not answer its own question. | ||
| + | **Treat that as the largest outstanding gap in this page's review coverage**, | ||
| + | and run it before treating the page as settled. | ||
| ===== 13. Related ===== | ===== 13. Related ===== | ||
provenance/privacy/cookies.1789057505.txt.gz · Last modified: by karel.kubicek.claude
