User Tools

Site Tools


provenance:privacy:cookies

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
provenance:privacy:cookies [2026/09/10 16:25] – Create: query log, denominators, folds and residues, quote checks, external verification with controls, judgement calls, and the two report scripts with their real output, behind privacy:cookies. Authored by Claude karel.kubicek.claudeprovenance:privacy:cookies [2026/09/10 16:44] (current) – Replace the literal citekey span count with the invariant it was checking; a snapshot count is stale after the next edit. Authored by Claude karel.kubicek.claude
Line 81: Line 81:
 ===== 4. The fold, and why it throws instead of leaving a residue ===== ===== 4. The fold, and why it throws instead of leaving a residue =====
  
-''resourceName'' on a cookie classification tuple is free text: **61 distinct strings across 79 tuples**, of which 31 begin with the word "custom". Aggregating by exact string gives a largest row of 14 (Cookiepedia) and a long tail of ones.+''resourceName'' on a cookie classification tuple is free text: **61 distinct strings across 79 tuples**, of which 27 begin with the word "custom" and 34 contain it (computed by the script — a hand count of this got 31). Aggregating by exact string gives a largest row of 14 (Cookiepedia) and a long tail of ones.
  
 The fold is a **hand-keyed map** in ''report_cookies.mjs'' (the ''RESOURCE'' constant) from the exact string to two values: a label-source ''family'' and the ''question'' the classifier answers. The script **throws** if a tuple's ''resourceName'' is not in the map. That is deliberate and is the opposite of the usual residue convention: The fold is a **hand-keyed map** in ''report_cookies.mjs'' (the ''RESOURCE'' constant) from the exact string to two values: a label-source ''family'' and the ''question'' the classifier answers. The script **throws** if a tuple's ''resourceName'' is not in the map. That is deliberate and is the opposite of the usual residue convention:
Line 169: Line 169:
 ^ Claim on the page ^ Primary source, and how it was checked ^ ^ Claim on the page ^ Primary source, and how it was checked ^
 | Cookiepedia holds **42,020,489** cookies; benchmark split 1% / 5% / 3% / 58% / **32% unknown** | ''cookiepedia.co.uk'' front page, read directly. Labelled on the page as a vendor self-report | | Cookiepedia holds **42,020,489** cookies; benchmark split 1% / 5% / 3% / 58% / **32% unknown** | ''cookiepedia.co.uk'' front page, read directly. Labelled on the page as a vendor self-report |
-| Cookiepedia rejects plain ''curl'' | ''cookiepedia.co.uk/cookies/user_id'' returns **403** to ''curl'' with a desktop User-Agent while the site root returns **200** |+| Cookiepedia rejects plain ''curl'' | ''cookiepedia.co.uk/cookies/user_id'' returns **403** to ''curl'' with a desktop User-Agent while the site root returns **200**. The currency pass reproduced this and additionally found that a headless Chromium from this datacentre address is blocked too, so the per-cookie pages could not be read at all — the ''user_id'' example on the page is now footnoted as unverified |
 | The CookieGraph Cookiepedia name table: **45,785** names, **39.3%** categorised, 35.1% ''Error'', 25.6% ''Unknown'' | Computed from the released artifact by ''scripts/cookiepedia_coverage.py'' (§7.1). Reproducible in one ''curl'' | | The CookieGraph Cookiepedia name table: **45,785** names, **39.3%** categorised, 35.1% ''Error'', 25.6% ''Unknown'' | Computed from the released artifact by ''scripts/cookiepedia_coverage.py'' (§7.1). Reproducible in one ''curl'' |
 | The CookieGraph ''cookiepedia.csv'' is **917,551 rows** of CMP-declared labels in the CookieBlock ''consent_data'' schema, not Cookiepedia classifications | Downloaded and counted; header is '',id,browser_id,visit_id,name,domain,cat_id,cat_name,purpose,expiry,type_name,type_id''. Confirmed against ''labelling_scripts/cookiepedia.py'', which merges it with ''tranco.csv'' into a ''declared_label'' | | The CookieGraph ''cookiepedia.csv'' is **917,551 rows** of CMP-declared labels in the CookieBlock ''consent_data'' schema, not Cookiepedia classifications | Downloaded and counted; header is '',id,browser_id,visit_id,name,domain,cat_id,cat_name,purpose,expiry,type_name,type_id''. Confirmed against ''labelling_scripts/cookiepedia.py'', which merges it with ''tranco.csv'' into a ''declared_label'' |
Line 329: Line 329:
 ^ Family ^ Status ^ What it rests on ^ ^ Family ^ Status ^ What it rests on ^
 | Purpose database lookup | current, the default | 8 of the 10 papers in 2023–2024 and 5 of 12 in 2025–2026 | | Purpose database lookup | current, the default | 8 of the 10 papers in 2023–2024 and 5 of 12 in 2025–2026 |
-| Own hand-written rule | current, largest family, mostly unreusable | 20 of 53; 31 of 61 distinct ''resourceName'' strings begin "custom" |+| Own hand-written rule | current, largest family, mostly unreusable | 20 of 53; 27 of 61 distinct ''resourceName'' strings begin "custom", 34 contain it |
 | Identifier heuristic | current, **but the predicate changed** | Family present 2014→2026 including 3 papers in 2025–2026; the supersession of length/expiry thresholds by entropy is argued on the sibling page from {[randall2022_uidsmuggling]} and {[calzavara2026_syntactic]}, not from a corpus count | | Identifier heuristic | current, **but the predicate changed** | Family present 2014→2026 including 3 papers in 2025–2026; the supersession of length/expiry thresholds by entropy is argued on the sibling page from {[randall2022_uidsmuggling]} and {[calzavara2026_syntactic]}, not from a corpus count |
 | Filter list provenance | current and growing | 8 papers, **3 of them 2025**. Small numbers, said so on the page | | Filter list provenance | current and growing | 8 papers, **3 of them 2025**. Small numbers, said so on the page |
Line 366: Line 366:
 Checked before saving: the key does not exist in ''literature:bibliography'' (0 matches), the DOI does not exist (0 matches), and ''scripts/bib_dedup_scan.py'' reports no new A/B/C/D candidate pair involving it. The one pre-existing [C] pair the scan reports (''bratton2019_replication'' / ''sumner2014_exaggeration'') is unrelated to this run and untouched. Checked before saving: the key does not exist in ''literature:bibliography'' (0 matches), the DOI does not exist (0 matches), and ''scripts/bib_dedup_scan.py'' reports no new A/B/C/D candidate pair involving it. The one pre-existing [C] pair the scan reports (''bratton2019_replication'' / ''sumner2014_exaggeration'') is unrelated to this run and untouched.
  
-Every other citekey on the page already existed. Twenty distinct keys resolve; the rendered page shows 128 ''bibtex_citekey'' spans, which is **64 markers** — the plugin emits two spans per marker — and 20 references.+Every other citekey on the page already existed. **20 distinct keys and 20 references.** The check that matters is the invariant, not a snapshot: the plugin emits **two ''bibtex_citekey'' spans per citekey //instance//** — not per marker, because three markers here are multi-key — so rendered spans must equal twice the number of comma-separated keys in the source's ''{[…]}'' markers, and distinct rendered references must equal distinct source keys. Both held at every save. A literal span count is not published here because it changes with every edit and would be stale within the hour. 
 + 
 +**The new key did not render until the bibliography's own cache was purged.** Immediately after saving, ''privacy:cookies'' showed 19 references instead of 20: ''cahn2016_cookies'' resolved to nothing, with no warning of any kind. ''?purge=true'' on ''privacy:cookies'' alone did not fix it; ''?purge=true'' on **''literature:bibliography''** did. Anyone adding a key should count rendered references against distinct source keys after purging both pages, not assume a save is enough.
  
 ===== 11. The report script and its output ===== ===== 11. The report script and its output =====
Line 488: Line 490:
 the script throws instead of silently dropping one). The listing in section 2.2 is what to the script throws instead of silently dropping one). The listing in section 2.2 is what to
 check: a zero residue means the map is complete, not that every judgement in it is right. check: a zero residue means the map is complete, not that every judgement in it is right.
 +
 +Of the 61 distinct strings, 27 BEGIN with the word "custom" and 34 contain it anywhere.
 +That is the reusability finding: these are one-off predicates written for one paper.
  
 ============================================================================== ==============================================================================
Line 517: Line 522:
 Security role: authentication or session cookie, attribute correctness      5       9.4%         7 Security role: authentication or session cookie, attribute correctness      5       9.4%         7
 Something else (named in the residue listing)                               12      22.6%        14 Something else (named in the residue listing)                               12      22.6%        14
 +
 +==============================================================================
 +2.1c NAMED LABEL SOURCES, PAPER-COUNTED — the figures the prose quotes
 +==============================================================================
 +resourceName (exact string)              Papers  Tuples  Family                Question
 +---------------------------------------  ------  ------  --------------------  ----------
 +Cookiepedia                              14      14      purpose-database      purpose
 +CookieBlock                              4             published-classifier  purpose
 +Cookie-Script                            2             purpose-database      purpose
 +justdomains blocklist                    2             filter-list           tracking
 +Cookie Database                          1             purpose-database      purpose
 +Cookie Script                            1             purpose-database      purpose
 +Cookiedatabase.org                                   purpose-database      purpose
 +CookieGraph                              1             published-classifier  tracking
 +Disconnect.me                            1             filter-list           tracking
 +EasyList                                             filter-list           tracking
 +EasyList/EasyPrivacy                                 filter-list           tracking
 +Ghostery                                             filter-list           tracking
 +Ghostery and Disconnect intersection                 filter-list           tracking
 +GPT-3.5                                  1             llm                   purpose
 +GPT-3.5-turbo (custom fine-tuned model)  1             llm                   purpose
 +justdomains                              1             filter-list           tracking
 +Public Suffix List                                   party-rule            party
 +WebGraph                                             published-classifier  tracking
 +zxcvbn                                               identifier-heuristic  identifier
 +
 +  Merged across spellings (papers, not tuples):
 +    Cookiepedia                      14
 +    Cookie-Script / "Cookie Script"  3
 +    cookiedatabase.org               2
 +    CookieBlock                      4
 +    any purpose database             17
 +    justdomains (both spellings)     3
  
 ============================================================================== ==============================================================================
Line 782: Line 820:
 7. LLM CLASSIFICATION OF COOKIES — the currency check 7. LLM CLASSIFICATION OF COOKIES — the currency check
 ============================================================================== ==============================================================================
-Classification target  Papers classifying it  of which use an LLM  Share +Classification target  Papers classifying it (used/produced)  ...of which an LLM  LLM share  <-- PUBLISHED  (any/any numerator)  (any/any denominator)  (any/any share) 
----------------------  ---------------------  -------------------  ----- +---------------------  -------------------------------------  ------------------  ------------------------  -------------------  ---------------------  --------------- 
-other                  2594                   116                  4.5+privacy-policy         102                                    12                  11.8%                     12                   102                    11.8% 
-user-generated-text    419                    32                   7.6% +dark-pattern           13                                                       7.7%                      1                    13                     7.7
-vulnerability          883                    14                   1.6+user-generated-text    419                                    30                  7.2%                      32                   419                    7.6% 
-privacy-policy         102                    12                   11.8+consent-notice         39                                                       5.1%                      2                    39                     5.1
-mobile-app             282                    6                    2.1% +other                  2592                                   115                 4.4%                      116                  2594                   4.5
-website-category       424                    5                    1.2% +mobile-app             280                                                      2.1%                      6                    282                    2.1
-domain                 351                    3                    0.9% +email-message          54                                                       1.9%                      1                    54                     1.9% 
-network-traffic        383                                       0.5% +cookie                 53                                                       1.9%                      1                    53                     1.9% 
-consent-notice         39                     2                    5.1% +vulnerability          880                                    14                  1.6%                      14                   883                    1.6
-ip-address             295                    1                    0.3+website-category       424                                                      1.2%                      5                    424                    1.2% 
-web-request            262                                       0.4% +domain                 351                                                      0.9%                                         351                    0.9% 
-email-message          54                     1                    1.9+network-traffic        382                                                      0.5%                      2                    383                    0.5
-cookie                 53                                        1.9% +web-request            258                                                      0.4%                      1                    262                    0.4
-dark-pattern           13                     1                    7.7+ip-address             295                                                      0.3                     1                    295                    0.3
-malware                160                                       0.0% +malware                159                                    0                   0.0%                      0                    160                    0.0% 
-sdk-or-library         77                     0                    0.0% +sdk-or-library         77                                     0                   0.0%                      0                    77                     0.0% 
-javascript             44                     0                    0.0% +javascript             44                                     0                   0.0%                      0                    44                     0.0% 
-fingerprinting-script  32                     0                    0.0% +fingerprinting-script  31                                                       0.0%                      0                    32                     0.0% 
-website-popularity     16                     0                    0.0%+website-popularity     15                                                       0.0%                      0                    16                     0.0%
  
-The cookie row is the one privacy:cookies and design:website_classification must agree on. +  The PUBLISHED column is columns 2-4 and matches report_llm_currency.mjs row for row. 
-  LLM + cookie: WWW/2025/semantics-aware-cookie-purpose-compliance+  Rows where the two definitions disagree: 
 +    user-generated-text: used/used 30/419 = 7.2%   any/any 32/419 = 7.6% 
 +    other: used/used 115/2592 = 4.4%   any/any 116/2594 = 4.5% 
 + 
 +  The cookie row is the one privacy:cookies and design:website_classification must agree on. 
 +    LLM + cookie: WWW/2025/semantics-aware-cookie-purpose-compliance
  
 ============================================================================== ==============================================================================
Line 1657: Line 1700:
 } }
 const distinct = new Set(allTuples.map((t) => t.c.resourceName)); const distinct = new Set(allTuples.map((t) => t.c.resourceName));
 +// Computed, not counted by hand: the page states this figure and a hand count
 +// of it was wrong by four on the first attempt.
 +const customPrefixed = [...distinct].filter((n) => /^custom\b/i.test(n));
 +const customAnywhere = [...distinct].filter((n) => /custom/i.test(n));
 console.log( console.log(
   `\n${allTuples.length} cookie classification tuples across ${CLASSIFY.length} papers, ` +   `\n${allTuples.length} cookie classification tuples across ${CLASSIFY.length} papers, ` +
     `${distinct.size} distinct resourceName strings, all mapped (residue 0 by construction —\n` +     `${distinct.size} distinct resourceName strings, all mapped (residue 0 by construction —\n` +
     `the script throws instead of silently dropping one). The listing in section 2.2 is what to\n` +     `the script throws instead of silently dropping one). The listing in section 2.2 is what to\n` +
-    `check: a zero residue means the map is complete, not that every judgement in it is right.`+    `check: a zero residue means the map is complete, not that every judgement in it is right.\n\n` + 
 +    `Of the ${distinct.size} distinct strings, ${customPrefixed.length} BEGIN with the word "custom" and ` + 
 +    `${customAnywhere.length} contain it anywhere.\nThat is the reusability finding: these are one-off ` + 
 +    `predicates written for one paper.`
 ); );
  
Line 1692: Line 1742:
   ])   ])
 ); );
 +
 +h('2.1c NAMED LABEL SOURCES, PAPER-COUNTED — the figures the prose quotes');
 +// Every named (non-"custom") label source with its distinct-paper count, so no
 +// sentence on the page has to hand-count one. Two hand counts were wrong on the
 +// first attempt: Cookiepedia (12, really 14) and Cookie-Script (4, really 3 —
 +// one paper contributes two tuples).
 +const NAMED = /^(Cookiepedia|Cookie[- ]?Script|Cookiedatabase\.org|Cookie Database|CookieBlock|CookieGraph|WebGraph|EasyList|EasyList\/EasyPrivacy|justdomains|justdomains blocklist|Ghostery|Ghostery and Disconnect intersection|Disconnect\.me|zxcvbn|Public Suffix List|GPT-3\.5|GPT-3\.5-turbo \(custom fine-tuned model\))$/;
 +const namedRows = new Map();
 +for (const t of allTuples) {
 +  if (!NAMED.test(t.c.resourceName)) continue;
 +  if (!namedRows.has(t.c.resourceName)) namedRows.set(t.c.resourceName, new Set());
 +  namedRows.get(t.c.resourceName).add(key(t.p));
 +}
 +T(
 +  ['resourceName (exact string)', 'Papers', 'Tuples', 'Family', 'Question'],
 +  [...namedRows.entries()]
 +    .map(([n, set]) => {
 +      const ts = allTuples.filter((t) => t.c.resourceName === n);
 +      return [n, set.size, ts.length, ts[0].family, ts[0].question];
 +    })
 +    .sort((a, b) => b[1] - a[1] || a[0].localeCompare(b[0]))
 +);
 +// The two spellings of Cookie-Script and of justdomains are one product each.
 +const merge = (re) => new Set(allTuples.filter((t) => re.test(t.c.resourceName)).map((t) => key(t.p))).size;
 +console.log(`
 +  Merged across spellings (papers, not tuples):
 +    Cookiepedia                      ${merge(/^Cookiepedia$/)}
 +    Cookie-Script / "Cookie Script"  ${merge(/^Cookie[- ]?Script$/i)}
 +    cookiedatabase.org               ${merge(/^(Cookiedatabase\.org|Cookie Database)$/i)}
 +    CookieBlock                      ${merge(/^CookieBlock$/)}
 +    any purpose database             ${merge(/^(Cookiepedia|Cookie[- ]?Script|Cookiedatabase\.org|Cookie Database)$/i)}
 +    justdomains (both spellings)     ${merge(/^justdomains( blocklist)?$/i)}`);
  
 h('2.2 EVERY TUPLE, GROUPED BY FAMILY — the listing to argue with'); h('2.2 EVERY TUPLE, GROUPED BY FAMILY — the listing to argue with');
Line 1852: Line 1934:
 // --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
 // Cross-checks scripts/report_llm_currency.mjs, which computes the same figure // Cross-checks scripts/report_llm_currency.mjs, which computes the same figure
-// for every target. If these two disagree, one of them is wrong.+// for every target and backs design:website_classification, privacy:javascript 
 +// and design:ip_classification. If these two disagree, one of them is wrong and 
 +// two published pages contradict each other. 
 +// 
 +// That script's per-target share is USED/USED: numerator = papers that used or 
 +// produced LLM labels at that target, denominator = papers that used or 
 +// produced ANY classifier at that target. `compared` and `mentioned` tuples are 
 +// a baseline the authors argued against; counting them as adoption overstates 
 +// it, and counting them in the denominator but not the numerator understates 
 +// the share. This script reproduces that definition EXACTLY so the pages cannot 
 +// drift, and prints the naive any/any variant beside it so the difference is 
 +// visible rather than hidden. The rows where they disagree are listed by the 
 +// script itself under the table rather than being asserted in this comment, 
 +// because a comment does not get re-derived when the corpus moves. The cookie 
 +// row is 1/53 under both definitions. 
 +const USED_OM = new Set(['used', 'produced']); 
 +const setAdd = (m, k, v) => { 
 +  if (!m.has(k)) m.set(k, new Set()); 
 +  m.get(k).add(v); 
 +}; 
 +const llmUsedByTarget = new Map();
 const llmAnyByTarget = new Map(); const llmAnyByTarget = new Map();
 +const usedByTarget = new Map();
 const anyByTarget = new Map(); const anyByTarget = new Map();
 for (const p of CLASSIFIED) { for (const p of CLASSIFIED) {
   for (const c of p.classification) {   for (const c of p.classification) {
-    const t = c.target === 'other' ? 'other' : c.target; +    if (isSentinel(c.target)) continue
-    if (!anyByTarget.has(t)) anyByTarget.set(t, new Set()); +    setAdd(anyByTarget, c.target, key(p)); 
-    anyByTarget.get(t).add(key(p)); +    if (c.method === 'llm'setAdd(llmAnyByTarget, c.target, key(p)); 
-    if (c.method === 'llm'{ +    if (USED_OM.has(c.usedOrMentioned)) 
-      if (!llmAnyByTarget.has(t)) llmAnyByTarget.set(tnew Set()); +      setAdd(usedByTargetc.target, key(p)); 
-      llmAnyByTarget.get(t).add(key(p));+      if (c.method === 'llm') setAdd(llmUsedByTarget, c.target, key(p));
     }     }
   }   }
 } }
 +const sz = (m, t) => (m.get(t) ?? new Set()).size;
 T( T(
-  ['Classification target', 'Papers classifying it', 'of which use an LLM', 'Share'], +  [ 
-  [...anyByTarget.entries()] +    'Classification target', 
-    .map(([t, s]) => [t, s.size, (llmAnyByTarget.get(t) ?? new Set()).size, pct((llmAnyByTarget.get(t) ?? new Set()).sizes.size)]) +    'Papers classifying it (used/produced)', 
-    .sort((a, b) => b[2] - a[2] || b[1] - a[1])+    '...of which an LLM', 
 +    'LLM share  <-- PUBLISHED'
 +    '(any/any numerator)', 
 +    '(any/any denominator)', 
 +    '(any/any share)', 
 +  ], 
 +  [...anyByTarget.keys()] 
 +    .map((t) => [ 
 +      t, 
 +      sz(usedByTargett), 
 +      sz(llmUsedByTarget, t), 
 +      pct(sz(llmUsedByTarget, t), sz(usedByTarget, t)), 
 +      sz(llmAnyByTarget, t), 
 +      sz(anyByTarget, t), 
 +      pct(sz(llmAnyByTargett), sz(anyByTarget, t)), 
 +    ]) 
 +    .sort((a, b) => parseFloat(b[3]parseFloat(a[3]|| b[1] - a[1])
 ); );
-console.log('\nThe cookie row is the one privacy:cookies and design:website_classification must agree on.'); +console.log(` 
-for (const k of llmAnyByTarget.get('cookie') ?? []) console.log('  LLM + cookie: ' + k);+  The PUBLISHED column is columns 2-4 and matches report_llm_currency.mjs row for row. 
 +  Rows where the two definitions disagree:`); 
 +for (const t of anyByTarget.keys()) { 
 +  const a = pct(sz(llmUsedByTarget, t), sz(usedByTarget, t)); 
 +  const b = pct(sz(llmAnyByTarget, t), sz(anyByTarget, t)); 
 +  if (a !== b) console.log(`    ${t}: used/used ${sz(llmUsedByTarget, t)}/${sz(usedByTarget, t)} = ${a}   any/any ${sz(llmAnyByTarget, t)}/${sz(anyByTarget, t)} = ${b}`); 
 +
 +console.log('\n  The cookie row is the one privacy:cookies and design:website_classification must agree on.'); 
 +for (const k of llmAnyByTarget.get('cookie') ?? []) console.log('    LLM + cookie: ' + k);
  
 // --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
Line 2056: Line 2184:
 ===== 12. Review log ===== ===== 12. Review log =====
  
-_(review pass not yet run)_+Four passes were planned: three focused (Sonnet) and one generic (Fable). **All 
 +three focused passes returned and every finding is logged below with whether it 
 +was accepted or rejected. The generic pass was not run** — see the last 
 +subsection, which is the largest remaining gap in this page'review coverage. 
 + 
 +Between the self-audit and the three passes, **21 published figures, citations 
 +or claims were corrected**. Most are the same underlying mistake in two 
 +flavours: a number that the report script did not produce, hand-carried into 
 +the prose; and a number read out of a paper's table without reading the column 
 +header. Every figure of the first kind has since been moved into the script. 
 + 
 +==== Self-audit, before any reviewer returned ==== 
 + 
 +Found by re-deriving figures the prose had hand-counted. All were live on the 
 +page for roughly one hour and are now fixed. Every one is a case of the same 
 +mistake — **a number in the prose that the report script did not produce** — so 
 +the script was extended to compute each of them (§2.1c and §2 of its output). 
 + 
 +^ Figure ^ Published ^ Correct ^ How it was caught ^ 
 +| Distinct ''resourceName'' strings beginning "custom" | 31 | **27** (34 contain it) | Recounted with ''grep -c '^custom'''
 +| Papers naming Cookiepedia | 12 | **14** | Paper-counted with ''jq''
 +| Papers naming Cookie-Script | 4 | **3** | Same; one paper contributes two tuples | 
 +| Papers reporting more than one consent state | "at most 9 of 49" | **7 of 49** state accept-and-reject; 3 more state a single interaction | ''reject-all'' and ''accept-all'' are single states, so 7+2+1 is the wrong sum | 
 +| Cookie papers vs field on crawl reporting | "two to three times" | 2.0x on consent action, 3.5x on statefulness | Divided the two columns | 
 +| PoPETs vs IEEE S&P publication rate | "eight times" | more than ten times, on n=2 | Divided the two shares | 
 +| LLM share for ''user-generated-text'' | 7.6% | **7.2%** | See below | 
 +| "Nobody has trained a new cookie purpose model since 2023" | — | **false** | {[chen2025_semantics]} fine-tuned one in 2025; the ''llm'' family is separate from ''own-supervised-ml'' in the fold, and the prose read the latter as if it covered both | 
 + 
 +**The LLM one is the interesting failure.** The page's per-target table is 
 +supposed to be the same table as [[Design:Website classification#which_task_though]], 
 +"generated once so the pages cannot drift". This script re-derived it 
 +independently and got 32/419 = 7.6% for ''user-generated-text'' where the 
 +published cell says 7.2%. Neither was stale: ''report_llm_currency.mjs'' 
 +computes the share as **used/used** — numerator and denominator both restricted 
 +to ''usedOrMentioned'' in {used, produced} — while the re-derivation used 
 +any/any. The two definitions differ for exactly two targets 
 +(''user-generated-text'' 30/419 vs 32/419, ''other'' 115/2592 vs 116/2594) and 
 +agree everywhere else, including the cookie row, which is why it nearly went 
 +unnoticed. ''report_cookies.mjs'' §7 now reproduces the used/used definition 
 +exactly, prints the any/any variant beside it, and prints the rows where the two 
 +disagree. **Sharing a script is a claim about provenance, not about definition.** 
 + 
 +**Also caught: the new citekey silently did not render.** See §10. 
 + 
 +==== Pass 1 — figures against the script (Sonnet) ==== 
 + 
 +Given the page text, both scripts, both committed outputs, the provenance draft 
 +and the dataset; asked to re-run everything and hunt for stale or mis-scoped 
 +numbers. 
 + 
 +  * **Re-ran both scripts and diffed against the committed outputs: byte-identical.** Also ran the ''curl | python3'' one-liner the page publishes and reproduced 39.3% / 35.1% / 25.6%. 
 +  * **Accepted — "31 of 61 begin with custom" is 27.** Independently recomputed, and correctly noted that the script did not print the figure at all, which is why it was never checked. Already fixed by the self-audit; the script now prints it. 
 +  * **Accepted — the span arithmetic in §10 was 128/64 and the live page has 130 spans.** Its diagnosis is better than the self-audit's: the plugin emits two spans per **key instance**, not per marker, and three markers on this page are multi-key. §10 now states 62 markers / 65 instances / 130 spans. 
 +  * **Rejected — "at most 9 of 49 is correct, being 7 accept-and-reject plus 2 reject-all".** A paper that rejects on every visit measured one consent state, not more than one. The sentence was already being corrected to "7 state accept-and-reject, 3 more state a single interaction", which is what the enum supports; the reviewer's arithmetic reproduces the wrong claim rather than testing it. 
 +  * **Noted, no change — the CookieGraph 89.86%.** It flagged that the figure is over a 20% sample of the top million rather than the whole top million, then concluded the page's wording matches the paper's own abstract. Agreed: the page says "top-million", which is the paper's framing, and the provenance page records the distinction from the 20K stratified crawl. 
 +  * Everything else it checked — every cell of the methods, questions, categories, population, venue, year, coverage, quiet, crawl-config, consentAction and LLM tables, the 82% / 16pp / 38pp / "8 of the 10" arithmetic, sentinel handling, paper-vs-tuple counting, and the fold's throw-on-unmapped behaviour — reproduced exactly. It also independently re-verified six external sources. 
 + 
 +==== Pass 2 — citations and quotes (Sonnet) ==== 
 + 
 +Given both pages, the bibliography, the report output and the papers; asked to 
 +check that every key resolves, every attributed claim is supported, and every 
 +quotation is verbatim. **Six findings accepted, four of them material.** Each 
 +was re-verified against the source before the page was changed. 
 + 
 +  * **Accepted, and the worst error on the page — {[lin2024_browsing]}'s "28.50%".** Its Table 1 columns are //Total / Mean / Median//, not Total / Percent. The row reads ''Unclassified 14619 28.50 18'', so 28.50 is a **mean per site**, not a share; the paper states no percentage. Summing the five category totals gives 18,074 cookies, of which 14,619 are unclassified — **80.9%**, not 28.5%. The page's own adjacent quote ("the vast majority of cookies are unclassified") contradicted its own figure and that did not get noticed before publication. Corrected, and the 80.9% is labelled as computed here rather than as stated by the paper. 
 +  * **Accepted — {[jiwani2024crumbling]}'s 67% named the wrong term.** 7% is the score for the original term "functional"; 67% is the score for the candidate "personalized experience". "Anonymous analytics" also scores 67%, but against "performance" (24%). Two identical numbers in adjacent paragraphs, and the page paired the wrong one. Corrected, and the paper's actual recommendations are now named. 
 +  * **Accepted after independent verification — the opening sentence's citations.** The page carried "between 80% in 2012 {[roesner2012_detecting]} and 90% in 2019 {[solomos2019_clash,sanchezrola2019can]}", a faithful reproduction of the introduction of {[bollinger2022automating]}, which says exactly that. The reviewer reported that neither primary contains its figure. **Checked before acting rather than taken on trust:** the Roesner NSDI 2012 PDF was fetched from ''usenix.org/system/files/conference/nsdi12/nsdi12-final17.pdf'' and scanned — it contains **no percentage between 70% and 99% anywhere in its text**. The sentence is rewritten around {[sanchezrola2019can]}, which does support "more than 90%", with a footnote recording where the trend line comes from and that it could not be confirmed. The two shaky citations are kept //in the footnote// rather than deleted, so the next reader can re-open the question instead of re-discovering it. 
 +  * **Accepted — the {[bollinger2022automating]} discrepancy footnote was itself wrong twice.** The 83.4% is in **§4.1 "Baseline"**, not §4.4, and the **abstract states neither figure** — it gives 84.4%, CookieBlock's own balanced accuracy, a different quantity. A footnote written to flag someone else's inconsistency contained two of its own. 
 +  * **Accepted — {[sanchezrola2021_journey]}'s population.** "137,997,677 cookies collected across 387K websites" pairs the total-cookie count with the **cookie-sharing** site count from a different section ("8.97M cookie sharing events over 387K websites"). The crawl is 6.2M pages from a 1M-domain seed. Corrected in both places. 
 +  * **Accepted, minor — the 7.2% label-noise figure** is a lower bound on **third-party** cookie labels; the page had dropped the scope. 
 +  * **Noted, no change — README quote casing.** The page quotes "its outputs should not be used with the CookieBlock extension directly"; the README has a capital I and a trailing exclamation mark. Mid-sentence quoting convention, not a misquote. 
 +  * **Noted, no change — the Chrome Web Store string.** This reviewer could not reach the store page (its fetch hit Google's consent interstitial) and confirmed the claim only via the CRX endpoint. §7.2 already treats that as the decisive evidence, and the currency pass did reach the page. 
 +  * Everything else reproduced: sixteen papers' worth of figures, the bibliography's freedom from duplicate keys, DOIs and titles, ''cahn2016_cookies'' against the corpus index, and the verbatim Chrome MV2, Privacy Sandbox and Cookiepedia quotations. 
 + 
 +**What this pass says about the run.** Three of its findings are the same 
 +mistake: **a number read out of a table without reading the column header or 
 +the caption.** The automated quote check in §5 passed all three, because the 
 +quote really is in the paper — it is the //interpretation// of the quote that 
 +was wrong, and no substring match can catch that. That is the argument for this 
 +reviewer slot existing, and it is recorded here rather than smoothed over. 
 + 
 +==== Pass 3 — external currency (Sonnet) ==== 
 + 
 +Asked to fetch, not recall, every external claim as of 2026-09-10, with the 
 +CookieBlock removal singled out for controls. 
 + 
 +  * **Accepted — "12 purposes by IAB" is stale.** Verified independently against the live policy: it now defines **Purposes 1–11, Special Purposes 1–3, Features 1–3 and Special Features 1–2**. The archive.org snapshot the page linked is from 2021 and has 10 + 2 = 12. TCF v2.2 added Purpose 11. The page now gives the current counts, links the live canonical, keeps the snapshot as the version most of the literature actually used, and says to cite the version rather than the number. 
 +  * **Accepted — the Cookiepedia ''user_id'' example cannot be re-verified.** The per-cookie pages sit behind a Cloudflare interstitial that returns 403 to ''curl'' **and** to a headless Chromium from this address. The claim is now footnoted as last-observed and unverified rather than left reading as a live fact. 
 +  * **Accepted — ''privacysandbox.com'' is retired.** It redirects to ''privacysandbox.google.com/blog/…'' with the quoted text unchanged. Footnote repointed and the redirect recorded. 
 +  * **Noted, no change here — the 2025-10-17 Privacy Sandbox retirements.** Real (Topics, Attribution Reporting, Protected Audience, Related Website Sets and six more retired; CHIPS, FedCM and Private State Tokens kept) and correctly out of scope for this page, which defers all of it to [[Privacy:Privacy sandbox]]. The reviewer's suggestion to confirm that neighbour actually carries the date is fair and is left as a TODO for whoever owns it — **not** something this run should reach across and edit. 
 +  * **Noted, no change — "There will be soon a new release based on December 2024 crawl."** The reviewer points out that "soon" is now 21 months old and no such Zenodo release is visible. That sentence is Karel's own note to readers and only he can retire it; flagged here rather than removed. 
 +  * Everything else it fetched reproduced exactly, including all three CookieBlock availability checks with their controls, the Chrome MV2 quote, the AMO and Edge listings, all four label-source sites, the Cookie-Script 404, every CookieGraph artifact count, the Classifier README quotes and the Zenodo file path. It also found the correct live URL for Mozilla's manifest-version distribution page, which §7 had recorded under a guessed slug. 
 + 
 +==== Pass 4 — generic (Fable— not run ==== 
 + 
 +The generic pass was **not run**: the three focused passes returned late in the 
 +session and the run closed after acting on them. This page has therefore had no 
 +reader without a checklist — the pass that historically catches overstated 
 +claims, structural problems and a page that does not answer its own question. 
 +**Treat that as the largest outstanding gap in this page's review coverage**, 
 +and run it before treating the page as settled.
  
 ===== 13. Related ===== ===== 13. Related =====
provenance/privacy/cookies.1789057505.txt.gz · Last modified: by karel.kubicek.claude