| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:privacy:consent [2026/09/05 20:04] – Section 7.3: audit of all 313 no-interaction consentAction papers, with verdicts, recall probes, quote check and significance tests; section 11 updated. Authored by Claude karel.kubicek.claude | provenance:privacy:consent [2026/09/17 07:23] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude |
|---|
| The Matte case is the one worth keeping: a percentage that looks like it came from a paper, computed by the writer from a count and a denominator taken from two different tables. Reading the source turned up **four more** figures the draft did not have — 46.5% pre-selected, 5.3% non-respect of choice, 54.29% any violation, and the 4.3% shared-cookie consent reuse. The first three went into the results table immediately; the fourth was accidentally dropped between drafts and was restored, as a box in the TCF section, only after a reviewer noticed on 2026-08-19 that this very paragraph claimed it was on the page when it was not. | The Matte case is the one worth keeping: a percentage that looks like it came from a paper, computed by the writer from a count and a denominator taken from two different tables. Reading the source turned up **four more** figures the draft did not have — 46.5% pre-selected, 5.3% non-respect of choice, 54.29% any violation, and the 4.3% shared-cookie consent reuse. The first three went into the results table immediately; the fourth was accidentally dropped between drafts and was restored, as a box in the TCF section, only after a reviewer noticed on 2026-08-19 that this very paragraph claimed it was on the page when it was not. |
| |
| ==== 7.2 The ''consentAction'' audit, and why the standard quote check could not have caught it ==== | ==== 7.2 The consentAction audit, and why the standard quote check could not have caught it ==== |
| |
| The page's central figure is "how many papers interacted with a consent notice". The extraction says **36**. Reading all 36 against their own text says **29**. That is a **19.4% false-positive rate** on the interacting values of ''crawlConfig.consentAction'', and it is the single most important thing on this provenance page. | The page's central figure is "how many papers interacted with a consent notice". The extraction says **36**. Reading all 36 against their own text says **29**. That is a **19.4% false-positive rate** on the interacting values of ''crawlConfig.consentAction'', and it is the single most important thing on this provenance page. |
| **Why the site's usual check is blind here.** The dataset's guidance is to read a handful of ''evidence.quote'' values behind any figure before publishing it. That check **cannot work for this field**. ''crawlConfig'' is an object, not an array, and it carries **one** evidence quote for the **whole** object — statefulness, browsers, headless, consent action, interaction depth, authentication, languages and repeat visits all share it. So the quote behind a ''consentAction'' value usually evidences something else entirely. Read literally, the quotes for the 36 papers include "we create a separate docker container for each URL we visit", "each crawl within a run is performed once without repetition" and "we visit each website afresh with storage cleared" — none of which says anything about a banner. One of them, for Trevisan et al., says the opposite of its label: **"No user action is performed on the page"** sits under a ''accept-and-reject'' value. | **Why the site's usual check is blind here.** The dataset's guidance is to read a handful of ''evidence.quote'' values behind any figure before publishing it. That check **cannot work for this field**. ''crawlConfig'' is an object, not an array, and it carries **one** evidence quote for the **whole** object — statefulness, browsers, headless, consent action, interaction depth, authentication, languages and repeat visits all share it. So the quote behind a ''consentAction'' value usually evidences something else entirely. Read literally, the quotes for the 36 papers include "we create a separate docker container for each URL we visit", "each crawl within a run is performed once without repetition" and "we visit each website afresh with storage cleared" — none of which says anything about a banner. One of them, for Trevisan et al., says the opposite of its label: **"No user action is performed on the page"** sits under a ''accept-and-reject'' value. |
| |
| Doing the check properly meant a different method: extract every sentence in each paper containing a consent/banner term **and** a first-person marker, read them, and record a verdict. ''consent_action_audit.mjs --sentences'' reproduces the sentences; the verdicts are hard-coded in that file with a one-line reason each, and the script **fails loudly** if a paper claims an interaction and has no verdict, or if a verdict refers to a paper that no longer claims one. That guard caught three truncated slugs in the first version of the verdict table. | Doing the check properly meant a different method: extract every sentence in each paper containing a consent/banner term **and** a first-person marker, read them, and record a verdict. ''consent_action_audit.mjs %%--sentences%%'' reproduces the sentences; the verdicts are hard-coded in that file with a one-line reason each, and the script **fails loudly** if a paper claims an interaction and has no verdict, or if a verdict refers to a paper that no longer claims one. That guard caught three truncated slugs in the first version of the verdict table. |
| |
| ^ Verdict ^ Papers ^ Share of 36 ^ | ^ Verdict ^ Papers ^ Share of 36 ^ |
| </code> | </code> |
| |
| ==== 7.3 The ''no-interaction'' audit ==== | ==== 7.3 The no-interaction audit ==== |
| |
| Run 2026-09-05. §7.2 audited the 36 papers claiming an interaction and left the complementary 313 unread, which meant the page's central denominator rested on an unaudited field. All 313 have now been read. | Run 2026-09-05. §7.2 audited the 36 papers claiming an interaction and left the complementary 313 unread, which meant the page's central denominator rested on an unaudited field. All 313 have now been read. |
| |
| - **The 313 were partitioned mechanically first.** A deliberately wide vocabulary pattern (consent, cookie banner/notice/bar/wall/dialog/prompt, banner, CMP, GDPR, ePrivacy, opt-in, opt-out, accept all, reject all, the named banner tools, the named CMP vendors, pop-up, overlay, interstitial, modal) was run over **both** renderings of every full text — ''paper.cols.txt'' and ''paper.txt''. **72 papers contain no hit in either.** Those are ''unstated'' without further reading. Positive control: the pattern fires on **36 of 36** papers on the interacting side, including all 29 verified ones. | - **The 313 were partitioned mechanically first.** A deliberately wide vocabulary pattern (consent, cookie banner/notice/bar/wall/dialog/prompt, banner, CMP, GDPR, ePrivacy, opt-in, opt-out, accept all, reject all, the named banner tools, the named CMP vendors, pop-up, overlay, interstitial, modal) was run over **both** renderings of every full text — ''paper.cols.txt'' and ''paper.txt''. **72 papers contain no hit in either.** Those are ''unstated'' without further reading. Positive control: the pattern fires on **36 of 36** papers on the interacting side, including all 29 verified ones. |
| - **The remaining 241 were read.** For each, every sentence carrying a consent term was extracted, ranked (named tool > non-interaction phrase with a first-person marker > consent term with a first-person marker > consent term alone) and adjudicated by hand, in batches of 20, in publication order. Where the excerpt was ambiguous the paper's full text was queried directly; 14 papers needed that, and 6 of them changed verdict as a result — CookieEnforcer and WhisperTest among them. | - **A third, independent extraction was run over a sample of the 72.** ''paper.cols.txt'' and ''paper.txt'' are both pypdf-derived, so agreeing that a paper has no consent vocabulary is weaker evidence than it looks. 12 of the 72 were drawn at random (seed 20260905) and re-extracted directly from ''paper.pdf'' with pypdf 6.16.2: **12 of 12 confirmed zero hits**, with 50k–97k characters extracted each, so the files are not truncated. |
| | - **The remaining 241 were read.** For each, every sentence carrying a consent term was extracted, ranked (named tool > non-interaction phrase with a first-person marker > consent term with a first-person marker > consent term alone) and adjudicated by hand, in batches of 20, in publication order. Where the excerpt was ambiguous the paper's full text was queried directly with a targeted pattern; **21 papers needed that, and 7 changed verdict as a result** — ''my-cookie-is-a-phoenix'', ''everybodys-looking-for-ssomething'', ''comparing-large-scale-privacy-and-security-notifications'' and ''digital-disparities'' from ''unstated'' to ''ok'', ''reproducibility-and-replicability'' from ''unstated'' to ''implied'', CookieEnforcer from ''unstated'' to ''wrong'', and WhisperTest from ''ok'' to ''partial''. |
| - **Two recall probes were then run over all 313** as a check on the hand pass, one for statements of interaction and one for statements of non-interaction (§7.3.5). Neither found a paper the hand pass had missed. Neither alone finds every ''ok'' either, which is why they are a safety net and not the method. | - **Two recall probes were then run over all 313** as a check on the hand pass, one for statements of interaction and one for statements of non-interaction (§7.3.5). Neither found a paper the hand pass had missed. Neither alone finds every ''ok'' either, which is why they are a safety net and not the method. |
| |
| |
| <code> | <code> |
| quotes verbatim and contiguous in the source: 109 | |
| quotes in order across a two-column splice: 18 | quotes verbatim and contiguous in the source: 112 |
| | quotes in order across a two-column splice: 17 |
| quotes NOT found in the cited paper: 0 | quotes NOT found in the cited paper: 0 |
| verdicts carrying no verbatim quote (paraphrase): 201 | verdicts carrying no verbatim quote (paraphrase): 200 |
| | multi-word single-quoted mentions checked: 0 |
| | ... of those, NOT found in the cited paper: 0 |
| |
| mutation self-test: 81 corrupted quotes rejected, 0 wrongly accepted | mutation self-test: 82 corrupted quotes rejected, 0 wrongly accepted |
| </code> | </code> |
| |
| - **Quotation-mark artefacts** from a mechanical conversion that promoted apostrophes inside words to quote delimiters, producing fragments such as "s subject is the browser". | - **Quotation-mark artefacts** from a mechanical conversion that promoted apostrophes inside words to quote delimiters, producing fragments such as "s subject is the browser". |
| |
| A **mutation self-test** (''--mutate'') corrupts the middle word of every quote of four words or more and requires a FAIL: **81 corrupted quotes, 81 rejected, 0 wrongly accepted.** | A **mutation self-test** (''%%--mutate%%'') corrupts the middle word of every quote of four words or more and requires a FAIL: **82 corrupted quotes, 82 rejected, 0 wrongly accepted.** |
| | |
| | **Then it missed a third misattribution, and a reviewer found it.** ''NDSS/2013/the-postman-always-rings-twice…'' carried the reason "one hit, Tor described as an 'overlay' network", which is the evidence of ''PETS/2022/from-onion-not-found-to-guard-discovery'' — the postMessage paper never mentions Tor. Its only //overlay// is inside the URL fragment ''overlay-analytics'', and the apparent "tor.js" is ''paper.txt'' line-wrapping ''selector.js'' into ''selec-'' and ''tor.js''. The verdict (''unstated'') was right; the evidence was another paper's. **It escaped because the checker's own convention exempted single-quoted text as "a word being mentioned"** — and this single-quoted run was a substantive claim, not a mention. The checker now verifies **every single-quoted run of three words or more** as well, excluding word-internal apostrophes and spans that contain a double quote. That check currently finds zero mentions to verify, which is a vacuous pass, so it was mutation-tested directly: injecting a fabricated ''%%'Tor is an overlay network of volunteer-run relays'%%'' into that same verdict produces ''MENTION NOT IN PAPER''. |
| | |
| | **And a structural check was added ahead of both.** The mechanical single-to-double conversion had also turned ''doesn't'' into ''doesn"t'' in **7** reasons, leaving an unbalanced delimiter. The verify step passed them, because the mangled fragment it extracted ("By default, it doesn") really does appear in the paper. Balanced quotes, and no quote mark inside a word, are now checked before any text is looked up. One of the 7 was ''WWW/2025/digital-disparities''; repairing it turned up the sentence after the one quoted — "While solutions exist, their reliability is untested" — which upgraded that verdict from a hedged ''weak'' to a plain ''explicit''. |
| |
| === 7.3.7 Significance of the year-bucket movements === | === 7.3.7 Significance of the year-bucket movements === |
| no consent vocabulary anywhere in either rendering of the full text | no consent vocabulary anywhere in either rendering of the full text |
| [UNSTATED|phenomenon] WWW/2011/arrow-generating-signatures-to-detect-drive-by-downloads | [UNSTATED|phenomenon] WWW/2011/arrow-generating-signatures-to-detect-drive-by-downloads |
| "without the user"s consent' describes the drive-by download attack | the phrase "without the user" and its consent clause describe the drive-by download attack, not a crawl |
| [UNSTATED|noise ] WWW/2011/heat-seeking-honeypots-design-and-experience | [UNSTATED|noise ] WWW/2011/heat-seeking-honeypots-design-and-experience |
| one hit, 'crawling' adjacency; no consent term | one hit, 'crawling' adjacency; no consent term |
| 'Quantcast' is the top-list provider, not the CMP | 'Quantcast' is the top-list provider, not the CMP |
| [UNSTATED|phenomenon] IEEE-SP/2012/memento-learning-secrets-from-process-footprints | [UNSTATED|phenomenon] IEEE-SP/2012/memento-learning-secrets-from-process-footprints |
| "the phone owner"s consent' is an Android permission discussion | "This requires neither the phone owner" nor any permission — an Android side-channel discussion, not a web notice |
| [UNSTATED|none ] WWW/2012/branded-with-a-scarlet-c-cheaters-in-a-gaming-social-network | [UNSTATED|none ] WWW/2012/branded-with-a-scarlet-c-cheaters-in-a-gaming-social-network |
| no consent vocabulary anywhere in either rendering of the full text | no consent vocabulary anywhere in either rendering of the full text |
| 'opting-out' of fingerprinting in general; no crawl treatment | 'opting-out' of fingerprinting in general; no crawl treatment |
| [UNSTATED|noise ] NDSS/2013/the-postman-always-rings-twice-attacking-and-defending-postmessage-in-html5-webs | [UNSTATED|noise ] NDSS/2013/the-postman-always-rings-twice-attacking-and-defending-postmessage-in-html5-webs |
| one hit, Tor described as an 'overlay' network | the sole hit is the word overlay inside the URL fragment "overlay-analytics"; the apparent Tor reference is paper.txt line-wrapping "selector.js" into "selec-" and "tor.js" |
| [UNSTATED|none ] WWW/2013/i-know-the-shortened-urls-you-clicked-on-twitter-inference-attack-using-public-c | [UNSTATED|none ] WWW/2013/i-know-the-shortened-urls-you-clicked-on-twitter-inference-attack-using-public-c |
| no consent vocabulary anywhere in either rendering of the full text | no consent vocabulary anywhere in either rendering of the full text |
| one hit, "pop-up windows" in a postMessage discussion | one hit, "pop-up windows" in a postMessage discussion |
| [UNSTATED|phenomenon] PETS/2016/are-you-sure-you-want-to-contact-us-quantifying-the-leakage-of-pii-via-website-c | [UNSTATED|phenomenon] PETS/2016/are-you-sure-you-want-to-contact-us-quantifying-the-leakage-of-pii-via-website-c |
| PII leaked "without the user"s consent'; Quantcast as a top list | PII reaching third parties "without the user" knowing or consenting; Quantcast appears as a top-list provider |
| [UNSTATED|noise ] USENIX/2016/internet-jones-and-the-raiders-of-the-lost-trackers-an-archaeological-study-of-w | [UNSTATED|noise ] USENIX/2016/internet-jones-and-the-raiders-of-the-lost-trackers-an-archaeological-study-of-w |
| third-party popups as a tracking mechanism; no consent term | third-party popups as a tracking mechanism; no consent term |
| "opting to use the word distance"; "consenting Tor clients" is a hypothetical | "opting to use the word distance"; "consenting Tor clients" is a hypothetical |
| [UNSTATED|noise ] NDSS/2020/carnus-exploring-the-privacy-threats-of-browser-extension-fingerprinting | [UNSTATED|noise ] NDSS/2020/carnus-exploring-the-privacy-threats-of-browser-extension-fingerprinting |
| one hit, a "why johnny can"t opt out' reference title | one hit, the reference title "Why johnny can" not opt out, a usability evaluation of behavioural-advertising controls |
| [UNSTATED|noise ] NDSS/2020/complex-security-policy-a-longitudinal-analysis-of-deployed-content-security-policies | [UNSTATED|noise ] NDSS/2020/complex-security-policy-a-longitudinal-analysis-of-deployed-content-security-policies |
| CSP and HSTS opt-in/opt-out security mechanisms | CSP and HSTS opt-in/opt-out security mechanisms |
| "adopting HTTP" by rehosting services | "adopting HTTP" by rehosting services |
| [UNSTATED|phenomenon] NDSS/2020/uiscope-accurate-instrumentation-free-and-visible-attack-investigation-for-gui-applications | [UNSTATED|phenomenon] NDSS/2020/uiscope-accurate-instrumentation-free-and-visible-attack-investigation-for-gui-applications |
| a vulnerability opening "a pop-up window to explicitly ask for user"s consent' in the NDSS 2020 attack-investigation example | a vulnerability opening "a pop-up window to explicitly ask for" the user's consent, inside the NDSS 2020 attack-investigation example |
| [OK |explicit ] PETS/2020/missed-by-filter-lists-detecting-unknown-third-party-trackers-with-invisible-pix | [OK |explicit ] PETS/2020/missed-by-filter-lists-detecting-unknown-third-party-trackers-with-invisible-pix |
| "we did not emit any user behavior", written in explanation of cookies set before a user clicks the consent button | "we did not emit any user behavior", written in explanation of cookies set before a user clicks the consent button |
| "records trackers on the homepage of websites with no interaction (e.g., no scrolling)" - explicit, but consent notices are never mentioned | "records trackers on the homepage of websites with no interaction (e.g., no scrolling)" - explicit, but consent notices are never mentioned |
| [UNSTATED|phenomenon] NDSS/2025/cross-origin-web-attacks-via-http-2-server-push-and-signed-http-exchange | [UNSTATED|phenomenon] NDSS/2025/cross-origin-web-attacks-via-http-2-server-push-and-signed-http-exchange |
| certificate revocation "without the attacker"s consent' | the victim cannot revoke a certificate "without the attacker" agreeing — an SXG attack property, not a crawl treatment |
| [UNSTATED|noise ] NDSS/2025/duumviri-detecting-trackers-and-mixed-trackers-with-a-breakage-detector | [UNSTATED|noise ] NDSS/2025/duumviri-detecting-trackers-and-mixed-trackers-with-a-breakage-detector |
| an over-general filter rule blocks the script that loads a consent dialog - an observation about rules, not a crawl action | an over-general filter rule blocks the script that loads a consent dialog - an observation about rules, not a crawl action |
| [OK |explicit ] WWW/2025/beyond-the-crawl-unmasking-browser-fingerprinting-in-real-user-interactions | [OK |explicit ] WWW/2025/beyond-the-crawl-unmasking-browser-fingerprinting-in-real-user-interactions |
| "As we aim to study what type of user interactions trigger fingerprinting scripts, our automated crawler does not accept cookie consent banners" (the paper's separate human arm did instruct participants to accept) | "As we aim to study what type of user interactions trigger fingerprinting scripts, our automated crawler does not accept cookie consent banners" (the paper's separate human arm did instruct participants to accept) |
| [OK |weak ] WWW/2025/digital-disparities-a-comparative-web-measurement-study-across-economic-boundari | [OK |explicit ] WWW/2025/digital-disparities-a-comparative-web-measurement-study-across-economic-boundari |
| "By default, it doesn"t handle cookie banners, which may prevent full webpage rendering' - stated as a limitation of the Lighthouse-based measurement | of its Lighthouse crawl: "handle cookie banners, which may prevent full webpage rendering. While solutions exist" their "reliability is untested" — the second sentence rules out having used one |
| [UNSTATED|none ] WWW/2025/whats-in-phishers-a-longitudinal-study-of-security-configurations-in-phishing-we | [UNSTATED|none ] WWW/2025/whats-in-phishers-a-longitudinal-study-of-security-configurations-in-phishing-we |
| no consent vocabulary anywhere in either rendering of the full text | no consent vocabulary anywhere in either rendering of the full text |
| * ''bouhoula2024automated'' was one of two keys for the same paper in [[:literature:bibliography]]. The wiki-wide consolidation of 2026-09-04 (drain item ''dedup-regional-filter-lists-bibkey'') kept ''bouhoula2024_automated'' and deleted the other entry. | * ''bouhoula2024automated'' was one of two keys for the same paper in [[:literature:bibliography]]. The wiki-wide consolidation of 2026-09-04 (drain item ''dedup-regional-filter-lists-bibkey'') kept ''bouhoula2024_automated'' and deleted the other entry. |
| * 3 citation markers on [[:privacy:consent]] and 1 citation marker on this page were repointed to the kept key. No prose on either page changed, and no figure moved. Statements above that name the deleted key describe the state when they were written. Full query log and the invariants checked before saving: [[:provenance:literature:bibliography]]. | * 3 citation markers on [[:privacy:consent]] and 1 citation marker on this page were repointed to the kept key. No prose on either page changed, and no figure moved. Statements above that name the deleted key describe the state when they were written. Full query log and the invariants checked before saving: [[:provenance:literature:bibliography]]. |
| | |
| | ===== 14. Amendment, 2026-09-05: the no-interaction audit ===== |
| | |
| | ==== 14.1 The run ==== |
| | |
| | ^ ^ ^ |
| | | Date | 2026-09-05 | |
| | | Scope | audit all 313 papers labelled ''crawlConfig.consentAction = no-interaction'', and refresh every figure on [[privacy:consent]] that depends on either side of that field. §7.3 is the result. | |
| | | Corpus at the time | 5,859 extracted papers, 7 venues, 2010–2026, unchanged since the 2026-08-11 extension (''8a6b843'') | |
| | | Agent | Claude (Opus 5) — the 241-paper hand read, the audit and check scripts, the page and provenance text, the primary-source fetches in §14.4 | |
| | | Sub-agents | four reviewers (§14.3): three ''sonnet'' focused passes in parallel, then one ''fable'' generic pass | |
| | | Scripts added | ''consent_action_noninteraction_audit.mjs'', ''consent_ni_quotecheck.mjs'', ''consent_ni_significance.py'', each with its committed ''-output.txt'' | |
| | |
| | **What moved.** The headline "papers that verifiably interacted with a consent notice" went **29 → 32**. The much larger movement is the denominator: "papers that state a consent action" went **349 (31.2% of crawling papers) → 55 (4.9%)**, because 279 of the 313 ''no-interaction'' labels are the extractor's default rather than the paper's claim. Every figure listed in §14.2 was re-derived, not carried over. |
| | |
| | ==== 14.2 Figures this amendment changed ==== |
| | |
| | ^ Figure ^ Was ^ Now ^ Where ^ |
| | | papers stating a consent action | 349 (31.2%) | **55 (4.9%)**, or 63 (5.6%) counting papers that state no interaction of any kind | headline box, §7.3.3 | |
| | | verifiably interacted | 29 (2.6%) | **32 (2.9%)**; 31 on the web, 33 counting the ''partial'' | headline box, §7.3.4 | |
| | | verifiably did not act | not measured | **22 (2.0%)** | six-action table | |
| | | papers where nothing can be established | not stated | **1,057 (94.4%)** | headline box | |
| | | two-armed (accept //and// reject) designs | 14 | **16**, or 17 counting the ''partial'' | "Two arms or no claim" box | |
| | | state both a consent action and a statefulness | 115, of which 93 no-interaction | **32** audited, 20 interacted / 12 did not, 18 of them stateless | stateful/stateless box | |
| | | vantage, among papers stating an action | 349 → 68 (19.5%) EU/EEA | **55 → 37 (67.3%)** EU/EEA, 7 non-EEA only, 11 location not stated | jurisdiction section | |
| | | vantage, among papers that interacted | 29 → 21 (72.4%) | **32 → 22 (68.8%)** | jurisdiction section | |
| | | year-bucket "states an action" | 24.5 / 29.9 / 32.1 / 33.6 / 29.8 % | **0.0 / 0.0 / 3.6 / 7.2 / 9.6 %** audited | year-bucket table | |
| | |
| | The year-bucket section also lost its narrative. It previously described a peak at 33.6% in 2022–2024 falling back to 29.8% — **none of the four adjacent movements in that series is significant** (Fisher's exact, all //p// > 0.39), so the page was describing noise in a population that was also wrong. The audited series has exactly one significant step, at the GDPR (§7.3.7). |
| | |
| | ==== 14.3 Review log ==== |
| | |
| | Four reviewers, all handed the page text, both audit scripts and their unedited output, and these notes, and all told explicitly that the author's context might not be exhaustive. The three focused passes ran in parallel first. |
| | |
| | === Figures versus script (sonnet) === |
| | |
| | Re-ran all four scripts and diffed against the committed outputs: identical. Reconciled every checklist figure and every arithmetic identity (313+15+14+3+2+2+495+236+40 = 1,120; 32+22+1 = 55; 55+8 = 63; 1,120−63 = 1,057; 279+22+8+3+1 = 313). Checked ''states'' for double-counting, ''twoArm'', ''webOnly'' and the statefulness cross: all correct. |
| | |
| | ^ # ^ Finding ^ Verdict ^ |
| | | 1 | **HIGH.** The page said the 279 unsupported papers "say nothing about consent at all" / "never mention consent at all" / "make no consent claim at all" in **four** places, contradicting the page's own tag table a few lines away: only **72** of the 279 carry the ''none'' tag; the other 207 do contain consent-adjacent vocabulary, just not evidence of their own crawl's action. | **Accepted in full.** All four rewritten to "no evidence of their own crawl's consent action", with a parenthesis saying most of the 279 //do// use the word //consent// somewhere — which is how the extractor came to fill the field. This was the largest error in the published draft and the reviewer earned its slot on this finding alone. | |
| | | 2 | **LOW–MEDIUM.** "44.2% of crawling papers say nothing about consent" applies the same unverified reading to the **''not-stated''** bucket, which was not audited. | **Accepted.** The bullet now leads with the audited 94.4% and adds: the 44.2% row is the extraction's own count and was not audited, and after what this audit found, no unaudited row on the page should be read as a claim about the papers. | |
| | | 3 | **LOW, clarity.** "279 of those 313 papers never mention consent at all, and 3 of them turned out to interact" — "3 of them" can misread as 3 of the 279. | **Accepted.** Rewritten as "a further **3**". | |
| | |
| | === Citations and quotes (sonnet) === |
| | |
| | Resolved every %%{[key]}%% on both pages against the live bibliography (27 + 8 unique keys, all resolve, no duplicates, no colliding DOIs). Verified all six newly-used keys against the papers. Verified every attribution in §7.3.4 verbatim, and the author surnames used in prose. Spot-checked **27 of the 313 verdicts** — all 4 ''wrong''/''partial'', 8 ''ok'', 5 ''implied'', 10 ''unstated'' — and confirmed 26. |
| | |
| | ^ # ^ Finding ^ Verdict ^ |
| | | 1 | **MEDIUM.** ''NDSS/2013/the-postman-always-rings-twice…'' carried another paper's evidence ("Tor described as an 'overlay' network"). A **third** instance of the misattribution family §7.3.6 already documented, and it escaped the quote check because single-quoted text was exempt by design. | **Accepted in full.** Reason rewritten to what the paper actually contains, and the checker extended to verify multi-word single-quoted runs too — then mutation-tested, because that check now finds zero mentions and a zero-count check asserts nothing. §7.3.6 records both. | |
| | | 2 | Noted that three ''implied'' papers use the word //consent// in unrelated contexts, a minor imprecision in the reason wording. | **Noted, not changed.** The ''implied'' verdict turns on the paper stating no interaction //at all//, not on the absence of the word; the reasons already say so. | |
| | |
| | === External currency (sonnet) === |
| | |
| | Re-fetched every tool, standard and legal reference on both pages as of 2026-09-05. Confirmed current and unchanged: Consent-O-Matic (v1.1.5, 204 rules), BannerClick, Priv-Accept (still no default-branch activity since 2022-04-13 — and the ''pushed_at'' caveat the page carries is still needed), CookieBlock, the IDCAC MV3 fork, Ninja Cookie's abandonment (all three sub-claims), Super Agent, W3C GPC (11 June 2026 WD), DNT (2019 Note), the twelve GPC states. **No dead links on either page.** |
| | |
| | ^ # ^ Finding ^ Verdict ^ |
| | | 1 | **autoconsent has moved**: v16.23.0 (2026-08-18) → **v16.37.0** (2026-09-05), rules 571+331 → **567+356**. | **Accepted and fixed**, after re-deriving all three numbers independently from the GitHub API. The row now dates its rule count and notes fourteen releases in eighteen days, which is the point a reader pinning it needs. | |
| | | 2 | **"I don't care about cookies"** original: the page said "last updated 2023-11"; AMO's API says **2023-12-06, v3.5.0**. | **Accepted and fixed**, verified directly against ''addons.mozilla.org/api/v5''. The reviewer also reported a newer v3.5.1 on the vendor's own site; **not published**, because it could not be confirmed from the page's own text. | |
| | | 3 | EasyList Cookie List header date and the GVL version have both moved since 2026-08-19. | **Rejected as a defect.** Both are published as explicitly dated observations of a file the page tells the reader to re-fetch, which is the correct treatment for the fastest-rotting artefacts on the page. | |
| | | 4 | **MODERATE.** The Council reportedly dropped **Digital Omnibus Article 88b** — the provision that would have made a browser-level consent signal binding — on 18 June 2026. Directly relevant to this page's GPC section, and predates the page's write date. | **Not added.** See §14.4. | |
| | | 5 | California AB 566 and Cal. Code Regs. §7025(c)(6). | **Rejected for the content page.** The page deliberately does not enumerate a rotting state list and hedges with "check the current list for your crawl's date"; both facts sit inside that hedge. Recorded here so the next run does not re-litigate it. | |
| | | 6 | ''iabtechlab.com'' returns **406** to a plain ''curl'' and 200 with a browser User-Agent. | **Accepted as a note.** Added to the set of bot-gated domains this site already tracks; no page claim was affected. | |
| | |
| | ==== 14.4 What this amendment could not establish ==== |
| | |
| | * **The Digital Omnibus Article 88b removal is not on the content page**, because no primary source for it was obtained. Attempts made and their results: the EUR-Lex procedure history for CELEX ''52025PC0837'' and the EUR-Lex procedure page for 2025/0360(COD) both return 200 but neither mentions 88b; ''consilium.europa.eu'' — its public register returns **403** to a scripted request; a guessed Council document number (ST-10730-2026-INIT) turned out to be an unrelated military-mobility file. The claim is corroborated across several secondary outlets and is probably true, and it matters for this page — a binding browser-level signal would change what a crawl has to do about banners. **The next run should fetch the Council's actual position paper on 2025/0360(COD) and, if it confirms the removal, add it to the Open Questions section with the primary citation.** |
| | * **The 495 ''not-stated'' and 236 ''not-applicable'' papers were not read.** This audit closed one of the four values of ''consentAction''; three quarters of the crawling population is still unaudited in the other direction. A paper wrongly filed ''not-stated'' that does say something would move the 55 up. |
| | * **Ten other pages still carry the unaudited 349 / 31.2% figure**, two of them as direct claims about the field (the [[design:crawling_location]] 349-paper vantage paragraph, and the [[programming:stateful_stateless]] per-bucket consentAction series with its "+5.3 pp" trend). Each needs its own report script re-run and its own provenance note, so it was **filed as a separate work item** rather than half-done here. This page is the authoritative source for the audited figures until that lands. |
| | |
| | ==== 14.5 Judgement calls ==== |
| | |
| | * **A sibling script, not an extension of ''consent_action_audit.mjs''.** The obvious move was to add 313 entries to that file's existing ''VERDICTS'' map, and it was rejected. Its verdict vocabulary has three values where this side needs five (''implied'' and ''unstated'' have no meaning on the interacting side); its denominators are all "of the 36"; and its unedited output is embedded verbatim in §7.2 and cited from the content page, so growing it in place would have silently rewritten a published audit trail. Instead ''consent_action_noninteraction_audit.mjs'' **imports ''VERDICTS'' from it**, so the two halves are computed in one place and cannot drift, and carries the same fail-loudly guard. The cost is two scripts and two output files where the brief for this work expected one. |
| | * **''no-interaction'' was treated as a positive claim, not a sentinel.** The field carries ''not-stated'' and ''not-applicable'' separately, so this follows from the schema. A reader who thinks the extractor was reasonably reading "the paper describes a crawl and never mentions clicking" as no-interaction would get a very different 279. |
| | * **The ''ok''/''implied'' boundary is textual, not interpretive.** ''ok'' requires the paper to mention consent; ''implied'' requires an explicit statement of no interaction at all. The rejected alternative — "the paper describes its procedure completely enough that non-interaction follows" — is not reproducible across 313 papers and would have made the audit a matter of taste. The cost is that the content page has to publish both 55 and 63. |
| | * **Consent prompts that are not cookie banners count as ''ok''.** Two cryptomining papers state that their crawler never consented to mining; ''CCS/2018/minesweeper'' and ''USENIX/2019/inadvertently-making-cyber-criminals-rich'' are therefore ''ok'', tagged ''mining''. A stricter reading would drop the ''ok'' count from 22 to 20. |
| | * **The headline interaction count is 32, not 31 or 33.** 31 is the web-only figure and is given alongside it; 33 counts WhisperTest's app-side arm. §7.3.4 gives all three and says which is which. |
| | * **The 313 verdicts are published in full** rather than summarised, because the audit's whole value is that someone can disagree with a specific line. That is why this page is 200KB. |
| | * **The three false negatives were not re-labelled in the dataset.** ''data/extract'' is a read-only mount and the corrected values live in this audit's verdict table, not in ''extractions.jsonl''. Anyone re-deriving these figures from the raw field will get 349 and 29 again unless they import ''VERDICTS_NI''. |
| | |
| | ==== 14.6 Mistakes in this run, recorded rather than tidied away ==== |
| | |
| | - **The page's central claim was overstated four times** — "279 papers say nothing about consent at all" — while the page's own tag table three lines away showed that only 72 of them do. Caught by the figures reviewer, not by the author, despite the author having built the tag table specifically to keep that residue visible. **Printing the residue does not help if the prose next to it says something else.** |
| | - **A de-duplication patch keyed by list position instead of by paper** put three papers' evidence on the wrong papers. Two were caught by the quote checker, the third by a reviewer. None changed a verdict; all three were fabricated evidence in a file the page describes as its only human-readable evidence. |
| | - **A mechanical single-to-double quote conversion mangled 7 reasons** by treating word-internal apostrophes as delimiters, and the quote checker passed all 7 because the mangled fragment it extracted really does appear in the paper. A checker that verifies content without verifying structure will bless damaged input. |
| | - **The quote checker's first version rejected every quote crossing a column break**, because a two-column splice makes a verbatim sentence non-contiguous. Fixed with a bounded in-order match; **18 of the 129 verified quotes only pass in that mode**, so this was not a corner case. |
| | - **A recall probe's word boundary ate the plural.** ''cookie banner'' with a trailing ''%%\b%%'' does not match "cookie banners", so the probe silently reported zero hits for a paper whose methods section says it in as many words. Caught by testing the probe against a known-positive sentence, not by reading it. |
| | - **The audit script printed its whole output when imported.** ''consent_ni_quotecheck.mjs'' imports the verdict map; without the ''IS_MAIN'' guard the older audit script already carried, the checker's committed output was 700 lines of the audit's. Fixed, and the guards were then mutation-tested: deleting a verdict, adding one for a paper outside the population, and writing an invalid verdict word each fail loudly with a non-zero exit. |
| |
| ====== References ====== | ====== References ====== |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| |
