| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:privacy:consent [2026/09/05 20:17] – Section 14: run and review log for the no-interaction audit; 7.3 corrected after review (third misattribution, structural quote check, PDF re-extraction control, 21/7 requery count). Authored by Claude karel.kubicek.claude | provenance:privacy:consent [2026/09/17 07:23] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude |
|---|
| The Matte case is the one worth keeping: a percentage that looks like it came from a paper, computed by the writer from a count and a denominator taken from two different tables. Reading the source turned up **four more** figures the draft did not have — 46.5% pre-selected, 5.3% non-respect of choice, 54.29% any violation, and the 4.3% shared-cookie consent reuse. The first three went into the results table immediately; the fourth was accidentally dropped between drafts and was restored, as a box in the TCF section, only after a reviewer noticed on 2026-08-19 that this very paragraph claimed it was on the page when it was not. | The Matte case is the one worth keeping: a percentage that looks like it came from a paper, computed by the writer from a count and a denominator taken from two different tables. Reading the source turned up **four more** figures the draft did not have — 46.5% pre-selected, 5.3% non-respect of choice, 54.29% any violation, and the 4.3% shared-cookie consent reuse. The first three went into the results table immediately; the fourth was accidentally dropped between drafts and was restored, as a box in the TCF section, only after a reviewer noticed on 2026-08-19 that this very paragraph claimed it was on the page when it was not. |
| |
| ==== 7.2 The ''consentAction'' audit, and why the standard quote check could not have caught it ==== | ==== 7.2 The consentAction audit, and why the standard quote check could not have caught it ==== |
| |
| The page's central figure is "how many papers interacted with a consent notice". The extraction says **36**. Reading all 36 against their own text says **29**. That is a **19.4% false-positive rate** on the interacting values of ''crawlConfig.consentAction'', and it is the single most important thing on this provenance page. | The page's central figure is "how many papers interacted with a consent notice". The extraction says **36**. Reading all 36 against their own text says **29**. That is a **19.4% false-positive rate** on the interacting values of ''crawlConfig.consentAction'', and it is the single most important thing on this provenance page. |
| **Why the site's usual check is blind here.** The dataset's guidance is to read a handful of ''evidence.quote'' values behind any figure before publishing it. That check **cannot work for this field**. ''crawlConfig'' is an object, not an array, and it carries **one** evidence quote for the **whole** object — statefulness, browsers, headless, consent action, interaction depth, authentication, languages and repeat visits all share it. So the quote behind a ''consentAction'' value usually evidences something else entirely. Read literally, the quotes for the 36 papers include "we create a separate docker container for each URL we visit", "each crawl within a run is performed once without repetition" and "we visit each website afresh with storage cleared" — none of which says anything about a banner. One of them, for Trevisan et al., says the opposite of its label: **"No user action is performed on the page"** sits under a ''accept-and-reject'' value. | **Why the site's usual check is blind here.** The dataset's guidance is to read a handful of ''evidence.quote'' values behind any figure before publishing it. That check **cannot work for this field**. ''crawlConfig'' is an object, not an array, and it carries **one** evidence quote for the **whole** object — statefulness, browsers, headless, consent action, interaction depth, authentication, languages and repeat visits all share it. So the quote behind a ''consentAction'' value usually evidences something else entirely. Read literally, the quotes for the 36 papers include "we create a separate docker container for each URL we visit", "each crawl within a run is performed once without repetition" and "we visit each website afresh with storage cleared" — none of which says anything about a banner. One of them, for Trevisan et al., says the opposite of its label: **"No user action is performed on the page"** sits under a ''accept-and-reject'' value. |
| |
| Doing the check properly meant a different method: extract every sentence in each paper containing a consent/banner term **and** a first-person marker, read them, and record a verdict. ''consent_action_audit.mjs --sentences'' reproduces the sentences; the verdicts are hard-coded in that file with a one-line reason each, and the script **fails loudly** if a paper claims an interaction and has no verdict, or if a verdict refers to a paper that no longer claims one. That guard caught three truncated slugs in the first version of the verdict table. | Doing the check properly meant a different method: extract every sentence in each paper containing a consent/banner term **and** a first-person marker, read them, and record a verdict. ''consent_action_audit.mjs %%--sentences%%'' reproduces the sentences; the verdicts are hard-coded in that file with a one-line reason each, and the script **fails loudly** if a paper claims an interaction and has no verdict, or if a verdict refers to a paper that no longer claims one. That guard caught three truncated slugs in the first version of the verdict table. |
| |
| ^ Verdict ^ Papers ^ Share of 36 ^ | ^ Verdict ^ Papers ^ Share of 36 ^ |
| </code> | </code> |
| |
| ==== 7.3 The ''no-interaction'' audit ==== | ==== 7.3 The no-interaction audit ==== |
| |
| Run 2026-09-05. §7.2 audited the 36 papers claiming an interaction and left the complementary 313 unread, which meant the page's central denominator rested on an unaudited field. All 313 have now been read. | Run 2026-09-05. §7.2 audited the 36 papers claiming an interaction and left the complementary 313 unread, which meant the page's central denominator rested on an unaudited field. All 313 have now been read. |
| - **Quotation-mark artefacts** from a mechanical conversion that promoted apostrophes inside words to quote delimiters, producing fragments such as "s subject is the browser". | - **Quotation-mark artefacts** from a mechanical conversion that promoted apostrophes inside words to quote delimiters, producing fragments such as "s subject is the browser". |
| |
| A **mutation self-test** (''--mutate'') corrupts the middle word of every quote of four words or more and requires a FAIL: **82 corrupted quotes, 82 rejected, 0 wrongly accepted.** | A **mutation self-test** (''%%--mutate%%'') corrupts the middle word of every quote of four words or more and requires a FAIL: **82 corrupted quotes, 82 rejected, 0 wrongly accepted.** |
| |
| **Then it missed a third misattribution, and a reviewer found it.** ''NDSS/2013/the-postman-always-rings-twice…'' carried the reason "one hit, Tor described as an 'overlay' network", which is the evidence of ''PETS/2022/from-onion-not-found-to-guard-discovery'' — the postMessage paper never mentions Tor. Its only //overlay// is inside the URL fragment ''overlay-analytics'', and the apparent "tor.js" is ''paper.txt'' line-wrapping ''selector.js'' into ''selec-'' and ''tor.js''. The verdict (''unstated'') was right; the evidence was another paper's. **It escaped because the checker's own convention exempted single-quoted text as "a word being mentioned"** — and this single-quoted run was a substantive claim, not a mention. The checker now verifies **every single-quoted run of three words or more** as well, excluding word-internal apostrophes and spans that contain a double quote. That check currently finds zero mentions to verify, which is a vacuous pass, so it was mutation-tested directly: injecting a fabricated ''%%'Tor is an overlay network of volunteer-run relays'%%'' into that same verdict produces ''MENTION NOT IN PAPER''. | **Then it missed a third misattribution, and a reviewer found it.** ''NDSS/2013/the-postman-always-rings-twice…'' carried the reason "one hit, Tor described as an 'overlay' network", which is the evidence of ''PETS/2022/from-onion-not-found-to-guard-discovery'' — the postMessage paper never mentions Tor. Its only //overlay// is inside the URL fragment ''overlay-analytics'', and the apparent "tor.js" is ''paper.txt'' line-wrapping ''selector.js'' into ''selec-'' and ''tor.js''. The verdict (''unstated'') was right; the evidence was another paper's. **It escaped because the checker's own convention exempted single-quoted text as "a word being mentioned"** — and this single-quoted run was a substantive claim, not a mention. The checker now verifies **every single-quoted run of three words or more** as well, excluding word-internal apostrophes and spans that contain a double quote. That check currently finds zero mentions to verify, which is a vacuous pass, so it was mutation-tested directly: injecting a fabricated ''%%'Tor is an overlay network of volunteer-run relays'%%'' into that same verdict produces ''MENTION NOT IN PAPER''. |
| * 3 citation markers on [[:privacy:consent]] and 1 citation marker on this page were repointed to the kept key. No prose on either page changed, and no figure moved. Statements above that name the deleted key describe the state when they were written. Full query log and the invariants checked before saving: [[:provenance:literature:bibliography]]. | * 3 citation markers on [[:privacy:consent]] and 1 citation marker on this page were repointed to the kept key. No prose on either page changed, and no figure moved. Statements above that name the deleted key describe the state when they were written. Full query log and the invariants checked before saving: [[:provenance:literature:bibliography]]. |
| |
| ===== 14. Amendment, 2026-09-05: the ''no-interaction'' audit ===== | ===== 14. Amendment, 2026-09-05: the no-interaction audit ===== |
| |
| ==== 14.1 The run ==== | ==== 14.1 The run ==== |
| ==== 14.5 Judgement calls ==== | ==== 14.5 Judgement calls ==== |
| |
| | * **A sibling script, not an extension of ''consent_action_audit.mjs''.** The obvious move was to add 313 entries to that file's existing ''VERDICTS'' map, and it was rejected. Its verdict vocabulary has three values where this side needs five (''implied'' and ''unstated'' have no meaning on the interacting side); its denominators are all "of the 36"; and its unedited output is embedded verbatim in §7.2 and cited from the content page, so growing it in place would have silently rewritten a published audit trail. Instead ''consent_action_noninteraction_audit.mjs'' **imports ''VERDICTS'' from it**, so the two halves are computed in one place and cannot drift, and carries the same fail-loudly guard. The cost is two scripts and two output files where the brief for this work expected one. |
| * **''no-interaction'' was treated as a positive claim, not a sentinel.** The field carries ''not-stated'' and ''not-applicable'' separately, so this follows from the schema. A reader who thinks the extractor was reasonably reading "the paper describes a crawl and never mentions clicking" as no-interaction would get a very different 279. | * **''no-interaction'' was treated as a positive claim, not a sentinel.** The field carries ''not-stated'' and ''not-applicable'' separately, so this follows from the schema. A reader who thinks the extractor was reasonably reading "the paper describes a crawl and never mentions clicking" as no-interaction would get a very different 279. |
| * **The ''ok''/''implied'' boundary is textual, not interpretive.** ''ok'' requires the paper to mention consent; ''implied'' requires an explicit statement of no interaction at all. The rejected alternative — "the paper describes its procedure completely enough that non-interaction follows" — is not reproducible across 313 papers and would have made the audit a matter of taste. The cost is that the content page has to publish both 55 and 63. | * **The ''ok''/''implied'' boundary is textual, not interpretive.** ''ok'' requires the paper to mention consent; ''implied'' requires an explicit statement of no interaction at all. The rejected alternative — "the paper describes its procedure completely enough that non-interaction follows" — is not reproducible across 313 papers and would have made the audit a matter of taste. The cost is that the content page has to publish both 55 and 63. |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| |
