| Next revision | Previous revision |
| provenance:programming:stateful_stateless [2026/08/19 11:32] – Create the provenance log for Programming:Stateful stateless: populations and denominators, every query with unedited script output, the two folding rules and their full residue, the shared-evidence-quote validity problem and the two checks built for it ( karel.kubicek.claude | provenance:programming:stateful_stateless [2026/08/19 12:02] (current) – Add the re-review pass (figures and citations re-run against the final text: 0 wrong), record the author's own independent checks of the two new tables and the published code, correct this log's own mischaracterisation of a GitHub reporter comment as a ma karel.kubicek.claude |
|---|
| | Date | 2026-08-19 | | | Date | 2026-08-19 | |
| | Corpus | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P), 2010–2026 | | | Corpus | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P), 2010–2026 | |
| | Page before | 4,479 bytes of notes (rev ''1742374572''), a key-message pair and a short reading list, marked ''<wrap todo>This page only contains notes</wrap>'' | | | Page before | 4,479 bytes of notes (rev ''1742374572''): a two-bullet key message, a short reading list, a "Shallow vs Deep crawling" section, the unedited page template comment, and a todo box reading "This page only contains notes" | |
| | Page after | 46,218 bytes (rev ''1787137126'') | | | Page after | 60,763 bytes (rev ''1787140910''), after three rounds of review fixes plus a re-review pass | |
| | Decision | **Extend, not create.** The title is already the right scope for the design question. The page was a stub; nothing was broadened or narrowed. Overlap with neighbours was deliberate and is recorded under [[#Judgement calls]]. | | | Decision | **Extend, not create.** The title is already the right scope for the design question. The page was a stub; nothing was broadened or narrowed. Overlap with neighbours was deliberate and is recorded under [[#Judgement calls]]. | |
| | Who | Claude (Opus 5) end to end, with four review sub-agents (see [[#Review]]) | | | Who | Claude (Opus 5) end to end, with four review sub-agents (see [[#Review]]) | |
| | | Neighbours patched | ''design:crawling_location'' (false Jueckstock claim, rev ''1787137202''), ''privacy:cookies'' (third-party cookies were not discontinued, rev ''1787140005''), ''programming:crawler:openwpm'' (stale "notes stub" pointer, rev ''1787139249''), ''literature:bibliography'' (duplicate key removed, rev ''1787138947'') | |
| | Scripts added | ''scripts/report_stateful_stateless.mjs'', ''scripts/statefulness_probe.mjs'', ''scripts/statefulness_audit.mjs'', ''scripts/state_quotecheck.mjs'', ''scripts/pdf_grep.py'', ''scripts/state_probe/{server,probe,thirdparty,launch_args}.mjs'' | | | Scripts added | ''scripts/report_stateful_stateless.mjs'', ''scripts/statefulness_probe.mjs'', ''scripts/statefulness_audit.mjs'', ''scripts/state_quotecheck.mjs'', ''scripts/pdf_grep.py'', ''scripts/state_probe/{server,probe,thirdparty,launch_args}.mjs'' | |
| | Bibliography | 13 new keys added in two saves (revs ''1787136793'' and ''1787136845''), one of them **later deleted as a duplicate of an existing key** (rev ''1787138947''); see reviewer 3, finding 1 | | | Bibliography | 13 new keys added in two saves (revs ''1787136793'' and ''1787136845''), one of them **later deleted as a duplicate of an existing key** (rev ''1787138947''); see reviewer 3, finding 1 | |
| |
| <file text stateful_stateless_report.txt> | <file text stateful_stateless_report.txt> |
| claimed an interaction: 36 | |
| adjudicated: 36 | |
| |
| ── Verdicts ── | |
| Verdict Papers Share of the 36 | |
| ------------------------------------------ ------ --------------- | |
| supported by the paper text 28 77.8% | |
| supported but the enum value overstates it 1 2.8% | |
| NOT supported — extraction false positive 7 19.4% | |
| |
| FALSE-POSITIVE RATE of crawlConfig.consentAction on the interacting values: 7/36 = 19.4% | |
| |
| ── Figures recomputed on the AUDITED set ── | |
| papers that ran a crawl: 1120 | |
| state a consent action (incl. no-interaction): 349 (31.2%) | |
| ... of which no-interaction: 313 | |
| ... claim an interaction (extraction): 36 (3.2% of crawled) | |
| ... interaction VERIFIED in the paper: 29 (2.6% of crawled, 8.3% of those stating) | |
| |
| ── Audited interaction by enum value ── | |
| Value Claimed Verified False positives | |
| -------------------- ------- -------- --------------- | |
| accept-all 15 11 4 | |
| accept-and-reject 14 14 0 | |
| cmp-specific-choices 2 2 0 | |
| dismiss-or-remove 3 0 3 | |
| reject-all 2 2 0 | |
| |
| ── Audited interaction by year bucket ── | |
| Bucket Crawling papers Claimed Verified Share of bucket | |
| ---------- --------------- ------- -------- --------------- | |
| 2010–2013 102 0 0 0.0% | |
| 2014–2017 167 0 0 0.0% | |
| 2018–2021 308 6 3 1.0% | |
| 2022–2024 345 17 14 4.1% | |
| 2025–2026* 198 13 12 6.1% | |
| |
| ── Vantage point of the VERIFIED interacting papers ── | |
| Vantage Papers Share of 29 | |
| ---------------------------------- ------ ----------- | |
| EU/EEA vantage 21 72.4% | |
| vantage tuple, location not-stated 5 17.2% | |
| stated a non-EU/EEA vantage only 3 10.3% | |
| |
| ── Every verdict, in full ── | |
| [WRONG ] CCS/2018/pride-and-prejudice-in-progressive-web-apps-abusing-native-app-like-features-in | |
| claimed: accept-all | |
| the only consent in the paper is the PUSH NOTIFICATION permission prompt | |
| [WRONG ] WWW/2018/hiding-in-the-crowd-an-analysis-of-the-effectiveness-of-browser-fingerprinting-a | |
| claimed: accept-all | |
| no sentence in the paper mentions a consent notice or banner at all | |
| [PARTIAL] PETS/2019/4-years-of-eu-cookie-law-results-and-lessons-learned | |
| claimed: accept-and-reject | |
| main crawl states "No user action is performed on the page"; a separate experiment gives consent, so an ACCEPT arm exists but no reject arm is described | |
| [WRONG ] IMC/2020/when-push-comes-to-ads-measuring-the-rise-of-malicious-push-advertising | |
| claimed: accept-all | |
| "banner" in this paper means BANNER ADS; no consent interaction described | |
| [OK ] IEEE-SP/2020/do-cookie-banners-respect-my-choice-measuring-legal-compliance-of-banners-from-i | |
| claimed: accept-and-reject | |
| semi-automatic crawl clicking through banners on 560 sites | |
| [OK ] WWW/2021/user-tracking-in-the-post-cookie-era-how-websites-bypass-gdpr-consent-to-track-u | |
| claimed: accept-and-reject | |
| "we leverage the Consent-O-matic tool"; three crawls, one per consent action | |
| [OK ] PETS/2022/how-can-and-would-people-protect-from-online-tracking | |
| claimed: accept-and-reject | |
| "On the second visit, we accept the cookies... on the next visit, we try to opt-out" | |
| [OK ] USENIX/2022/automating-cookie-consent-and-gdpr-violation-detection | |
| claimed: accept-and-reject | |
| crawler consents to all purposes and separately denies consent | |
| [OK ] USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission | |
| claimed: accept-and-reject | |
| "three consent modes... accept all, reject all, and no action" | |
| [OK ] IMC/2023/the-prevalence-of-single-sign-on-on-the-web-towards-the-next-generation-of-web-c | |
| claimed: accept-all | |
| "We use a plugin to auto-accept cookie banners" | |
| [WRONG ] IMC/2023/understanding-the-privacy-risks-of-popular-search-engine-advertising-systems | |
| claimed: accept-all | |
| no first-person sentence mentions consent or a banner | |
| [WRONG ] PETS/2023/a-utility-preserving-obfuscation-approach-for-youtube-recommendations | |
| claimed: dismiss-or-remove | |
| the only "consent" is IRB participant consent for the user dataset | |
| [OK ] IEEE-SP/2023/the-leaky-web-automated-discovery-of-cross-site-information-leaks-in-browsers-an | |
| claimed: accept-all | |
| "the cookie banners accepted by our module" | |
| [WRONG ] IMC/2024/analyzing-the-in-accessibility-of-online-advertisements | |
| claimed: dismiss-or-remove | |
| the only "consent" is IRB informed consent; "banner" means banner ads | |
| [OK ] USENIX/2024/automated-large-scale-analysis-of-cookie-notice-compliance | |
| claimed: accept-and-reject | |
| five crawling steps: accepting, rejecting, closing, saving defaults, not interacting | |
| [OK ] NDSS/2024/fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting | |
| claimed: accept-all | |
| "...solving CAPTCHAs, and consenting to all cookie notices" (a manual sub-crawl) | |
| [OK ] USENIX/2024/dissecting-privacy-perspectives-of-websites-around-the-world-aceptar-todo-alle-a | |
| claimed: cmp-specific-choices | |
| measures "depth to reach the reject option", i.e. navigates the notice's own layers | |
| [OK ] PETS/2024/a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users | |
| claimed: accept-and-reject | |
| the paper IS a comparison of banner-interaction extensions | |
| [OK ] PETS/2024/fp-tracer-fine-grained-browser-fingerprinting-detection-via-taint-tracking-and-e | |
| claimed: accept-and-reject | |
| "we measure whether fingerprinters respect user consent banners using the Consent-O-Matic plugin" | |
| [OK ] PETS/2024/opted-out-yet-tracked-are-regulations-enough-to-protect-your-privacy | |
| claimed: accept-and-reject | |
| audits four CMPs with consent conveyed and not conveyed | |
| [OK ] WWW/2024/a-study-of-gdpr-compliance-under-the-transparency-and-consent-framework | |
| claimed: reject-all | |
| "accepted the automatic decline of user consent by our data collection system" | |
| [OK ] WWW/2024/the-double-edged-sword-identifying-authentication-pages-and-their-fingerprinting | |
| claimed: accept-all | |
| "To automatically interact with cookie consent banners, we integrated code derived from Priv-Accept" | |
| [OK ] IEEE-SP/2024/targeted-and-troublesome-tracking-and-advertising-on-childrens-websites | |
| claimed: accept-all | |
| "provide affirmative consent to all data processing request options (accept all)" via autoconsent | |
| [OK ] CCS/2025/piixel-leaks-passive-identification-of-personally-identifiable-information-leaka | |
| claimed: accept-all | |
| "we simulate a real user's choice of 'Accept All' using the Consent-O-Matic extension" | |
| [OK ] IMC/2025/canvassing-the-fingerprinters-characterizing-canvas-fingerprinting-use-across-th | |
| claimed: accept-all | |
| "uses the autoconsent library to opt-in to common consent banners" | |
| [OK ] PETS/2025/johnny-can-t-revoke-consent-either-measuring-compliance-of-consent-revocation-on | |
| claimed: accept-and-reject | |
| the paper accepts and then revokes consent on 200 sites | |
| [OK ] PETS/2025/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and | |
| claimed: accept-and-reject | |
| "using BannerClick... accepting cookie banners in the first half and... rejected domains in the second" | |
| [OK ] PETS/2025/referrer-policy-implementation-and-circumvention | |
| claimed: accept-and-reject | |
| "we employed DuckDuckGo's autoconsent library... comparison of RP implementations in different consent modes" | |
| [OK ] WWW/2025/before-after-the-effect-of-eus-2022-code-of-practice-on-disinformation | |
| claimed: accept-all | |
| "we make use of Consent-O-Matic ... to automatically accept all cookies in consent banners" | |
| [OK ] USENIX/2025/navigating-cookie-consent-violations-across-the-globe | |
| claimed: reject-all | |
| ConsentChk drives the banner across eight regions | |
| [OK ] WWW/2025/semantics-aware-cookie-purpose-compliance | |
| claimed: accept-all | |
| "We use two browser extensions, namely Consent-O-Matic and Cookie-Editor" | |
| [OK ] WWW/2025/the-first-early-evidence-of-the-use-of-browser-fingerprinting-for-online-trackin | |
| claimed: cmp-specific-choices | |
| "executing window.OneTrust.RejectAll to opt out"; per-CMP reject clicking | |
| [WRONG ] PETS/2026/the-masks-we-think-we-wear-privacy-threats-of-browser-extension-wallets-in-the-w | |
| claimed: dismiss-or-remove | |
| "explicit consent" is a RECOMMENDATION about wallet permissions, not a crawl action | |
| [OK ] PETS/2026/clicking-into-exposure-uncovering-privacy-risks-of-google-click-identifier-in-yo | |
| claimed: accept-and-reject | |
| "(i) a baseline run that accepts cookie banners... (ii) a rejection run that declines them" | |
| [OK ] PETS/2026/privacy-vs-profit-the-impact-of-googles-manifest-version-3-mv3-update-on-ad-bloc | |
| claimed: accept-all | |
| "Accept consent notice via Super Agent-Automatic Cookie Consent extension" | |
| [OK ] USENIX/2026/bridges-to-self-silent-web-to-app-tracking-on-mobile-via-localhost | |
| claimed: accept-all | |
| "We modify the crawler to automatically accept cookies using Priv-Accept" | |
| corpus: 5859 papers | corpus: 5859 papers |
| population 'crawled' (crawlConfig object present OR studyTypes includes automated-web-crawl): 1120 | population 'crawled' (crawlConfig object present OR studyTypes includes automated-web-crawl): 1120 |
| 2022–2024 71/345 = 20.6% 1270/1955 = 65.0% 681/1649 = 41.3% | 2022–2024 71/345 = 20.6% 1270/1955 = 65.0% 681/1649 = 41.3% |
| 2025–2026* 36/198 = 18.2% 907/1185 = 76.5% 467/1019 = 45.8% | 2025–2026* 36/198 = 18.2% 907/1185 = 76.5% 467/1019 = 45.8% |
| | |
| | |
| | === Every crawl-configuration field, reporting rate per bucket (of the crawling papers in each bucket) === |
| | |
| | Field 2010–2013 2014–2017 2018–2021 2022–2024 2025–2026* max-min last - first |
| | -------------------- -------------- --------------- --------------- --------------- --------------- ------- ------------ |
| | interactionDepth 80/102 = 78.4% 126/167 = 75.4% 237/308 = 76.9% 255/345 = 73.9% 143/198 = 72.2% 6.2 pp -6.2 pp |
| | authentication 61/102 = 59.8% 117/167 = 70.1% 213/308 = 69.2% 251/345 = 72.8% 137/198 = 69.2% 12.9 pp +9.4 pp |
| | browsers (>=1 named) 33/102 = 32.4% 81/167 = 48.5% 153/308 = 49.7% 162/345 = 47.0% 100/198 = 50.5% 18.2 pp +18.2 pp |
| | consentAction 25/102 = 24.5% 50/167 = 29.9% 99/308 = 32.1% 116/345 = 33.6% 59/198 = 29.8% 9.1 pp +5.3 pp |
| | **statefulness** 16/102 = 15.7% 35/167 = 21.0% 61/308 = 19.8% 71/345 = 20.6% 36/198 = 18.2% 5.3 pp +2.5 pp |
| | headless 1/102 = 1.0% 26/167 = 15.6% 41/308 = 13.3% 50/345 = 14.5% 22/198 = 11.1% 14.6 pp +10.1 pp |
| | |
| | The last two columns are the test of the page's claim: which field moved, and by how much. |
| |
| |
| Whitespace, quote characters, dashes and ligatures are normalised; nothing else is. Each quote is tried against ''paper.cols.txt'', then ''paper.norm.txt'', then — because both text renderings splice two-column text — against ''paper.pdf'' itself via ''scripts/pdf_grep.py'' (pypdf 6.16.1, de-hyphenated). The rendering that matched is printed, because **a quote that only the PDF has is a quote whose ''.cols'' text is spliced**, and that is worth knowing. | Whitespace, quote characters, dashes and ligatures are normalised; nothing else is. Each quote is tried against ''paper.cols.txt'', then ''paper.norm.txt'', then — because both text renderings splice two-column text — against ''paper.pdf'' itself via ''scripts/pdf_grep.py'' (pypdf 6.16.1, de-hyphenated). The rendering that matched is printed, because **a quote that only the PDF has is a quote whose ''.cols'' text is spliced**, and that is worth knowing. |
| |
| **Result: 29 of 29 pass.** Four passed only against the PDF: | **Result: 37 of 37 pass** (29 when the checker was first written; eight more were added with the comparison-studies table and the tightened denominators). Four passed only against the PDF: |
| |
| ^ Quote ^ Paper ^ | ^ Quote ^ Paper ^ |
| | 4 | Everything else — Mozilla TCP, Chrome 115 storage partitioning, both Privacy Sandbox posts (no 2026 reversal), Incognito third-party blocking, CHIPS/FedCM, GPC now on the W3C Privacy WG track, Playwright 1.62.1 being the actual latest, Chromium 151.0.7922.34 being its pinned build, pypdf 6.16.1 current, OpenWPM actively maintained with ''b9dd4c3a'' being current HEAD and all three code claims byte-for-byte — verified correct. | — | Noted; no change needed. | | | 4 | Everything else — Mozilla TCP, Chrome 115 storage partitioning, both Privacy Sandbox posts (no 2026 reversal), Incognito third-party blocking, CHIPS/FedCM, GPC now on the W3C Privacy WG track, Playwright 1.62.1 being the actual latest, Chromium 151.0.7922.34 being its pinned build, pypdf 6.16.1 current, OpenWPM actively maintained with ''b9dd4c3a'' being current HEAD and all three code claims byte-for-byte — verified correct. | — | Noted; no change needed. | |
| |
| **On finding 3, the reviewer was partly wrong and it was checked before publishing.** The GitHub API says issue 38455 was closed with ''state_reason: completed'', not "as infeasible", and the Chromium bug involved is ''crbug.com/468317746'', not the ''crbug.com/410491202'' the reviewer cited. Reading the actual thread gave a better and more on-topic story than the reviewer's summary: Playwright disables partitioning **because its own ''storageState'' API cannot represent partitioned storage** — "Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now" — and issue 38455 was closed after the Chromium-side request for bulk storage-key APIs was declined as "infeasible — too far outside of the product scope". The "flag will be removed" expectation is the //requester's// remark in that thread, not a cited Chromium roadmap, and the page now says so in those terms. This is the run's clearest illustration of the standing rule that a sub-agent's findings are leads, not facts. | **On finding 3, the reviewer was partly wrong and it was checked before publishing.** The GitHub API says issue 38455 was closed with ''state_reason: completed'', not "as infeasible", and the Chromium bug involved is ''crbug.com/468317746'', not the ''crbug.com/410491202'' the reviewer cited. Reading the actual thread gave a better and more on-topic story than the reviewer's summary: Playwright disables partitioning **because its own ''storageState'' API cannot represent partitioned storage** — "Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now" — and issue 38455 was closed after the Chromium-side request for bulk storage-key APIs was declined as "infeasible - too far outside of the product scope". The "flag will be removed" expectation is the //requester's// remark in that thread, not a cited Chromium roadmap, and the page now says so in those terms. This is the run's clearest illustration of the standing rule that a sub-agent's findings are leads, not facts. |
| |
| ==== Reviewer 3 — citations and quotations (''model: sonnet'') ==== | ==== Reviewer 3 — citations and quotations (''model: sonnet'') ==== |
| * **Citations (sonnet)** — yes, and it found the single most damaging defect of the run, a duplicate bibliography key that a naive key-equality check had passed. It also caught a wrong date range that had been copied from nowhere. | * **Citations (sonnet)** — yes, and it found the single most damaging defect of the run, a duplicate bibliography key that a naive key-equality check had passed. It also caught a wrong date range that had been copied from nowhere. |
| * **Currency (sonnet)** — yes on coverage: it fetched and confirmed 15-odd external claims that would otherwise rest on recall, and it found the "flag is going away" story. But its summary of that story was **wrong in two specifics** (closure reason and Chromium bug number), and using it verbatim would have put two false facts on the page. Its value is as a lead generator, not as a source. | * **Currency (sonnet)** — yes on coverage: it fetched and confirmed 15-odd external claims that would otherwise rest on recall, and it found the "flag is going away" story. But its summary of that story was **wrong in two specifics** (closure reason and Chromium bug number), and using it verbatim would have put two false facts on the page. Its value is as a lead generator, not as a source. |
| * **Generic (fable)** — see below. | * **Generic (fable)** — see [[#Reviewer 4 — generic, no checklist (model: fable)|reviewer 4]]; it was the most valuable of the four. |
| | |
| | ==== Reviewer 4 — generic, no checklist (''model: fable'') ==== |
| | |
| | Given both pages, the script outputs, and the neighbouring pages, with no list of things to look for. |
| | |
| | ^ # ^ Finding ^ Severity ^ Verdict ^ |
| | | 1 | **The page's first substantive sentence was false.** "a crawl that … visits each target once cannot see cookie syncing … because none of those exist without accumulated state" — tracker A can set an ID and sync it to B in the same page load, and the page's own Zeber quote says a fresh profile "would be a clear target for cookie syncing", i.e. fresh profiles over-trigger it. | HIGH | **Accepted in full.** The reviewer is right and the error was self-refuting: the page quoted the evidence against its own claim two sections later. Both the intro and the "what each design can measure" row now say that a stateless crawl sees //first-contact// syncing and over-triggers it, and that what accumulation buys is the aged identity and the sync graph — with {[englehardt2016online]} running its sync analysis on the //stateful// 100k crawl as the supporting example. | |
| | | 2 | **"the only crawl-configuration reporting rate in this corpus with no trend at all"** was never computed. ''report_stateful_stateless.mjs'' computed bucket trends for statefulness and for two outside norms, and snapshot rates for the other configuration fields — not their trends. These notes made it worse by claiming the superlative was one "which the report script does check". | HIGH | **Accepted, and fixed by computing it rather than by deleting it.** The script now prints every crawl-configuration field's reporting rate per bucket with max−min and last−first columns. The claim survives in a narrower and now-checkable form: statefulness has the narrowest range (5.3 pp) and the smallest first-to-last change (+2.5 pp) of the six fields, against +18.2 pp for naming a browser and +10.1 pp for headless. Interaction depth moved less in one sense and moved //down//. The page carries the table. | |
| | | 3 | The "complete" published script was **not runnable as published**: the file blocks were named ''state_probe_server.mjs'' / ''state_probe.mjs'' while the code does ''import { startServer } from './server.mjs''', so saving under the page's names gives ''ERR_MODULE_NOT_FOUND''. Same defect class as reviewer 1's finding, reintroduced by the fix for it. | HIGH | **Accepted.** Blocks renamed ''server.mjs'' and ''probe.mjs'', matching the repo, with a sentence telling the reader to save them side by side. | |
| | | 4 | **A neighbouring page contradicted this page's centrepiece currency claim.** [[privacy:cookies]] opened "This trend persists despite the discontinuation of third-party cookies", which this page verified against Google's own announcement did not happen. | HIGH | **Accepted.** [[privacy:cookies]] patched in the same sitting (rev ''1787140005'') with the primary source, the current Chrome/Firefox/Safari positions, and a pointer to this page's engine-defaults section — the same treatment ''design:crawling_location'' got earlier in the run. | |
| | | 5 | These notes said "Result: 29 of 29 pass" and "all 29 quotations" while the embedded checker output said 37. | MEDIUM | **Accepted.** The prose predated eight quotes added with the comparison-studies table. Now 37, and the count is read from the embedded output rather than restated. | |
| | | 6 | These notes promised "four review sub-agents (see #Review)" and documented three, with "**Generic (fable)** — see below" followed by nothing. | MEDIUM | **Accepted.** This section is that slot. | |
| | | 7 | "Every number here is produced by ''scripts/report_stateful_stateless.mjs''" overcommitted: the 16/10/3 audit table comes from ''statefulness_audit.mjs'' and the 178/41/72 corroboration counts from ''statefulness_probe.mjs''. | MEDIUM | **Accepted.** All three are named on the page. | |
| | | 8 | The "real, unedited output" of the report script **began with about a hundred lines of another page's audit**, because ''consent_action_audit.mjs'' prints at module top level and this report imports its ''VERDICTS''. | MEDIUM | **Accepted, and fixed in the code rather than papered over.** ''consent_action_audit.mjs'' now guards its printing behind an is-main check. Its own direct-run output was diffed before and after and is **byte-identical** to the committed ''scripts/consent_action_audit-output.txt'', so [[privacy:consent]]'s audit trail is unaffected. The report output embedded above was regenerated. | |
| | | 9 | "accumulates an unpartitioned cross-site profile that resembles a default Chrome user's" overstates: Chrome has partitioned third-party //storage// since 115, so the resemblance is to the cookie jar only. | MEDIUM | **Accepted.** Split into cookie jar (resembles) and storage (does not). | |
| | | 10 | Five nits: "the 28 that survived that audit" (29 survived; 28 were //fully// supported); "43.8% to about 55–59%" when the last bucket is 52.8%; "as of January 2026 the maintainers had no replacement design" when the cited footnote dates stop at 2025-12-22; the Flash respawning figures not dated as historical; no Puppeteer recipe in "How to do it" despite Puppeteer having 76 corpus papers to Playwright's 34. | NIT | **All accepted.** The January 2026 claim was kept but re-evidenced: the thread carries a **maintainer** comment of 2026-01-06 asking the reporter for a design that keeps ''storageState'' working, and a **reporter** reply of 2026-01-14. (The re-review caught this sentence describing both as maintainer comments; corrected here.) The page says only that "as late as January 2026 the maintainers were still asking the reporter for a design", which the 6 January comment supports on its own. Flash is dated with its 2020 end of life and the modern successors. A Puppeteer ''userDataDir'' line is added, with the note that it has no ''storageState'' equivalent. | |
| | | 11 | "If you are writing for PETS, the reviewers are used to seeing it" turns a reporting rate into a claim about reviewer expectations. | NIT | **Accepted** — cut to "PETS is where this reporting norm is strongest", which is what the data says. | |
| | | 12 | ''programming:crawler:openwpm'' still carries an "OpenWPM 0.35.0 (Firefox 152)" example while current stable Firefox is 154. | NIT | **Rejected as out of scope, and recorded here instead.** That is a factual staleness on another page's own illustrative example, not a claim this page makes or relies on, and editing another page's examples is scope creep. Whoever next touches that page should fix it. (The stale "Programming:Stateful stateless — currently a notes stub" pointer on the same page //was// fixed, rev ''1787139249'', because it was a false statement about //this// page.) | |
| | |
| | Its clean checks, recorded because they bound what remains in doubt: it re-derived the ratios 51.6%, 57.3%, 1.61×, 2.51×, 2.9×, 5.9× and 55.7% and all hold; it verified that the suspicious-looking 6.91× is verbatim in Rasaii et al. (which uses both "6.9 times" and "6.91 times"); and it independently recomputed the five OpenWPM papers labelled ''both'' and found all five among the 16 adjudicated-ok, so [[programming:crawler:openwpm]]'s "those five are the ones to read" survives this page's audit. It judged the voice consistent with the neighbours and the boundaries with [[programming:crawler]] and [[programming:crawler:openwpm]] clean. |
| | |
| | **Was it worth its slot: yes, and it was the most valuable of the four.** Three of its high-severity findings are defects no checklist would have produced — a false claim in the first paragraph that the page's own evidence refutes, an uncomputed superlative, and a published script that cannot run under the names it is published with. Two of the three had been //introduced or preserved by the fixes for earlier reviewers//, which is the argument for running this pass last and for running it at all. |
| | |
| | ==== Re-review, after all fixes ==== |
| | |
| | Reviewers 1 and 3 were re-run against the final text, because their findings had been acted on and because the report script had gained two tables since they last saw it. Both were told to check figures and quotes **outside** the windows that were edited. |
| | |
| | ^ Pass ^ Result ^ |
| | | Figures (''sonnet'') | Re-ran every script (byte-identical to the committed outputs), independently recomputed all six rows of the new per-field trend table from the raw JSONL, confirmed the 16-row comparison table is exactly the audit's 16 ''ok'' papers with no ''partial'' or ''wrong'' leakage, extracted the two published ''<file>'' blocks verbatim and ran them (all nine rows, matching the page), and verified the is-main guard: importing ''VERDICTS'' now emits nothing, and ''consent_action_audit.mjs'' run directly is still byte-identical to its committed output. **0 wrong**, 1 nit. | |
| | | Citations (''sonnet'') | Confirmed the duplicate key is gone and the bibliography has 0 duplicate keys and one ''</bibtex>''; checked all 16 comparison-table row descriptions against the papers' own text; re-fetched every external footnote including the two GitHub issues via the API rather than a summary; independently re-verified about 20 of the 37 quotations by hand. **0 wrong**, 1 misleading, 3 nits. | |
| | |
| | Fixed from that pass: the em dash in the quoted Chromium closure reason (the comment uses a hyphen); the lower-cased "Removes cookies from context" in the quoted Playwright docs; "primed cache against unprimed" reworded for {[mirheidari2022_cache]}, whose cache control is per-URL hit-versus-miss inside one detection step rather than two population arms; and the ambiguous sentence about interaction depth "moving less", which was true of the //range// and false of the net change. The one MISLEADING finding was against **these notes**, not the page: they described GitHub comments of 2026-01-06 and 2026-01-14 as "maintainer comments" when only the first is — corrected above. |
| | |
| | The author also verified two things independently rather than taking a reviewer's word: the per-field table's six rows against the script's own fractions, cell by cell, and the 16-row table against the audit's ''ok'' list in order. Both matched. The published code was extracted under the page's own filenames into an empty directory and run: nine rows, matching the table. |
| | |
| | **Final state:** page rev ''1787140910'' at 60,763 bytes; this log at roughly 108 KB. Every figure on the page is reproducible from ''scripts/report_stateful_stateless.mjs'', ''statefulness_audit.mjs'' and ''statefulness_probe.mjs''; every quotation from ''state_quotecheck.mjs'' (37/37); every measured browser claim from ''scripts/state_probe/''. |
| |
| ===== Pre-review fixes the author caught ===== | ===== Pre-review fixes the author caught ===== |
| |
| - **The page originally compared statefulness reporting against artifact release "24.3% to 72.7%" and ethics review "8.3% to 45.2%", quoted from ''OVERVIEW.md''.** Those are computed on ''OVERVIEW.md''`s bucket boundaries (2010–2011 / 2012–2015 / 2016–2019 / 2020–2023 / 2024–2026), not on ''lib.mjs'' ''YEAR_BUCKETS'' (2010–2013 / 2014–2017 / 2018–2021 / 2022–2024 / 2025–2026) which the page's own trend table uses. Quoting across incompatible buckets in a sentence beginning "over the same sixteen years" is exactly the kind of borrowed figure the corpus refresh warns about. Both indicators are now recomputed in ''report_stateful_stateless.mjs'' on the page's own buckets — **23.7% → 76.5%** and **10.2% → 45.8%** — and published as a table beside the statefulness column. (Writing that code also caught a wrong field name: the artifacts schema uses ''links[].belongsToAuthors'', not ''isAuthorsOwn'', which had silently produced 0.0% in every bucket.) | - **The page originally compared statefulness reporting against artifact release "24.3% to 72.7%" and ethics review "8.3% to 45.2%", quoted from ''OVERVIEW.md''.** Those are computed on ''OVERVIEW.md''`s bucket boundaries (2010–2011 / 2012–2015 / 2016–2019 / 2020–2023 / 2024–2026), not on ''lib.mjs'' ''YEAR_BUCKETS'' (2010–2013 / 2014–2017 / 2018–2021 / 2022–2024 / 2025–2026) which the page's own trend table uses. Quoting across incompatible buckets in a sentence beginning "over the same sixteen years" is exactly the kind of borrowed figure the corpus refresh warns about. Both indicators are now recomputed in ''report_stateful_stateless.mjs'' on the page's own buckets — **23.7% → 76.5%** and **10.2% → 45.8%** — and published as a table beside the statefulness column. (Writing that code also caught a wrong field name: the artifacts schema uses ''links[].belongsToAuthors'', not ''isAuthorsOwn'', which had silently produced 0.0% in every bucket.) |
| - **"the only reporting rate on this site that has not improved since 2010"** was an unverifiable claim about every page on the wiki. Narrowed to the crawl-configuration fields, which the report script does check. | - **"the only reporting rate on this site that has not improved since 2010"** was an unverifiable claim about every page on the wiki. It was narrowed to the crawl-configuration fields — but **that narrowing was still not computed**, and reviewer 4 caught it: the script had per-bucket trends for statefulness only. It is computed now; see reviewer 4, finding 2. |
| - **The consent-interaction subset (36 papers, 61.1% reporting) is the same 36 papers [[privacy:consent]] audited**, where 7 (19.4%) are extraction false positives. Rather than repeat an unaudited figure, ''report_stateful_stateless.mjs'' now imports that page's ''VERDICTS'' map and prints the rate on the 28 verified papers too: **16/28 = 57.1%**, i.e. the finding survives. The page carries both rows and the caveat. | - **The consent-interaction subset (36 papers, 61.1% reporting) is the same 36 papers [[privacy:consent]] audited**, where 7 (19.4%) are extraction false positives. Rather than repeat an unaudited figure, ''report_stateful_stateless.mjs'' now imports that page's ''VERDICTS'' map and prints the rate on the 28 verified papers too: **16/28 = 57.1%**, i.e. the finding survives. The page carries both rows and the caveat. |
| |
| |
| ^ Path ^ What it does ^ | ^ Path ^ What it does ^ |
| | ''scripts/report_stateful_stateless.mjs'' | every figure on the page, each with its denominator; ''--wiki'' emits DokuWiki tables | | | ''scripts/report_stateful_stateless.mjs'' | the page's tables, each with its denominator; ''--wiki'' emits DokuWiki markup. Imports ''VERDICTS'' from ''consent_action_audit.mjs'', whose printing is guarded behind an is-main check so the import has no side effect | |
| | ''scripts/statefulness_probe.mjs'' | extracts state-management sentences from the 219 stating papers' own text | | | ''scripts/statefulness_probe.mjs'' | extracts state-management sentences from the 219 stating papers' own text | |
| | ''scripts/statefulness_audit.mjs'' | the 29-paper hand adjudication and its effect on the page's figures | | | ''scripts/statefulness_audit.mjs'' | the 29-paper hand adjudication and its effect on the page's figures | |
| | ''scripts/state_quotecheck.mjs'' | all 29 quotations against ''.cols'', ''.norm'' and the PDF | | | ''scripts/state_quotecheck.mjs'' | all 37 quotations against ''.cols'', ''.norm'' and the PDF | |
| | ''scripts/pdf_grep.py'' | the PDF rendering used by the above (pypdf, de-hyphenated) | | | ''scripts/pdf_grep.py'' | the PDF rendering used by the above (pypdf, de-hyphenated) | |
| | ''scripts/state_probe/server.mjs'' + ''probe.mjs'' | the "what a reset actually resets" table | | | ''scripts/state_probe/server.mjs'' + ''probe.mjs'' | the "what a reset actually resets" table | |
