| Next revision | Previous revision |
| provenance:programming:stateful_stateless [2026/08/19 11:32] – Create the provenance log for Programming:Stateful stateless: populations and denominators, every query with unedited script output, the two folding rules and their full residue, the shared-evidence-quote validity problem and the two checks built for it ( karel.kubicek.claude | provenance:programming:stateful_stateless [2026/09/29 11:39] (current) – Log Ramakrishnan 2026 additions, suggestion-list verdicts, review. Authored by Claude karel.kubicek.claude |
|---|
| | Date | 2026-08-19 | | | Date | 2026-08-19 | |
| | Corpus | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P), 2010–2026 | | | Corpus | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P), 2010–2026 | |
| | Page before | 4,479 bytes of notes (rev ''1742374572''), a key-message pair and a short reading list, marked ''<wrap todo>This page only contains notes</wrap>'' | | | Page before | 4,479 bytes of notes (rev ''1742374572''): a two-bullet key message, a short reading list, a "Shallow vs Deep crawling" section, the unedited page template comment, and a todo box reading "This page only contains notes" | |
| | Page after | 46,218 bytes (rev ''1787137126'') | | | Page after | 60,763 bytes (rev ''1787140910''), after three rounds of review fixes plus a re-review pass | |
| | Decision | **Extend, not create.** The title is already the right scope for the design question. The page was a stub; nothing was broadened or narrowed. Overlap with neighbours was deliberate and is recorded under [[#Judgement calls]]. | | | Decision | **Extend, not create.** The title is already the right scope for the design question. The page was a stub; nothing was broadened or narrowed. Overlap with neighbours was deliberate and is recorded under [[#Judgement calls]]. | |
| | Who | Claude (Opus 5) end to end, with four review sub-agents (see [[#Review]]) | | | Who | Claude (Opus 5) end to end, with four review sub-agents (see [[#Review]]) | |
| | | Neighbours patched | ''design:crawling_location'' (false Jueckstock claim, rev ''1787137202''), ''privacy:cookies'' (third-party cookies were not discontinued, rev ''1787140005''), ''programming:crawler:openwpm'' (stale "notes stub" pointer, rev ''1787139249''), ''literature:bibliography'' (duplicate key removed, rev ''1787138947'') | |
| | Scripts added | ''scripts/report_stateful_stateless.mjs'', ''scripts/statefulness_probe.mjs'', ''scripts/statefulness_audit.mjs'', ''scripts/state_quotecheck.mjs'', ''scripts/pdf_grep.py'', ''scripts/state_probe/{server,probe,thirdparty,launch_args}.mjs'' | | | Scripts added | ''scripts/report_stateful_stateless.mjs'', ''scripts/statefulness_probe.mjs'', ''scripts/statefulness_audit.mjs'', ''scripts/state_quotecheck.mjs'', ''scripts/pdf_grep.py'', ''scripts/state_probe/{server,probe,thirdparty,launch_args}.mjs'' | |
| | Bibliography | 13 new keys added in two saves (revs ''1787136793'' and ''1787136845''), one of them **later deleted as a duplicate of an existing key** (rev ''1787138947''); see reviewer 3, finding 1 | | | Bibliography | 13 new keys added in two saves (revs ''1787136793'' and ''1787136845''), one of them **later deleted as a duplicate of an existing key** (rev ''1787138947''); see reviewer 3, finding 1 | |
| |
| <file text stateful_stateless_report.txt> | <file text stateful_stateless_report.txt> |
| claimed an interaction: 36 | |
| adjudicated: 36 | |
| |
| ── Verdicts ── | |
| Verdict Papers Share of the 36 | |
| ------------------------------------------ ------ --------------- | |
| supported by the paper text 28 77.8% | |
| supported but the enum value overstates it 1 2.8% | |
| NOT supported — extraction false positive 7 19.4% | |
| |
| FALSE-POSITIVE RATE of crawlConfig.consentAction on the interacting values: 7/36 = 19.4% | |
| |
| ── Figures recomputed on the AUDITED set ── | |
| papers that ran a crawl: 1120 | |
| state a consent action (incl. no-interaction): 349 (31.2%) | |
| ... of which no-interaction: 313 | |
| ... claim an interaction (extraction): 36 (3.2% of crawled) | |
| ... interaction VERIFIED in the paper: 29 (2.6% of crawled, 8.3% of those stating) | |
| |
| ── Audited interaction by enum value ── | |
| Value Claimed Verified False positives | |
| -------------------- ------- -------- --------------- | |
| accept-all 15 11 4 | |
| accept-and-reject 14 14 0 | |
| cmp-specific-choices 2 2 0 | |
| dismiss-or-remove 3 0 3 | |
| reject-all 2 2 0 | |
| |
| ── Audited interaction by year bucket ── | |
| Bucket Crawling papers Claimed Verified Share of bucket | |
| ---------- --------------- ------- -------- --------------- | |
| 2010–2013 102 0 0 0.0% | |
| 2014–2017 167 0 0 0.0% | |
| 2018–2021 308 6 3 1.0% | |
| 2022–2024 345 17 14 4.1% | |
| 2025–2026* 198 13 12 6.1% | |
| |
| ── Vantage point of the VERIFIED interacting papers ── | |
| Vantage Papers Share of 29 | |
| ---------------------------------- ------ ----------- | |
| EU/EEA vantage 21 72.4% | |
| vantage tuple, location not-stated 5 17.2% | |
| stated a non-EU/EEA vantage only 3 10.3% | |
| |
| ── Every verdict, in full ── | |
| [WRONG ] CCS/2018/pride-and-prejudice-in-progressive-web-apps-abusing-native-app-like-features-in | |
| claimed: accept-all | |
| the only consent in the paper is the PUSH NOTIFICATION permission prompt | |
| [WRONG ] WWW/2018/hiding-in-the-crowd-an-analysis-of-the-effectiveness-of-browser-fingerprinting-a | |
| claimed: accept-all | |
| no sentence in the paper mentions a consent notice or banner at all | |
| [PARTIAL] PETS/2019/4-years-of-eu-cookie-law-results-and-lessons-learned | |
| claimed: accept-and-reject | |
| main crawl states "No user action is performed on the page"; a separate experiment gives consent, so an ACCEPT arm exists but no reject arm is described | |
| [WRONG ] IMC/2020/when-push-comes-to-ads-measuring-the-rise-of-malicious-push-advertising | |
| claimed: accept-all | |
| "banner" in this paper means BANNER ADS; no consent interaction described | |
| [OK ] IEEE-SP/2020/do-cookie-banners-respect-my-choice-measuring-legal-compliance-of-banners-from-i | |
| claimed: accept-and-reject | |
| semi-automatic crawl clicking through banners on 560 sites | |
| [OK ] WWW/2021/user-tracking-in-the-post-cookie-era-how-websites-bypass-gdpr-consent-to-track-u | |
| claimed: accept-and-reject | |
| "we leverage the Consent-O-matic tool"; three crawls, one per consent action | |
| [OK ] PETS/2022/how-can-and-would-people-protect-from-online-tracking | |
| claimed: accept-and-reject | |
| "On the second visit, we accept the cookies... on the next visit, we try to opt-out" | |
| [OK ] USENIX/2022/automating-cookie-consent-and-gdpr-violation-detection | |
| claimed: accept-and-reject | |
| crawler consents to all purposes and separately denies consent | |
| [OK ] USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission | |
| claimed: accept-and-reject | |
| "three consent modes... accept all, reject all, and no action" | |
| [OK ] IMC/2023/the-prevalence-of-single-sign-on-on-the-web-towards-the-next-generation-of-web-c | |
| claimed: accept-all | |
| "We use a plugin to auto-accept cookie banners" | |
| [WRONG ] IMC/2023/understanding-the-privacy-risks-of-popular-search-engine-advertising-systems | |
| claimed: accept-all | |
| no first-person sentence mentions consent or a banner | |
| [WRONG ] PETS/2023/a-utility-preserving-obfuscation-approach-for-youtube-recommendations | |
| claimed: dismiss-or-remove | |
| the only "consent" is IRB participant consent for the user dataset | |
| [OK ] IEEE-SP/2023/the-leaky-web-automated-discovery-of-cross-site-information-leaks-in-browsers-an | |
| claimed: accept-all | |
| "the cookie banners accepted by our module" | |
| [WRONG ] IMC/2024/analyzing-the-in-accessibility-of-online-advertisements | |
| claimed: dismiss-or-remove | |
| the only "consent" is IRB informed consent; "banner" means banner ads | |
| [OK ] USENIX/2024/automated-large-scale-analysis-of-cookie-notice-compliance | |
| claimed: accept-and-reject | |
| five crawling steps: accepting, rejecting, closing, saving defaults, not interacting | |
| [OK ] NDSS/2024/fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting | |
| claimed: accept-all | |
| "...solving CAPTCHAs, and consenting to all cookie notices" (a manual sub-crawl) | |
| [OK ] USENIX/2024/dissecting-privacy-perspectives-of-websites-around-the-world-aceptar-todo-alle-a | |
| claimed: cmp-specific-choices | |
| measures "depth to reach the reject option", i.e. navigates the notice's own layers | |
| [OK ] PETS/2024/a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users | |
| claimed: accept-and-reject | |
| the paper IS a comparison of banner-interaction extensions | |
| [OK ] PETS/2024/fp-tracer-fine-grained-browser-fingerprinting-detection-via-taint-tracking-and-e | |
| claimed: accept-and-reject | |
| "we measure whether fingerprinters respect user consent banners using the Consent-O-Matic plugin" | |
| [OK ] PETS/2024/opted-out-yet-tracked-are-regulations-enough-to-protect-your-privacy | |
| claimed: accept-and-reject | |
| audits four CMPs with consent conveyed and not conveyed | |
| [OK ] WWW/2024/a-study-of-gdpr-compliance-under-the-transparency-and-consent-framework | |
| claimed: reject-all | |
| "accepted the automatic decline of user consent by our data collection system" | |
| [OK ] WWW/2024/the-double-edged-sword-identifying-authentication-pages-and-their-fingerprinting | |
| claimed: accept-all | |
| "To automatically interact with cookie consent banners, we integrated code derived from Priv-Accept" | |
| [OK ] IEEE-SP/2024/targeted-and-troublesome-tracking-and-advertising-on-childrens-websites | |
| claimed: accept-all | |
| "provide affirmative consent to all data processing request options (accept all)" via autoconsent | |
| [OK ] CCS/2025/piixel-leaks-passive-identification-of-personally-identifiable-information-leaka | |
| claimed: accept-all | |
| "we simulate a real user's choice of 'Accept All' using the Consent-O-Matic extension" | |
| [OK ] IMC/2025/canvassing-the-fingerprinters-characterizing-canvas-fingerprinting-use-across-th | |
| claimed: accept-all | |
| "uses the autoconsent library to opt-in to common consent banners" | |
| [OK ] PETS/2025/johnny-can-t-revoke-consent-either-measuring-compliance-of-consent-revocation-on | |
| claimed: accept-and-reject | |
| the paper accepts and then revokes consent on 200 sites | |
| [OK ] PETS/2025/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and | |
| claimed: accept-and-reject | |
| "using BannerClick... accepting cookie banners in the first half and... rejected domains in the second" | |
| [OK ] PETS/2025/referrer-policy-implementation-and-circumvention | |
| claimed: accept-and-reject | |
| "we employed DuckDuckGo's autoconsent library... comparison of RP implementations in different consent modes" | |
| [OK ] WWW/2025/before-after-the-effect-of-eus-2022-code-of-practice-on-disinformation | |
| claimed: accept-all | |
| "we make use of Consent-O-Matic ... to automatically accept all cookies in consent banners" | |
| [OK ] USENIX/2025/navigating-cookie-consent-violations-across-the-globe | |
| claimed: reject-all | |
| ConsentChk drives the banner across eight regions | |
| [OK ] WWW/2025/semantics-aware-cookie-purpose-compliance | |
| claimed: accept-all | |
| "We use two browser extensions, namely Consent-O-Matic and Cookie-Editor" | |
| [OK ] WWW/2025/the-first-early-evidence-of-the-use-of-browser-fingerprinting-for-online-trackin | |
| claimed: cmp-specific-choices | |
| "executing window.OneTrust.RejectAll to opt out"; per-CMP reject clicking | |
| [WRONG ] PETS/2026/the-masks-we-think-we-wear-privacy-threats-of-browser-extension-wallets-in-the-w | |
| claimed: dismiss-or-remove | |
| "explicit consent" is a RECOMMENDATION about wallet permissions, not a crawl action | |
| [OK ] PETS/2026/clicking-into-exposure-uncovering-privacy-risks-of-google-click-identifier-in-yo | |
| claimed: accept-and-reject | |
| "(i) a baseline run that accepts cookie banners... (ii) a rejection run that declines them" | |
| [OK ] PETS/2026/privacy-vs-profit-the-impact-of-googles-manifest-version-3-mv3-update-on-ad-bloc | |
| claimed: accept-all | |
| "Accept consent notice via Super Agent-Automatic Cookie Consent extension" | |
| [OK ] USENIX/2026/bridges-to-self-silent-web-to-app-tracking-on-mobile-via-localhost | |
| claimed: accept-all | |
| "We modify the crawler to automatically accept cookies using Priv-Accept" | |
| corpus: 5859 papers | corpus: 5859 papers |
| population 'crawled' (crawlConfig object present OR studyTypes includes automated-web-crawl): 1120 | population 'crawled' (crawlConfig object present OR studyTypes includes automated-web-crawl): 1120 |
| === Statefulness against the other crawl-configuration fields it competes with === | === Statefulness against the other crawl-configuration fields it competes with === |
| |
| Field Papers stating it Share of 1120 crawling papers | Field Papers stating it Share of 1120 crawling papers |
| -------------------------- ----------------- ----------------------------- | ------------------------------------------------------------------- ----------------- ----------------------------- |
| Interaction depth 841 75.1% | Interaction depth 841 75.1% |
| Authentication 779 69.6% | Authentication 779 69.6% |
| At least one browser named 529 47.2% | At least one browser named 529 47.2% |
| Consent action 349 31.2% | Consent action field populated (schema; not an audited paper claim) 349 31.2% |
| **Stateful or stateless** 219 19.6% | **Stateful or stateless** 219 19.6% |
| Headless or headful 140 12.5% | Headless or headful 140 12.5% |
| |
| |
| 2022–2024 71/345 = 20.6% 1270/1955 = 65.0% 681/1649 = 41.3% | 2022–2024 71/345 = 20.6% 1270/1955 = 65.0% 681/1649 = 41.3% |
| 2025–2026* 36/198 = 18.2% 907/1185 = 76.5% 467/1019 = 45.8% | 2025–2026* 36/198 = 18.2% 907/1185 = 76.5% 467/1019 = 45.8% |
| | |
| | |
| | === Every crawl-configuration field, reporting rate per bucket (of the crawling papers in each bucket) === |
| | |
| | Field 2010–2013 2014–2017 2018–2021 2022–2024 2025–2026* max-min last - first |
| | ---------------------------------------------------------------- -------------- --------------- --------------- --------------- --------------- ------- ------------ |
| | interactionDepth 80/102 = 78.4% 126/167 = 75.4% 237/308 = 76.9% 255/345 = 73.9% 143/198 = 72.2% 6.2 pp -6.2 pp |
| | authentication 61/102 = 59.8% 117/167 = 70.1% 213/308 = 69.2% 251/345 = 72.8% 137/198 = 69.2% 12.9 pp +9.4 pp |
| | browsers (>=1 named) 33/102 = 32.4% 81/167 = 48.5% 153/308 = 49.7% 162/345 = 47.0% 100/198 = 50.5% 18.2 pp +18.2 pp |
| | consentAction (schema field populated; not a paper-claim series) 25/102 = 24.5% 50/167 = 29.9% 99/308 = 32.1% 116/345 = 33.6% 59/198 = 29.8% 9.1 pp +5.3 pp |
| | **statefulness** 16/102 = 15.7% 35/167 = 21.0% 61/308 = 19.8% 71/345 = 20.6% 36/198 = 18.2% 5.3 pp +2.5 pp |
| | headless 1/102 = 1.0% 26/167 = 15.6% 41/308 = 13.3% 50/345 = 14.5% 22/198 = 11.1% 14.6 pp +10.1 pp |
| | |
| | The last two columns are the test of the page's claim: which field moved, and by how much. |
| | |
| | Audited consent-action papers by bucket (paper claims, not raw field population): |
| | Bucket Crawling papers Audited papers stating a consent action Share Of those, interacted |
| | ---------- --------------- --------------------------------------- ----- -------------------- |
| | 2010–2013 102 0 0.0% 0 |
| | 2014–2017 167 0 0.0% 0 |
| | 2018–2021 308 11 3.6% 3 |
| | 2022–2024 345 25 7.2% 16 |
| | 2025–2026* 198 19 9.6% 13 |
| | The schema row above is field population only; this audited row is the paper-claim series. |
| |
| |
| === Designs whose result is uninterpretable without the statefulness: do they state it? === | === Designs whose result is uninterpretable without the statefulness: do they state it? === |
| |
| Subset of crawling papers N State statefulness Share stating stateless stateful both | Subset of crawling papers N State statefulness Share stating stateless stateful both |
| ------------------------------------------------------------------------------ ---- ------------------ ------------- --------- -------- ---- | ------------------------------------------------------------------------- ---- ------------------ ------------- --------- -------- ---- |
| Repeat-visit designs (repeatVisits >= 2) 199 85 42.7% 49 21 15 | Repeat-visit designs (repeatVisits >= 2) 199 85 42.7% 49 21 15 |
| Crawls that acted on a consent notice 36 22 61.1% 11 4 7 | Crawls that an audit found acted on a consent notice 32 20 62.5% 10 4 6 |
| Crawls that logged in (authentication stated, not "none") 90 54 60.0% 7 39 8 | Crawls with an audited consent action (interaction or explicit no-action) 55 32 58.2% 18 8 6 |
| Deep crawls (interactionDepth = deep-crawl) 157 40 25.5% 8 26 6 | Crawls that logged in (authentication stated, not "none") 90 54 60.0% 7 39 8 |
| Crawls beyond the landing page (subpages or deep) 303 83 27.4% 30 40 13 | Deep crawls (interactionDepth = deep-crawl) 157 40 25.5% 8 26 6 |
| Crawls whose consent interaction was //verified// by the privacy:consent audit 28 16 57.1% 8 3 5 | Crawls beyond the landing page (subpages or deep) 303 83 27.4% 30 40 13 |
| //all crawling papers, for comparison// 1120 219 19.6% 113 77 29 | //all crawling papers, for comparison// 1120 219 19.6% 113 77 29 |
| |
| |
| One clean confirmed false positive found in that set: ''USENIX/2023/automated-cookie-notice-analysis-and-enforcement'' is labelled ''stateless'' and the string "stateful"/"stateless" appears **zero** times in its full text. | One clean confirmed false positive found in that set: ''USENIX/2023/automated-cookie-notice-analysis-and-enforcement'' is labelled ''stateless'' and the string "stateful"/"stateless" appears **zero** times in its full text. |
| |
| ==== Check 2: full hand adjudication of the 29 ''both'' papers ==== | ==== Check 2: full hand adjudication of the 29 both papers ==== |
| |
| The ''both'' bucket is the page's most load-bearing set — it is the reading list of comparison studies — so all 29 were read rather than sampled, using the probe's ''--only both --sentences'' output plus the paper text where that was thin. | The ''both'' bucket is the page's most load-bearing set — it is the reading list of comparison studies — so all 29 were read rather than sampled, using the probe's ''%%--only%% both %%--sentences%%'' output plus the paper text where that was thin. |
| |
| <code> | <code> |
| Whitespace, quote characters, dashes and ligatures are normalised; nothing else is. Each quote is tried against ''paper.cols.txt'', then ''paper.norm.txt'', then — because both text renderings splice two-column text — against ''paper.pdf'' itself via ''scripts/pdf_grep.py'' (pypdf 6.16.1, de-hyphenated). The rendering that matched is printed, because **a quote that only the PDF has is a quote whose ''.cols'' text is spliced**, and that is worth knowing. | Whitespace, quote characters, dashes and ligatures are normalised; nothing else is. Each quote is tried against ''paper.cols.txt'', then ''paper.norm.txt'', then — because both text renderings splice two-column text — against ''paper.pdf'' itself via ''scripts/pdf_grep.py'' (pypdf 6.16.1, de-hyphenated). The rendering that matched is printed, because **a quote that only the PDF has is a quote whose ''.cols'' text is spliced**, and that is worth knowing. |
| |
| **Result: 29 of 29 pass.** Four passed only against the PDF: | **Result: 37 of 37 pass** (29 when the checker was first written; eight more were added with the comparison-studies table and the tightened denominators). Four passed only against the PDF: |
| |
| ^ Quote ^ Paper ^ | ^ Quote ^ Paper ^ |
| * **First design used ''localhost'' as the second site.** In this container ''getent hosts localhost'' resolves only to ''::1'', so a server bound to ''127.0.0.1:8132'' was unreachable and the script hung until killed. Switched to numeric loopback aliases. | * **First design used ''localhost'' as the second site.** In this container ''getent hosts localhost'' resolves only to ''::1'', so a server bound to ''127.0.0.1:8132'' was unreachable and the script hung until killed. Switched to numeric loopback aliases. |
| * **Playwright's Firefox will not launch here.** Headless dies with ''RenderCompositorSWGL failed mapping default framebuffer, no dt''; headed under a manually started ''Xvfb :99'' dies for want of ''dbus-launch'' (''xvfb-run'' itself fails earlier: ''xauth command not found''). So **the Firefox row is absent from the output rather than guessed**, the script records the launch failure instead of crashing, and the page's Firefox claim rests on Mozilla's documentation, explicitly labelled as such. This is the one measurement the page wanted and could not take. | * **Playwright's Firefox will not launch here.** Headless dies with ''RenderCompositorSWGL failed mapping default framebuffer, no dt''; headed under a manually started ''Xvfb :99'' dies for want of ''dbus-launch'' (''xvfb-run'' itself fails earlier: ''xauth command not found''). So **the Firefox row is absent from the output rather than guessed**, the script records the launch failure instead of crashing, and the page's Firefox claim rests on Mozilla's documentation, explicitly labelled as such. This is the one measurement the page wanted and could not take. |
| * **''headless: !process.env.DISPLAY'' is wrong in this container.** ''DISPLAY'' is set even though no X server is reachable, so Playwright launched headed and died with "Looks like you launched a headed browser without having a XServer running". Replaced with an explicit ''--headed'' flag. | * **''headless: !process.env.DISPLAY'' is wrong in this container.** ''DISPLAY'' is set even though no X server is reachable, so Playwright launched headed and died with "Looks like you launched a headed browser without having a XServer running". Replaced with an explicit ''%%--headed%%'' flag. |
| |
| ==== 3. What flags the crawler actually launches with ==== | ==== 3. What flags the crawler actually launches with ==== |
| | CHIPS and FedCM continue; other Privacy Sandbox APIs are phased out | ''privacysandbox.com/news/update-on-plans-for-privacy-sandbox-technologies/'' (HTTP 200) | 17 October 2025: "CHIPS and FedCM … have seen broad adoption … We'll continue to support those APIs" | | | CHIPS and FedCM continue; other Privacy Sandbox APIs are phased out | ''privacysandbox.com/news/update-on-plans-for-privacy-sandbox-technologies/'' (HTTP 200) | 17 October 2025: "CHIPS and FedCM … have seen broad adoption … We'll continue to support those APIs" | |
| | Chrome Incognito blocks third-party cookies by default | same 22 April 2025 post | "Chrome's Incognito mode, which already blocks third-party cookies by default" | | | Chrome Incognito blocks third-party cookies by default | same 22 April 2025 post | "Chrome's Incognito mode, which already blocks third-party cookies by default" | |
| | Playwright disables ''ThirdPartyStoragePartitioning'' | the installed ''playwright-core'' 1.62.1 default ''--disable-features'' list, **and** the live launch command line via ''spawnargs'' | present in both; the inline source comment points at ''github.com/microsoft/playwright/issues/32230'' | | | Playwright disables ''ThirdPartyStoragePartitioning'' | the installed ''playwright-core'' 1.62.1 default ''%%--disable-features%%'' list, **and** the live launch command line via ''spawnargs'' | present in both; the inline source comment points at ''github.com/microsoft/playwright/issues/32230'' | |
| | Puppeteer does not disable it | ''puppeteer-core'' 25.5.0 ''ChromeLauncher.js'' ''disabledFeatures'' array, and ''puppeteer.defaultArgs()'' output | absent from both | | | Puppeteer does not disable it | ''puppeteer-core'' 25.5.0 ''ChromeLauncher.js'' ''disabledFeatures'' array, and ''puppeteer.defaultArgs()'' output | absent from both | |
| | OpenWPM allows unpartitioned third-party cookies by default | ''raw.githubusercontent.com/openwpm/OpenWPM/master/openwpm/config.py'' and ''.../deploy_browsers/configure_firefox.py'' | ''tp_cookies: str = "always"'' → ''fo.set_preference("network.cookie.cookieBehavior", 0)'' | | | OpenWPM allows unpartitioned third-party cookies by default | ''raw.githubusercontent.com/openwpm/OpenWPM/master/openwpm/config.py'' and ''.../deploy_browsers/configure_firefox.py'' | ''tp_cookies: str = "always"'' → ''fo.set_preference("network.cookie.cookieBehavior", 0)'' | |
| |
| |
| ==== Reviewer 1 — figures against the script (''model: sonnet'') ==== | ==== Reviewer 1 — figures against the script (model: sonnet) ==== |
| |
| Re-ran all three dataset scripts (byte-identical to the committed outputs), re-ran the three browser probes, independently recomputed ~35 figures across all 13 tables with its own jq/node queries, and pulled the primary-source PDFs for every quoted paper. | Re-ran all three dataset scripts (byte-identical to the committed outputs), re-ran the three browser probes, independently recomputed ~35 figures across all 13 tables with its own jq/node queries, and pulled the primary-source PDFs for every quoted paper. |
| Its own summary is worth recording: //"No error was found in any script-derived number itself — every discrepancy found is in page prose layered on top of correct script output."// That is where the next run should look first. | Its own summary is worth recording: //"No error was found in any script-derived number itself — every discrepancy found is in page prose layered on top of correct script output."// That is where the next run should look first. |
| |
| ==== Reviewer 2 — external currency (''model: sonnet'') ==== | ==== Reviewer 2 — external currency (model: sonnet) ==== |
| |
| Fetched every external claim live; also re-ran all three probe scripts against the installed toolchain and reproduced every published row. | Fetched every external claim live; also re-ran all three probe scripts against the installed toolchain and reproduced every published row. |
| |
| ^ # ^ Finding ^ Severity ^ Verdict ^ | ^ # ^ Finding ^ Severity ^ Verdict ^ |
| | 1 | Puppeteer's latest is 25.8.0 (2026-08-17), not the cited 25.5.0 — but the reviewer installed 25.8.0 and found its ''--disable-features'' list **byte-identical**, so the comparison stands. | NIT | **Accepted.** A footnote on the table now records that 25.8.0 is identical, so the comparison is not an artefact of the pinned version. | | | 1 | Puppeteer's latest is 25.8.0 (2026-08-17), not the cited 25.5.0 — but the reviewer installed 25.8.0 and found its ''%%--disable-features%%'' list **byte-identical**, so the comparison stands. | NIT | **Accepted.** A footnote on the table now records that 25.8.0 is identical, so the comparison is not an artefact of the pinned version. | |
| | 2 | The illustrative "what to report" sentence said "OpenWPM 0.35.0 (Firefox 152)"; current stable Firefox is 154.0. | NIT | **Accepted** — already changed to "(its pinned unbranded Firefox build)" in the author's pass, which is what the example needed anyway. | | | 2 | The illustrative "what to report" sentence said "OpenWPM 0.35.0 (Firefox 152)"; current stable Firefox is 154.0. | NIT | **Accepted** — already changed to "(its pinned unbranded Firefox build)" in the author's pass, which is what the example needed anyway. | |
| | 3 | Playwright issue 32230 is **closed** (resolved by PR 32701), and a later issue asking to re-enable partitioning was closed because Chromium plans to remove the flag entirely — so the page's workaround is on borrowed time, and the page does not say so. Called out as "the single most relevant 'about to move' fact" missing. | (substantive addition) | **Accepted, but not as reported.** See below. | | | 3 | Playwright issue 32230 is **closed** (resolved by PR 32701), and a later issue asking to re-enable partitioning was closed because Chromium plans to remove the flag entirely — so the page's workaround is on borrowed time, and the page does not say so. Called out as "the single most relevant 'about to move' fact" missing. | (substantive addition) | **Accepted, but not as reported.** See below. | |
| | 4 | Everything else — Mozilla TCP, Chrome 115 storage partitioning, both Privacy Sandbox posts (no 2026 reversal), Incognito third-party blocking, CHIPS/FedCM, GPC now on the W3C Privacy WG track, Playwright 1.62.1 being the actual latest, Chromium 151.0.7922.34 being its pinned build, pypdf 6.16.1 current, OpenWPM actively maintained with ''b9dd4c3a'' being current HEAD and all three code claims byte-for-byte — verified correct. | — | Noted; no change needed. | | | 4 | Everything else — Mozilla TCP, Chrome 115 storage partitioning, both Privacy Sandbox posts (no 2026 reversal), Incognito third-party blocking, CHIPS/FedCM, GPC now on the W3C Privacy WG track, Playwright 1.62.1 being the actual latest, Chromium 151.0.7922.34 being its pinned build, pypdf 6.16.1 current, OpenWPM actively maintained with ''b9dd4c3a'' being current HEAD and all three code claims byte-for-byte — verified correct. | — | Noted; no change needed. | |
| |
| **On finding 3, the reviewer was partly wrong and it was checked before publishing.** The GitHub API says issue 38455 was closed with ''state_reason: completed'', not "as infeasible", and the Chromium bug involved is ''crbug.com/468317746'', not the ''crbug.com/410491202'' the reviewer cited. Reading the actual thread gave a better and more on-topic story than the reviewer's summary: Playwright disables partitioning **because its own ''storageState'' API cannot represent partitioned storage** — "Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now" — and issue 38455 was closed after the Chromium-side request for bulk storage-key APIs was declined as "infeasible — too far outside of the product scope". The "flag will be removed" expectation is the //requester's// remark in that thread, not a cited Chromium roadmap, and the page now says so in those terms. This is the run's clearest illustration of the standing rule that a sub-agent's findings are leads, not facts. | **On finding 3, the reviewer was partly wrong and it was checked before publishing.** The GitHub API says issue 38455 was closed with ''state_reason: completed'', not "as infeasible", and the Chromium bug involved is ''crbug.com/468317746'', not the ''crbug.com/410491202'' the reviewer cited. Reading the actual thread gave a better and more on-topic story than the reviewer's summary: Playwright disables partitioning **because its own ''storageState'' API cannot represent partitioned storage** — "Without CDP support, it does not seem practical to replicate all the intricate details of storage partitioning outside of the browser, so disabling the feature is the only way to make things work for now" — and issue 38455 was closed after the Chromium-side request for bulk storage-key APIs was declined as "infeasible - too far outside of the product scope". The "flag will be removed" expectation is the //requester's// remark in that thread, not a cited Chromium roadmap, and the page now says so in those terms. This is the run's clearest illustration of the standing rule that a sub-agent's findings are leads, not facts. |
| |
| ==== Reviewer 3 — citations and quotations (''model: sonnet'') ==== | ==== Reviewer 3 — citations and quotations (model: sonnet) ==== |
| |
| Checked all citekeys on both pages, verified all 13 new bibliography entries against Crossref / DOI resolvers / venue pages, re-ran the quote checker and then **independently** re-verified every quotation against the source text or PDF, and re-fetched all five non-paper footnote sources. | Checked all citekeys on both pages, verified all 13 new bibliography entries against Crossref / DOI resolvers / venue pages, re-ran the quote checker and then **independently** re-verified every quotation against the source text or PDF, and re-fetched all five non-paper footnote sources. |
| * **Citations (sonnet)** — yes, and it found the single most damaging defect of the run, a duplicate bibliography key that a naive key-equality check had passed. It also caught a wrong date range that had been copied from nowhere. | * **Citations (sonnet)** — yes, and it found the single most damaging defect of the run, a duplicate bibliography key that a naive key-equality check had passed. It also caught a wrong date range that had been copied from nowhere. |
| * **Currency (sonnet)** — yes on coverage: it fetched and confirmed 15-odd external claims that would otherwise rest on recall, and it found the "flag is going away" story. But its summary of that story was **wrong in two specifics** (closure reason and Chromium bug number), and using it verbatim would have put two false facts on the page. Its value is as a lead generator, not as a source. | * **Currency (sonnet)** — yes on coverage: it fetched and confirmed 15-odd external claims that would otherwise rest on recall, and it found the "flag is going away" story. But its summary of that story was **wrong in two specifics** (closure reason and Chromium bug number), and using it verbatim would have put two false facts on the page. Its value is as a lead generator, not as a source. |
| * **Generic (fable)** — see below. | * **Generic (fable)** — see [[#Reviewer 4 — generic, no checklist (model: fable)|reviewer 4]]; it was the most valuable of the four. |
| | |
| | ==== Reviewer 4 — generic, no checklist (model: fable) ==== |
| | |
| | Given both pages, the script outputs, and the neighbouring pages, with no list of things to look for. |
| | |
| | ^ # ^ Finding ^ Severity ^ Verdict ^ |
| | | 1 | **The page's first substantive sentence was false.** "a crawl that … visits each target once cannot see cookie syncing … because none of those exist without accumulated state" — tracker A can set an ID and sync it to B in the same page load, and the page's own Zeber quote says a fresh profile "would be a clear target for cookie syncing", i.e. fresh profiles over-trigger it. | HIGH | **Accepted in full.** The reviewer is right and the error was self-refuting: the page quoted the evidence against its own claim two sections later. Both the intro and the "what each design can measure" row now say that a stateless crawl sees //first-contact// syncing and over-triggers it, and that what accumulation buys is the aged identity and the sync graph — with {[englehardt2016online]} running its sync analysis on the //stateful// 100k crawl as the supporting example. | |
| | | 2 | **"the only crawl-configuration reporting rate in this corpus with no trend at all"** was never computed. ''report_stateful_stateless.mjs'' computed bucket trends for statefulness and for two outside norms, and snapshot rates for the other configuration fields — not their trends. These notes made it worse by claiming the superlative was one "which the report script does check". | HIGH | **Accepted, and fixed by computing it rather than by deleting it.** The script now prints every crawl-configuration field's reporting rate per bucket with max−min and last−first columns. The claim survives in a narrower and now-checkable form: statefulness has the narrowest range (5.3 pp) and the smallest first-to-last change (+2.5 pp) of the six fields, against +18.2 pp for naming a browser and +10.1 pp for headless. Interaction depth moved less in one sense and moved //down//. The page carries the table. | |
| | | 3 | The "complete" published script was **not runnable as published**: the file blocks were named ''state_probe_server.mjs'' / ''state_probe.mjs'' while the code does ''import { startServer } from './server.mjs''', so saving under the page's names gives ''ERR_MODULE_NOT_FOUND''. Same defect class as reviewer 1's finding, reintroduced by the fix for it. | HIGH | **Accepted.** Blocks renamed ''server.mjs'' and ''probe.mjs'', matching the repo, with a sentence telling the reader to save them side by side. | |
| | | 4 | **A neighbouring page contradicted this page's centrepiece currency claim.** [[privacy:cookies]] opened "This trend persists despite the discontinuation of third-party cookies", which this page verified against Google's own announcement did not happen. | HIGH | **Accepted.** [[privacy:cookies]] patched in the same sitting (rev ''1787140005'') with the primary source, the current Chrome/Firefox/Safari positions, and a pointer to this page's engine-defaults section — the same treatment ''design:crawling_location'' got earlier in the run. | |
| | | 5 | These notes said "Result: 29 of 29 pass" and "all 29 quotations" while the embedded checker output said 37. | MEDIUM | **Accepted.** The prose predated eight quotes added with the comparison-studies table. Now 37, and the count is read from the embedded output rather than restated. | |
| | | 6 | These notes promised "four review sub-agents (see #Review)" and documented three, with "**Generic (fable)** — see below" followed by nothing. | MEDIUM | **Accepted.** This section is that slot. | |
| | | 7 | "Every number here is produced by ''scripts/report_stateful_stateless.mjs''" overcommitted: the 16/10/3 audit table comes from ''statefulness_audit.mjs'' and the 178/41/72 corroboration counts from ''statefulness_probe.mjs''. | MEDIUM | **Accepted.** All three are named on the page. | |
| | | 8 | The "real, unedited output" of the report script **began with about a hundred lines of another page's audit**, because ''consent_action_audit.mjs'' prints at module top level and this report imports its ''VERDICTS''. | MEDIUM | **Accepted, and fixed in the code rather than papered over.** ''consent_action_audit.mjs'' now guards its printing behind an is-main check. Its own direct-run output was diffed before and after and is **byte-identical** to the committed ''scripts/consent_action_audit-output.txt'', so [[privacy:consent]]'s audit trail is unaffected. The report output embedded above was regenerated. | |
| | | 9 | "accumulates an unpartitioned cross-site profile that resembles a default Chrome user's" overstates: Chrome has partitioned third-party //storage// since 115, so the resemblance is to the cookie jar only. | MEDIUM | **Accepted.** Split into cookie jar (resembles) and storage (does not). | |
| | | 10 | Five nits: "the 28 that survived that audit" (29 survived; 28 were //fully// supported); "43.8% to about 55–59%" when the last bucket is 52.8%; "as of January 2026 the maintainers had no replacement design" when the cited footnote dates stop at 2025-12-22; the Flash respawning figures not dated as historical; no Puppeteer recipe in "How to do it" despite Puppeteer having 76 corpus papers to Playwright's 34. | NIT | **All accepted.** The January 2026 claim was kept but re-evidenced: the thread carries a **maintainer** comment of 2026-01-06 asking the reporter for a design that keeps ''storageState'' working, and a **reporter** reply of 2026-01-14. (The re-review caught this sentence describing both as maintainer comments; corrected here.) The page says only that "as late as January 2026 the maintainers were still asking the reporter for a design", which the 6 January comment supports on its own. Flash is dated with its 2020 end of life and the modern successors. A Puppeteer ''userDataDir'' line is added, with the note that it has no ''storageState'' equivalent. | |
| | | 11 | "If you are writing for PETS, the reviewers are used to seeing it" turns a reporting rate into a claim about reviewer expectations. | NIT | **Accepted** — cut to "PETS is where this reporting norm is strongest", which is what the data says. | |
| | | 12 | ''programming:crawler:openwpm'' still carries an "OpenWPM 0.35.0 (Firefox 152)" example while current stable Firefox is 154. | NIT | **Rejected as out of scope, and recorded here instead.** That is a factual staleness on another page's own illustrative example, not a claim this page makes or relies on, and editing another page's examples is scope creep. Whoever next touches that page should fix it. (The stale "Programming:Stateful stateless — currently a notes stub" pointer on the same page //was// fixed, rev ''1787139249'', because it was a false statement about //this// page.) | |
| | |
| | Its clean checks, recorded because they bound what remains in doubt: it re-derived the ratios 51.6%, 57.3%, 1.61×, 2.51×, 2.9×, 5.9× and 55.7% and all hold; it verified that the suspicious-looking 6.91× is verbatim in Rasaii et al. (which uses both "6.9 times" and "6.91 times"); and it independently recomputed the five OpenWPM papers labelled ''both'' and found all five among the 16 adjudicated-ok, so [[programming:crawler:openwpm]]'s "those five are the ones to read" survives this page's audit. It judged the voice consistent with the neighbours and the boundaries with [[programming:crawler]] and [[programming:crawler:openwpm]] clean. |
| | |
| | **Was it worth its slot: yes, and it was the most valuable of the four.** Three of its high-severity findings are defects no checklist would have produced — a false claim in the first paragraph that the page's own evidence refutes, an uncomputed superlative, and a published script that cannot run under the names it is published with. Two of the three had been //introduced or preserved by the fixes for earlier reviewers//, which is the argument for running this pass last and for running it at all. |
| | |
| | ==== Re-review, after all fixes ==== |
| | |
| | Reviewers 1 and 3 were re-run against the final text, because their findings had been acted on and because the report script had gained two tables since they last saw it. Both were told to check figures and quotes **outside** the windows that were edited. |
| | |
| | ^ Pass ^ Result ^ |
| | | Figures (''sonnet'') | Re-ran every script (byte-identical to the committed outputs), independently recomputed all six rows of the new per-field trend table from the raw JSONL, confirmed the 16-row comparison table is exactly the audit's 16 ''ok'' papers with no ''partial'' or ''wrong'' leakage, extracted the two published ''%%<file>%%'' blocks verbatim and ran them (all nine rows, matching the page), and verified the is-main guard: importing ''VERDICTS'' now emits nothing, and ''consent_action_audit.mjs'' run directly is still byte-identical to its committed output. **0 wrong**, 1 nit. | |
| | | Citations (''sonnet'') | Confirmed the duplicate key is gone and the bibliography has 0 duplicate keys and one ''</bibtex>''; checked all 16 comparison-table row descriptions against the papers' own text; re-fetched every external footnote including the two GitHub issues via the API rather than a summary; independently re-verified about 20 of the 37 quotations by hand. **0 wrong**, 1 misleading, 3 nits. | |
| | |
| | Fixed from that pass: the em dash in the quoted Chromium closure reason (the comment uses a hyphen); the lower-cased "Removes cookies from context" in the quoted Playwright docs; "primed cache against unprimed" reworded for {[mirheidari2022_cache]}, whose cache control is per-URL hit-versus-miss inside one detection step rather than two population arms; and the ambiguous sentence about interaction depth "moving less", which was true of the //range// and false of the net change. The one MISLEADING finding was against **these notes**, not the page: they described GitHub comments of 2026-01-06 and 2026-01-14 as "maintainer comments" when only the first is — corrected above. |
| | |
| | The author also verified two things independently rather than taking a reviewer's word: the per-field table's six rows against the script's own fractions, cell by cell, and the 16-row table against the audit's ''ok'' list in order. Both matched. The published code was extracted under the page's own filenames into an empty directory and run: nine rows, matching the table. |
| | |
| | **Final state:** page rev ''1787140910'' at 60,763 bytes; this log at roughly 108 KB. Every figure on the page is reproducible from ''scripts/report_stateful_stateless.mjs'', ''statefulness_audit.mjs'' and ''statefulness_probe.mjs''; every quotation from ''state_quotecheck.mjs'' (37/37); every measured browser claim from ''scripts/state_probe/''. |
| |
| ===== Pre-review fixes the author caught ===== | ===== Pre-review fixes the author caught ===== |
| |
| - **The page originally compared statefulness reporting against artifact release "24.3% to 72.7%" and ethics review "8.3% to 45.2%", quoted from ''OVERVIEW.md''.** Those are computed on ''OVERVIEW.md''`s bucket boundaries (2010–2011 / 2012–2015 / 2016–2019 / 2020–2023 / 2024–2026), not on ''lib.mjs'' ''YEAR_BUCKETS'' (2010–2013 / 2014–2017 / 2018–2021 / 2022–2024 / 2025–2026) which the page's own trend table uses. Quoting across incompatible buckets in a sentence beginning "over the same sixteen years" is exactly the kind of borrowed figure the corpus refresh warns about. Both indicators are now recomputed in ''report_stateful_stateless.mjs'' on the page's own buckets — **23.7% → 76.5%** and **10.2% → 45.8%** — and published as a table beside the statefulness column. (Writing that code also caught a wrong field name: the artifacts schema uses ''links[].belongsToAuthors'', not ''isAuthorsOwn'', which had silently produced 0.0% in every bucket.) | - **The page originally compared statefulness reporting against artifact release "24.3% to 72.7%" and ethics review "8.3% to 45.2%", quoted from ''OVERVIEW.md''.** Those are computed on ''OVERVIEW.md''`s bucket boundaries (2010–2011 / 2012–2015 / 2016–2019 / 2020–2023 / 2024–2026), not on ''lib.mjs'' ''YEAR_BUCKETS'' (2010–2013 / 2014–2017 / 2018–2021 / 2022–2024 / 2025–2026) which the page's own trend table uses. Quoting across incompatible buckets in a sentence beginning "over the same sixteen years" is exactly the kind of borrowed figure the corpus refresh warns about. Both indicators are now recomputed in ''report_stateful_stateless.mjs'' on the page's own buckets — **23.7% → 76.5%** and **10.2% → 45.8%** — and published as a table beside the statefulness column. (Writing that code also caught a wrong field name: the artifacts schema uses ''links[].belongsToAuthors'', not ''isAuthorsOwn'', which had silently produced 0.0% in every bucket.) |
| - **"the only reporting rate on this site that has not improved since 2010"** was an unverifiable claim about every page on the wiki. Narrowed to the crawl-configuration fields, which the report script does check. | - **"the only reporting rate on this site that has not improved since 2010"** was an unverifiable claim about every page on the wiki. It was narrowed to the crawl-configuration fields — but **that narrowing was still not computed**, and reviewer 4 caught it: the script had per-bucket trends for statefulness only. It is computed now; see reviewer 4, finding 2. |
| - **The consent-interaction subset (36 papers, 61.1% reporting) is the same 36 papers [[privacy:consent]] audited**, where 7 (19.4%) are extraction false positives. Rather than repeat an unaudited figure, ''report_stateful_stateless.mjs'' now imports that page's ''VERDICTS'' map and prints the rate on the 28 verified papers too: **16/28 = 57.1%**, i.e. the finding survives. The page carries both rows and the caveat. | - **The consent-interaction subset is now taken from the complete 2026-09-05 audit:** **32** papers were found to have acted on a consent notice, and **55** papers were audited as stating a consent action. The audit found **279/313 = 89.1%** unsupported ''no-interaction'' labels and **7/36 = 19.4%** false positives among labels claiming interaction. ''report_stateful_stateless.mjs'' imports the exported audit sets and prints both audited rows. |
| |
| ===== Where this page's figures cross-check against neighbours ===== | ===== Where this page's figures cross-check against neighbours ===== |
| |
| ^ Figure ^ Here ^ Neighbour ^ Reconciled? ^ | ^ Figure ^ Here ^ Neighbour ^ Reconciled? ^ |
| | statefulness stated, of crawling papers | 219 / 1,120 = 19.6% | [[programming:crawler]]: "Stateful or stateless \| 219 \| 19.6%" | identical | | | statefulness stated, of crawling papers | 219 / 1,120 = 19.6% | [[programming:crawler]]: same 219 / 1,120 = 19.6% row | identical | |
| | OpenWPM papers stating it | 32/58 = 55.2% (category-restricted fold) | [[programming:crawler:openwpm]]: 33/60 = 55.0%, and 186/1,061 for everyone else | differ by one paper; both definitions printed by this page's script and named on the page | | | OpenWPM papers stating it | 32/58 = 55.2% (category-restricted fold) | [[programming:crawler:openwpm]]: 33/60 = 55.0%, and 186/1,061 for everyone else | differ by one paper; both definitions printed by this page's script and named on the page | |
| | Consent-interaction subset | 36 papers, 22 state statefulness | [[privacy:consent]]: same 36 papers, 7 false positives | this page prints both the raw and the audited row | | | Audited interaction subset | 32 papers, 20 state statefulness | [[privacy:consent]]: 7/36 interaction labels false positive | audited row printed by this page's report | |
| | Papers with a consent action stated | 349 / 1,120 = 31.2% | [[privacy:requests]] reports 52.3% on its own 197-paper subset | different populations; not compared on the page | | | Audited consent-action papers | 55 / 1,120 = 4.9% | [[privacy:requests]] reports 52.3% on its own 197-paper subset | different populations; not compared on the page | |
| |
| ===== Artefacts ===== | ===== Artefacts ===== |
| |
| ^ Path ^ What it does ^ | ^ Path ^ What it does ^ |
| | ''scripts/report_stateful_stateless.mjs'' | every figure on the page, each with its denominator; ''--wiki'' emits DokuWiki tables | | | ''scripts/report_stateful_stateless.mjs'' | the page's tables, each with its denominator; ''%%--wiki%%'' emits DokuWiki markup. Imports the audited paper-key sets from ''consent_action_noninteraction_audit.mjs'', whose printing is guarded behind an is-main check so the import has no side effect | |
| | ''scripts/statefulness_probe.mjs'' | extracts state-management sentences from the 219 stating papers' own text | | | ''scripts/statefulness_probe.mjs'' | extracts state-management sentences from the 219 stating papers' own text | |
| | ''scripts/statefulness_audit.mjs'' | the 29-paper hand adjudication and its effect on the page's figures | | | ''scripts/statefulness_audit.mjs'' | the 29-paper hand adjudication and its effect on the page's figures | |
| | ''scripts/state_quotecheck.mjs'' | all 29 quotations against ''.cols'', ''.norm'' and the PDF | | | ''scripts/state_quotecheck.mjs'' | all 37 quotations against ''.cols'', ''.norm'' and the PDF | |
| | ''scripts/pdf_grep.py'' | the PDF rendering used by the above (pypdf, de-hyphenated) | | | ''scripts/pdf_grep.py'' | the PDF rendering used by the above (pypdf, de-hyphenated) | |
| | ''scripts/state_probe/server.mjs'' + ''probe.mjs'' | the "what a reset actually resets" table | | | ''scripts/state_probe/server.mjs'' + ''probe.mjs'' | the "what a reset actually resets" table | |
| * [[programming:crawler:openwpm|Programming:Crawler:OpenWPM]] and [[programming:crawler|Programming:Crawler]] — where the OpenWPM figures and the framework fold come from. | * [[programming:crawler:openwpm|Programming:Crawler:OpenWPM]] and [[programming:crawler|Programming:Crawler]] — where the OpenWPM figures and the framework fold come from. |
| * [[privacy:consent]] — source of the consent-interaction adjudication reused here. | * [[privacy:consent]] — source of the consent-interaction adjudication reused here. |
| | |
| | ===== Amendment, 2026-09-04: citekey consolidation ===== |
| | |
| | * ''fouad2022my'' was one of two keys for the same paper in [[:literature:bibliography]]. The wiki-wide consolidation of 2026-09-04 (drain item ''dedup-regional-filter-lists-bibkey'') kept ''fouad2022_cookie'' and deleted the other entry. |
| | * 3 citation markers on [[:programming:stateful_stateless]] were repointed to the kept key. No prose on either page changed, and no figure moved. Statements above that name the deleted key describe the state when they were written. Full query log and the invariants checked before saving: [[:provenance:literature:bibliography]]. |
| |
| ====== References ====== | ====== References ====== |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| | |
| | ===== ConsentAction audit propagation, 2026-09-17 ===== |
| | |
| | **What was wrong.** The page's ''consentAction'' row and its 24.5% / 29.9% / 32.1% / 33.6% / 29.8% series were presented without saying that they measure raw field population, not paper claims; the old +5.3 pp reading was therefore not a valid consent-action trend. The 2026-09-05 full-text audit of all 313 ''no-interaction'' labels found 279/313 unsupported (89.1%). |
| | |
| | **Command and real output.** ''node scripts/report_stateful_stateless.mjs'' → raw schema row 349/1,120 = 31.2%; audited paper-claim buckets **0/102 = 0.0%, 0/167 = 0.0%, 11/308 = 3.6%, 25/345 = 7.2%, 19/198 = 9.6%**, with interacted counts 0, 0, 3, 16, 13. ''python3 scripts/consent_ni_significance.py'' → old raw-series adjacent p-values **0.4008, 0.679, 0.7387, 0.3911**, all not significant; audited-series p-values **1, 0.009883, 0.05751, 0.3324**, so only the 2014–2017 → 2018–2021 audited movement is significant and 2025–2026 remains provisional. |
| | |
| | **What changed.** The raw row is explicitly labelled schema field population, a separate audited paper-claim table is published, and the consent subset now uses **32** audited-interacting papers (20 state statefulness, 62.5%) and **55** audited-action papers (32 state statefulness, 58.2%). The page also records the 89.1% unsupported rate alongside the older 19.4% interacting-side false-positive rate. |
| | |
| | **Review.** One **model: 'sonnet'** figures-vs-script pass re-ran ''report_stateful_stateless.mjs'' and ''consent_ni_significance.py'', checked the edited table and the figures immediately before and after it, and found no mismatch. No citation pass was needed: no citekey or quoted claim changed. |
| | |
| | **Rejected finding.** The raw schema series was not accepted as evidence that papers stated consent actions; the eight ''no interaction at all'' papers without consent vocabulary were not folded into the 55-action series. |
| | |
| | ===== Ramakrishnan et al. (IMC 2026) added, 2026-09-29 ===== |
| | |
| | **What.** Karel pointed to Ramakrishnan, Xie, Anand and Li, //Crawling in the Deep: Evaluating Web Crawling Configurations for Web Privacy Measurements//, IMC 2026 {[ramakrishnan2026_crawling]}, with a list of suggested guidance. The paper was read in full and the pages were extended from the paper's own text, not from the suggestion list. Edits are in one refuse-on-mismatch script, ''scripts/apply_ramakrishnan2026_edits.py'', run against a fresh ''export_raw'' of each page. |
| | |
| | **Source and its status.** PDF at ''karthikr.in/data/papers/privacy_diff_imc.pdf'' (15 pages), text extracted with ''pypdf''. **Not in the corpus**: IMC 2026 (12–16 October 2026) had not been held when the corpus was built, and there is no ''data/fulltext/2026/IMC'' directory. So none of its figures enter any corpus count, and the pages say so. Bibliography entry built from Crossref (''10.1145/3777912.3839794'': title, four authors, pages 374–388, //Proceedings of the 2026 ACM Internet Measurement Conference//), key ''ramakrishnan2026_crawling''; no DOI or key collision in ''literature:bibliography'' before adding. |
| | |
| | **The suggestion list, checked against the paper.** Kept here because it is what stops the next edit re-adding the rejected parts. |
| | |
| | ^ Suggested claim ^ Verdict ^ What the paper says ^ |
| | | cited as "IMC 2024" | **wrong** | IMC 2026, Karlsruhe; CrUX list of October 2024 is probably the source of the year | |
| | | stop after 8–13 consecutive pages with nothing new | **kept** | //k// = 11 (domains), 12–13 (APIs), 8 (cookies); //k// = 13 covers all six metrics (§4.3.2) | |
| | | … "yields 90%+ coverage" | **wrong unit** | the criterion is a 0% //median// per-site error relative to a 50-page crawl, not a coverage share | |
| | | … "reduces pages crawled by 30–50%" | **overstated** | fixed-depth crawls need 4.2–52.8% //more// pages; read the other way that is 4.0–34.6% //fewer// pages with adaptive stopping, and only about 5% at a looser error target (footnote 1) | |
| | | small or static sites "saturate after 2–3 pages" | **not in the paper** | dropped | |
| | | over 90% of tracking requests, cookie writes and fingerprinting fire within 15–20 s | **true for tracking domains only** | 93–95% of (site, tracking domain) pairs by 15 s; per site, 65.4% of sites miss no tracking cookie at 15 s and 70.9% at 20 s; fingerprinting APIs 83.8% / over 87% (§4.4) | |
| | | recommend a 15–20 s dwell | **kept, qualified** | the paper recommends 15 s for time-constrained crawls; the page adds that cookie studies should wait longer | |
| | | BFS beats DFS because DFS "gets trapped in homogeneous sub-trees … same tracking templates", finds more "CMP banners" | **mechanism not in the paper** | the paper's evidence is that deep links are more often dead (BFS 50 live pages on ~79% of sites, DFS short on ~half) and that BFS still leads on sites where all strategies found 50 pages; CMP banners were not measured and banners were never interacted with | |
| | | intra-site statefulness needed to see "cookie syncing, stateful retargeting, cookies set once consent state is established" | **not measured** | the paper measures tracking domains, fingerprinting APIs and cookies; state was kept only within a site; stateless crawls gave the //most// raw unique cookies (§4.2) | |
| | | interaction "non-negotiable", "scrolling and cursor movement", trackers defer execution "to verify human presence" | **overstated / not in the paper** | interaction was Tab keystrokes plus scroll-to-bottom every 5 s, no mouse movement; state + interaction gave 0.8–11.6% more tracking domains than other configurations of the same strategy; the paper makes no claim about human-presence checks | |
| | | a new "Configuring Crawl Budgets" section | **declined** | the material fits existing sections of [[programming:interaction]] (depth, page choice, on-page wait) and [[programming:stateful_stateless]]; a parallel page would split them | |
| | |
| | **Figures quoted and where.** All from the paper's text: §3.1 (50 live pages / 100 requests cap, breadth-limited BFS branching 3, 30 s max wait), §3.1.3 (interaction = Tab keystrokes + scroll every 5 s, no clicks), §3.3 (CrUX October 2024, 9,673 of 10K reached, OpenWPM 0.30.0, Firefox 130, headful, one Georgia academic vantage, no bot mitigation), §4.1 (79% / 67% / ~half), §4.2 (1–14% tracking domains, 1–26% tracking cookies, 0.8–11.6% state+interaction, 9.5% stateless cookie total, Wilcoxon signed-rank with Holm), §4.3.1 (fixed depths 33/39/23/24/37/43), §4.3.2 (//k//, median stopping depth and page totals per metric; footnote 1 for the 10%-error operating point), §4.4 (timeout figures), §4.5 (1K-site re-run five months later). The Stafeev and Pellegrino sentence on randomised BFS was checked verbatim in ''data/fulltext/2024/USENIX/sok-state-of-the-krawlers-…/paper.cols.txt''. |
| | |
| | **What this does not establish.** Everything past page 50 (the paper's ceiling); the long tail of the ranking; the post-consent state; state carried across sites; the effect of scrolling alone. |
| | |
| | **On this page's content page.** ''programming:stateful_stateless'' gains one bullet in //What a stateful design buys you// (the within-site comparison and the stateless cookie-count inversion), one reading-list entry, and an update note on the first open question. |
| | |
| | **Review.** One **model: 'sonnet'** pass checked every new figure and quote against the paper text, the Stafeev and Pellegrino ''paper.cols.txt'' and Crossref, then re-checked after fixes. All table figures, the recomputed "fewer" percentages and both verbatim quotes were correct on the first pass. Accepted, all 13 plus two conflicts: the Stafeev sentence overstated support for plain BFS (their Table 8 has BFS and DFS within about 1%, DFS marginally ahead on JS source) — reworded, citation dropped from the breadth-first row; the 9.5% cookie margin is stateless-with-interaction over stateless-without, not stateless over stateful; the 1–14% / 1–26% BFS margins are over all sites, not at equal page counts; "the only measurement at scale of interaction" contradicted the paper's own citation of Demir et al. 2022/2023 — removed; the state+interaction effect is a combination the paper never splits, significance is on per-site distributions, fingerprinting-API margin only 0.5–1.2%; "twice, nothing else changed" misdescribed twelve configurations; "the clean experiment now exists" softened to "closest so far"; per-site cookie medians are similar, only the 75th percentile is higher; the dead-deep-links explanation is the authors' untested reading; the depth analysis configuration, //k// scoping of the 95th-percentile and 10%-error points, and "inefficient on many sites"; the re-run covers adaptive thresholds only and BFS coverage fell 79% → 72%; the cookie-wait caution is labelled as our reading and the paper's single 15 s recommendation is stated; "no mouse movement" is not in the paper; "replication" heading renamed; two pre-existing "only paper" sentences on the interaction page scoped to the corpus; booktitle aligned with Crossref. Rejected: none. Optional qualifier not added: BFS stateful+interaction was not significantly different from BFS stateless+interaction for fingerprinting APIs at depth 50 — the 0.5–1.2% margin on the page already signals it. |
| |