Differences

This shows you the differences between two versions of the page.

--- provenance:programming:interaction [2026/09/17 08:14] – Qualify root start link; Authored by Claude karel.kubicek.claude
+++ provenance:programming:interaction [2026/09/29 11:39] (current) – Log Ramakrishnan 2026 additions, suggestion-list verdicts, review. Authored by Claude karel.kubicek.claude
@@ Line 120: / Line 120: @@
 ===== The validity problem specific to this page =====
-**''crawlConfig'' carries ONE evidence quote for the WHOLE object.** ''statefulness'', ''browsers'', ''headless'', ''consentAction'', ''interactionDepth'', ''subpagesPerSite'', ''authentication'', ''languages'' and ''repeatVisits'' all share it. So the site's usual check — "read a handful of ''evidence.quote'' values before publishing a figure" — is **structurally blind** here: the quote behind an ''interactionDepth'' value usually evidences the browser or the statefulness instead. Spot-checking it validates nothing. (The same trap cost ''privacy:consent'' a 19.4% false-positive rate on ''consentAction'' before it was audited.)
+**''crawlConfig'' carries ONE evidence quote for the WHOLE object.** ''statefulness'', ''browsers'', ''headless'', ''consentAction'', ''interactionDepth'', ''subpagesPerSite'', ''authentication'', ''languages'' and ''repeatVisits'' all share it. So the site's usual check — "read a handful of ''evidence.quote'' values before publishing a figure" — is **structurally blind** here: the quote behind an ''interactionDepth'' value usually evidences the browser or the statefulness instead. Spot-checking it validates nothing. (The same trap cost ''privacy:consent'' a 19.4% false-positive rate among labels claiming interaction; the complete audit found **279 of 313 no-interaction labels unsupported (89.1%)**.)
 **How blind, measured.** On the subset where it is cheapest to judge — the 71 web crawls labelled ''landing-plus-subpages'' that //also// state a ''subpagesPerSite'' — every 6th was read on 2026-08-27 and its single shared quote classified by what it actually evidences:
@@ Line 416: / Line 416: @@
 The fresh rendered-DOM sweep found this page's bare ''start'' link resolved to ''provenance:programming:start''. It is now root-anchored; the post-save DOM was re-checked for red links. No figures or citations changed.
+===== ConsentAction audit propagation, 2026-09-17 =====
+**What was wrong.** The crawl-configuration comparison called 349/1,120 = 31.2% ''consentAction'' papers without stating that this is raw field population. It was not a supported paper-claim count.
+**Command and real output.** ''node scripts/report_interaction.mjs'' → ''consentAction (field populated; not an audited paper claim) | 349 | 31.2%''. ''node scripts/consent_action_noninteraction_audit.mjs'' → **55/1,120 = 4.9%** audited paper claims; **279/313 = 89.1%** unsupported ''no-interaction'' labels and **7/36 = 19.4%** false positives among labels claiming interaction.
+**What changed.** The page and the report output now qualify the row as schema population. Its limitations paragraph names both sides of the audit: the 89.1% unsupported no-interaction rate and the 19.4% interacting-label false-positive rate.
+**Review.** One **model: 'sonnet'** figures-vs-script pass re-ran ''report_interaction.mjs'' and checked the consent row and adjacent crawl-configuration rows. No mismatch was found. No citation pass was needed: no citekey or quoted claim changed.
+**Rejected finding.** The 349 schema value was not replaced by 55 inside this schema comparison; the page now says what the row means and points to the separate audit.
+===== Ramakrishnan et al. (IMC 2026) added, 2026-09-29 =====
+**What.** Karel pointed to Ramakrishnan, Xie, Anand and Li, //Crawling in the Deep: Evaluating Web Crawling Configurations for Web Privacy Measurements//, IMC 2026 {[ramakrishnan2026_crawling]}, with a list of suggested guidance. The paper was read in full and the pages were extended from the paper's own text, not from the suggestion list. Edits are in one refuse-on-mismatch script, ''scripts/apply_ramakrishnan2026_edits.py'', run against a fresh ''export_raw'' of each page.
+**Source and its status.** PDF at ''karthikr.in/data/papers/privacy_diff_imc.pdf'' (15 pages), text extracted with ''pypdf''. **Not in the corpus**: IMC 2026 (12–16 October 2026) had not been held when the corpus was built, and there is no ''data/fulltext/2026/IMC'' directory. So none of its figures enter any corpus count, and the pages say so. Bibliography entry built from Crossref (''10.1145/3777912.3839794'': title, four authors, pages 374–388, //Proceedings of the 2026 ACM Internet Measurement Conference//), key ''ramakrishnan2026_crawling''; no DOI or key collision in ''literature:bibliography'' before adding.
+**The suggestion list, checked against the paper.** Kept here because it is what stops the next edit re-adding the rejected parts.
+^ Suggested claim ^ Verdict ^ What the paper says ^
+| cited as "IMC 2024" | **wrong** | IMC 2026, Karlsruhe; CrUX list of October 2024 is probably the source of the year |
+| stop after 8–13 consecutive pages with nothing new | **kept** | //k// = 11 (domains), 12–13 (APIs), 8 (cookies); //k// = 13 covers all six metrics (§4.3.2) |
+| … "yields 90%+ coverage" | **wrong unit** | the criterion is a 0% //median// per-site error relative to a 50-page crawl, not a coverage share |
+| … "reduces pages crawled by 30–50%" | **overstated** | fixed-depth crawls need 4.2–52.8% //more// pages; read the other way that is 4.0–34.6% //fewer// pages with adaptive stopping, and only about 5% at a looser error target (footnote 1) |
+| small or static sites "saturate after 2–3 pages" | **not in the paper** | dropped |
+| over 90% of tracking requests, cookie writes and fingerprinting fire within 15–20 s | **true for tracking domains only** | 93–95% of (site, tracking domain) pairs by 15 s; per site, 65.4% of sites miss no tracking cookie at 15 s and 70.9% at 20 s; fingerprinting APIs 83.8% / over 87% (§4.4) |
+| recommend a 15–20 s dwell | **kept, qualified** | the paper recommends 15 s for time-constrained crawls; the page adds that cookie studies should wait longer |
+| BFS beats DFS because DFS "gets trapped in homogeneous sub-trees … same tracking templates", finds more "CMP banners" | **mechanism not in the paper** | the paper's evidence is that deep links are more often dead (BFS 50 live pages on ~79% of sites, DFS short on ~half) and that BFS still leads on sites where all strategies found 50 pages; CMP banners were not measured and banners were never interacted with |
+| intra-site statefulness needed to see "cookie syncing, stateful retargeting, cookies set once consent state is established" | **not measured** | the paper measures tracking domains, fingerprinting APIs and cookies; state was kept only within a site; stateless crawls gave the //most// raw unique cookies (§4.2) |
+| interaction "non-negotiable", "scrolling and cursor movement", trackers defer execution "to verify human presence" | **overstated / not in the paper** | interaction was Tab keystrokes plus scroll-to-bottom every 5 s, no mouse movement; state + interaction gave 0.8–11.6% more tracking domains than other configurations of the same strategy; the paper makes no claim about human-presence checks |
+| a new "Configuring Crawl Budgets" section | **declined** | the material fits existing sections of [[programming:interaction]] (depth, page choice, on-page wait) and [[programming:stateful_stateless]]; a parallel page would split them |
+**Figures quoted and where.** All from the paper's text: §3.1 (50 live pages / 100 requests cap, breadth-limited BFS branching 3, 30 s max wait), §3.1.3 (interaction = Tab keystrokes + scroll every 5 s, no clicks), §3.3 (CrUX October 2024, 9,673 of 10K reached, OpenWPM 0.30.0, Firefox 130, headful, one Georgia academic vantage, no bot mitigation), §4.1 (79% / 67% / ~half), §4.2 (1–14% tracking domains, 1–26% tracking cookies, 0.8–11.6% state+interaction, 9.5% stateless cookie total, Wilcoxon signed-rank with Holm), §4.3.1 (fixed depths 33/39/23/24/37/43), §4.3.2 (//k//, median stopping depth and page totals per metric; footnote 1 for the 10%-error operating point), §4.4 (timeout figures), §4.5 (1K-site re-run five months later). The Stafeev and Pellegrino sentence on randomised BFS was checked verbatim in ''data/fulltext/2024/USENIX/sok-state-of-the-krawlers-…/paper.cols.txt''.
+**What this does not establish.** Everything past page 50 (the paper's ceiling); the long tail of the ranking; the post-consent state; state carried across sites; the effect of scrolling alone.
+**On this page's content page.** ''programming:interaction'' gains subsections on the second saturation study and adaptive stopping, traversal order, and per-page wait, plus rows in the current-methods table, the reporting list, the default design and three open questions; two pre-existing "only paper" sentences were scoped to the corpus.
+**Review.** One **model: 'sonnet'** pass checked every new figure and quote against the paper text, the Stafeev and Pellegrino ''paper.cols.txt'' and Crossref, then re-checked after fixes. All table figures, the recomputed "fewer" percentages and both verbatim quotes were correct on the first pass. Accepted, all 13 plus two conflicts: the Stafeev sentence overstated support for plain BFS (their Table 8 has BFS and DFS within about 1%, DFS marginally ahead on JS source) — reworded, citation dropped from the breadth-first row; the 9.5% cookie margin is stateless-with-interaction over stateless-without, not stateless over stateful; the 1–14% / 1–26% BFS margins are over all sites, not at equal page counts; "the only measurement at scale of interaction" contradicted the paper's own citation of Demir et al. 2022/2023 — removed; the state+interaction effect is a combination the paper never splits, significance is on per-site distributions, fingerprinting-API margin only 0.5–1.2%; "twice, nothing else changed" misdescribed twelve configurations; "the clean experiment now exists" softened to "closest so far"; per-site cookie medians are similar, only the 75th percentile is higher; the dead-deep-links explanation is the authors' untested reading; the depth analysis configuration, //k// scoping of the 95th-percentile and 10%-error points, and "inefficient on many sites"; the re-run covers adaptive thresholds only and BFS coverage fell 79% → 72%; the cookie-wait caution is labelled as our reading and the paper's single 15 s recommendation is stated; "no mouse movement" is not in the paper; "replication" heading renamed; two pre-existing "only paper" sentences on the interaction page scoped to the corpus; booktitle aligned with Crossref. Rejected: none. Optional qualifier not added: BFS stateful+interaction was not significantly different from BFS stateless+interaction for fingerprinting APIs at depth 50 — the 0.5–1.2% margin on the page already signals it.