| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:programming:interaction [2026/08/27 12:51] – Write up the silent render truncation and the guard that certified it: ''..'' is monospace, not nowiki, so a literal <WRAP todo> in a table cell opened a never-closed div; check_wrap.mjs masked the wrong construct and reported OK. Record the bisect, the f karel.kubicek.claude | provenance:programming:interaction [2026/09/29 11:39] (current) – Log Ramakrishnan 2026 additions, suggestion-list verdicts, review. Authored by Claude karel.kubicek.claude |
|---|
| ===== The validity problem specific to this page ===== | ===== The validity problem specific to this page ===== |
| |
| **''crawlConfig'' carries ONE evidence quote for the WHOLE object.** ''statefulness'', ''browsers'', ''headless'', ''consentAction'', ''interactionDepth'', ''subpagesPerSite'', ''authentication'', ''languages'' and ''repeatVisits'' all share it. So the site's usual check — "read a handful of ''evidence.quote'' values before publishing a figure" — is **structurally blind** here: the quote behind an ''interactionDepth'' value usually evidences the browser or the statefulness instead. Spot-checking it validates nothing. (The same trap cost ''privacy:consent'' a 19.4% false-positive rate on ''consentAction'' before it was audited.) | **''crawlConfig'' carries ONE evidence quote for the WHOLE object.** ''statefulness'', ''browsers'', ''headless'', ''consentAction'', ''interactionDepth'', ''subpagesPerSite'', ''authentication'', ''languages'' and ''repeatVisits'' all share it. So the site's usual check — "read a handful of ''evidence.quote'' values before publishing a figure" — is **structurally blind** here: the quote behind an ''interactionDepth'' value usually evidences the browser or the statefulness instead. Spot-checking it validates nothing. (The same trap cost ''privacy:consent'' a 19.4% false-positive rate among labels claiming interaction; the complete audit found **279 of 313 no-interaction labels unsupported (89.1%)**.) |
| |
| **How blind, measured.** On the subset where it is cheapest to judge — the 71 web crawls labelled ''landing-plus-subpages'' that //also// state a ''subpagesPerSite'' — every 6th was read on 2026-08-27 and its single shared quote classified by what it actually evidences: | **How blind, measured.** On the subset where it is cheapest to judge — the 71 web crawls labelled ''landing-plus-subpages'' that //also// state a ''subpagesPerSite'' — every 6th was read on 2026-08-27 and its single shared quote classified by what it actually evidences: |
| ===== One deliberate red link ===== | ===== One deliberate red link ===== |
| |
| //Related Pages// links to [[design:automated_measurements]], which does not exist. It is left in because the wiki already promises it from [[start]] and from four other pages, so removing it here would be inconsistent rather than helpful. Every other outbound link on both pages was fetched and returns a real page. | //Related Pages// links to [[design:automated_measurements]], which does not exist. It is left in because the wiki already promises it from [[:start]] and from four other pages, so removing it here would be inconsistent rather than helpful. Every other outbound link on both pages was fetched and returns a real page. |
| |
| ===== External sources: verified, and rejected ===== | ===== External sources: verified, and rejected ===== |
| **The guard was worse than useless.** ''scripts/check_wrap.mjs'' masked ''%%''%%…''%%''%%'' before counting tags — on the assumption that a tag written in inline code is a mention — and did **not** mask ''%%%%%%…%%%%%%''. That is backwards on both counts: it blinded the guard to the one construct that fires, and it would have raised a false alarm on the one that does not. It reported ''OK'' on the broken page every time it was run. | **The guard was worse than useless.** ''scripts/check_wrap.mjs'' masked ''%%''%%…''%%''%%'' before counting tags — on the assumption that a tag written in inline code is a mention — and did **not** mask ''%%%%%%…%%%%%%''. That is backwards on both counts: it blinded the guard to the one construct that fires, and it would have raised a false alarm on the one that does not. It reported ''OK'' on the broken page every time it was run. |
| |
| Fixed on 2026-08-27: mask ''%%<nowiki>%%'' and ''%%%%%%…%%%%%%'' only, and flag a plugin tag inside ''%%''%%…''%%''%%'' as an error in its own right, with the reason in the message. Re-run across every page source in the workdir it now reports **71 instances**, including three pages whose %%<WRAP>%% counts are unbalanced. Two of those were checked live and still render to the end, so this is a lead for whoever owns those pages rather than an emergency — but the class of defect is real and was previously invisible. | Fixed on 2026-08-27: mask ''%%<nowiki>%%'' and ''%%%%%%…%%%%%%'' only, and flag a plugin tag inside ''%%''%%…''%%''%%'' as an error in its own right, with the reason in the message. Re-run across every page source in the workdir it now reports **71 instances**, including three pages whose %%<WRAP>%% counts are unbalanced. Two of those were checked live and still render to the end, so this is a lead for whoever owns those pages rather than an emergency — but the class of defect is real and was previously invisible. Filed as the ''wrap-escape sweep'' item in ''/workspace/drain/work.db'' rather than fixed here, because editing ten other people's pages is outside this item's scope. |
| |
| ===== What could not be established ===== | ===== What could not be established ===== |
| * **How much an LLM agent changes a measurement.** Five papers use one as the browsing instrument; none reports a same-site-list comparison against a scripted crawl. This is on the page as an open question. | * **How much an LLM agent changes a measurement.** Five papers use one as the browsing instrument; none reports a same-site-list comparison against a scripted crawl. This is on the page as an open question. |
| * **Whether ''subpagesPerSite'' means the same thing across papers.** "10 subpages" could be 10 attempted, 10 successfully loaded, or 10 distinct URLs. Nothing in the schema distinguishes them and few papers say. The page reports the distribution and does not compute a mean. | * **Whether ''subpagesPerSite'' means the same thing across papers.** "10 subpages" could be 10 attempted, 10 successfully loaded, or 10 distinct URLs. Nothing in the schema distinguishes them and few papers say. The page reports the distribution and does not compute a mean. |
| * **''design:sampling'' now disagrees with this page and was not updated.** It publishes its own depth figures — 24.8% landing-only of 680 papers, "15.0% state how many subpages" — against this page's 37.2% of 417 and 12.1% of 857. The populations are different and each page states its own, but nothing reconciles them for a reader who reads both. Its Hispar box, checked 2026-08-12, says there is "no maintained equivalent … no tooling to address it", which is now stale against this page's HTTP Archive secondary-page paragraph. **Deferred:** editing that page is outside this item's scope. The change it needs is one sentence pointing here. | * **''design:sampling'' now disagrees with this page and was not updated.** It publishes its own depth figures — 24.8% landing-only of 680 papers, "15.0% state how many subpages" — against this page's 37.2% of 417 and 12.1% of 857. The populations are different and each page states its own, but nothing reconciles them for a reader who reads both. Its Hispar box, checked 2026-08-12, says there is "no maintained equivalent … no tooling to address it", which is now stale against this page's HTTP Archive secondary-page paragraph. **Deferred**, and filed: editing that page is outside this item's scope, so it is recorded as the ''sampling depth reconcile'' item in ''/workspace/drain/work.db''. The change it needs is one sentence pointing here plus a scope note on the two depth figures. |
| * **Why a literal %%<WRAP todo>%% in a table cell truncates the page, while the same literal in prose elsewhere on this wiki does not, is not established.** The fix is verified causally — the page truncated before the escape and rendered in full immediately after, with nothing else changed — but two other provenance pages carry unbalanced %%<WRAP>%% counts and render to the end, so the table cell is doing something the prose case is not. Not chased further. | * **Why a literal %%<WRAP todo>%% in a table cell truncates the page, while the same literal in prose elsewhere on this wiki does not, is not established.** The fix is verified causally — the page truncated before the escape and rendered in full immediately after, with nothing else changed — but two other provenance pages carry unbalanced %%<WRAP>%% counts and render to the end, so the table cell is doing something the prose case is not. Not chased further. |
| * **No discriminant was run for ''crawlConfig.authentication''.** The page's 553 ''none'' values are treated as an upper bound in the prose, but the same ''LANDING''/''DEEPER''-style check that validated ''interactionDepth'' would take an afternoon and has not been done. | * **No discriminant was run for ''crawlConfig.authentication''.** The page's 553 ''none'' values are treated as an upper bound in the prose, but the same ''LANDING''/''DEEPER''-style check that validated ''interactionDepth'' would take an afternoon and has not been done. |
| Four reviewers, all told explicitly that the author's context might not be exhaustive, all handed the page text, the report script and its unedited output, the fold module and the provenance notes. The three focused ones ran in parallel first and every finding they raised was accepted; the generic reviewer then ran on the corrected page, and its findings are logged below with what was done about each. | Four reviewers, all told explicitly that the author's context might not be exhaustive, all handed the page text, the report script and its unedited output, the fold module and the provenance notes. The three focused ones ran in parallel first and every finding they raised was accepted; the generic reviewer then ran on the corrected page, and its findings are logged below with what was done about each. |
| |
| ==== Reviewer 1 — figures against the script (''model: sonnet'') ==== | ==== Reviewer 1 — figures against the script (model: sonnet) ==== |
| |
| ^ Finding ^ Verdict ^ What was done ^ | ^ Finding ^ Verdict ^ What was done ^ |
| | The page said 81 papers "got past a login of any kind — 42 by registering an account, 24 by logging in manually, 15 with automated login and **one via SSO**". The script's ''authentication'' breakdown for the 857 web crawls has **no ''sso'' row**, and 42+24+15 already sums to 81, so the SSO clause made the page contradict its own total | **accepted; a real error** | The ''sso'' count was carried over from an early ''site_queries.mjs --page crawl-config'' run, whose population is the 1,080 papers with a ''crawlConfig'' object across **all** platforms, not this page's 857 web crawls. There is exactly one ''authentication: sso'' paper in the whole corpus and it is an Android study. Clause removed; the page now says "Not one of the 857 used SSO" with a footnote naming why. The report now prints the check explicitly so it cannot recur | | | The page said 81 papers "got past a login of any kind — 42 by registering an account, 24 by logging in manually, 15 with automated login and **one via SSO**". The script's ''authentication'' breakdown for the 857 web crawls has **no ''sso'' row**, and 42+24+15 already sums to 81, so the SSO clause made the page contradict its own total | **accepted; a real error** | The ''sso'' count was carried over from an early ''site_queries.mjs %%--page%% crawl-config'' run, whose population is the 1,080 papers with a ''crawlConfig'' object across **all** platforms, not this page's 857 web crawls. There is exactly one ''authentication: sso'' paper in the whole corpus and it is an Android study. Clause removed; the page now says "Not one of the 857 used SSO" with a footnote naming why. The report now prints the check explicitly so it cannot recur | |
| | The published %%<file javascript interaction_probe.mjs>%% block was **not byte-identical** to the committed script — it dropped the trailing console block, so the code as printed would not produce the output printed beside it | **accepted** | The committed script's header was changed to a generic invocation, and both %%<file>%% blocks are now generated from the files and asserted byte-identical | | | The published %%<file javascript interaction_probe.mjs>%% block was **not byte-identical** to the committed script — it dropped the trailing console block, so the code as printed would not produce the output printed beside it | **accepted** | The committed script's header was changed to a generic invocation, and both %%<file>%% blocks are now generated from the files and asserted byte-identical | |
| | "Playwright 1.62.1, Chromium 151.0.7922.34, Linux" was asserted **beside** the output rather than printed **by** it — the script only queried the Playwright version and never the Chromium build or the platform | **accepted** | The probe now records ''browser.version()'' and ''process.platform'' and prints both. The version line on the page is now part of the quoted output | | | "Playwright 1.62.1, Chromium 151.0.7922.34, Linux" was asserted **beside** the output rather than printed **by** it — the script only queried the Playwright version and never the Chromium build or the platform | **accepted** | The probe now records ''browser.version()'' and ''process.platform'' and prints both. The version line on the page is now part of the quoted output | |
| That first finding is the exact failure mode the task spec warns about: **a number carried over from a different population**. It survived the author's own number guard because the guard checks that a figure appears //in the report//, and "1" appears everywhere. | That first finding is the exact failure mode the task spec warns about: **a number carried over from a different population**. It survived the author's own number guard because the guard checks that a figure appears //in the report//, and "1" appears everywhere. |
| |
| ==== Reviewer 2 — citations and quotes (''model: sonnet'') ==== | ==== Reviewer 2 — citations and quotes (model: sonnet) ==== |
| |
| ^ Finding ^ Verdict ^ What was done ^ | ^ Finding ^ Verdict ^ What was done ^ |
| One thing the reviewer noticed that is **not** a wiki error and is worth recording: the Sun et al. paper states "87% in Phase 2 and 79% in Phase 3" in one place and "87% in Phase 1 and 79% in Phase 2" in another. **The inconsistency is in the source paper.** The page quotes the first form, which is the one in the results section. | One thing the reviewer noticed that is **not** a wiki error and is worth recording: the Sun et al. paper states "87% in Phase 2 and 79% in Phase 3" in one place and "87% in Phase 1 and 79% in Phase 2" in another. **The inconsistency is in the source paper.** The page quotes the first form, which is the one in the results section. |
| |
| ==== Reviewer 3 — external currency (''model: sonnet'') ==== | ==== Reviewer 3 — external currency (model: sonnet) ==== |
| |
| ^ Finding ^ Verdict ^ What was done ^ | ^ Finding ^ Verdict ^ What was done ^ |
| Two of these — the un-computed median and the probe-zero — passed ''check_page_numbers.mjs'' cleanly, because "10" and "0" both appear in the report attached to something else. **The number guard proves a digit is in the report, not that it is the right digit for its sentence.** | Two of these — the un-computed median and the probe-zero — passed ''check_page_numbers.mjs'' cleanly, because "10" and "0" both appear in the report attached to something else. **The number guard proves a digit is in the report, not that it is the right digit for its sentence.** |
| |
| ==== Reviewer 4 — generic, no checklist (''model: fable'') ==== | ==== Reviewer 4 — generic, no checklist (model: fable) ==== |
| |
| Given both pages, the scripts and the fixture, told the reader's profile and the corpus's scope, and told explicitly not to repeat the other three briefs. It reviewed the provenance page as well as the content page. Thirteen findings; twelve accepted in full, one accepted in part. | Given both pages, the scripts and the fixture, told the reader's profile and the corpus's scope, and told explicitly not to repeat the other three briefs. It reviewed the provenance page as well as the content page. Thirteen findings; twelve accepted in full, one accepted in part. |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| | |
| | ===== Link-hygiene sweep, 2026-09-17 ===== |
| | |
| | The fresh rendered-DOM sweep found this page's bare ''start'' link resolved to ''provenance:programming:start''. It is now root-anchored; the post-save DOM was re-checked for red links. No figures or citations changed. |
| | |
| | ===== ConsentAction audit propagation, 2026-09-17 ===== |
| | |
| | **What was wrong.** The crawl-configuration comparison called 349/1,120 = 31.2% ''consentAction'' papers without stating that this is raw field population. It was not a supported paper-claim count. |
| | |
| | **Command and real output.** ''node scripts/report_interaction.mjs'' → ''consentAction (field populated; not an audited paper claim) | 349 | 31.2%''. ''node scripts/consent_action_noninteraction_audit.mjs'' → **55/1,120 = 4.9%** audited paper claims; **279/313 = 89.1%** unsupported ''no-interaction'' labels and **7/36 = 19.4%** false positives among labels claiming interaction. |
| | |
| | **What changed.** The page and the report output now qualify the row as schema population. Its limitations paragraph names both sides of the audit: the 89.1% unsupported no-interaction rate and the 19.4% interacting-label false-positive rate. |
| | |
| | **Review.** One **model: 'sonnet'** figures-vs-script pass re-ran ''report_interaction.mjs'' and checked the consent row and adjacent crawl-configuration rows. No mismatch was found. No citation pass was needed: no citekey or quoted claim changed. |
| | |
| | **Rejected finding.** The 349 schema value was not replaced by 55 inside this schema comparison; the page now says what the row means and points to the separate audit. |
| | |
| | ===== Ramakrishnan et al. (IMC 2026) added, 2026-09-29 ===== |
| | |
| | **What.** Karel pointed to Ramakrishnan, Xie, Anand and Li, //Crawling in the Deep: Evaluating Web Crawling Configurations for Web Privacy Measurements//, IMC 2026 {[ramakrishnan2026_crawling]}, with a list of suggested guidance. The paper was read in full and the pages were extended from the paper's own text, not from the suggestion list. Edits are in one refuse-on-mismatch script, ''scripts/apply_ramakrishnan2026_edits.py'', run against a fresh ''export_raw'' of each page. |
| | |
| | **Source and its status.** PDF at ''karthikr.in/data/papers/privacy_diff_imc.pdf'' (15 pages), text extracted with ''pypdf''. **Not in the corpus**: IMC 2026 (12–16 October 2026) had not been held when the corpus was built, and there is no ''data/fulltext/2026/IMC'' directory. So none of its figures enter any corpus count, and the pages say so. Bibliography entry built from Crossref (''10.1145/3777912.3839794'': title, four authors, pages 374–388, //Proceedings of the 2026 ACM Internet Measurement Conference//), key ''ramakrishnan2026_crawling''; no DOI or key collision in ''literature:bibliography'' before adding. |
| | |
| | **The suggestion list, checked against the paper.** Kept here because it is what stops the next edit re-adding the rejected parts. |
| | |
| | ^ Suggested claim ^ Verdict ^ What the paper says ^ |
| | | cited as "IMC 2024" | **wrong** | IMC 2026, Karlsruhe; CrUX list of October 2024 is probably the source of the year | |
| | | stop after 8–13 consecutive pages with nothing new | **kept** | //k// = 11 (domains), 12–13 (APIs), 8 (cookies); //k// = 13 covers all six metrics (§4.3.2) | |
| | | … "yields 90%+ coverage" | **wrong unit** | the criterion is a 0% //median// per-site error relative to a 50-page crawl, not a coverage share | |
| | | … "reduces pages crawled by 30–50%" | **overstated** | fixed-depth crawls need 4.2–52.8% //more// pages; read the other way that is 4.0–34.6% //fewer// pages with adaptive stopping, and only about 5% at a looser error target (footnote 1) | |
| | | small or static sites "saturate after 2–3 pages" | **not in the paper** | dropped | |
| | | over 90% of tracking requests, cookie writes and fingerprinting fire within 15–20 s | **true for tracking domains only** | 93–95% of (site, tracking domain) pairs by 15 s; per site, 65.4% of sites miss no tracking cookie at 15 s and 70.9% at 20 s; fingerprinting APIs 83.8% / over 87% (§4.4) | |
| | | recommend a 15–20 s dwell | **kept, qualified** | the paper recommends 15 s for time-constrained crawls; the page adds that cookie studies should wait longer | |
| | | BFS beats DFS because DFS "gets trapped in homogeneous sub-trees … same tracking templates", finds more "CMP banners" | **mechanism not in the paper** | the paper's evidence is that deep links are more often dead (BFS 50 live pages on ~79% of sites, DFS short on ~half) and that BFS still leads on sites where all strategies found 50 pages; CMP banners were not measured and banners were never interacted with | |
| | | intra-site statefulness needed to see "cookie syncing, stateful retargeting, cookies set once consent state is established" | **not measured** | the paper measures tracking domains, fingerprinting APIs and cookies; state was kept only within a site; stateless crawls gave the //most// raw unique cookies (§4.2) | |
| | | interaction "non-negotiable", "scrolling and cursor movement", trackers defer execution "to verify human presence" | **overstated / not in the paper** | interaction was Tab keystrokes plus scroll-to-bottom every 5 s, no mouse movement; state + interaction gave 0.8–11.6% more tracking domains than other configurations of the same strategy; the paper makes no claim about human-presence checks | |
| | | a new "Configuring Crawl Budgets" section | **declined** | the material fits existing sections of [[programming:interaction]] (depth, page choice, on-page wait) and [[programming:stateful_stateless]]; a parallel page would split them | |
| | |
| | **Figures quoted and where.** All from the paper's text: §3.1 (50 live pages / 100 requests cap, breadth-limited BFS branching 3, 30 s max wait), §3.1.3 (interaction = Tab keystrokes + scroll every 5 s, no clicks), §3.3 (CrUX October 2024, 9,673 of 10K reached, OpenWPM 0.30.0, Firefox 130, headful, one Georgia academic vantage, no bot mitigation), §4.1 (79% / 67% / ~half), §4.2 (1–14% tracking domains, 1–26% tracking cookies, 0.8–11.6% state+interaction, 9.5% stateless cookie total, Wilcoxon signed-rank with Holm), §4.3.1 (fixed depths 33/39/23/24/37/43), §4.3.2 (//k//, median stopping depth and page totals per metric; footnote 1 for the 10%-error operating point), §4.4 (timeout figures), §4.5 (1K-site re-run five months later). The Stafeev and Pellegrino sentence on randomised BFS was checked verbatim in ''data/fulltext/2024/USENIX/sok-state-of-the-krawlers-…/paper.cols.txt''. |
| | |
| | **What this does not establish.** Everything past page 50 (the paper's ceiling); the long tail of the ranking; the post-consent state; state carried across sites; the effect of scrolling alone. |
| | |
| | **On this page's content page.** ''programming:interaction'' gains subsections on the second saturation study and adaptive stopping, traversal order, and per-page wait, plus rows in the current-methods table, the reporting list, the default design and three open questions; two pre-existing "only paper" sentences were scoped to the corpus. |
| | |
| | **Review.** One **model: 'sonnet'** pass checked every new figure and quote against the paper text, the Stafeev and Pellegrino ''paper.cols.txt'' and Crossref, then re-checked after fixes. All table figures, the recomputed "fewer" percentages and both verbatim quotes were correct on the first pass. Accepted, all 13 plus two conflicts: the Stafeev sentence overstated support for plain BFS (their Table 8 has BFS and DFS within about 1%, DFS marginally ahead on JS source) — reworded, citation dropped from the breadth-first row; the 9.5% cookie margin is stateless-with-interaction over stateless-without, not stateless over stateful; the 1–14% / 1–26% BFS margins are over all sites, not at equal page counts; "the only measurement at scale of interaction" contradicted the paper's own citation of Demir et al. 2022/2023 — removed; the state+interaction effect is a combination the paper never splits, significance is on per-site distributions, fingerprinting-API margin only 0.5–1.2%; "twice, nothing else changed" misdescribed twelve configurations; "the clean experiment now exists" softened to "closest so far"; per-site cookie medians are similar, only the 75th percentile is higher; the dead-deep-links explanation is the authors' untested reading; the depth analysis configuration, //k// scoping of the 95th-percentile and 10%-error points, and "inefficient on many sites"; the re-run covers adaptive thresholds only and BFS coverage fell 79% → 72%; the cookie-wait caution is labelled as our reading and the paper's single 15 s recommendation is stated; "no mouse movement" is not in the paper; "replication" heading renamed; two pre-existing "only paper" sentences on the interaction page scoped to the corpus; booktitle aligned with Crossref. Rejected: none. Optional qualifier not added: BFS stateful+interaction was not significantly different from BFS stateless+interaction for fingerprinting APIs at depth 50 — the 0.5–1.2% margin on the page already signals it. |
| |
