| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:programming:interaction [2026/08/27 12:46] – Move the five raw script outputs to provenance:programming:interaction:outputs and link each from the point that produced it: with them inline this page silently truncated its own rendering, losing the Related and References sections. Record the bisect an karel.kubicek.claude | provenance:programming:interaction [2026/08/27 12:52] (current) – Name the two deferred follow-ups by their drain item keys. Authored by Claude karel.kubicek.claude |
|---|
| | Date | 2026-08-27 | | | Date | 2026-08-27 | |
| | Corpus at the time | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues, 2010–2026 (the 2026-08-11 extension, commit ''8a6b843'') | | | Corpus at the time | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues, 2010–2026 (the 2026-08-11 extension, commit ''8a6b843'') | |
| | Page before | a 2,411-byte stub: a ''<WRAP todo>'' box, three bullet topics, and one claim about Urban et al. | | | Page before | a 2,411-byte stub: a %%<WRAP todo>%% box, three bullet topics, and one claim about Urban et al. | |
| | Page after | 56,675 bytes, rev 1787834365 — eight saves: the initial publish, one per review round, and five self-review corrections | | | Page after | 56,675 bytes, rev 1787834365 — eight saves: the initial publish, one per review round, and five self-review corrections | |
| | Model | Claude Opus 5, one session, no sub-agents used for research | | | Model | Claude Opus 5, one session, no sub-agents used for research | |
| * The ladder is **monotone and each rung adds exactly one beacon**, which is the point: depth does not substitute for scrolling and scrolling does not substitute for clicking. | * The ladder is **monotone and each rung adds exactly one beacon**, which is the point: depth does not substitute for scrolling and scrolling does not substitute for clicking. |
| |
| The two files are published on the page in ''<file>'' blocks and are byte-identical to ''sandbox/interaction_site.mjs'' and ''sandbox/interaction_probe.mjs'' — asserted by a diff on every save, and checked end-to-end (see [[#The published code was checked end-to-end]]). | The two files are published on the page in %%<file>%% blocks and are byte-identical to ''sandbox/interaction_site.mjs'' and ''sandbox/interaction_probe.mjs'' — asserted by a diff on every save, and checked end-to-end (see [[#The published code was checked end-to-end]]). |
| |
| ===== Quotes checked ===== | ===== Quotes checked ===== |
| ===== The published code was checked end-to-end ===== | ===== The published code was checked end-to-end ===== |
| |
| Not just "the ''<file>'' block matches the committed file", which is where this check usually stops. Both code blocks were downloaded back **off the rendered page** through DokuWiki's own export, diffed against the committed scripts, copied into an empty directory and run: | Not just "the %%<file>%% block matches the committed file", which is where this check usually stops. Both code blocks were downloaded back **off the rendered page** through DokuWiki's own export, diffed against the committed scripts, copied into an empty directory and run: |
| |
| <code> | <code> |
| | ''design_mobile_and_app_measurement.txt'' / ''report_mobile-output.txt'' | identical (both fail — pre-existing, unrelated) | | | ''design_mobile_and_app_measurement.txt'' / ''report_mobile-output.txt'' | identical (both fail — pre-existing, unrelated) | |
| | ''design_longitudinal.txt'' / ''report_longitudinal-output.txt'' | **improved**: unaccounted 24 → 20, and three four-part Chromium versions (''113.0.5672.0'', ''152.0.7977.54'', ''154.0.8026.0'') now reported as version identifiers instead of as digit fragments. That pair was already failing before the change and still fails | | | ''design_longitudinal.txt'' / ''report_longitudinal-output.txt'' | **improved**: unaccounted 24 → 20, and three four-part Chromium versions (''113.0.5672.0'', ''152.0.7977.54'', ''154.0.8026.0'') now reported as version identifiers instead of as digit fragments. That pair was already failing before the change and still fails | |
| | |
| | ===== The page truncated itself, silently, and the guard said OK ===== |
| | |
| | Worth a section rather than a footnote, because it is a failure mode with no error message anywhere and a guard that actively certified it. |
| | |
| | **Symptom.** After a routine save, the rendered page simply stopped mid-table. The //Related// and //References// sections were not in the HTML at all — no purple warning, no PHP notice, no truncated response; DokuWiki emitted a complete page that happened to contain only part of the content. The only way to notice is to count what rendered against what you wrote. |
| | |
| | **What it was not.** Two hours of bisecting on ''playground:playground'' ruled out: total size (a neighbouring provenance page renders 143 KB), the biggest embedded output block (removing 27 KB changed nothing), the review section (removing 16 KB changed nothing), the specific table cell it stopped at (rewriting it moved nothing), and every section on its own (all 13 rendered fine in isolation). Cumulative prefixes gave the decisive clue by being **non-monotone** — the first 11 KB failed while the first 34 KB passed — which cannot happen under any size or complexity budget, and can only happen if something is left **open** and something later closes it. |
| | |
| | **What it was.** One table cell in the run table at the top of the page: |
| | |
| | <code> |
| | | Page before | a 2,411-byte stub: a ''<WRAP todo>'' box, three bullet topics, ... | |
| | </code> |
| | |
| | (with real angle brackets, of course — they are escaped here so this page does not reproduce the bug while describing it.) |
| | |
| | ''%%''%%…''%%''%%'' is **monospace, not nowiki.** It changes how the text looks and does nothing whatever to stop the wrap plugin. That literal opened a real, never-closed ''%%<div>%%'', and everything after it vanished. Replacing the quotes with ''%%%%%%…%%%%%%'' fixed it on the next save, with nothing else changed. |
| | |
| | **The guard was worse than useless.** ''scripts/check_wrap.mjs'' masked ''%%''%%…''%%''%%'' before counting tags — on the assumption that a tag written in inline code is a mention — and did **not** mask ''%%%%%%…%%%%%%''. That is backwards on both counts: it blinded the guard to the one construct that fires, and it would have raised a false alarm on the one that does not. It reported ''OK'' on the broken page every time it was run. |
| | |
| | Fixed on 2026-08-27: mask ''%%<nowiki>%%'' and ''%%%%%%…%%%%%%'' only, and flag a plugin tag inside ''%%''%%…''%%''%%'' as an error in its own right, with the reason in the message. Re-run across every page source in the workdir it now reports **71 instances**, including three pages whose %%<WRAP>%% counts are unbalanced. Two of those were checked live and still render to the end, so this is a lead for whoever owns those pages rather than an emergency — but the class of defect is real and was previously invisible. Filed as the ''wrap-escape sweep'' item in ''/workspace/drain/work.db'' rather than fixed here, because editing ten other people's pages is outside this item's scope. |
| |
| ===== What could not be established ===== | ===== What could not be established ===== |
| * **How much an LLM agent changes a measurement.** Five papers use one as the browsing instrument; none reports a same-site-list comparison against a scripted crawl. This is on the page as an open question. | * **How much an LLM agent changes a measurement.** Five papers use one as the browsing instrument; none reports a same-site-list comparison against a scripted crawl. This is on the page as an open question. |
| * **Whether ''subpagesPerSite'' means the same thing across papers.** "10 subpages" could be 10 attempted, 10 successfully loaded, or 10 distinct URLs. Nothing in the schema distinguishes them and few papers say. The page reports the distribution and does not compute a mean. | * **Whether ''subpagesPerSite'' means the same thing across papers.** "10 subpages" could be 10 attempted, 10 successfully loaded, or 10 distinct URLs. Nothing in the schema distinguishes them and few papers say. The page reports the distribution and does not compute a mean. |
| * **''design:sampling'' now disagrees with this page and was not updated.** It publishes its own depth figures — 24.8% landing-only of 680 papers, "15.0% state how many subpages" — against this page's 37.2% of 417 and 12.1% of 857. The populations are different and each page states its own, but nothing reconciles them for a reader who reads both. Its Hispar box, checked 2026-08-12, says there is "no maintained equivalent … no tooling to address it", which is now stale against this page's HTTP Archive secondary-page paragraph. **Deferred:** editing that page is outside this item's scope. The change it needs is one sentence pointing here. | * **''design:sampling'' now disagrees with this page and was not updated.** It publishes its own depth figures — 24.8% landing-only of 680 papers, "15.0% state how many subpages" — against this page's 37.2% of 417 and 12.1% of 857. The populations are different and each page states its own, but nothing reconciles them for a reader who reads both. Its Hispar box, checked 2026-08-12, says there is "no maintained equivalent … no tooling to address it", which is now stale against this page's HTTP Archive secondary-page paragraph. **Deferred**, and filed: editing that page is outside this item's scope, so it is recorded as the ''sampling depth reconcile'' item in ''/workspace/drain/work.db''. The change it needs is one sentence pointing here plus a scope note on the two depth figures. |
| * **Why the render truncates is not established, only where.** The bisect isolates a threshold between about 51 KB and 60 KB of source for this page's content mix, but not the mechanism — most likely a PCRE backtrack or instruction limit in the parser rather than anything wrong with the markup, since every construct on the page is balanced and the truncated tail renders perfectly in isolation. Anyone adding to this log should re-check that the page still ends with its //References// heading. | * **Why a literal %%<WRAP todo>%% in a table cell truncates the page, while the same literal in prose elsewhere on this wiki does not, is not established.** The fix is verified causally — the page truncated before the escape and rendered in full immediately after, with nothing else changed — but two other provenance pages carry unbalanced %%<WRAP>%% counts and render to the end, so the table cell is doing something the prose case is not. Not chased further. |
| * **No discriminant was run for ''crawlConfig.authentication''.** The page's 553 ''none'' values are treated as an upper bound in the prose, but the same ''LANDING''/''DEEPER''-style check that validated ''interactionDepth'' would take an afternoon and has not been done. | * **No discriminant was run for ''crawlConfig.authentication''.** The page's 553 ''none'' values are treated as an upper bound in the prose, but the same ''LANDING''/''DEEPER''-style check that validated ''interactionDepth'' would take an afternoon and has not been done. |
| * **The stability figures for ''interactionDepth'' (97% run-to-run) were measured on the old 4,322-paper corpus** and have not been re-measured. They are quoted from ''data/extract/OVERVIEW.md'' as the right order of magnitude. | * **The stability figures for ''interactionDepth'' (97% run-to-run) were measured on the old 4,322-paper corpus** and have not been re-measured. They are quoted from ''data/extract/OVERVIEW.md'' as the right order of magnitude. |
| - **Broadening rather than splitting.** The stub proposed three sub-pages. Two already exist elsewhere; a third (registration) exists as its own stub. Writing a fourth would have left the hub empty. A reasonable person could instead have made this a pure index page — the argument against is that depth and on-page action have no other home, and the neighbours already point here for them. | - **Broadening rather than splitting.** The stub proposed three sub-pages. Two already exist elsewhere; a third (registration) exists as its own stub. Writing a fourth would have left the hub empty. A reasonable person could instead have made this a pure index page — the argument against is that depth and on-page action have no other home, and the neighbours already point here for them. |
| - **Excluding ''single-target-page'' from every depth ratio.** This is the single biggest decision on the page, and it moves the headline from 37.3% to 62.8%. The justification is a read sample, printed above. A reasonable person could report both denominators; the page reports the value's size and says explicitly why it is out. | - **Excluding ''single-target-page'' from every depth ratio.** This is the single biggest decision on the page, and it moves the headline from 37.3% to 62.8%. The justification is a read sample, printed above. A reasonable person could report both denominators; the page reports the value's size and says explicitly why it is out. |
| - **Reporting the depth ratio at all, given the selection bias.** The alternative was to publish only the reporting rate. The ratio is published with an explicit upper-bound warning in its own ''<WRAP important>'' box. | - **Reporting the depth ratio at all, given the selection bias.** The alternative was to publish only the reporting rate. The ratio is published with an explicit upper-bound warning in its own %%<WRAP important>%% box. |
| - **Calling mouse-movement emulation "never established".** ~2% throughout with no trend, over 857 papers. That is a claim about seven venues and is labelled as such. | - **Calling mouse-movement emulation "never established".** ~2% throughout with no trend, over 857 papers. That is a claim about seven venues and is labelled as such. |
| - **Calling landing-page-only crawls "still defensible"** rather than outdated. Aqeel et al. and Urban et al. both argue for internal pages, and it would have been easy to write the currency table as "landing-only is superseded". It is not: 155 of 417 depth-stating papers still do it, Lauinger et al. {[lauinger2017_thou]} give a good reason, and for many questions the landing page is the right unit. The claim the page makes is that it is no longer defensible //silently//. | - **Calling landing-page-only crawls "still defensible"** rather than outdated. Aqeel et al. and Urban et al. both argue for internal pages, and it would have been easy to write the currency table as "landing-only is superseded". It is not: 155 of 417 depth-stating papers still do it, Lauinger et al. {[lauinger2017_thou]} give a good reason, and for many questions the landing page is the right unit. The claim the page makes is that it is no longer defensible //silently//. |
| - **Publishing a synthetic fixture rather than a live crawl.** A live measurement of "third parties visible only after scrolling" over a real site list would be a stronger result. It would also be a crawl of third-party sites run from this container without an ethics review or a robots check. The fixture makes the mechanism visible; the open-questions box names the real measurement as missing. | - **Publishing a synthetic fixture rather than a live crawl.** A live measurement of "third parties visible only after scrolling" over a real site list would be a stronger result. It would also be a crawl of third-party sites run from this container without an ethics review or a robots check. The fixture makes the mechanism visible; the open-questions box names the real measurement as missing. |
| - **Splitting the raw script outputs onto [[provenance:programming:interaction:outputs|a sub-page]].** Forced, not chosen. With all five ''<file>'' blocks inline this page **silently truncated its own rendering**: the parser stopped mid-table and the //Related// and //References// sections were simply absent from the HTML, with no error and no purple warning — the failure mode this wiki has been bitten by before but from a different cause. Bisected on ''playground:playground'' on 2026-08-27: renders in full at ~51 KB of source, truncates at ~60 KB, on a wiki where ''provenance:programming:stateful_stateless'' renders 111 KB fine — so it is a content-mix limit, not a byte limit, and this page's 12 tables and dense inline markup are the likely cost. Every line of output is still published; it is one click away and linked from each point that produced it. **A reasonable person could instead have trimmed the outputs**; that would have broken the rule that the output on the provenance page is unedited. | - **Splitting the raw script outputs onto [[provenance:programming:interaction:outputs|a sub-page]].** Done while chasing a render bug, and kept afterwards. The bug turned out to be something else entirely (below), so the split is no longer forced — it is kept because the log reads better without 39 KB of raw output between its sections, and every line of that output is still published, one click away, linked from the point that produced it. **A reasonable person would keep them inline**; the rule that matters — the output is unedited — holds either way. |
| - **Not adding ''~~DISCUSSION~~'' to this provenance page.** Following the convention set by the existing ''provenance:'' pages: comments belong on the content page. | - **Not adding ''~~DISCUSSION~~'' to this provenance page.** Following the convention set by the existing ''provenance:'' pages: comments belong on the content page. |
| |
| ^ Finding ^ Verdict ^ What was done ^ | ^ Finding ^ Verdict ^ What was done ^ |
| | The page said 81 papers "got past a login of any kind — 42 by registering an account, 24 by logging in manually, 15 with automated login and **one via SSO**". The script's ''authentication'' breakdown for the 857 web crawls has **no ''sso'' row**, and 42+24+15 already sums to 81, so the SSO clause made the page contradict its own total | **accepted; a real error** | The ''sso'' count was carried over from an early ''site_queries.mjs --page crawl-config'' run, whose population is the 1,080 papers with a ''crawlConfig'' object across **all** platforms, not this page's 857 web crawls. There is exactly one ''authentication: sso'' paper in the whole corpus and it is an Android study. Clause removed; the page now says "Not one of the 857 used SSO" with a footnote naming why. The report now prints the check explicitly so it cannot recur | | | The page said 81 papers "got past a login of any kind — 42 by registering an account, 24 by logging in manually, 15 with automated login and **one via SSO**". The script's ''authentication'' breakdown for the 857 web crawls has **no ''sso'' row**, and 42+24+15 already sums to 81, so the SSO clause made the page contradict its own total | **accepted; a real error** | The ''sso'' count was carried over from an early ''site_queries.mjs --page crawl-config'' run, whose population is the 1,080 papers with a ''crawlConfig'' object across **all** platforms, not this page's 857 web crawls. There is exactly one ''authentication: sso'' paper in the whole corpus and it is an Android study. Clause removed; the page now says "Not one of the 857 used SSO" with a footnote naming why. The report now prints the check explicitly so it cannot recur | |
| | The published ''<file javascript interaction_probe.mjs>'' block was **not byte-identical** to the committed script — it dropped the trailing console block, so the code as printed would not produce the output printed beside it | **accepted** | The committed script's header was changed to a generic invocation, and both ''<file>'' blocks are now generated from the files and asserted byte-identical | | | The published %%<file javascript interaction_probe.mjs>%% block was **not byte-identical** to the committed script — it dropped the trailing console block, so the code as printed would not produce the output printed beside it | **accepted** | The committed script's header was changed to a generic invocation, and both %%<file>%% blocks are now generated from the files and asserted byte-identical | |
| | "Playwright 1.62.1, Chromium 151.0.7922.34, Linux" was asserted **beside** the output rather than printed **by** it — the script only queried the Playwright version and never the Chromium build or the platform | **accepted** | The probe now records ''browser.version()'' and ''process.platform'' and prints both. The version line on the page is now part of the quoted output | | | "Playwright 1.62.1, Chromium 151.0.7922.34, Linux" was asserted **beside** the output rather than printed **by** it — the script only queried the Playwright version and never the Chromium build or the platform | **accepted** | The probe now records ''browser.version()'' and ''process.platform'' and prints both. The version line on the page is now part of the quoted output | |
| | Everything else — every table, bucket, percentage, denominator, sentinel exclusion, provisional-year label, and the ''interaction_site.mjs'' file block | **clean** | — | | | Everything else — every table, bucket, percentage, denominator, sentinel exclusion, provisional-year label, and the ''interaction_site.mjs'' file block | **clean** | — | |
| * **The two published sweeps of the same quantity disagreed.** ''report_interaction.mjs'' and ''interaction_fulltext_probe.mjs'' each defined their own patterns and first-person marker, and they had drifted: 282 clicks against 285, 69 scrolls against 71, 11.1% against 12.4% for the 2025–2026 scrolling trend. **Both were published** — the page quoted one, this provenance page reproduced the other, and the prose here claimed the page quoted the number it did not | * **The two published sweeps of the same quantity disagreed.** ''report_interaction.mjs'' and ''interaction_fulltext_probe.mjs'' each defined their own patterns and first-person marker, and they had drifted: 282 clicks against 285, 69 scrolls against 71, 11.1% against 12.4% for the 2025–2026 scrolling trend. **Both were published** — the page quoted one, this provenance page reproduced the other, and the prose here claimed the page quoted the number it did not |
| * **What was done:** Definitions moved into ''interaction_fold.mjs'' and imported by both scripts. Outputs now identical wherever they overlap. The page's figures did not change; the probe's did. The episode is written up in a box above rather than quietly corrected | * **What was done:** Definitions moved into ''interaction_fold.mjs'' and imported by both scripts. Outputs now identical wherever they overlap. The page's figures did not change; the probe's did. The episode is written up in a box above rather than quietly corrected |
| * **This page contradicted itself about the published code**, still carrying a parenthetical from before reviewer 1's fix saying the ''<file>'' block "drops only the trailing console block" — it no longer does | * **This page contradicted itself about the published code**, still carrying a parenthetical from before reviewer 1's fix saying the %%<file>%% block "drops only the trailing console block" — it no longer does |
| * **What was done:** Parenthetical deleted; it now points at the end-to-end check | * **What was done:** Parenthetical deleted; it now points at the end-to-end check |
| * **The review log narrated a review that had not happened.** The "worth its slot" table had ''4, generic — see below'' with nothing below, and "Nothing was rejected in this round" was written before round 4 produced anything. The run table's page size was two saves stale | * **The review log narrated a review that had not happened.** The "worth its slot" table had ''4, generic — see below'' with nothing below, and "Nothing was rejected in this round" was written before round 4 produced anything. The run table's page size was two saves stale |
