Measure The Web

User Tools

Site Tools

Trace: • interaction
provenance:programming:interaction

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
provenance:programming:interaction [2026/08/27 12:32] – Add the self-review sweep for absolute claims (five overstatements found and fixed between review rounds, two of which passed the number guard), and refresh the embedded report output. Authored by Claude karel.kubicek.claudeprovenance:programming:interaction [2026/08/27 12:52] (current) – Name the two deferred follow-ups by their drain item keys. Authored by Claude karel.kubicek.claude
Line 10: Line 10:
 | Date | 2026-08-27 | | Date | 2026-08-27 |
 | Corpus at the time | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues, 2010–2026 (the 2026-08-11 extension, commit ''8a6b843'') | | Corpus at the time | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues, 2010–2026 (the 2026-08-11 extension, commit ''8a6b843'') |
-| Page before | a 2,411-byte stub: a ''<WRAP todo>'' box, three bullet topics, and one claim about Urban et al. | +| Page before | a 2,411-byte stub: a %%<WRAP todo>%% box, three bullet topics, and one claim about Urban et al. | 
-| Page after | 50,697 bytes, rev 1787833934 |+| Page after | 56,675 bytes, rev 1787834365 — eight saves: the initial publish, one per review round, and five self-review corrections |
 | Model | Claude Opus 5, one session, no sub-agents used for research | | Model | Claude Opus 5, one session, no sub-agents used for research |
 | Sub-agents | four reviewers only (see [[#Review]]) | | Sub-agents | four reviewers only (see [[#Review]]) |
Line 63: Line 63:
 </code> </code>
  
-<file text report_interaction-output.txt> +**The full unedited output is on [[provenance:programming:interaction:outputs#The report: every figure with its denominator|the outputs page]]**, in ''report_interaction-output.txt''.
- +
-============================================================================== +
-A. POPULATION +
-============================================================================== +
-corpus                                              5859 +
-crawled  (crawlConfig OR studyTypes automated-web-crawl)   1120 +
-webCrawled  (crawled AND platforms includes 'web')         857   <- this page's denominator +
-  of which carry a crawlConfig object                      838 +
-  crawled but NOT web (excluded: app stores, network scans, social graphs)  263 +
- +
-============================================================================== +
-B. interactionDepth — what the crawling literature says about depth +
-============================================================================== +
- +
-── crawled — 1120 papers ── +
-interactionDepth         papers  share +
------------------------  ------  ----- +
-single-target-page       376     33.6% +
-not-stated               239     21.3% +
-landing-page-only        162     14.5% +
-deep-crawl               157     14.0% +
-landing-plus-subpages    146     13.0% +
-(no crawlConfig object)  40      3.6% +
-  stated (any non-sentinel value): 841 / 1120 = 75.1% +
- +
-── webCrawled — 857 papers ── +
-interactionDepth         papers  share +
------------------------  ------  ----- +
-single-target-page       285     33.3% +
-landing-page-only        155     18.1% +
-landing-plus-subpages    137     16.0% +
-not-stated               136     15.9% +
-deep-crawl               125     14.6% +
-(no crawlConfig object)  19      2.2% +
-  stated (any non-sentinel value): 702 / 857 = 81.9% +
- +
-── Where interactionDepth ranks among the crawl-configuration fields ── +
-crawlConfig field  papers stating it  share of 1120 crawling papers +
------------------  -----------------  ----------------------------- +
-interactionDepth   841                75.1% +
-authentication     779                69.6% +
-browsers           529                47.2% +
-consentAction      349                31.2% +
-statefulness       219                19.6% +
-headless           140                12.5% +
-  gap between the best-reported field and the second: 62 papers, 5.5 percentage points. +
- +
-── The depth axis proper — webCrawled papers giving one of the three SITE-depth values ── +
-  denominator: 417 papers (NOT 857; single-target-page and not-stated are excluded) +
-depth                  papers  share of 417 +
----------------------  ------  ------------ +
-landing-page-only      155     37.2% +
-landing-plus-subpages  137     32.9% +
-deep-crawl             125     30.0% +
-  went beyond the landing page: 262 / 417 = 62.8% +
-  CAVEAT: papers that went deeper have more reason to say so, so this ratio is +
-  an upper bound on the share of the whole field that crawls beyond the landing page. +
- +
-============================================================================== +
-C. Reporting rate over time (webCrawled, four-year buckets) +
-============================================================================== +
-bucket      web crawls  states a depth  on the site-depth axis  of those, beyond landing  gives subpages/site +
-----------  ----------  --------------  ----------------------  ------------------------  ------------------- +
-2010–2013   80          67 (83.8%)      35                      27 (77.1%)                8 (10.0%) +
-2014–2017   130         109 (83.8%)     57                      35 (61.4%)                11 (8.5%) +
-2018–2021   241         198 (82.2%)     121                     72 (59.5%)                37 (15.4%) +
-2022–2024   253         207 (81.8%)     125                     82 (65.6%)                33 (13.0%) +
-2025–2026*  153         121 (79.1%)     79                      46 (58.2%)                15 (9.8%) +
- +
-* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and +
-  IEEE S&P 2026 / WWW 2026 abstracts are incompletely indexed, so selection under-covers them. +
- +
-============================================================================== +
-D. The label discriminant — does interactionDepth track the paper's own text? +
-============================================================================== +
-crawlConfig carries ONE evidence quote for the WHOLE object, so reading it cannot +
-validate interactionDepth. Instead: does the paper contain a first-person sentence +
-naming the ROOT of a site as the unit of a visit (LANDING), something BELOW the root +
-(DEEPER), both, or neither? `not-stated` is the control row. +
- +
-label                  papers with text  landing phrase only  deeper phrase only  both        neither +
----------------------  ----------------  -------------------  ------------------  ----------  ----------- +
-landing-page-only      155               85 (54.8%)           4 (2.6%)            24 (15.5%)  42 (27.1%) +
-landing-plus-subpages  137               57 (41.6%)           12 (8.8%)           42 (30.7%)  26 (19.0%) +
-deep-crawl             125               27 (21.6%)           21 (16.8%)          27 (21.6%)  50 (40.0%) +
-single-target-page     285               78 (27.4%)           11 (3.9%)           9 (3.2%)    187 (65.6%) +
-not-stated             136               16 (11.8%)           5 (3.7%)            1 (0.7%)    114 (83.8%) +
- +
-── What the shared crawlConfig quote actually evidences ── +
-  pool: 71 web crawls labelled landing-plus-subpages that also state a subpage count. +
-  every 6th read by hand on 2026-08-27 = 12 papers. +
-what the ONE shared quote evidences  papers of 12 +
------------------------------------  ------------ +
-depth                                6 +
-other                                5 +
-partial                              1 +
-    [depth] CCS/2011/fashion-crimes-trending-term-exploitation-on-the-web +
-        "parsing the front page of the domain and a set of five additional web pages within the same domain" +
-    [depth] CCS/2015/sunlight-fine-grained-targeting-detection-at-scale-with-statistical-confidence +
-        "For each site, we visit the top 10 pages returned from a site-specific search on Google" +
-    [other] IMC/2018/how-tracking-companies-circumvented-ad-blockers-using-websockets +
-        "we performed four crawls over our sampled 100K websites" — evidences repeatVisits and the population, not depth +
-    [other] WWW/2019/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private +
-        "We did not erase any cookie after a harvest was performed" — evidences statefulness +
-    [other] PETS/2020/no-boundaries-data-exfiltration-by-third-parties-embedded-on-web-pages +
-        "from US-based EC2 cloud instances using stateless browsers" — evidences vantage and statefulness +
-    [partial] NDSS/2021/cv-inspector-towards-automating-detection-of-adblock-circumvention +
-        "We visit each site for a total of eight times ... four page visits per case" — visit structure, not site depth +
-    [depth] USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild +
-        "we therefore additionally selected three random links to an internal subpage and visited these as well" +
-    [other] IMC/2023/on-the-similarity-of-web-measurements-under-different-experimental-setups +
-        "We choose to use a stateless approach" — evidences statefulness +
-    [depth] PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework +
-        "select nine internal hyperlinks at random from the homepage" +
-    [depth] PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing +
-        "We added ten random sub-pages per domain" +
-    [depth] NDSS/2026/dom-xss-detection-via-webpage-interaction-fuzzing-and-url-component-synthesis +
-        "extracted a maximum of 5 subpages from each page" +
-    [other] PETS/2025/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic +
-        "Browser state was purged after every crawl" — evidences statefulness +
-  Read this as the size of the blindness, not as an error rate: a quote that +
-  evidences statefulness does not make the depth label wrong, it makes the +
-  quote useless as a check on it. +
- +
-── Internal consistency: does subpagesPerSite ever contradict interactionDepth? ── +
-  104 web crawls state BOTH an interactionDepth and a subpagesPerSite. +
-  contradictions (landing-page-only with n>0, or landing-plus-subpages with n=0): 0 +
-  This is a corroborating check on the enum, independent of the text discriminant above: +
-  the two fields are extracted from the same shared quote but mean different things, +
-  so a systematically wrong depth label would show up here as disagreement. +
- +
-── Hand audit of the `landing-page-only` "neither" cell ── +
-  42 papers in the cell; every 4th read by hand on 2026-08-27 = 11 papers. +
-verdict           papers of 11 +
-----------------  ------------ +
-inferred          10 +
-stated-elsewhere  1 +
-    [inferred] IMC/2011/measuring-the-state-of-ecn-readiness-in-servers-clients-and-routers +
-        "For each domain, we retrieve the complete web page" — the unit is the domain, the page is never named +
-    [inferred] IMC/2018/needle-in-a-haystack-tracking-down-elite-phishing-domains-in-the-wild +
-        "we crawled 6,755 unique phishing URLs" — a URL list, not a site depth +
-    [inferred] IMC/2019/visiblev8-in-browser-monitoring-of-javascript-in-the-wild +
-        "We visited each domain in our target list 5 times" — domain granularity only +
-    [stated-elsewhere] WWW/2019/the-chain-of-implicit-trust-an-analysis-of-the-web-third-party-resources-loading +
-        "the Alexa top-200K websites' main pages" — states the root, but not in a first-person sentence +
-    [inferred] WWW/2021/cookie-swap-party-abusing-first-party-cookies-for-web-tracking +
-        "the Alexa top 10K websites that we crawled" — site granularity only +
-    [inferred] IMC/2023/ptperf-on-the-performance-evaluation-of-tor-pluggable-transports +
-        "we accessed each website five times using curl" — site granularity only +
-    [inferred] CCS/2024/a-first-look-at-security-and-privacy-risks-in-the-rapidapi-ecosystem +
-        "We crawl all the content within the API listing page" — a listing page, not a site root +
-    [inferred] WWW/2024/quic-is-not-quick-enough-over-fast-internet +
-        no first-person sentence names the page unit at all +
-    [inferred] USENIX/2025/dns-flare-a-flush-reload-attack-on-dns-forwarders +
-        no first-person sentence names the page unit at all +
-    [inferred] WWW/2026/webgeoinfer-structure-free-multi-stage-framework-for-geolocation-inference-from +
-        "We access their web interfaces ... comparable to how an ordinary user visits a public site" +
-    [inferred] IMC/2025/somesite-i-used-to-crawl-awareness-agency-and-efficacy-in-protecting-content-cre +
-        "We visit the site with a headless browser" — site granularity only +
- +
-============================================================================== +
-E. subpagesPerSite — the number people actually pick +
-============================================================================== +
-  denominator: 857 web crawls; 104 (12.1%) give a number. +
-  of the 417 on the site-depth axis, 102 (24.5%) give a number. +
-subpages per site      papers  share of 104 +
----------------------  ------  ------------ +
-0 (landing page only)  4       3.8% +
-1–4                    29      27.9% +
-5–9                    15      14.4% +
-10–19                  21      20.2% +
-20–49                  16      15.4% +
-50–99                  2       1.9% +
-100+                   17      16.3% +
-  per four-year bucket: +
-    2010–2013   n=  8  median=5 +
-    2014–2017   n= 11  median=5 +
-    2018–2021   n= 37  median=10 +
-    2022–2024   n= 33  median=15 +
-    2025–2026*  n= 15  median=5 +
-  median 10; the five most common values: +
-        5 subpages: 14 papers +
-        3 subpages: 12 papers +
-       10 subpages: 12 papers +
-        1 subpages: 11 papers +
-       20 subpages: 8 papers +
-  the largest values (a "deep crawl" of one application, not a site-list crawl): +
-     2000: 1 papers +
-     1000: 3 papers +
-      500: 2 papers +
-      300: 1 papers +
-      200: 3 papers +
-      100: 7 papers +
- +
-============================================================================== +
-F. repeatVisits and authentication — the other two interaction knobs +
-============================================================================== +
-repeatVisits stated: 233 / 857 web crawls = 27.2% +
-  of those, 50 (21.5%) visit exactly once; 183 more than once. +
- +
-── authentication — of 857 web crawls ── +
-authentication           papers  share +
------------------------  ------  ----- +
-none                     553     64.5% +
-not-stated               204     23.8% +
-account-registration     42      4.9% +
-manual-login             24      2.8% +
-(no crawlConfig object)  19      2.2% +
-automated-login          15      1.8% +
-  crawls that got past a login of any kind: 81 (9.5%) +
-  sso in THIS population: 0; sso anywhere in the corpus: 1 (USENIX/2024/darkfleece-probing-the-dark-side-of-android-subscription-apps, platforms mobile) +
- +
-============================================================================== +
-G. What the crawler does ON the page — full-text probe +
-============================================================================== +
-The schema has NO field for scrolling, clicking, hovering, typing or waiting. +
-These are UPPER BOUNDS on "the paper did this": a first-person sentence matching +
-the pattern. "we did not scroll" matches too. Denominator: the 857 web crawls. +
- +
-papers with full text on disk: 857 of 857 +
- +
-what the paper says it does            papers  share of 857 +
--------------------------------------  ------  ------------ +
-clicks something                       282     32.9% +
-scrolls                                69      8.1% +
-waits / dwells a stated time           56      6.5% +
-types on the keyboard                  39      4.6% +
-fills a form                           28      3.3% +
-moves the mouse                        18      2.1% +
-hovers                                 14      1.6% +
-picks links at random                  14      1.6% +
-says it aims for human-like behaviour  14      1.6% +
-mentions bot / crawler detection       61      7.1% +
- +
-  bot/crawler detection, 2010-2013 bucket: 3 of 80 papers match ANYWHERE in the text, 0 in a first-person sentence. +
- +
-Same, by four-year bucket (share of that bucket's web crawls): +
-pattern                                2010–2013    2014–2017     2018–2021     2022–2024     2025–2026* +
--------------------------------------  -----------  ------------  ------------  ------------  ------------ +
-clicks something                       35/80 43.8%  41/130 31.5%  72/241 29.9%  80/253 31.6%  54/153 35.3% +
-scrolls                                2/80 2.5%    5/130 3.8%    18/241 7.5%   27/253 10.7%  17/153 11.1% +
-waits / dwells a stated time           0/80 0.0%    12/130 9.2%   14/241 5.8%   24/253 9.5%   6/153 3.9% +
-types on the keyboard                  4/80 5.0%    6/130 4.6%    7/241 2.9%    17/253 6.7%   5/153 3.3% +
-fills a form                           1/80 1.3%    1/130 0.8%    8/241 3.3%    10/253 4.0%   8/153 5.2% +
-moves the mouse                        3/80 3.8%    2/130 1.5%    7/241 2.9%    4/253 1.6%    2/153 1.3% +
-hovers                                 2/80 2.5%    0/130 0.0%    6/241 2.5%    4/253 1.6%    2/153 1.3% +
-picks links at random                  2/80 2.5%    1/130 0.8%    6/241 2.5%    1/253 0.4%    4/153 2.6% +
-says it aims for human-like behaviour  0/80 0.0%    4/130 3.1%    1/241 0.4%    3/253 1.2%    6/153 3.9% +
-mentions bot / crawler detection       0/80 0.0%    2/130 1.5%    17/241 7.1%   24/253 9.5%   18/153 11.8% +
- +
-============================================================================== +
-H. LLM-agent-driven browsing — sweep, then hand verdicts +
-============================================================================== +
-sweep hits over the 857 web crawls: 35 +
-NOTE: 1 hand verdicts are not returned by the current sweep +
-   USENIX/2024/understanding-help-seeking-and-help-giving-on-social-media-for-image-based-sexua +
-role            papers of 35 +
---------------  ------------ +
-not-browsing    14 +
-subject         9 +
-citation        5 +
-instrument      4 +
-captcha-solver  2 +
-instrument-app  1 +
- +
-An LLM agent drove the browsing in 5 of 857 web crawls (0.6%). +
-   NDSS/2025/yurascanner-leveraging-llms-for-task-driven-web-app-scanning +
-      "YURASCANNER operates as a goal-based agent, suggesting actions to achieve predefined objectives by processing webpages" +
-   PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms +
-      "a multi-agent system powered by large language models (LLMs) to simulate persona-driven browsing behavior" +
-   CCS/2025/whispertest-a-voice-control-based-library-for-ios-ui-automation +
-      "we design an LLM-based navigation pipeline tailored to perform privacy-related measurements in 200 apps" +
-   USENIX/2026/the-state-of-passkeys-studying-the-adoption-and-security-of-passkeys-on-the-web +
-      "We built a prototype tool using Playwright and the agentic LLM-based Browser Use framework" +
-   PETS/2026/on-the-suitability-of-llm-driven-agents-for-dark-pattern-audits +
-      "We design and deploy an LLM-driven auditing agent capable of end-to-end traversal of rights-request workflows" +
-   years: 2025, 2025, 2026, 2026, 2026 — the earliest is 2025. +
- +
-============================================================================== +
-I. The measured consequences — per-paper figures with their own denominators +
-============================================================================== +
- +
-  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-     * subsites set 36% more cookies than landing pages: 78 vs 55 on average, over the top 10k websites (TLD+1), 100 subsites each +
-     * the mean number of accessed/set cookies "stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites" — pre-study on 100 randomly selected sites from the top 1,000, at 25/50/75/100/250/500/1,000 subsites +
-     * trackers (EasyPrivacy) increased ~6% on subsites; 2.5% of sites tracked ONLY on subsites +
-     * Fingerprint2 device fingerprinting increased 25% on subsites; present on 0.15% of landing pages +
- +
-  USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains +
-     * same Alexa top-10K, same cookie policy: homepage crawl 192,038 requests / 76,816 chains; interactive crawl (random internal pages via clicks on iframes and anchors) 575,550 / 229,151 — 3.0x +
-     * 302 redirects are 42.91% of AT redirect requests on the homepage crawl vs 28.56% interactive +
-     * AT requests navigating to a new domain: 51.32% homepage vs 47.49% interactive +
- +
-  USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild +
-     * front pages plus "three random links to an internal subpage": "visiting subpages did indeed significantly increase the prevalence by about 17% compared to only crawling the front pages" +
-     * denominator, from the paper's Table 2: 206,676 sites (24.42%) carry an indicator, of which 30,494 only on a subpage (+17% overall) +
-     * per technique, subpage-only increase: ShortCut +22%, TrigBreak +80%, ModBuilt +18%, WidthDiff +19%, LogGet +33% — the aggregate hides an 80% +
- +
-  NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search +
-     * against three state-of-the-art scanners on ten web applications: "a 59% increase in code coverage and successfully submits HTML forms 5x more frequently than the next best tool" +
- +
-  PETS/2026/on-the-suitability-of-llm-driven-agents-for-dark-pattern-audits +
-     * 456 data-broker websites; verified workflow completion 87% in Phase 2 and 79% in Phase 3 +
- +
-  IMC/2024/browsing-without-third-party-cookies-what-do-you-see +
-     * clickstream traversal, and its own bias statement: "our dataset is biased towards static inner pages ... we are less likely to explore the more dynamic areas of a website" +
- +
-  NDSS/2017/thou-shalt-not-depend-on-me-analysing-the-use-of-outdated-javascript-libraries-o +
-     * the reason a landing-page-only design is chosen: "We only crawl the homepage of each visited site due to the presence of many sites that thwart deeper traversal by requiring log-ins." +
- +
-  USENIX/2024/trust-me-if-you-can-how-usable-is-trusted-types-in-practice +
-     * a fully specified depth: "visited those up to two levels of links from the start page, while crawling at most 500 URLs per site" +
- +
-  USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission +
-     * typing is simulated against bot detection: "we simulate user typing behavior by using randomized intervals for each key press and dwell times, as well as the delay times between each press" +
- +
-  CCS/2020/the-cookie-hunter-automated-black-box-auditing-for-web-authentication-and-author +
-     * what stops an interacting crawler: "In 22 cases, there was some form of an anti-bot challenge that our system was not able to solve and, thus, could not proceed with registration." +
- +
-  PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework +
-     * the most complete interaction statement in the corpus, quoted whole on the page: "For each domain, we programmed the crawler to load the domain's homepage, scroll to the bottom of the page, then sleep for 25 seconds. Further, we programmed our crawler to select nine internal hyperlinks at random from the homepage and crawl them using the same load, scroll, and sleep approach." — two sentences, and they answer unit, depth, selection rule, on-page action, wait, and same-treatment-for-subpages +
- +
-  PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing +
-     * a stated subpage-selection rule: "We added ten random sub-pages per domain, filtering to exclude auxiliary pages like privacy policies or contact pages." +
- +
-============================================================================== +
-Y. Figures the page quotes from other papers or other pages +
-============================================================================== +
-  statefulness stated: 219 / 1120 crawling papers = 19.6% +
-    (the page cites this as the comparison for how well depth is reported) +
- +
-  Zeber et al., TheWebConf 2020, "The Representativeness of Automated Web Crawls as a +
-  Surrogate for Human Browsing" — IN the corpus; figures read from +
-  data/fulltext/2020/WWW/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing/paper.cols.txt +
-  on 2026-08-27, all four verbatim: +
-    "over 50,000 users of the Firefox Web browser"; "30 million site visits across 50,000 users" +
-    "crawler site visits issued requests to a median of 11.6 third-party domains, whereas for +
-     visits by humans, the median was 4.5 third parties" +
-    "The median number of tracking domains accessed by a user on visiting a Trexa list site is +
-     1.9, whereas for the crawler it is 6.1" +
- +
-  Urban et al.'s site list: "the top 10k websites" — the page's "10,000 websites" in the +
-  opening box is a rhetorical example of a methods sentence, not a corpus figure. +
- +
-  NOT FIGURES — literals inside the fixture the page publishes in <file> blocks, +
-  scanned only under check_page_numbers.mjs --code. Listed here so they trace to +
-  this report instead of polluting the guard's shared ALLOW map: +
-    127.0.0.1  the first-party origin the fixture serves +
-    127.0.0.2  the third-party origin the fixture serves, so the browser treats it as third-party +
-    8231       the first-party port +
-    8232       the third-party port +
-    3000       the CSS pixel height of the tall div that makes scrolling necessary +
-    50         the scroll-bottom tolerance in pixels +
-    200, 300   waitForTimeout values in milliseconds +
-    404, 200   HTTP status codes the fixture writes +
-    2          the maximum crawl depth in the last strategy +
- +
-  Tool versions in the measured probe, printed by the probe itself +
-  (sandbox/interaction_probe-output.txt): Playwright 1.62.1, Chromium 151.0.7922.34. +
- +
- +
-============================================================================== +
-Z. NON-CORPUS NUMBERS ON THE PAGE, with their primary source +
-============================================================================== +
- +
-  Aqeel et al., IMC 2020, "On Landing and Internal Web Pages" — NOT in the corpus +
-  (the selection stage labelled it neither a security nor a privacy measurement; +
-  see data/labels/run1/labels.jsonl). Read from the authors' own PDF at +
-  balakrishnanc.github.io/papers/aqeel-imc2020.pdf on 2026-08-27: +
-    119 web-performance publications reviewed (IMC/PAM/NSDI/SIGCOMM/CoNEXT, 2015–2019): +
-      41 (34.5%) need no revision, 48 (40.3%) minor, 30 (25.2%) major -> 65.5% need at least minor +
-    landing pages are on average 34% larger than internal pages (geometric mean of ratios, H1K) +
-    for 32% of H1K sites the landing page has FEWER objects than the median internal page +
-    internal pages' content visually loads 14% more slowly in the median (Speed Index, Ht30) +
-    median: internal pages collectively fetch from 18 third-party domains never seen on the +
-      landing page; for 10% of H1K sites, 80 or more +
-    at the 80th percentile, internal pages carry 20 tracking requests and landing pages 28 +
-    in about 10% of H1K sites, internal pages have no trackers while the landing page does +
-    header bidding, of 200 sites: 17 have HB ads on the landing page, a further 12 only on internal pages +
-    36 of the 1000 H1K sites serve their landing page over HTTP; among sites with a secure +
-      landing page, 170 have at least one HTTP internal page +
-    Hispar: H2K = 100,000 URLs, >=2000 sites x 50 URLs (1 landing + <=49 internal), weekly refresh, +
-      bootstrapped from Alexa Top 1M via Google "site:" search +
- +
-  Hispar is dead. Checked 2026-08-27: hispar.cs.duke.edu does not resolve (DNS NXDOMAIN); +
-  last Wayback capture that returned content 2024-11-16 (HTTP 200, 1,420 bytes); last capture +
-  of any kind 2024-12-03 (HTTP 403); github.com/waqaraqeel/hispar last pushed 2020-10-27 and +
-  github.com/waqaraqeel/lit_grabber 2020-05-26. Its bootstrap list, Alexa Top 1M, was itself +
-  retired in 2022. +
- +
-  HTTP Archive crawls exactly ONE secondary page per site, since April 2022. +
-  har.fyi (HTTP Archive's own BigQuery reference), "Pages table": "Pages are tested on a +
-  monthly basis and as of April 2022, both the root page and one secondary page are tested." +
-  The pages table carries is_root_page and root_page columns. +
-  How the secondary page is chosen, from github.com/HTTPArchive/crawl at commit 4b83ac0 +
-  (2025-08-20), crawl.py and crawl_pubsub.py: MAX_DEPTH = 1 and MAX_BREADTH = 1, and the +
-  child job is the FIRST link in the page's crawl_links whose hostname equals the parent's, +
-  whose extension is not in ['.jpg','.jpeg','.gif','.png','.webp','.avif','.webm','.pdf', +
-  '.tiff','.zip'], and which has not already been visited. +
-  NOTE: httparchive.org/faq still says "The HTTP Archive examines each URL in the list, but +
-  does not crawl the website's other pages" — the FAQ is stale; the BigQuery reference and +
-  the crawl controller agree with each other and not with it. Checked 2026-08-27. +
-</file>+
  
 ===== The full-text probe for on-page actions ===== ===== The full-text probe for on-page actions =====
Line 472: Line 69:
 The schema has **no field** for scrolling, clicking, hovering, typing or waiting. The only way to count them is to sweep the text, and the count is an **upper bound** by construction: it counts papers with a first-person sentence matching a pattern, and "we did not scroll" matches too. The schema has **no field** for scrolling, clicking, hovering, typing or waiting. The only way to count them is to sweep the text, and the count is an **upper bound** by construction: it counts papers with a first-person sentence matching a pattern, and "we did not scroll" matches too.
  
-Two columns are printed side by side so the width of the claim is visible: //any match anywhere in the paper// against //a match in a sentence that also contains a first-person marker//. The gap is large — 469 papers mention clicking, 285 mention it in a first-person sentence — and the page quotes only the tighter column. +Two columns are printed side by side so the width of the claim is visible: //any match anywhere in the paper// against //a match in a sentence that also contains a first-person marker//. The gap is large — **469** papers mention clicking, **282** mention it in a first-person sentence — and the page quotes only the tighter column.
- +
-<file text interaction_fulltext_probe-output.txt> +
-population: --pop web → 857 papers, 857 of them with full text on disk +
- +
-pattern                          papers (any match)  %      papers (1st-person sentence)  % +
--------------------------------  ------------------  -----  ----------------------------  ----- +
-scroll                           120                 14.0%  71                            8.3% +
-click                            469                 54.7%  285                           33.3% +
-hover                            48                  5.6%   14                            1.6% +
-mouse movement                   42                  4.9%   20                            2.3% +
-keyboard / typing                97                  11.3%  39                            4.6% +
-form fill                        59                  6.9%   28                            3.3% +
-dwell / wait                     87                  10.2%  55                            6.4% +
-random walk / link following     21                  2.5%   14                            1.6% +
-subpage / subsite                110                 12.8%  87                            10.2% +
-landing page only                22                  2.6%   14                            1.6% +
-human-like / realistic browsing  28                  3.3%   14                            1.6% +
-LLM / agent-driven browsing      32                  3.7%   14                            1.6% +
-bot / crawler detection          95                  11.1%  61                            7.1% +
-cloaking                         150                 17.5%  81                            9.5% +
- +
-Both columns are UPPER BOUNDS on "the paper did this". +
-Read the sentences with --hits "<pattern>" before quoting either.+
  
 +<WRAP important>
 +**These two scripts used to disagree, and both numbers were published.** Until 2026-08-27 ''report_interaction.mjs'' and ''interaction_fulltext_probe.mjs'' each carried their own copy of the patterns and of the first-person marker, and the copies had drifted: **282 clicks against 285, 69 scrolls against 71, 18 mouse-movement against 20, and 11.1% against 12.4%** for the 2025–2026 scrolling trend. The page quoted the report's figures; this provenance page reproduced the probe's, one section apart, with no explanation. Found by the generic reviewer.
  
-First-person-sentence hits by four-year bucket (share of that bucket's papers): +Fixed at the source rather than by reconciling the prose: ''PRIMITIVES'', ''CONTEXT_PATTERNS'' and ''FIRST_PERSON'' now live once, in ''interaction_fold.mjs'', and both scripts import them. The two outputs below are now identical wherever they overlap. **The lesson is not "check your numbers" — it is that a figure defined twice will eventually be two figures.** 
-pattern                          2010-2013    2014-2017     2018-2021     2022-2024     2025-2026* +</WRAP>
--------------------------------  -----------  ------------  ------------  ------------  ------------ +
-scroll                           2/80 2.5%    5/130 3.8%    18/241 7.5%   27/253 10.7%  19/153 12.4% +
-click                            35/80 43.8%  41/130 31.5%  72/241 29.9%  81/253 32.0%  56/153 36.6% +
-hover                            2/80 2.5%    0/130 0.0%    6/241 2.5%    4/253 1.6%    2/153 1.3% +
-mouse movement                   3/80 3.8%    2/130 1.5%    7/241 2.9%    4/253 1.6%    4/153 2.6% +
-keyboard / typing                4/80 5.0%    6/130 4.6%    7/241 2.9%    17/253 6.7%   5/153 3.3% +
-form fill                        1/80 1.3%    1/130 0.8%    8/241 3.3%    10/253 4.0%   8/153 5.2% +
-dwell / wait                     0/80 0.0%    12/130 9.2%   14/241 5.8%   23/253 9.1%   6/153 3.9% +
-random walk / link following     2/80 2.5%    1/130 0.8%    6/241 2.5%    1/253 0.4%    4/153 2.6% +
-subpage / subsite                2/80 2.5%    9/130 6.9%    19/241 7.9%   37/253 14.6%  20/153 13.1% +
-landing page only                0/80 0.0%    5/130 3.8%    2/241 0.8%    6/253 2.4%    1/153 0.7% +
-human-like / realistic browsing  0/80 0.0%    4/130 3.1%    1/241 0.4%    3/253 1.2%    6/153 3.9% +
-LLM / agent-driven browsing      0/80 0.0%    0/130 0.0%    0/241 0.0%    2/253 0.8%    12/153 7.8% +
-bot / crawler detection          0/80 0.0%    2/130 1.5%    17/241 7.1%   24/253 9.5%   18/153 11.8% +
-cloaking                         11/80 13.8%  18/130 13.8%  19/241 7.9%   25/253 9.9%   8/153 5.2%+
  
-* 2025-2026 is provisional: CCS/IMC 2026 have not been held and IEEE S&P/WWW 2026 are incompletely selected. +**The full unedited output is on [[provenance:programming:interaction:outputs#The on-page-action full-text probe|the outputs page]]**, in ''interaction_fulltext_probe-output.txt''.
-</file>+
  
 Read individual sentences behind any row with: Read individual sentences behind any row with:
Line 528: Line 88:
 ===== Folding, hand verdicts, and the residue ===== ===== Folding, hand verdicts, and the residue =====
  
-There is **no name fold on this page**. Nothing it counts is a free-text name: ''interactionDepth'' and ''authentication'' are enums, ''subpagesPerSite'' and ''repeatVisits'' are integers, and the on-page-action figures are regex sweeps whose patterns are published in full in the script. What this page has instead is **three hand classifications**, all guarded at run time.+There is **no name fold on this page**. Nothing it counts is a free-text name: ''interactionDepth'' and ''authentication'' are enums, ''subpagesPerSite'' and ''repeatVisits'' are integers, and the on-page-action figures are regex sweeps whose patterns are published in full in the script. What this page has instead is **three hand classifications**, all guarded at run time, and two published sweeps whose patterns live in one place so they cannot drift apart again.
  
 ==== 1. LLM-agent roles ==== ==== 1. LLM-agent roles ====
Line 598: Line 158:
 The page publishes a measurement rather than an assertion: a local instrumented site with five third-party beacons, each reachable only under a different condition, visited under six strategies by Playwright's own Chromium. It needs no network. The page publishes a measurement rather than an assertion: a local instrumented site with five third-party beacons, each reachable only under a different condition, visited under six strategies by Playwright's own Chromium. It needs no network.
  
-<file text interaction_probe-output.txt> +**The full unedited output is on [[provenance:programming:interaction:outputs#The measured strategy ladder|the outputs page]]**, in ''interaction_probe-output.txt''.
-strategy                                                             pages  t-landing  t-article   t-scroll    t-click     t-deep +
--------------------------------------------------------------------  -----  ---------  ---------  ---------  ---------  --------- +
-landing page only                                                        1          ✓          ✗          ✗          ✗          ✗ +
-landing + FIRST same-origin link (the HTTP Archive rule)                 2          ✓          ✗          ✗          ✗          ✗ +
-landing + ALL same-origin links from the landing page                    4          ✓          ✓          ✗          ✗          ✗ +
-landing + all links, and scroll to the bottom of each                    4          ✓          ✓          ✓          ✗          ✗ +
-landing + all links, scroll, and click every button                      4          ✓          ✓          ✓          ✓          ✗ +
-depth 2: landing, its links, and their links, with scroll and click      5          ✓          ✓          ✓          ✓          ✓ +
- +
-t-landing  on the landing page +
-t-article  on one subpage, at load +
-t-scroll   on that subpage, only after scrolling to the bottom +
-t-click    on another subpage, only after a button click +
-t-deep     on a page linked only from a subpage, never from the landing page +
- +
-Playwright 1.62.1, Chromium 151.0.7922.34, linux. Needs no network. +
-</file>+
  
 Design notes, because the fixture is doing real work: Design notes, because the fixture is doing real work:
Line 625: Line 168:
   * The ladder is **monotone and each rung adds exactly one beacon**, which is the point: depth does not substitute for scrolling and scrolling does not substitute for clicking.   * The ladder is **monotone and each rung adds exactly one beacon**, which is the point: depth does not substitute for scrolling and scrolling does not substitute for clicking.
  
-The two files are published on the page in ''<file>'' blocks and are byte-identical to ''sandbox/interaction_site.mjs'' and ''sandbox/interaction_probe.mjs'' (the ''<file>'' copy of the probe drops only the trailing console block that prints the beacon legend, which the page prints as prose instead).+The two files are published on the page in %%<file>%% blocks and are byte-identical to ''sandbox/interaction_site.mjs'' and ''sandbox/interaction_probe.mjs'' — asserted by a diff on every save, and checked end-to-end (see [[#The published code was checked end-to-end]]).
  
 ===== Quotes checked ===== ===== Quotes checked =====
Line 631: Line 174:
 **Two independent renderings, because they fail on different sentences.** ''paper.cols.txt'' repairs two-column reading order but still interleaves a fragment of the neighbouring column into some sentences; a ''pypdf'' extraction of ''paper.pdf'' keeps raw order (wrong across columns, right within a line). A quote present in either, with whitespace collapsed and ''-\n'' hyphenation undone, is present in the paper. **Two independent renderings, because they fail on different sentences.** ''paper.cols.txt'' repairs two-column reading order but still interleaves a fragment of the neighbouring column into some sentences; a ''pypdf'' extraction of ''paper.pdf'' keeps raw order (wrong across columns, right within a line). A quote present in either, with whitespace collapsed and ''-\n'' hyphenation undone, is present in the paper.
  
-<file text interaction_quotecheck-output.txt> +**The full unedited output is on [[provenance:programming:interaction:outputs#The dual-rendering quote check|the outputs page]]**, in ''interaction_quotecheck-output.txt''.
-  ok  cols=Y pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-      "subsites set considerably more (36 %) cookies than the respective landing pages" +
-  ok  cols=Y pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-      "On average, 55 cookies were set when loading a landing page while 78 were set when a subsi" +
-  ok  cols=n pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-      "the mean amount of accessed/set cookies stabilizes around 50 (SD: 100; median at 12) after" +
-  ok  cols=Y pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-      "visited 25, 50, 75, 100, 250, 500, and 1,000 subsites" +
-  ok  cols=n pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-      "we choose 100 random subsites that we used during the experiment crawls" +
-  ok  cols=Y pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-      "we randomly selected 100 websites (TLD+1) from the top 1,000 websites" +
-  ok  cols=Y pdf=Y  USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains +
-      "Interactive crawls additionally navigate to random internal pages by clicking on iframes a" +
-  ok  cols=Y pdf=Y  USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains +
-      "we visit the Alexa top-10K homepages" +
-  ok  cols=Y pdf=Y  NDSS/2017/thou-shalt-not-depend-on-me-analysing-the-use-of-outdated-javascript-libraries-o +
-      "We only crawl the homepage of each visited site due to the presence of many sites that thw" +
-  ok  cols=Y pdf=Y  IMC/2024/browsing-without-third-party-cookies-what-do-you-see +
-      "our dataset is biased towards static inner pages which may cause us to underestimate the i" +
-  ok  cols=Y pdf=Y  USENIX/2024/trust-me-if-you-can-how-usable-is-trusted-types-in-practice +
-      "visited those up to two levels of links from the start page, while crawling at most 500 UR" +
-  ok  cols=Y pdf=Y  USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission +
-      "we simulate user typing behavior by using randomized intervals for each key press and dwel" +
-  ok  cols=Y pdf=Y  CCS/2020/the-cookie-hunter-automated-black-box-auditing-for-web-authentication-and-author +
-      "In 22 cases, there was some form of an anti-bot challenge that our system was not able to " +
-  ok  cols=Y pdf=Y  PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework +
-      "For each domain, we programmed the crawler to load the domain's homepage,9 scroll to the b" +
-  ok  cols=Y pdf=Y  PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework +
-      "we programmed our crawler to select nine internal hyperlinks at random from the homepage a" +
-  ok  cols=Y pdf=Y  PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing +
-      "We added ten random sub-pages per domain, filtering to exclude auxiliary pages like privac" +
-  ok  cols=Y pdf=Y  WWW/2020/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing +
-      "crawler site visits issued requests to a median of 11.6 third-party domains, whereas for v" +
-  ok  cols=Y pdf=Y  WWW/2020/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing +
-      "The median number of tracking domains accessed by a user on visiting a Trexa list site is " +
-  ok  cols=Y pdf=n  USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild +
-      "we can see that visiting subpages did indeed significantly increase the prevalence by abou" +
-  ok  cols=Y pdf=Y  USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild +
-      "we therefore additionally selected three random links to an internal subpage and visited t" +
-  ok  cols=n pdf=Y  NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search +
-      "EvoCrawl achieves a 59% increase in code coverage and successfully submits HTML forms 5" +
-  ok  cols=Y pdf=Y  PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms +
-      "a multi-agent system powered by large language models (LLMs) to simulate persona-driven br" +
-  ok  cols=Y pdf=Y  USENIX/2026/the-state-of-passkeys-studying-the-adoption-and-security-of-passkeys-on-the-web +
-      "Browser Use framework" +
-23 quotes checked against two independent renderings. +
-  found in paper.cols.txt : 20 +
-  found in pypdf(paper.pdf): 22 +
-  found in at least one    : 23 +
-  found in NEITHER         : 0 +
-</file>+
  
-**23 of 23 quotes located; 20 in ''.cols'', 22 in the PDF rendering, 0 in neither.**+**26 of 26 quotes located; 23 in ''.cols'', 25 in the PDF rendering, 0 in neither.**
  
 The three that ''.cols'' misses are column splices, and the raw text is worth showing because it is what a splice looks like: The three that ''.cols'' misses are column splices, and the raw text is worth showing because it is what a splice looks like:
Line 708: Line 199:
 A separate needle check covers every literal per-paper **figure** on the page — not just the quoted sentences — against ''paper.cols.txt'': A separate needle check covers every literal per-paper **figure** on the page — not just the quoted sentences — against ''paper.cols.txt'':
  
-<file text verify_interaction_figures-output.txt> +**The full unedited output is on [[provenance:programming:interaction:outputs#The per-paper figure needle check|the outputs page]]**, in ''verify_interaction_figures-output.txt''.
-SPLICED  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-         "stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites" +
-         (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py) +
-SPLICED  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field +
-         "we choose 100 random subsites that we used during the experiment crawls" +
-         (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py) +
-SPLICED  NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search +
-         "a 59% increase in code coverage and successfully submits HTML forms 5" +
-         (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py) +
- +
-44 needles checked, 0 not located anywhere, 3 located only after undoing a column splice, 6 shorter than 20 chars (flagged, not trusted). +
- +
-Paraphrased figures (NOT verbatim in the paper — anchor check only): +
-</file>+
  
 **44 needles, 0 not located anywhere, 3 located only after undoing a column splice, 6 flagged as shorter than 20 characters** (the count of needles is unchanged by the review fixes; two Musch and Johns needles replaced one) (a short needle can pass for the wrong reason; they are flagged rather than trusted). **44 needles, 0 not located anywhere, 3 located only after undoing a column splice, 6 flagged as shorter than 20 characters** (the count of needles is unchanged by the review fixes; two Musch and Johns needles replaced one) (a short needle can pass for the wrong reason; they are flagged rather than trusted).
Line 730: Line 207:
 ===== The published code was checked end-to-end ===== ===== The published code was checked end-to-end =====
  
-Not just "the ''<file>'' block matches the committed file", which is where this check usually stops. Both code blocks were downloaded back **off the rendered page** through DokuWiki's own export, diffed against the committed scripts, copied into an empty directory and run:+Not just "the %%<file>%% block matches the committed file", which is where this check usually stops. Both code blocks were downloaded back **off the rendered page** through DokuWiki's own export, diffed against the committed scripts, copied into an empty directory and run:
  
 <code> <code>
Line 775: Line 252:
 | ''design_mobile_and_app_measurement.txt'' / ''report_mobile-output.txt'' | identical (both fail — pre-existing, unrelated) | | ''design_mobile_and_app_measurement.txt'' / ''report_mobile-output.txt'' | identical (both fail — pre-existing, unrelated) |
 | ''design_longitudinal.txt'' / ''report_longitudinal-output.txt'' | **improved**: unaccounted 24 → 20, and three four-part Chromium versions (''113.0.5672.0'', ''152.0.7977.54'', ''154.0.8026.0'') now reported as version identifiers instead of as digit fragments. That pair was already failing before the change and still fails | | ''design_longitudinal.txt'' / ''report_longitudinal-output.txt'' | **improved**: unaccounted 24 → 20, and three four-part Chromium versions (''113.0.5672.0'', ''152.0.7977.54'', ''154.0.8026.0'') now reported as version identifiers instead of as digit fragments. That pair was already failing before the change and still fails |
 +
 +===== The page truncated itself, silently, and the guard said OK =====
 +
 +Worth a section rather than a footnote, because it is a failure mode with no error message anywhere and a guard that actively certified it.
 +
 +**Symptom.** After a routine save, the rendered page simply stopped mid-table. The //Related// and //References// sections were not in the HTML at all — no purple warning, no PHP notice, no truncated response; DokuWiki emitted a complete page that happened to contain only part of the content. The only way to notice is to count what rendered against what you wrote.
 +
 +**What it was not.** Two hours of bisecting on ''playground:playground'' ruled out: total size (a neighbouring provenance page renders 143 KB), the biggest embedded output block (removing 27 KB changed nothing), the review section (removing 16 KB changed nothing), the specific table cell it stopped at (rewriting it moved nothing), and every section on its own (all 13 rendered fine in isolation). Cumulative prefixes gave the decisive clue by being **non-monotone** — the first 11 KB failed while the first 34 KB passed — which cannot happen under any size or complexity budget, and can only happen if something is left **open** and something later closes it.
 +
 +**What it was.** One table cell in the run table at the top of the page:
 +
 +<code>
 +| Page before | a 2,411-byte stub: a ''&lt;WRAP todo&gt;'' box, three bullet topics, ... |
 +</code>
 +
 +(with real angle brackets, of course — they are escaped here so this page does not reproduce the bug while describing it.)
 +
 +''%%''%%…''%%''%%'' is **monospace, not nowiki.** It changes how the text looks and does nothing whatever to stop the wrap plugin. That literal opened a real, never-closed ''%%<div>%%'', and everything after it vanished. Replacing the quotes with ''%%%%%%…%%%%%%'' fixed it on the next save, with nothing else changed.
 +
 +**The guard was worse than useless.** ''scripts/check_wrap.mjs'' masked ''%%''%%…''%%''%%'' before counting tags — on the assumption that a tag written in inline code is a mention — and did **not** mask ''%%%%%%…%%%%%%''. That is backwards on both counts: it blinded the guard to the one construct that fires, and it would have raised a false alarm on the one that does not. It reported ''OK'' on the broken page every time it was run.
 +
 +Fixed on 2026-08-27: mask ''%%<nowiki>%%'' and ''%%%%%%…%%%%%%'' only, and flag a plugin tag inside ''%%''%%…''%%''%%'' as an error in its own right, with the reason in the message. Re-run across every page source in the workdir it now reports **71 instances**, including three pages whose %%<WRAP>%% counts are unbalanced. Two of those were checked live and still render to the end, so this is a lead for whoever owns those pages rather than an emergency — but the class of defect is real and was previously invisible. Filed as the ''wrap-escape sweep'' item in ''/workspace/drain/work.db'' rather than fixed here, because editing ten other people's pages is outside this item's scope.
  
 ===== What could not be established ===== ===== What could not be established =====
Line 783: Line 282:
   * **How much an LLM agent changes a measurement.** Five papers use one as the browsing instrument; none reports a same-site-list comparison against a scripted crawl. This is on the page as an open question.   * **How much an LLM agent changes a measurement.** Five papers use one as the browsing instrument; none reports a same-site-list comparison against a scripted crawl. This is on the page as an open question.
   * **Whether ''subpagesPerSite'' means the same thing across papers.** "10 subpages" could be 10 attempted, 10 successfully loaded, or 10 distinct URLs. Nothing in the schema distinguishes them and few papers say. The page reports the distribution and does not compute a mean.   * **Whether ''subpagesPerSite'' means the same thing across papers.** "10 subpages" could be 10 attempted, 10 successfully loaded, or 10 distinct URLs. Nothing in the schema distinguishes them and few papers say. The page reports the distribution and does not compute a mean.
 +  * **''design:sampling'' now disagrees with this page and was not updated.** It publishes its own depth figures — 24.8% landing-only of 680 papers, "15.0% state how many subpages" — against this page's 37.2% of 417 and 12.1% of 857. The populations are different and each page states its own, but nothing reconciles them for a reader who reads both. Its Hispar box, checked 2026-08-12, says there is "no maintained equivalent … no tooling to address it", which is now stale against this page's HTTP Archive secondary-page paragraph. **Deferred**, and filed: editing that page is outside this item's scope, so it is recorded as the ''sampling depth reconcile'' item in ''/workspace/drain/work.db''. The change it needs is one sentence pointing here plus a scope note on the two depth figures.
 +  * **Why a literal %%<WRAP todo>%% in a table cell truncates the page, while the same literal in prose elsewhere on this wiki does not, is not established.** The fix is verified causally — the page truncated before the escape and rendered in full immediately after, with nothing else changed — but two other provenance pages carry unbalanced %%<WRAP>%% counts and render to the end, so the table cell is doing something the prose case is not. Not chased further.
 +  * **No discriminant was run for ''crawlConfig.authentication''.** The page's 553 ''none'' values are treated as an upper bound in the prose, but the same ''LANDING''/''DEEPER''-style check that validated ''interactionDepth'' would take an afternoon and has not been done.
   * **The stability figures for ''interactionDepth'' (97% run-to-run) were measured on the old 4,322-paper corpus** and have not been re-measured. They are quoted from ''data/extract/OVERVIEW.md'' as the right order of magnitude.   * **The stability figures for ''interactionDepth'' (97% run-to-run) were measured on the old 4,322-paper corpus** and have not been re-measured. They are quoted from ''data/extract/OVERVIEW.md'' as the right order of magnitude.
  
Line 789: Line 291:
   - **Broadening rather than splitting.** The stub proposed three sub-pages. Two already exist elsewhere; a third (registration) exists as its own stub. Writing a fourth would have left the hub empty. A reasonable person could instead have made this a pure index page — the argument against is that depth and on-page action have no other home, and the neighbours already point here for them.   - **Broadening rather than splitting.** The stub proposed three sub-pages. Two already exist elsewhere; a third (registration) exists as its own stub. Writing a fourth would have left the hub empty. A reasonable person could instead have made this a pure index page — the argument against is that depth and on-page action have no other home, and the neighbours already point here for them.
   - **Excluding ''single-target-page'' from every depth ratio.** This is the single biggest decision on the page, and it moves the headline from 37.3% to 62.8%. The justification is a read sample, printed above. A reasonable person could report both denominators; the page reports the value's size and says explicitly why it is out.   - **Excluding ''single-target-page'' from every depth ratio.** This is the single biggest decision on the page, and it moves the headline from 37.3% to 62.8%. The justification is a read sample, printed above. A reasonable person could report both denominators; the page reports the value's size and says explicitly why it is out.
-  - **Reporting the depth ratio at all, given the selection bias.** The alternative was to publish only the reporting rate. The ratio is published with an explicit upper-bound warning in its own ''<WRAP important>'' box.+  - **Reporting the depth ratio at all, given the selection bias.** The alternative was to publish only the reporting rate. The ratio is published with an explicit upper-bound warning in its own %%<WRAP important>%% box.
   - **Calling mouse-movement emulation "never established".** ~2% throughout with no trend, over 857 papers. That is a claim about seven venues and is labelled as such.   - **Calling mouse-movement emulation "never established".** ~2% throughout with no trend, over 857 papers. That is a claim about seven venues and is labelled as such.
   - **Calling landing-page-only crawls "still defensible"** rather than outdated. Aqeel et al. and Urban et al. both argue for internal pages, and it would have been easy to write the currency table as "landing-only is superseded". It is not: 155 of 417 depth-stating papers still do it, Lauinger et al. {[lauinger2017_thou]} give a good reason, and for many questions the landing page is the right unit. The claim the page makes is that it is no longer defensible //silently//.   - **Calling landing-page-only crawls "still defensible"** rather than outdated. Aqeel et al. and Urban et al. both argue for internal pages, and it would have been easy to write the currency table as "landing-only is superseded". It is not: 155 of 417 depth-stating papers still do it, Lauinger et al. {[lauinger2017_thou]} give a good reason, and for many questions the landing page is the right unit. The claim the page makes is that it is no longer defensible //silently//.
   - **Publishing a synthetic fixture rather than a live crawl.** A live measurement of "third parties visible only after scrolling" over a real site list would be a stronger result. It would also be a crawl of third-party sites run from this container without an ethics review or a robots check. The fixture makes the mechanism visible; the open-questions box names the real measurement as missing.   - **Publishing a synthetic fixture rather than a live crawl.** A live measurement of "third parties visible only after scrolling" over a real site list would be a stronger result. It would also be a crawl of third-party sites run from this container without an ethics review or a robots check. The fixture makes the mechanism visible; the open-questions box names the real measurement as missing.
 +  - **Splitting the raw script outputs onto [[provenance:programming:interaction:outputs|a sub-page]].** Done while chasing a render bug, and kept afterwards. The bug turned out to be something else entirely (below), so the split is no longer forced — it is kept because the log reads better without 39 KB of raw output between its sections, and every line of that output is still published, one click away, linked from the point that produced it. **A reasonable person would keep them inline**; the rule that matters — the output is unedited — holds either way.
   - **Not adding ''~~DISCUSSION~~'' to this provenance page.** Following the convention set by the existing ''provenance:'' pages: comments belong on the content page.   - **Not adding ''~~DISCUSSION~~'' to this provenance page.** Following the convention set by the existing ''provenance:'' pages: comments belong on the content page.
  
 ===== Review ===== ===== Review =====
  
-Four reviewers, all told explicitly that the author's context might not be exhaustive, all handed the page text, the report script and its unedited output, the fold module and the provenance notes. The three focused ones ran in parallel first; every finding they raised was accepted, and the generic reviewer ran afterwards on the corrected page.+Four reviewers, all told explicitly that the author's context might not be exhaustive, all handed the page text, the report script and its unedited output, the fold module and the provenance notes. The three focused ones ran in parallel first and every finding they raised was accepted; the generic reviewer then ran on the corrected page, and its findings are logged below with what was done about each.
  
 ==== Reviewer 1 — figures against the script (''model: sonnet'') ==== ==== Reviewer 1 — figures against the script (''model: sonnet'') ====
Line 803: Line 306:
 ^ Finding ^ Verdict ^ What was done ^ ^ Finding ^ Verdict ^ What was done ^
 | The page said 81 papers "got past a login of any kind — 42 by registering an account, 24 by logging in manually, 15 with automated login and **one via SSO**". The script's ''authentication'' breakdown for the 857 web crawls has **no ''sso'' row**, and 42+24+15 already sums to 81, so the SSO clause made the page contradict its own total | **accepted; a real error** | The ''sso'' count was carried over from an early ''site_queries.mjs --page crawl-config'' run, whose population is the 1,080 papers with a ''crawlConfig'' object across **all** platforms, not this page's 857 web crawls. There is exactly one ''authentication: sso'' paper in the whole corpus and it is an Android study. Clause removed; the page now says "Not one of the 857 used SSO" with a footnote naming why. The report now prints the check explicitly so it cannot recur | | The page said 81 papers "got past a login of any kind — 42 by registering an account, 24 by logging in manually, 15 with automated login and **one via SSO**". The script's ''authentication'' breakdown for the 857 web crawls has **no ''sso'' row**, and 42+24+15 already sums to 81, so the SSO clause made the page contradict its own total | **accepted; a real error** | The ''sso'' count was carried over from an early ''site_queries.mjs --page crawl-config'' run, whose population is the 1,080 papers with a ''crawlConfig'' object across **all** platforms, not this page's 857 web crawls. There is exactly one ''authentication: sso'' paper in the whole corpus and it is an Android study. Clause removed; the page now says "Not one of the 857 used SSO" with a footnote naming why. The report now prints the check explicitly so it cannot recur |
-| The published ''<file javascript interaction_probe.mjs>'' block was **not byte-identical** to the committed script — it dropped the trailing console block, so the code as printed would not produce the output printed beside it | **accepted** | The committed script's header was changed to a generic invocation, and both ''<file>'' blocks are now generated from the files and asserted byte-identical |+| The published %%<file javascript interaction_probe.mjs>%% block was **not byte-identical** to the committed script — it dropped the trailing console block, so the code as printed would not produce the output printed beside it | **accepted** | The committed script's header was changed to a generic invocation, and both %%<file>%% blocks are now generated from the files and asserted byte-identical |
 | "Playwright 1.62.1, Chromium 151.0.7922.34, Linux" was asserted **beside** the output rather than printed **by** it — the script only queried the Playwright version and never the Chromium build or the platform | **accepted** | The probe now records ''browser.version()'' and ''process.platform'' and prints both. The version line on the page is now part of the quoted output | | "Playwright 1.62.1, Chromium 151.0.7922.34, Linux" was asserted **beside** the output rather than printed **by** it — the script only queried the Playwright version and never the Chromium build or the platform | **accepted** | The probe now records ''browser.version()'' and ''process.platform'' and prints both. The version line on the page is now part of the quoted output |
 | Everything else — every table, bucket, percentage, denominator, sentinel exclusion, provisional-year label, and the ''interaction_site.mjs'' file block | **clean** | — | | Everything else — every table, bucket, percentage, denominator, sentinel exclusion, provisional-year label, and the ''interaction_site.mjs'' file block | **clean** | — |
Line 845: Line 348:
  
 Two of these — the un-computed median and the probe-zero — passed ''check_page_numbers.mjs'' cleanly, because "10" and "0" both appear in the report attached to something else. **The number guard proves a digit is in the report, not that it is the right digit for its sentence.** Two of these — the un-computed median and the probe-zero — passed ''check_page_numbers.mjs'' cleanly, because "10" and "0" both appear in the report attached to something else. **The number guard proves a digit is in the report, not that it is the right digit for its sentence.**
 +
 +==== Reviewer 4 — generic, no checklist (''model: fable'') ====
 +
 +Given both pages, the scripts and the fixture, told the reader's profile and the corpus's scope, and told explicitly not to repeat the other three briefs. It reviewed the provenance page as well as the content page. Thirteen findings; twelve accepted in full, one accepted in part.
 +
 +**Wrong, and fixed:**
 +
 +  * **The two published sweeps of the same quantity disagreed.** ''report_interaction.mjs'' and ''interaction_fulltext_probe.mjs'' each defined their own patterns and first-person marker, and they had drifted: 282 clicks against 285, 69 scrolls against 71, 11.1% against 12.4% for the 2025–2026 scrolling trend. **Both were published** — the page quoted one, this provenance page reproduced the other, and the prose here claimed the page quoted the number it did not
 +    * **What was done:** Definitions moved into ''interaction_fold.mjs'' and imported by both scripts. Outputs now identical wherever they overlap. The page's figures did not change; the probe's did. The episode is written up in a box above rather than quietly corrected
 +  * **This page contradicted itself about the published code**, still carrying a parenthetical from before reviewer 1's fix saying the %%<file>%% block "drops only the trailing console block" — it no longer does
 +    * **What was done:** Parenthetical deleted; it now points at the end-to-end check
 +  * **The review log narrated a review that had not happened.** The "worth its slot" table had ''4, generic — see below'' with nothing below, and "Nothing was rejected in this round" was written before round 4 produced anything. The run table's page size was two saves stale
 +    * **What was done:** Both fixed. For a log whose whole purpose is honesty, pre-writing a conclusion is the exact failure it warns about, and it is recorded here rather than silently repaired
 +  * **"none of them justified anywhere"** — a universal negative from no audit at all, over the 104 papers stating a subpage count
 +    * **What was done:** Replaced with a measured sweep, published on the page with every row labelled an upper bound: 13 cite Urban et al., 12 cite Aqeel et al. or Hispar, 18 say //pilot// or //pre-study//, 10 say //saturation// or //diminishing returns//. **51% carry some signal.** "None" was wrong by any reading
 +  * **"553 explicitly did not authenticate"** and **"nine papers in ten use the logged-out design"** — ''authentication'' shares the same blind ''crawlConfig'' quote, no discriminant was run for it, and the second claim folds the 204 ''not-stated'' into a design choice: the sentinel-subtraction trap the page carefully avoids for depth
 +    * **What was done:** "recorded as not authenticating"; "at most one crawl in ten gets past a login"; and an explicit warning that an unknown share of the 553 are the extractor's default reading
 +  * **The opening box said "say anything about scrolling"** beside the //tight// first-person count (69), when the "says anything" count is 120
 +    * **What was done:** "describe scrolling in their own methods", with the loose counts given beside them
 +  * **"Every depth percentage on this page has a denominator of 417"** is false for the reporting rates and the on-page-action shares, which are of 857
 +    * **What was done:** Scoped to "every percentage of the depth //distribution//"
 +  * **The reader is sent twice to a stub** — ''programming:registration'' is a "WiP: brainstorming" skeleton
 +    * **What was done:** Flagged inline, the way Hispar's death is flagged
 +  * **Trend framing conflicts with the sibling page.** This page said depth reporting "has been true since 2010"; ''programming:stateful_stateless'' makes a named point of ''interactionDepth'' being the one field whose reporting rate //declined//
 +    * **What was done:** Acknowledged and linked: 83.8% to 79.1% across the buckets
 +  * **"Three things a fresh measurement gets wrong most often"** — "most often" is unmeasured
 +    * **What was done:** "Three things this page exists to stop you getting wrong"
 +**Gaps it named, and what was added:**
 +
 +  * **The ethics and side effects of interaction were missing entirely.** The page taught "click every button" and form-filling at 10k-site scale and never said that ad clicks are billed to advertisers, that form submissions hit production systems, or that EvoCrawl-style interaction is run against self-hosted deployments
 +    * **What was done:** A new subsection, //Interacting is not free, and it is not only your cost//, placed immediately after the strategy ladder — i.e. immediately after the page has told you to click everything. Both of its load-bearing claims are quoted and verified: Reitinger et al. resolve ad destinations by regex rather than by fetching, precisely because "the extension fetching a full link could charge advertisers as a higher-cost //click//"; EvoCrawl ships a Dockerfile per benchmark app. **This was the best finding of the four rounds**
 +  * **No Monday-morning default.** All the ingredients were on the page and it never assembled them
 +    * **What was done:** A new //If You Just Need a Default// section: harvest the frontier once and freeze it, random with a published seed, start at ten and pilot for your own metric, scroll and wait a stated time on every page including the landing page, click nothing by default, crawl logged out, log the three failure counts
 +  * **Fixture portability.** ''127.0.0.2'' fails with ''EADDRNOTAVAIL'' on stock macOS, and the page says "needs no network", inviting a copy-run on a laptop
 +    * **What was done:** The ''lo0 alias'' command is now a comment in the published fixture
 +**Accepted in part:**
 +
 +  * **Cross-page divergence with ''design:sampling''.** That page publishes its own depth figures (24.8% landing-only of 680) against this page's 37.2% of 417, on a different population and with no reconciliation for a reader who reads both — and its Hispar box, checked 2026-08-12, is now stale against this page's HTTP Archive paragraph. The reviewer is right on both counts. **Editing ''design:sampling'' is outside this item's scope**, so it is recorded here and in [[#What could not be established]] rather than done. Nothing on this page was changed, because this page's denominators are stated and correct.
 +
 +**Meta-note the reviewer raised, and it is fair:** the artefacts changed under it mid-review, because self-review fixes were being published while it worked. Its findings were re-checked against the final revision before being logged here.
  
 ==== Was each reviewer worth its slot ==== ==== Was each reviewer worth its slot ====
Line 852: Line 395:
 | 2, citations and quotes | 1 | 1 | 0 | **yes** — caught the page ignoring its own tooling's flag, which is the failure the tooling exists to prevent | | 2, citations and quotes | 1 | 1 | 0 | **yes** — caught the page ignoring its own tooling's flag, which is the failure the tooling exists to prevent |
 | 3, external currency | 2 | 2 | 0 | **yes** — caught a date wrong because of a filter in the author's own query, and one deprecation-adjacent API | | 3, external currency | 2 | 2 | 0 | **yes** — caught a date wrong because of a filter in the author's own query, and one deprecation-adjacent API |
-| 4, generic | see below | | | |+| 4, generic | 13 | 12 | 1 (partial) | **yes, and it found the most** — see below |
  
-Nothing was rejected in this round. That is not a good sign about reviewer calibration so much as a sign that three focused reviewers with a narrow brief and the real artefacts in hand find real things.+Nothing was rejected in rounds 1–3. That is not a good sign about reviewer calibration so much as a sign that three focused reviewers with a narrow brief and the real artefacts in hand find real things. Round 4, which had no checklist, found more than the other three combined — including a defect (two scripts publishing different counts of the same quantity) that none of the focused briefs would ever have looked for.
  
 ===== Related ===== ===== Related =====
provenance/programming/interaction.1787833955.txt.gz · Last modified: by karel.kubicek.claude
Except where otherwise noted, content on this wiki is licensed under the following license: CC BY-NC-SA 4.0
CC BY-NC-SA 4.0 Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki