This is an old revision of the document!
Table of Contents
Provenance: Programming:Interaction
Working log behind Interaction with Websites. Every figure on that page, the query that produced it, its denominator, the folds and their residue, the quotes checked, the external sources verified and rejected, what could not be established, and the review. Corpus-wide caveats — the venue scope, the selection funnel, the stability of each field — are on corpus and are not restated here.
Voice is a working log, not prose. It is for someone checking a number.
The run
| Date | 2026-08-27 |
| Corpus at the time | data/extract/run1/extractions.jsonl, 5,859 papers, 7 venues, 2010–2026 (the 2026-08-11 extension, commit 8a6b843) |
| Page before | a 2,411-byte stub: a <WRAP todo> box, three bullet topics, and one claim about Urban et al. |
| Page after | 50,697 bytes, rev 1787833934 |
| Model | Claude Opus 5, one session, no sub-agents used for research |
| Sub-agents | four reviewers only (see Review) |
| Scripts written | interaction_fold.mjs, report_interaction.mjs, interaction_fulltext_probe.mjs, interaction_depth_sentences.mjs, interaction_context.mjs, verify_interaction_figures.mjs, interaction_quotecheck.py, sandbox/interaction_site.mjs, sandbox/interaction_probe.mjs |
| Bibliography | 7 entries added to bibliography (rev 1787832147); no duplicate keys, no duplicate titles |
Create, extend or overlap? Extend, and broaden. The stub was titled Interaction with website and listed three topics “to cover in separate pages”: form input, consent, and stateful/stateless crawling. Two of those three now have their own pages (consent, stateful_stateless) and the third is registration. That left the hub with nothing of its own. The decision taken here was to give it the subject the neighbours explicitly delegate to it: consent says “the crawler-side mechanics of clicking anything at all are on Interaction”, and stateful_stateless says “for what to do on the page once you are there, see Interaction”. So the page is now about depth, subpage selection, on-page action and the identity knobs, and it delegates consent and login back out. No content was removed except the one claim corrected below.
One inherited claim was wrong and is now corrected on the page. The stub said, of Urban et al. [1Urban, Tobias; Degeling, Martin; Holz, Thorsten; Pohlmann, Norbert (2020): "Beyond the Front Page:Measuring Third Party Dynamics in the Field", in: Proceedings of The Web Conference 2020, pp. 1275–1286. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)]: “From 10 visited subpages is increase small.” That number is not in the paper. Its pre-study visits 25, 50, 75, 100, 250, 500 and 1,000 subsites — 10 is not one of the steps — and its stated result is that the cookie count “stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites”. The stub understated the saturation point by an order of magnitude, in the direction that would let a reader justify a shallower crawl. Verified against both renderings of the paper (see Quotes checked).
Populations and denominators
Every figure on the page uses one of these four. None of them is 5,859.
| Name | Rule | N | Used for |
|---|---|---|---|
crawled | crawlConfig object present OR studyTypes includes automated-web-crawl | 1,120 | the corpus-wide 75.1% depth-reporting rate, and the statefulness comparison |
webCrawled | crawled AND platforms includes web | 857 | the page's main denominator |
— with a crawlConfig object | of those 857 | 838 | per-field distributions (the other 19 count as not-stated) |
| site-depth axis | webCrawled AND interactionDepth in {landing-page-only, landing-plus-subpages, deep-crawl} | 417 | every depth ratio |
The 263 crawled papers that are not webCrawled are app-store scraping, network probing and social-graph traversal. “How far into the site” does not mean the same thing there, so they are excluded and the count is printed by the report.
The written boundary, fixed before the first figure
crawlConfig.interactionDepth has five values. Only three of them answer “how far into the site did the crawl go”. The fourth, single-target-page (285 of the 857 web crawls — the single largest value), does not: read against the papers' own text it fires on website-fingerprinting trace collection, phishing-URL monitors, search-result scrapes, cloud-dashboard pulls and API polling. Those crawls have a page list, not a site list.
Folding it into the depth ratio would have moved “went beyond the landing page” from 262/417 (62.8%) to 262/702 (37.3%) and made the number meaningless in both directions. It is reported separately on the page and excluded from every depth denominator. A reproducible sample of its evidence quotes, every 12th of the 285, was read on 2026-08-27; representative entries:
CCS/2010/blade-... "Prior to visiting each URL, the testbed reverts the VM ..." USENIX/2011/show-me-... "The crawler traverses any redirection pages and then fetches ... the resulting page" IMC/2019/shamfinder-... "We accessed to the corresponding websites via the two schemes, HTTP and HTTPS" USENIX/2020/phishtime-.. "40 virtual machines ... to empirically monitor blacklisting of each website" CCS/2023/transformer-... "After each visit, we closed the browser and cleared the cookies" IMC/2025/roam-without-.. "We conducted daily retrievals of eSIM offers over a four-month period"
None of those is a statement about site depth.
A terminology trap that the exclusion does not fix, and that the page names instead. In the phishing, malvertising and spam literature “landing page” means the destination of a redirect or ad click. A detection[] sweep for /landing page/i over the web-crawl population returns mostly that sense: spam campaigns sharing a blacklisted landing page, rogue landing-page lifetime, LDA topics over advertiser landing pages. Anyone re-running a query on this page's topic must read the sense, not the string.
Every query, with its output
The whole report, unedited. Run it with:
cd /workspace/publications_dataset && node /workspace/artifacts/wiki/scripts/report_interaction.mjs node /workspace/artifacts/wiki/scripts/report_interaction.mjs --wiki # DokuWiki tables node /workspace/artifacts/wiki/scripts/report_interaction.mjs --list # the 417 site-depth papers node /workspace/artifacts/wiki/scripts/report_interaction.mjs --quotes <regex>
- report_interaction-output.txt
============================================================================== A. POPULATION ============================================================================== corpus 5859 crawled (crawlConfig OR studyTypes automated-web-crawl) 1120 webCrawled (crawled AND platforms includes 'web') 857 <- this page's denominator of which carry a crawlConfig object 838 crawled but NOT web (excluded: app stores, network scans, social graphs) 263 ============================================================================== B. interactionDepth — what the crawling literature says about depth ============================================================================== ── crawled — 1120 papers ── interactionDepth papers share ----------------------- ------ ----- single-target-page 376 33.6% not-stated 239 21.3% landing-page-only 162 14.5% deep-crawl 157 14.0% landing-plus-subpages 146 13.0% (no crawlConfig object) 40 3.6% stated (any non-sentinel value): 841 / 1120 = 75.1% ── webCrawled — 857 papers ── interactionDepth papers share ----------------------- ------ ----- single-target-page 285 33.3% landing-page-only 155 18.1% landing-plus-subpages 137 16.0% not-stated 136 15.9% deep-crawl 125 14.6% (no crawlConfig object) 19 2.2% stated (any non-sentinel value): 702 / 857 = 81.9% ── Where interactionDepth ranks among the crawl-configuration fields ── crawlConfig field papers stating it share of 1120 crawling papers ----------------- ----------------- ----------------------------- interactionDepth 841 75.1% authentication 779 69.6% browsers 529 47.2% consentAction 349 31.2% statefulness 219 19.6% headless 140 12.5% gap between the best-reported field and the second: 62 papers, 5.5 percentage points. ── The depth axis proper — webCrawled papers giving one of the three SITE-depth values ── denominator: 417 papers (NOT 857; single-target-page and not-stated are excluded) depth papers share of 417 --------------------- ------ ------------ landing-page-only 155 37.2% landing-plus-subpages 137 32.9% deep-crawl 125 30.0% went beyond the landing page: 262 / 417 = 62.8% CAVEAT: papers that went deeper have more reason to say so, so this ratio is an upper bound on the share of the whole field that crawls beyond the landing page. ============================================================================== C. Reporting rate over time (webCrawled, four-year buckets) ============================================================================== bucket web crawls states a depth on the site-depth axis of those, beyond landing gives subpages/site ---------- ---------- -------------- ---------------------- ------------------------ ------------------- 2010–2013 80 67 (83.8%) 35 27 (77.1%) 8 (10.0%) 2014–2017 130 109 (83.8%) 57 35 (61.4%) 11 (8.5%) 2018–2021 241 198 (82.2%) 121 72 (59.5%) 37 (15.4%) 2022–2024 253 207 (81.8%) 125 82 (65.6%) 33 (13.0%) 2025–2026* 153 121 (79.1%) 79 46 (58.2%) 15 (9.8%) * 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and IEEE S&P 2026 / WWW 2026 abstracts are incompletely indexed, so selection under-covers them. ============================================================================== D. The label discriminant — does interactionDepth track the paper's own text? ============================================================================== crawlConfig carries ONE evidence quote for the WHOLE object, so reading it cannot validate interactionDepth. Instead: does the paper contain a first-person sentence naming the ROOT of a site as the unit of a visit (LANDING), something BELOW the root (DEEPER), both, or neither? `not-stated` is the control row. label papers with text landing phrase only deeper phrase only both neither --------------------- ---------------- ------------------- ------------------ ---------- ----------- landing-page-only 155 85 (54.8%) 4 (2.6%) 24 (15.5%) 42 (27.1%) landing-plus-subpages 137 57 (41.6%) 12 (8.8%) 42 (30.7%) 26 (19.0%) deep-crawl 125 27 (21.6%) 21 (16.8%) 27 (21.6%) 50 (40.0%) single-target-page 285 78 (27.4%) 11 (3.9%) 9 (3.2%) 187 (65.6%) not-stated 136 16 (11.8%) 5 (3.7%) 1 (0.7%) 114 (83.8%) ── What the shared crawlConfig quote actually evidences ── pool: 71 web crawls labelled landing-plus-subpages that also state a subpage count. every 6th read by hand on 2026-08-27 = 12 papers. what the ONE shared quote evidences papers of 12 ----------------------------------- ------------ depth 6 other 5 partial 1 [depth] CCS/2011/fashion-crimes-trending-term-exploitation-on-the-web "parsing the front page of the domain and a set of five additional web pages within the same domain" [depth] CCS/2015/sunlight-fine-grained-targeting-detection-at-scale-with-statistical-confidence "For each site, we visit the top 10 pages returned from a site-specific search on Google" [other] IMC/2018/how-tracking-companies-circumvented-ad-blockers-using-websockets "we performed four crawls over our sampled 100K websites" — evidences repeatVisits and the population, not depth [other] WWW/2019/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private "We did not erase any cookie after a harvest was performed" — evidences statefulness [other] PETS/2020/no-boundaries-data-exfiltration-by-third-parties-embedded-on-web-pages "from US-based EC2 cloud instances using stateless browsers" — evidences vantage and statefulness [partial] NDSS/2021/cv-inspector-towards-automating-detection-of-adblock-circumvention "We visit each site for a total of eight times ... four page visits per case" — visit structure, not site depth [depth] USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild "we therefore additionally selected three random links to an internal subpage and visited these as well" [other] IMC/2023/on-the-similarity-of-web-measurements-under-different-experimental-setups "We choose to use a stateless approach" — evidences statefulness [depth] PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework "select nine internal hyperlinks at random from the homepage" [depth] PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing "We added ten random sub-pages per domain" [depth] NDSS/2026/dom-xss-detection-via-webpage-interaction-fuzzing-and-url-component-synthesis "extracted a maximum of 5 subpages from each page" [other] PETS/2025/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic "Browser state was purged after every crawl" — evidences statefulness Read this as the size of the blindness, not as an error rate: a quote that evidences statefulness does not make the depth label wrong, it makes the quote useless as a check on it. ── Internal consistency: does subpagesPerSite ever contradict interactionDepth? ── 104 web crawls state BOTH an interactionDepth and a subpagesPerSite. contradictions (landing-page-only with n>0, or landing-plus-subpages with n=0): 0 This is a corroborating check on the enum, independent of the text discriminant above: the two fields are extracted from the same shared quote but mean different things, so a systematically wrong depth label would show up here as disagreement. ── Hand audit of the `landing-page-only` "neither" cell ── 42 papers in the cell; every 4th read by hand on 2026-08-27 = 11 papers. verdict papers of 11 ---------------- ------------ inferred 10 stated-elsewhere 1 [inferred] IMC/2011/measuring-the-state-of-ecn-readiness-in-servers-clients-and-routers "For each domain, we retrieve the complete web page" — the unit is the domain, the page is never named [inferred] IMC/2018/needle-in-a-haystack-tracking-down-elite-phishing-domains-in-the-wild "we crawled 6,755 unique phishing URLs" — a URL list, not a site depth [inferred] IMC/2019/visiblev8-in-browser-monitoring-of-javascript-in-the-wild "We visited each domain in our target list 5 times" — domain granularity only [stated-elsewhere] WWW/2019/the-chain-of-implicit-trust-an-analysis-of-the-web-third-party-resources-loading "the Alexa top-200K websites' main pages" — states the root, but not in a first-person sentence [inferred] WWW/2021/cookie-swap-party-abusing-first-party-cookies-for-web-tracking "the Alexa top 10K websites that we crawled" — site granularity only [inferred] IMC/2023/ptperf-on-the-performance-evaluation-of-tor-pluggable-transports "we accessed each website five times using curl" — site granularity only [inferred] CCS/2024/a-first-look-at-security-and-privacy-risks-in-the-rapidapi-ecosystem "We crawl all the content within the API listing page" — a listing page, not a site root [inferred] WWW/2024/quic-is-not-quick-enough-over-fast-internet no first-person sentence names the page unit at all [inferred] USENIX/2025/dns-flare-a-flush-reload-attack-on-dns-forwarders no first-person sentence names the page unit at all [inferred] WWW/2026/webgeoinfer-structure-free-multi-stage-framework-for-geolocation-inference-from "We access their web interfaces ... comparable to how an ordinary user visits a public site" [inferred] IMC/2025/somesite-i-used-to-crawl-awareness-agency-and-efficacy-in-protecting-content-cre "We visit the site with a headless browser" — site granularity only ============================================================================== E. subpagesPerSite — the number people actually pick ============================================================================== denominator: 857 web crawls; 104 (12.1%) give a number. of the 417 on the site-depth axis, 102 (24.5%) give a number. subpages per site papers share of 104 --------------------- ------ ------------ 0 (landing page only) 4 3.8% 1–4 29 27.9% 5–9 15 14.4% 10–19 21 20.2% 20–49 16 15.4% 50–99 2 1.9% 100+ 17 16.3% per four-year bucket: 2010–2013 n= 8 median=5 2014–2017 n= 11 median=5 2018–2021 n= 37 median=10 2022–2024 n= 33 median=15 2025–2026* n= 15 median=5 median 10; the five most common values: 5 subpages: 14 papers 3 subpages: 12 papers 10 subpages: 12 papers 1 subpages: 11 papers 20 subpages: 8 papers the largest values (a "deep crawl" of one application, not a site-list crawl): 2000: 1 papers 1000: 3 papers 500: 2 papers 300: 1 papers 200: 3 papers 100: 7 papers ============================================================================== F. repeatVisits and authentication — the other two interaction knobs ============================================================================== repeatVisits stated: 233 / 857 web crawls = 27.2% of those, 50 (21.5%) visit exactly once; 183 more than once. ── authentication — of 857 web crawls ── authentication papers share ----------------------- ------ ----- none 553 64.5% not-stated 204 23.8% account-registration 42 4.9% manual-login 24 2.8% (no crawlConfig object) 19 2.2% automated-login 15 1.8% crawls that got past a login of any kind: 81 (9.5%) sso in THIS population: 0; sso anywhere in the corpus: 1 (USENIX/2024/darkfleece-probing-the-dark-side-of-android-subscription-apps, platforms mobile) ============================================================================== G. What the crawler does ON the page — full-text probe ============================================================================== The schema has NO field for scrolling, clicking, hovering, typing or waiting. These are UPPER BOUNDS on "the paper did this": a first-person sentence matching the pattern. "we did not scroll" matches too. Denominator: the 857 web crawls. papers with full text on disk: 857 of 857 what the paper says it does papers share of 857 ------------------------------------- ------ ------------ clicks something 282 32.9% scrolls 69 8.1% waits / dwells a stated time 56 6.5% types on the keyboard 39 4.6% fills a form 28 3.3% moves the mouse 18 2.1% hovers 14 1.6% picks links at random 14 1.6% says it aims for human-like behaviour 14 1.6% mentions bot / crawler detection 61 7.1% bot/crawler detection, 2010-2013 bucket: 3 of 80 papers match ANYWHERE in the text, 0 in a first-person sentence. Same, by four-year bucket (share of that bucket's web crawls): pattern 2010–2013 2014–2017 2018–2021 2022–2024 2025–2026* ------------------------------------- ----------- ------------ ------------ ------------ ------------ clicks something 35/80 43.8% 41/130 31.5% 72/241 29.9% 80/253 31.6% 54/153 35.3% scrolls 2/80 2.5% 5/130 3.8% 18/241 7.5% 27/253 10.7% 17/153 11.1% waits / dwells a stated time 0/80 0.0% 12/130 9.2% 14/241 5.8% 24/253 9.5% 6/153 3.9% types on the keyboard 4/80 5.0% 6/130 4.6% 7/241 2.9% 17/253 6.7% 5/153 3.3% fills a form 1/80 1.3% 1/130 0.8% 8/241 3.3% 10/253 4.0% 8/153 5.2% moves the mouse 3/80 3.8% 2/130 1.5% 7/241 2.9% 4/253 1.6% 2/153 1.3% hovers 2/80 2.5% 0/130 0.0% 6/241 2.5% 4/253 1.6% 2/153 1.3% picks links at random 2/80 2.5% 1/130 0.8% 6/241 2.5% 1/253 0.4% 4/153 2.6% says it aims for human-like behaviour 0/80 0.0% 4/130 3.1% 1/241 0.4% 3/253 1.2% 6/153 3.9% mentions bot / crawler detection 0/80 0.0% 2/130 1.5% 17/241 7.1% 24/253 9.5% 18/153 11.8% ============================================================================== H. LLM-agent-driven browsing — sweep, then hand verdicts ============================================================================== sweep hits over the 857 web crawls: 35 NOTE: 1 hand verdicts are not returned by the current sweep USENIX/2024/understanding-help-seeking-and-help-giving-on-social-media-for-image-based-sexua role papers of 35 -------------- ------------ not-browsing 14 subject 9 citation 5 instrument 4 captcha-solver 2 instrument-app 1 An LLM agent drove the browsing in 5 of 857 web crawls (0.6%). NDSS/2025/yurascanner-leveraging-llms-for-task-driven-web-app-scanning "YURASCANNER operates as a goal-based agent, suggesting actions to achieve predefined objectives by processing webpages" PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms "a multi-agent system powered by large language models (LLMs) to simulate persona-driven browsing behavior" CCS/2025/whispertest-a-voice-control-based-library-for-ios-ui-automation "we design an LLM-based navigation pipeline tailored to perform privacy-related measurements in 200 apps" USENIX/2026/the-state-of-passkeys-studying-the-adoption-and-security-of-passkeys-on-the-web "We built a prototype tool using Playwright and the agentic LLM-based Browser Use framework" PETS/2026/on-the-suitability-of-llm-driven-agents-for-dark-pattern-audits "We design and deploy an LLM-driven auditing agent capable of end-to-end traversal of rights-request workflows" years: 2025, 2025, 2026, 2026, 2026 — the earliest is 2025. ============================================================================== I. The measured consequences — per-paper figures with their own denominators ============================================================================== WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field * subsites set 36% more cookies than landing pages: 78 vs 55 on average, over the top 10k websites (TLD+1), 100 subsites each * the mean number of accessed/set cookies "stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites" — pre-study on 100 randomly selected sites from the top 1,000, at 25/50/75/100/250/500/1,000 subsites * trackers (EasyPrivacy) increased ~6% on subsites; 2.5% of sites tracked ONLY on subsites * Fingerprint2 device fingerprinting increased 25% on subsites; present on 0.15% of landing pages USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains * same Alexa top-10K, same cookie policy: homepage crawl 192,038 requests / 76,816 chains; interactive crawl (random internal pages via clicks on iframes and anchors) 575,550 / 229,151 — 3.0x * 302 redirects are 42.91% of AT redirect requests on the homepage crawl vs 28.56% interactive * AT requests navigating to a new domain: 51.32% homepage vs 47.49% interactive USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild * front pages plus "three random links to an internal subpage": "visiting subpages did indeed significantly increase the prevalence by about 17% compared to only crawling the front pages" * denominator, from the paper's Table 2: 206,676 sites (24.42%) carry an indicator, of which 30,494 only on a subpage (+17% overall) * per technique, subpage-only increase: ShortCut +22%, TrigBreak +80%, ModBuilt +18%, WidthDiff +19%, LogGet +33% — the aggregate hides an 80% NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search * against three state-of-the-art scanners on ten web applications: "a 59% increase in code coverage and successfully submits HTML forms 5x more frequently than the next best tool" PETS/2026/on-the-suitability-of-llm-driven-agents-for-dark-pattern-audits * 456 data-broker websites; verified workflow completion 87% in Phase 2 and 79% in Phase 3 IMC/2024/browsing-without-third-party-cookies-what-do-you-see * clickstream traversal, and its own bias statement: "our dataset is biased towards static inner pages ... we are less likely to explore the more dynamic areas of a website" NDSS/2017/thou-shalt-not-depend-on-me-analysing-the-use-of-outdated-javascript-libraries-o * the reason a landing-page-only design is chosen: "We only crawl the homepage of each visited site due to the presence of many sites that thwart deeper traversal by requiring log-ins." USENIX/2024/trust-me-if-you-can-how-usable-is-trusted-types-in-practice * a fully specified depth: "visited those up to two levels of links from the start page, while crawling at most 500 URLs per site" USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission * typing is simulated against bot detection: "we simulate user typing behavior by using randomized intervals for each key press and dwell times, as well as the delay times between each press" CCS/2020/the-cookie-hunter-automated-black-box-auditing-for-web-authentication-and-author * what stops an interacting crawler: "In 22 cases, there was some form of an anti-bot challenge that our system was not able to solve and, thus, could not proceed with registration." PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework * the most complete interaction statement in the corpus, quoted whole on the page: "For each domain, we programmed the crawler to load the domain's homepage, scroll to the bottom of the page, then sleep for 25 seconds. Further, we programmed our crawler to select nine internal hyperlinks at random from the homepage and crawl them using the same load, scroll, and sleep approach." — two sentences, and they answer unit, depth, selection rule, on-page action, wait, and same-treatment-for-subpages PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing * a stated subpage-selection rule: "We added ten random sub-pages per domain, filtering to exclude auxiliary pages like privacy policies or contact pages." ============================================================================== Y. Figures the page quotes from other papers or other pages ============================================================================== statefulness stated: 219 / 1120 crawling papers = 19.6% (the page cites this as the comparison for how well depth is reported) Zeber et al., TheWebConf 2020, "The Representativeness of Automated Web Crawls as a Surrogate for Human Browsing" — IN the corpus; figures read from data/fulltext/2020/WWW/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing/paper.cols.txt on 2026-08-27, all four verbatim: "over 50,000 users of the Firefox Web browser"; "30 million site visits across 50,000 users" "crawler site visits issued requests to a median of 11.6 third-party domains, whereas for visits by humans, the median was 4.5 third parties" "The median number of tracking domains accessed by a user on visiting a Trexa list site is 1.9, whereas for the crawler it is 6.1" Urban et al.'s site list: "the top 10k websites" — the page's "10,000 websites" in the opening box is a rhetorical example of a methods sentence, not a corpus figure. NOT FIGURES — literals inside the fixture the page publishes in <file> blocks, scanned only under check_page_numbers.mjs --code. Listed here so they trace to this report instead of polluting the guard's shared ALLOW map: 127.0.0.1 the first-party origin the fixture serves 127.0.0.2 the third-party origin the fixture serves, so the browser treats it as third-party 8231 the first-party port 8232 the third-party port 3000 the CSS pixel height of the tall div that makes scrolling necessary 50 the scroll-bottom tolerance in pixels 200, 300 waitForTimeout values in milliseconds 404, 200 HTTP status codes the fixture writes 2 the maximum crawl depth in the last strategy Tool versions in the measured probe, printed by the probe itself (sandbox/interaction_probe-output.txt): Playwright 1.62.1, Chromium 151.0.7922.34. ============================================================================== Z. NON-CORPUS NUMBERS ON THE PAGE, with their primary source ============================================================================== Aqeel et al., IMC 2020, "On Landing and Internal Web Pages" — NOT in the corpus (the selection stage labelled it neither a security nor a privacy measurement; see data/labels/run1/labels.jsonl). Read from the authors' own PDF at balakrishnanc.github.io/papers/aqeel-imc2020.pdf on 2026-08-27: 119 web-performance publications reviewed (IMC/PAM/NSDI/SIGCOMM/CoNEXT, 2015–2019): 41 (34.5%) need no revision, 48 (40.3%) minor, 30 (25.2%) major -> 65.5% need at least minor landing pages are on average 34% larger than internal pages (geometric mean of ratios, H1K) for 32% of H1K sites the landing page has FEWER objects than the median internal page internal pages' content visually loads 14% more slowly in the median (Speed Index, Ht30) median: internal pages collectively fetch from 18 third-party domains never seen on the landing page; for 10% of H1K sites, 80 or more at the 80th percentile, internal pages carry 20 tracking requests and landing pages 28 in about 10% of H1K sites, internal pages have no trackers while the landing page does header bidding, of 200 sites: 17 have HB ads on the landing page, a further 12 only on internal pages 36 of the 1000 H1K sites serve their landing page over HTTP; among sites with a secure landing page, 170 have at least one HTTP internal page Hispar: H2K = 100,000 URLs, >=2000 sites x 50 URLs (1 landing + <=49 internal), weekly refresh, bootstrapped from Alexa Top 1M via Google "site:" search Hispar is dead. Checked 2026-08-27: hispar.cs.duke.edu does not resolve (DNS NXDOMAIN); last Wayback capture that returned content 2024-11-16 (HTTP 200, 1,420 bytes); last capture of any kind 2024-12-03 (HTTP 403); github.com/waqaraqeel/hispar last pushed 2020-10-27 and github.com/waqaraqeel/lit_grabber 2020-05-26. Its bootstrap list, Alexa Top 1M, was itself retired in 2022. HTTP Archive crawls exactly ONE secondary page per site, since April 2022. har.fyi (HTTP Archive's own BigQuery reference), "Pages table": "Pages are tested on a monthly basis and as of April 2022, both the root page and one secondary page are tested." The pages table carries is_root_page and root_page columns. How the secondary page is chosen, from github.com/HTTPArchive/crawl at commit 4b83ac0 (2025-08-20), crawl.py and crawl_pubsub.py: MAX_DEPTH = 1 and MAX_BREADTH = 1, and the child job is the FIRST link in the page's crawl_links whose hostname equals the parent's, whose extension is not in ['.jpg','.jpeg','.gif','.png','.webp','.avif','.webm','.pdf', '.tiff','.zip'], and which has not already been visited. NOTE: httparchive.org/faq still says "The HTTP Archive examines each URL in the list, but does not crawl the website's other pages" — the FAQ is stale; the BigQuery reference and the crawl controller agree with each other and not with it. Checked 2026-08-27.
The full-text probe for on-page actions
The schema has no field for scrolling, clicking, hovering, typing or waiting. The only way to count them is to sweep the text, and the count is an upper bound by construction: it counts papers with a first-person sentence matching a pattern, and “we did not scroll” matches too.
Two columns are printed side by side so the width of the claim is visible: any match anywhere in the paper against a match in a sentence that also contains a first-person marker. The gap is large — 469 papers mention clicking, 285 mention it in a first-person sentence — and the page quotes only the tighter column.
- interaction_fulltext_probe-output.txt
population: --pop web → 857 papers, 857 of them with full text on disk pattern papers (any match) % papers (1st-person sentence) % ------------------------------- ------------------ ----- ---------------------------- ----- scroll 120 14.0% 71 8.3% click 469 54.7% 285 33.3% hover 48 5.6% 14 1.6% mouse movement 42 4.9% 20 2.3% keyboard / typing 97 11.3% 39 4.6% form fill 59 6.9% 28 3.3% dwell / wait 87 10.2% 55 6.4% random walk / link following 21 2.5% 14 1.6% subpage / subsite 110 12.8% 87 10.2% landing page only 22 2.6% 14 1.6% human-like / realistic browsing 28 3.3% 14 1.6% LLM / agent-driven browsing 32 3.7% 14 1.6% bot / crawler detection 95 11.1% 61 7.1% cloaking 150 17.5% 81 9.5% Both columns are UPPER BOUNDS on "the paper did this". Read the sentences with --hits "<pattern>" before quoting either. First-person-sentence hits by four-year bucket (share of that bucket's papers): pattern 2010-2013 2014-2017 2018-2021 2022-2024 2025-2026* ------------------------------- ----------- ------------ ------------ ------------ ------------ scroll 2/80 2.5% 5/130 3.8% 18/241 7.5% 27/253 10.7% 19/153 12.4% click 35/80 43.8% 41/130 31.5% 72/241 29.9% 81/253 32.0% 56/153 36.6% hover 2/80 2.5% 0/130 0.0% 6/241 2.5% 4/253 1.6% 2/153 1.3% mouse movement 3/80 3.8% 2/130 1.5% 7/241 2.9% 4/253 1.6% 4/153 2.6% keyboard / typing 4/80 5.0% 6/130 4.6% 7/241 2.9% 17/253 6.7% 5/153 3.3% form fill 1/80 1.3% 1/130 0.8% 8/241 3.3% 10/253 4.0% 8/153 5.2% dwell / wait 0/80 0.0% 12/130 9.2% 14/241 5.8% 23/253 9.1% 6/153 3.9% random walk / link following 2/80 2.5% 1/130 0.8% 6/241 2.5% 1/253 0.4% 4/153 2.6% subpage / subsite 2/80 2.5% 9/130 6.9% 19/241 7.9% 37/253 14.6% 20/153 13.1% landing page only 0/80 0.0% 5/130 3.8% 2/241 0.8% 6/253 2.4% 1/153 0.7% human-like / realistic browsing 0/80 0.0% 4/130 3.1% 1/241 0.4% 3/253 1.2% 6/153 3.9% LLM / agent-driven browsing 0/80 0.0% 0/130 0.0% 0/241 0.0% 2/253 0.8% 12/153 7.8% bot / crawler detection 0/80 0.0% 2/130 1.5% 17/241 7.1% 24/253 9.5% 18/153 11.8% cloaking 11/80 13.8% 18/130 13.8% 19/241 7.9% 25/253 9.9% 8/153 5.2% * 2025-2026 is provisional: CCS/IMC 2026 have not been held and IEEE S&P/WWW 2026 are incompletely selected.
Read individual sentences behind any row with:
node scripts/interaction_fulltext_probe.mjs --pop web --hits "scrolls" node scripts/interaction_fulltext_probe.mjs --pop web --hits "LLM / agent-driven browsing"
Folding, hand verdicts, and the residue
There is no name fold on this page. Nothing it counts is a free-text name: interactionDepth and authentication are enums, subpagesPerSite and repeatVisits are integers, and the on-page-action figures are regex sweeps whose patterns are published in full in the script. What this page has instead is three hand classifications, all guarded at run time.
1. LLM-agent roles
A full-text sweep for agent frameworks and LLM-driven browsing over the 857 web crawls returns 35 papers. A sweep hit is an upper bound and nothing more: “LLM” and “agent” appear in papers that classify text with a model, in papers that study agents as a threat, and in reference lists citing Mind2Web. Each of the 35 was read for its deciding sentence and given one of six roles.
| Role | Papers | Meaning |
|---|---|---|
not-browsing | 14 | the LLM classifies or scores text; no crawler is driven by it |
subject | 9 | agents are what the paper measures, not how it measures |
citation | 5 | the name appears only in related work or the reference list |
instrument | 4 | an LLM agent drove the browsing the paper measured with |
captcha-solver | 2 | a vision-language agent defeats a bot check |
instrument-app | 1 | as instrument, but the target is a mobile app |
The report throws a FAILURE line if the sweep returns a paper with no hand verdict, so the map cannot silently go stale as the corpus grows. It printed 8 such lines on the first run; all 8 were read and added. One verdict (USENIX/2024/understanding-help-seeking-…) is carried over from a wider earlier sweep and is not returned by the current one; the report notes it rather than dropping it.
Two of the citation verdicts are homographs rather than citations, and are marked as such in the fold: in USENIX/2016/internet-jones-… and USENIX/2023/policycomp-… the string “browser use” is ordinary English, not the Browser Use framework.
Residue: zero. Every sweep hit has a verdict.
2. The `landing-page-only` "neither" cell
See The validity problem specific to this page below. 42 papers; every 4th read by hand (11 papers); 10 verdicts inferred, 1 stated-elsewhere; residue zero within the sample, and the report fails loudly if a sampled paper has no verdict or a verdict no longer corresponds to a sampled paper. The 31 unsampled papers in the cell are not claimed to be anything — the page says “a hand-read sample of those found”, not “all of those are”.
3. What the shared quote evidences
12 papers, verdicts depth / other / partial, each with the deciding quote. See The validity problem specific to this page. Residue zero within the sample; the report fails loudly on divergence.
The hand maps, with the deciding sentence for every entry, are in scripts/interaction_fold.mjs.
The validity problem specific to this page
crawlConfig carries ONE evidence quote for the WHOLE object. statefulness, browsers, headless, consentAction, interactionDepth, subpagesPerSite, authentication, languages and repeatVisits all share it. So the site's usual check — “read a handful of evidence.quote values before publishing a figure” — is structurally blind here: the quote behind an interactionDepth value usually evidences the browser or the statefulness instead. Spot-checking it validates nothing. (The same trap cost privacy:consent a 19.4% false-positive rate on consentAction before it was audited.)
How blind, measured. On the subset where it is cheapest to judge — the 71 web crawls labelled landing-plus-subpages that also state a subpagesPerSite — every 6th was read on 2026-08-27 and its single shared quote classified by what it actually evidences:
| What the ONE shared quote evidences | Papers of 12 |
|---|---|
| the depth or the subpage count | 6 |
a different crawlConfig key entirely (statefulness ×3, vantage, repeat visits) | 5 |
| visit structure, but not site depth | 1 |
So about half the quotes behind a depth value evidence something else. Read that as the size of the blindness, not as an error rate: a quote about statefulness does not make the depth label wrong, it makes the quote useless as a check on it. The 12 verdicts, each with the deciding quote, are in SHARED_QUOTE_AUDIT in scripts/interaction_fold.mjs and printed by the report, which fails loudly if the sample and the map diverge.
In place of the quote check, a discriminant. For each paper, does its own text contain a first-person sentence naming the root of a site as the unit of a visit (LANDING), something below the root (DEEPER), both, or neither? If the label tracks the text there will be a gradient across the enum, and not-stated is the control.
| Label | Papers with text | landing phrase only | deeper phrase only | both | neither |
|---|---|---|---|---|---|
landing-page-only | 155 | 85 (54.8%) | 4 (2.6%) | 24 (15.5%) | 42 (27.1%) |
landing-plus-subpages | 137 | 57 (41.6%) | 12 (8.8%) | 42 (30.7%) | 26 (19.0%) |
deep-crawl | 125 | 27 (21.6%) | 21 (16.8%) | 27 (21.6%) | 50 (40.0%) |
single-target-page | 285 | 78 (27.4%) | 11 (3.9%) | 9 (3.2%) | 187 (65.6%) |
not-stated (control) | 136 | 16 (11.8%) | 5 (3.7%) | 1 (0.7%) | 114 (83.8%) |
What this supports, and what it does not:
- It supports the labels.
landing-page-onlypapers name a deeper unit and nothing else in 4 of 155 cases.landing-plus-subpageshas by far the highest “both” rate, which is the signature you would predict for a design that visits the root and pages below it. And thenot-statedcontrol has neither phrase 83.8% of the time — the sentinel is behaving. - It does not validate any individual paper. It is a distributional check.
- It shows a real soft spot. 42 of 155 (27.1%)
landing-page-onlypapers contain neither phrase. A hand-read sample of 11 of those found 10 in which the paper describes visits at site granularity (“we visited each domain in our target list 5 times”, “we accessed each website five times using curl”, “the Alexa top 10K websites that we crawled”) and never says which part of the site — so the label is a defensible default inference, not a stated value. One (WWW/2019/the-chain-of-implicit-trust-…) does state the root, in wording both regexes miss (“the Alexa top-200K websites' main pages”, outside a first-person sentence).
A second, independent corroboration
subpagesPerSite and interactionDepth are extracted from the same shared quote but mean different things, so a systematically wrong depth label would show up as disagreement between them. Of the 104 web crawls that state both, contradictions = 0 — no landing-page-only paper carries a positive subpage count, and no landing-plus-subpages paper carries zero. Printed by the report.
Consequence, stated on the page: the 81.9% “states a depth” figure is an upper bound. It is still the best-reported crawl-configuration field in the schema, and that is the claim the page makes.
The regexes matter more than they look. A first attempt used a narrower “explicit landing-page phrase” pattern and reported that only 47.1% of landing-page-only papers had one. Hand-reading the misses showed the pattern was simply missing “root page”, “main page”, “frontpage” and plural-with-a-number forms (“crawl the landing pages of 100K websites”). That 47.1% was a regex artefact and was never published. The widened patterns are in scripts/interaction_fold.mjs as LANDING and DEEPER, in full.
The measured strategy ladder
The page publishes a measurement rather than an assertion: a local instrumented site with five third-party beacons, each reachable only under a different condition, visited under six strategies by Playwright's own Chromium. It needs no network.
- interaction_probe-output.txt
strategy pages t-landing t-article t-scroll t-click t-deep ------------------------------------------------------------------- ----- --------- --------- --------- --------- --------- landing page only 1 ✓ ✗ ✗ ✗ ✗ landing + FIRST same-origin link (the HTTP Archive rule) 2 ✓ ✗ ✗ ✗ ✗ landing + ALL same-origin links from the landing page 4 ✓ ✓ ✗ ✗ ✗ landing + all links, and scroll to the bottom of each 4 ✓ ✓ ✓ ✗ ✗ landing + all links, scroll, and click every button 4 ✓ ✓ ✓ ✓ ✗ depth 2: landing, its links, and their links, with scroll and click 5 ✓ ✓ ✓ ✓ ✓ t-landing on the landing page t-article on one subpage, at load t-scroll on that subpage, only after scrolling to the bottom t-click on another subpage, only after a button click t-deep on a page linked only from a subpage, never from the landing page Playwright 1.62.1, Chromium 151.0.7922.34, linux. Needs no network.
Design notes, because the fixture is doing real work:
- The third parties are served from a second origin (127.0.0.2) so the browser treats them as third-party the way a real measurement would.
/aboutis deliberately empty of third parties and is deliberately the first link in DOM order. That is what makes row 2 informative: the HTTP Archive's first-same-origin-link rule lands on it and learns nothing. On a real site the first nav item is usually institutional, so this is the rule's normal behaviour, not a contrived worst case./commentsis linked only from/article, so no landing-page-plus-one-hop strategy can reach it.- The scroll beacon fires on a
scrolllistener at the bottom of a 3000px page; the click beacon on aclicklistener. Both are the ordinary lazy-loading shapes. - The ladder is monotone and each rung adds exactly one beacon, which is the point: depth does not substitute for scrolling and scrolling does not substitute for clicking.
The two files are published on the page in <file> blocks and are byte-identical to sandbox/interaction_site.mjs and sandbox/interaction_probe.mjs (the <file> copy of the probe drops only the trailing console block that prints the beacon legend, which the page prints as prose instead).
Quotes checked
Two independent renderings, because they fail on different sentences. paper.cols.txt repairs two-column reading order but still interleaves a fragment of the neighbouring column into some sentences; a pypdf extraction of paper.pdf keeps raw order (wrong across columns, right within a line). A quote present in either, with whitespace collapsed and -\n hyphenation undone, is present in the paper.
- interaction_quotecheck-output.txt
ok cols=Y pdf=Y WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field "subsites set considerably more (36 %) cookies than the respective landing pages" ok cols=Y pdf=Y WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field "On average, 55 cookies were set when loading a landing page while 78 were set when a subsi" ok cols=n pdf=Y WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field "the mean amount of accessed/set cookies stabilizes around 50 (SD: 100; median at 12) after" ok cols=Y pdf=Y WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field "visited 25, 50, 75, 100, 250, 500, and 1,000 subsites" ok cols=n pdf=Y WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field "we choose 100 random subsites that we used during the experiment crawls" ok cols=Y pdf=Y WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field "we randomly selected 100 websites (TLD+1) from the top 1,000 websites" ok cols=Y pdf=Y USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains "Interactive crawls additionally navigate to random internal pages by clicking on iframes a" ok cols=Y pdf=Y USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains "we visit the Alexa top-10K homepages" ok cols=Y pdf=Y NDSS/2017/thou-shalt-not-depend-on-me-analysing-the-use-of-outdated-javascript-libraries-o "We only crawl the homepage of each visited site due to the presence of many sites that thw" ok cols=Y pdf=Y IMC/2024/browsing-without-third-party-cookies-what-do-you-see "our dataset is biased towards static inner pages which may cause us to underestimate the i" ok cols=Y pdf=Y USENIX/2024/trust-me-if-you-can-how-usable-is-trusted-types-in-practice "visited those up to two levels of links from the start page, while crawling at most 500 UR" ok cols=Y pdf=Y USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission "we simulate user typing behavior by using randomized intervals for each key press and dwel" ok cols=Y pdf=Y CCS/2020/the-cookie-hunter-automated-black-box-auditing-for-web-authentication-and-author "In 22 cases, there was some form of an anti-bot challenge that our system was not able to " ok cols=Y pdf=Y PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework "For each domain, we programmed the crawler to load the domain's homepage,9 scroll to the b" ok cols=Y pdf=Y PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework "we programmed our crawler to select nine internal hyperlinks at random from the homepage a" ok cols=Y pdf=Y PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing "We added ten random sub-pages per domain, filtering to exclude auxiliary pages like privac" ok cols=Y pdf=Y WWW/2020/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing "crawler site visits issued requests to a median of 11.6 third-party domains, whereas for v" ok cols=Y pdf=Y WWW/2020/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing "The median number of tracking domains accessed by a user on visiting a Trexa list site is " ok cols=Y pdf=n USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild "we can see that visiting subpages did indeed significantly increase the prevalence by abou" ok cols=Y pdf=Y USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild "we therefore additionally selected three random links to an internal subpage and visited t" ok cols=n pdf=Y NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search "EvoCrawl achieves a 59% increase in code coverage and successfully submits HTML forms 5" ok cols=Y pdf=Y PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms "a multi-agent system powered by large language models (LLMs) to simulate persona-driven br" ok cols=Y pdf=Y USENIX/2026/the-state-of-passkeys-studying-the-adoption-and-security-of-passkeys-on-the-web "Browser Use framework" 23 quotes checked against two independent renderings. found in paper.cols.txt : 20 found in pypdf(paper.pdf): 22 found in at least one : 23 found in NEITHER : 0
23 of 23 quotes located; 20 in .cols, 22 in the PDF rendering, 0 in neither.
The three that .cols misses are column splices, and the raw text is worth showing because it is what a splice looks like:
found that the mean amount ON THE CLIENT, BECAUSE EACH PAIR CAN BE USED FOR DIFFERENT PURPOSES. of accessed/set cookies stabilizes around 50 (SD: 100; median at 12) WE HEURISTICALLY GROUP COOKIES IN DIFFERENT CATEGORIES BASED ON after visiting 100 subsites (see Figure 2).
(capitals mark the interleaved neighbouring column). The pypdf rendering of the same passage:
mber of visited subsites, we found that the mean amount of accessed/set cookies stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites (see Figure 2).
The page quotes the reconstructed sentence. That reconstruction is the whole basis of the page's central correction, which is why it was checked twice.
One elision, declared. The page quotes Aziz and Wilson [2Aziz, Muhammad Abu Bakar; Wilson, Christo (2024): "Johnny Still Can't Opt-out: Assessing the IAB CCPA Compliance Framework", in: Proceedings on Privacy Enhancing Technologies. (DOI)] whole as the model reporting statement. The PDF renders “the domain's homepage,9” with a superscript footnote marker; the page drops the marker and nothing else. The quotecheck carries the marker form so the elision is visible in the audit trail rather than invisible in the prose.
A separate needle check covers every literal per-paper figure on the page — not just the quoted sentences — against paper.cols.txt:
- verify_interaction_figures-output.txt
SPLICED WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field "stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites" (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py) SPLICED WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field "we choose 100 random subsites that we used during the experiment crawls" (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py) SPLICED NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search "a 59% increase in code coverage and successfully submits HTML forms 5" (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py) 44 needles checked, 0 not located anywhere, 3 located only after undoing a column splice, 6 shorter than 20 chars (flagged, not trusted). Paraphrased figures (NOT verbatim in the paper — anchor check only):
44 needles, 0 not located anywhere, 3 located only after undoing a column splice, 6 flagged as shorter than 20 characters (the count of needles is unchanged by the review fixes; two Musch and Johns needles replaced one) (a short needle can pass for the wrong reason; they are flagged rather than trusted).
One paraphrase was caught in review and removed. The first draft presented “Visiting subpages increased prevalence by about 17%” in quotation marks. That string is a detection[].prevalence summary of Musch and Johns [3Musch, Marius; Johns, Martin (2021): "U Can't Debug This: Detecting JavaScript Anti-Debugging Techniques in the Wild", in: Proceedings of the USENIX Security Symposium. (Link)], not a sentence they wrote — the checker's own PARAPHRASED block flagged it and the draft did not act on the flag. The page now quotes the paper's sentence, “we can see that visiting subpages did indeed significantly increase the prevalence by about 17% compared to only crawling the front pages”, and adds the denominator and per-technique spread from the paper's Table 2. PARAPHRASED is now empty: every quoted string on the page is verbatim.
The published code was checked end-to-end
Not just “the <file> block matches the committed file”, which is where this check usually stops. Both code blocks were downloaded back off the rendered page through DokuWiki's own export, diffed against the committed scripts, copied into an empty directory and run:
curl -s "https://measuretheweb.org/_export/code/programming/interaction?codeblock=1" -o interaction_site.mjs curl -s "https://measuretheweb.org/_export/code/programming/interaction?codeblock=2" -o interaction_probe.mjs node interaction_probe.mjs
Both downloads are byte-identical to sandbox/interaction_site.mjs and sandbox/interaction_probe.mjs, and running the downloaded pair reproduces the table on the page, ending with the version line the page quotes. A reader who copies the page gets the thing that produced the result.
One deliberate red link
Related Pages links to automated_measurements, which does not exist. It is left in because the wiki already promises it from start and from four other pages, so removing it here would be inconsistent rather than helpful. Every other outbound link on both pages was fetched and returns a real page.
External sources: verified, and rejected
Everything below was fetched on 2026-08-27. Recall was not used.
| Claim on the page | How it was verified | Verdict |
|---|---|---|
| Aqeel et al., IMC 2020 [4Aqeel, Waqar; Chandrasekaran, Balakrishnan; Feldmann, Anja; Maggs, Bruce M. (2020): "On Landing and Internal Web Pages: The Strange Case of Jekyll and Hyde in Web Performance Measurement", in: Proceedings of the ACM Internet Measurement Conference, pp. 680-695. (DOI)] is not in this corpus | data/labels/run1/labels.jsonl has the record with securityMeasurement: false, privacyMeasurement: false — the selection stage screened it out as a web-performance paper. It is in the bibliographic index (data/corpus2/.meta/IMC-2020.json) and has no data/fulltext directory | confirmed; the paper was read externally and every figure quoted from it is attributed to the PDF, not to the corpus |
| All Aqeel et al. figures | downloaded balakrishnanc.github.io/papers/aqeel-imc2020.pdf (the first author's institutional copy; ACM's DL returns 403 to this container) and read the text with pypdf. 119 papers / 41 / 48 / 30; 34% larger; 32% fewer objects; 14% slower Speed Index; median 18 unseen third parties, 80+ for 10%; 20 vs 28 tracking requests at the 80th percentile; 17 + 12 header-bidding sites; 36 HTTP landing pages, 170 sites with an HTTP internal page; Hispar H2K = 100,000 URLs, ≥2000 sites × 50 | all confirmed verbatim |
| Hispar is dead | hispar.cs.duke.edu → DNS NXDOMAIN (socket.gethostbyname fails; cs.duke.edu resolves fine, so it is the host and not the network). Wayback CDX: last 200 capture 20241116044231. GitHub API: waqaraqeel/hispar pushed_at 2020-10-27, waqaraqeel/lit_grabber 2020-05-26, neither archived | confirmed |
| HTTP Archive tests one secondary page since April 2022 | har.fyi/reference/tables/pages/ (HTTP Archive's own BigQuery reference): “Pages are tested on a monthly basis and as of April 2022, both the root page and one secondary page are tested.” Schema carries is_root_page and root_page | confirmed |
httparchive.org/faq is stale | fetched: “The HTTP Archive examines each URL in the list, but does not crawl the website's other pages.” Directly contradicts the BigQuery reference and the crawl controller | confirmed; the page footnotes it as a warning |
| How the secondary page is chosen | cloned github.com/HTTPArchive/crawl at tip 4b83ac03178bddacbf5c79204d8b896b5b1c8c1c (2025-08-20). crawl.py and crawl_pubsub.py both set MAX_DEPTH = 1, MAX_BREADTH = 1. The child-job loop takes links from crawl_links in order, keeps those whose hostname equals the parent's and whose extension is not in ['.jpg','.jpeg','.gif','.png','.webp','.avif','.webm','.pdf','.tiff','.zip'] and not already visited, and breaks at width > MAX_BREADTH | confirmed |
| Playwright / Chromium versions in the probe | printed by the probe itself: Playwright 1.62.1, Chromium 151.0.7922.34 | printed, not recalled |
Rejected, and why:
- The 2022 Web Almanac methodology page as the source for the secondary-page claim. It says the right thing (“This year, we introduced secondary pages”), but it is a yearly report about a snapshot, not the crawler's documentation, and it does not state the selection rule. Replaced with the BigQuery reference plus the crawl source.
- A search-engine summary that asserted “as of April 2022, both the root page and one secondary page are tested” without a link to the primary source. The sentence turned out to be correct, but it was re-derived from
har.fyibefore use. Search summaries are not sources here. dl.acm.orgfor Aqeel et al. — returns HTTP 403 to this container. Used the first author's institutional PDF instead, and said so.httparchive.org/faqas a statement of current behaviour — kept only as a documented example of stale vendor documentation.
A change to a shared guard
scripts/check_page_numbers.mjs treated version identifiers as three-part semver only. Chromium builds are four-part (“151.0.7922.34”), so the guard shredded them into “151.0” and “7922.34” and reported both as unaccounted figures forever. Both the detection regex and the removal regex were widened to \d+\.\d+\.\d+(?:\.\d+)?.
Checked for behaviour change against four existing page/report pairs:
| Pair | Before vs after |
|---|---|
privacy_consent.CURRENT.txt / report_consent-output.txt | identical (both OK) |
programming_crawler_foxhound.txt / report_foxhound-output.txt | identical (both OK) |
design_mobile_and_app_measurement.txt / report_mobile-output.txt | identical (both fail — pre-existing, unrelated) |
design_longitudinal.txt / report_longitudinal-output.txt | improved: unaccounted 24 → 20, and three four-part Chromium versions (113.0.5672.0, 152.0.7977.54, 154.0.8026.0) now reported as version identifiers instead of as digit fragments. That pair was already failing before the change and still fails |
What could not be established
- Whether the 42-paper “neither” cell is representative. 11 were read. The other 31 were not, and no claim is made about them.
- Whether the depth labels are right for any individual paper. The discriminant is distributional. The object-level shared quote makes a per-paper check impossible without reading every paper, and 417 was too many for this run.
- Whether the 62.8% “went beyond the landing page” ratio has a selection bias, and how large. The page says it is an upper bound and gives the mechanism (a deeper crawl has more reason to describe itself). It does not quantify it. Closing it would need the 136
not-statedpapers read by hand. - How much an LLM agent changes a measurement. Five papers use one as the browsing instrument; none reports a same-site-list comparison against a scripted crawl. This is on the page as an open question.
- Whether
subpagesPerSitemeans the same thing across papers. “10 subpages” could be 10 attempted, 10 successfully loaded, or 10 distinct URLs. Nothing in the schema distinguishes them and few papers say. The page reports the distribution and does not compute a mean. - The stability figures for
interactionDepth(97% run-to-run) were measured on the old 4,322-paper corpus and have not been re-measured. They are quoted fromdata/extract/OVERVIEW.mdas the right order of magnitude.
Judgement calls
- Broadening rather than splitting. The stub proposed three sub-pages. Two already exist elsewhere; a third (registration) exists as its own stub. Writing a fourth would have left the hub empty. A reasonable person could instead have made this a pure index page — the argument against is that depth and on-page action have no other home, and the neighbours already point here for them.
- Excluding
single-target-pagefrom every depth ratio. This is the single biggest decision on the page, and it moves the headline from 37.3% to 62.8%. The justification is a read sample, printed above. A reasonable person could report both denominators; the page reports the value's size and says explicitly why it is out. - Reporting the depth ratio at all, given the selection bias. The alternative was to publish only the reporting rate. The ratio is published with an explicit upper-bound warning in its own
<WRAP important>box. - Calling mouse-movement emulation “never established”. ~2% throughout with no trend, over 857 papers. That is a claim about seven venues and is labelled as such.
- Calling landing-page-only crawls “still defensible” rather than outdated. Aqeel et al. and Urban et al. both argue for internal pages, and it would have been easy to write the currency table as “landing-only is superseded”. It is not: 155 of 417 depth-stating papers still do it, Lauinger et al. [5Lauinger, Tobias; Chaabane, Abdelberi; Arshad, Sajjad; Robertson, William; Wilson, Christo; Kirda, Engin (2017): "Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web", in: Proceedings of the Network and Distributed System Security Symposium. (Link)] give a good reason, and for many questions the landing page is the right unit. The claim the page makes is that it is no longer defensible silently.
- Publishing a synthetic fixture rather than a live crawl. A live measurement of “third parties visible only after scrolling” over a real site list would be a stronger result. It would also be a crawl of third-party sites run from this container without an ethics review or a robots check. The fixture makes the mechanism visible; the open-questions box names the real measurement as missing.
- Not adding
~~DISCUSSION~~to this provenance page. Following the convention set by the existingprovenance:pages: comments belong on the content page.
Review
Four reviewers, all told explicitly that the author's context might not be exhaustive, all handed the page text, the report script and its unedited output, the fold module and the provenance notes. The three focused ones ran in parallel first; every finding they raised was accepted, and the generic reviewer ran afterwards on the corrected page.
Reviewer 1 — figures against the script (''model: sonnet'')
| Finding | Verdict | What was done |
|---|---|---|
The page said 81 papers “got past a login of any kind — 42 by registering an account, 24 by logging in manually, 15 with automated login and one via SSO”. The script's authentication breakdown for the 857 web crawls has no sso row, and 42+24+15 already sums to 81, so the SSO clause made the page contradict its own total | accepted; a real error | The sso count was carried over from an early site_queries.mjs –page crawl-config run, whose population is the 1,080 papers with a crawlConfig object across all platforms, not this page's 857 web crawls. There is exactly one authentication: sso paper in the whole corpus and it is an Android study. Clause removed; the page now says “Not one of the 857 used SSO” with a footnote naming why. The report now prints the check explicitly so it cannot recur |
The published <file javascript interaction_probe.mjs> block was not byte-identical to the committed script — it dropped the trailing console block, so the code as printed would not produce the output printed beside it | accepted | The committed script's header was changed to a generic invocation, and both <file> blocks are now generated from the files and asserted byte-identical |
| “Playwright 1.62.1, Chromium 151.0.7922.34, Linux” was asserted beside the output rather than printed by it — the script only queried the Playwright version and never the Chromium build or the platform | accepted | The probe now records browser.version() and process.platform and prints both. The version line on the page is now part of the quoted output |
Everything else — every table, bucket, percentage, denominator, sentinel exclusion, provisional-year label, and the interaction_site.mjs file block | clean | — |
That first finding is the exact failure mode the task spec warns about: a number carried over from a different population. It survived the author's own number guard because the guard checks that a figure appears in the report, and “1” appears everywhere.
Reviewer 2 — citations and quotes (''model: sonnet'')
| Finding | Verdict | What was done |
|---|---|---|
“Visiting subpages increased prevalence by about 17%” was presented in quotation marks but is a detection[].prevalence summary, not a sentence Musch and Johns wrote — and the author's own verify_interaction_figures.mjs already flagged it, and the draft shipped anyway | accepted; a real error | Replaced with the paper's sentence, verbatim in both renderings. Took the opportunity to add what the summary had thrown away: the denominator (206,676 sites with an indicator, of which 30,494 only on a subpage) and the per-technique spread, which runs to +80% for triggered breakpoints against the +17% aggregate. The page is better for the correction than it was for the shortcut |
| All 18 citekeys resolve uniquely; no duplicate keys, no DOI or title collision under a different key | clean | — |
| Every “Name et al.” matches the first author of the entry cited beside it | clean | — |
| Every Aqeel et al. figure re-derived from an independently downloaded copy of the PDF | clean | — |
The Hispar and HTTP Archive claims, re-checked independently against DNS, the GitHub API, har.fyi and the crawl source | clean | — |
One thing the reviewer noticed that is not a wiki error and is worth recording: the Sun et al. paper states “87% in Phase 2 and 79% in Phase 3” in one place and “87% in Phase 1 and 79% in Phase 2” in another. The inconsistency is in the source paper. The page quotes the first form, which is the one in the results section.
Reviewer 3 — external currency (''model: sonnet'')
| Finding | Verdict | What was done |
|---|---|---|
| “its last Internet Archive capture is 2024-11-16” — the CDX API shows a later capture, a 403 on 2024-12-03. 2024-11-16 is the last capture that returned any content | accepted; imprecise as written | The author's original CDX query carried filter=statuscode:200, which is exactly why the later 403 was invisible. The page now gives both dates and says which is which. The substance — the site has been dead for about two years — is unchanged |
page.$$eval() and page.$$() in the published fixture still work and are not deprecated, but current Playwright docs mark both “Discouraged” in favour of locator.evaluateAll() and page.locator().all() | accepted, though the reviewer offered it as a style note | Changed anyway, and the probe re-run. A fresh PhD student copies published code; publishing the discouraged form teaches the discouraged form |
Hispar host does not resolve; both repos' push dates; Alexa retired 1 May 2022; the har.fyi sentence; the stale httparchive.org/faq sentence; 4b83ac0 still the tip of HTTPArchive/crawl with MAX_DEPTH = 1 / MAX_BREADTH = 1 and the exact skip_extensions list; locator.pressSequentially() current and locator.fill() firing a single input event; Puppeteer ElementHandle.type() with delay current; Playwright 1.62.1 the current npm latest and Chromium 151.0.7922.34 the build it bundles; Browser Use, OpenWPM and the Public Suffix List all still live under those names | all confirmed live | — |
Caught by the author, between review rounds
Worth recording because it is the same class of error the reviewers were looking for and none of them had reached it yet. The draft said depth is “by a wide margin, the best-reported crawl-configuration field in this corpus”. It is the best-reported, but the margin over authentication is 5.5 percentage points (841 against 779 of 1,120), which is not a wide margin. Replaced with the measured ranking of all six fields, which makes the real point better: there is no cliff between depth and authentication, and a large one below them (47.2%, then 31.2%, 19.6%, 12.5%).
A self-review sweep for absolute claims, between rounds
After the focused reviewers, the draft was swept for words that assert more than a corpus of seven venues can carry — unchanged, always, never, no paper, nobody, every. Five things were wrong or overstated and were fixed:
| Claim as drafted | What the data says | What it says now |
|---|---|---|
| depth is “by a wide margin, the best-reported crawl-configuration field” | best-reported, but 75.1% against 69.6% for authentication — 5.5 percentage points | the measured ranking of all six fields, which shows the real cliff is below those two |
| “Median 10, unchanged across all five buckets” | never computed. Per bucket: 5, 5, 10, 15, 5 on 8, 11, 37, 33, 15 papers | the real series, called small-sample noise rather than a trend |
| “Clicking is flat and has been since 2010” | 43.8%, 31.5%, 29.9%, 31.6%, 35.3% — the highest bucket is the earliest | “has not risen: it sits between 29.9% and 43.8% in every bucket, and the highest value is the earliest one” |
| “no paper in the 2010–2013 bucket mentions bot or crawler detection” (twice) | 0 in a first-person sentence, but 3 of 80 match the pattern somewhere in the text: USENIX/2010/searching-the-searchers-with-searchaudit, WWW/2011/heat-seeking-honeypots-design-and-experience, IEEE-SP/2012/rozzle-de-cloaking-internet-malware | the probe's scope is stated and the loose count given. A probe's zero is not “nobody” |
“Almost nothing on a modern page loads at load” | unmeasured and false as written | names what actually does not: lazy ad slots, tabbed content, form-triggered behaviour |
The Open Questions box's remaining “nobody” claims now carry an explicit scope line: “nobody” means no paper in this corpus's seven venues, 2010–2026.
Two of these — the un-computed median and the probe-zero — passed check_page_numbers.mjs cleanly, because “10” and “0” both appear in the report attached to something else. The number guard proves a digit is in the report, not that it is the right digit for its sentence.
Was each reviewer worth its slot
| Reviewer | Findings | Accepted | Rejected | Worth it |
|---|---|---|---|---|
| 1, figures vs script | 3 | 3 | 0 | yes — caught the cross-population figure, which no automated guard here can catch |
| 2, citations and quotes | 1 | 1 | 0 | yes — caught the page ignoring its own tooling's flag, which is the failure the tooling exists to prevent |
| 3, external currency | 2 | 2 | 0 | yes — caught a date wrong because of a filter in the author's own query, and one deprecation-adjacent API |
| 4, generic | see below |
Nothing was rejected in this round. That is not a good sign about reviewer calibration so much as a sign that three focused reviewers with a narrow brief and the real artefacts in hand find real things.
Related
- Corpus-level provenance — venue scope, selection funnel, field stability
- stateful_stateless and consent — the two neighbouring logs, both of which hit the same shared-
crawlConfig-quote problem
References
- [1]
- Urban, Tobias; Degeling, Martin; Holz, Thorsten; Pohlmann, Norbert (2020): "Beyond the Front Page:Measuring Third Party Dynamics in the Field", in: Proceedings of The Web Conference 2020, pp. 1275–1286. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)
- [2]
- Aziz, Muhammad Abu Bakar; Wilson, Christo (2024): "Johnny Still Can't Opt-out: Assessing the IAB CCPA Compliance Framework", in: Proceedings on Privacy Enhancing Technologies. (DOI)
- [3]
- Musch, Marius; Johns, Martin (2021): "U Can't Debug This: Detecting JavaScript Anti-Debugging Techniques in the Wild", in: Proceedings of the USENIX Security Symposium. (Link)
- [4]
- Aqeel, Waqar; Chandrasekaran, Balakrishnan; Feldmann, Anja; Maggs, Bruce M. (2020): "On Landing and Internal Web Pages: The Strange Case of Jekyll and Hyde in Web Performance Measurement", in: Proceedings of the ACM Internet Measurement Conference, pp. 680-695. (DOI)
- [5]
- Lauinger, Tobias; Chaabane, Abdelberi; Arshad, Sajjad; Robertson, William; Wilson, Christo; Kirda, Engin (2017): "Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web", in: Proceedings of the Network and Distributed System Security Symposium. (Link)
