User Tools

Site Tools


provenance:programming:interaction

This is an old revision of the document!


Provenance: Programming:Interaction

Working log behind Interaction with Websites. Every figure on that page, the query that produced it, its denominator, the folds and their residue, the quotes checked, the external sources verified and rejected, what could not be established, and the review. Corpus-wide caveats — the venue scope, the selection funnel, the stability of each field — are on corpus and are not restated here.

Voice is a working log, not prose. It is for someone checking a number.

The run

Date 2026-08-27
Corpus at the time data/extract/run1/extractions.jsonl, 5,859 papers, 7 venues, 2010–2026 (the 2026-08-11 extension, commit 8a6b843)
Page before a 2,411-byte stub: a <WRAP todo> box, three bullet topics, and one claim about Urban et al.
Page after 48,067 bytes, rev 1787833364 (three saves: initial, then one per review round)
Model Claude Opus 5, one session, no sub-agents used for research
Sub-agents four reviewers only (see Review)
Scripts written interaction_fold.mjs, report_interaction.mjs, interaction_fulltext_probe.mjs, interaction_depth_sentences.mjs, interaction_context.mjs, verify_interaction_figures.mjs, interaction_quotecheck.py, sandbox/interaction_site.mjs, sandbox/interaction_probe.mjs
Bibliography 7 entries added to bibliography (rev 1787832147); no duplicate keys, no duplicate titles

Create, extend or overlap? Extend, and broaden. The stub was titled Interaction with website and listed three topics “to cover in separate pages”: form input, consent, and stateful/stateless crawling. Two of those three now have their own pages (consent, stateful_stateless) and the third is registration. That left the hub with nothing of its own. The decision taken here was to give it the subject the neighbours explicitly delegate to it: consent says “the crawler-side mechanics of clicking anything at all are on Interaction”, and stateful_stateless says “for what to do on the page once you are there, see Interaction”. So the page is now about depth, subpage selection, on-page action and the identity knobs, and it delegates consent and login back out. No content was removed except the one claim corrected below.

One inherited claim was wrong and is now corrected on the page. The stub said, of Urban et al. [1Urban, Tobias; Degeling, Martin; Holz, Thorsten; Pohlmann, Norbert (2020): "Beyond the Front Page:Measuring Third Party Dynamics in the Field", in: Proceedings of The Web Conference 2020, pp. 1275–1286. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)]: “From 10 visited subpages is increase small.” That number is not in the paper. Its pre-study visits 25, 50, 75, 100, 250, 500 and 1,000 subsites — 10 is not one of the steps — and its stated result is that the cookie count “stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites”. The stub understated the saturation point by an order of magnitude, in the direction that would let a reader justify a shallower crawl. Verified against both renderings of the paper (see Quotes checked).

Populations and denominators

Every figure on the page uses one of these four. None of them is 5,859.

Name Rule N Used for
crawled crawlConfig object present OR studyTypes includes automated-web-crawl 1,120 the corpus-wide 75.1% depth-reporting rate, and the statefulness comparison
webCrawled crawled AND platforms includes web 857 the page's main denominator
— with a crawlConfig object of those 857 838 per-field distributions (the other 19 count as not-stated)
site-depth axis webCrawled AND interactionDepth in {landing-page-only, landing-plus-subpages, deep-crawl} 417 every depth ratio

The 263 crawled papers that are not webCrawled are app-store scraping, network probing and social-graph traversal. “How far into the site” does not mean the same thing there, so they are excluded and the count is printed by the report.

The written boundary, fixed before the first figure

crawlConfig.interactionDepth has five values. Only three of them answer “how far into the site did the crawl go”. The fourth, single-target-page (285 of the 857 web crawls — the single largest value), does not: read against the papers' own text it fires on website-fingerprinting trace collection, phishing-URL monitors, search-result scrapes, cloud-dashboard pulls and API polling. Those crawls have a page list, not a site list.

Folding it into the depth ratio would have moved “went beyond the landing page” from 262/417 (62.8%) to 262/702 (37.3%) and made the number meaningless in both directions. It is reported separately on the page and excluded from every depth denominator. A reproducible sample of its evidence quotes, every 12th of the 285, was read on 2026-08-27; representative entries:

CCS/2010/blade-...       "Prior to visiting each URL, the testbed reverts the VM ..."
USENIX/2011/show-me-...  "The crawler traverses any redirection pages and then fetches ... the resulting page"
IMC/2019/shamfinder-...  "We accessed to the corresponding websites via the two schemes, HTTP and HTTPS"
USENIX/2020/phishtime-.. "40 virtual machines ... to empirically monitor blacklisting of each website"
CCS/2023/transformer-... "After each visit, we closed the browser and cleared the cookies"
IMC/2025/roam-without-.. "We conducted daily retrievals of eSIM offers over a four-month period"

None of those is a statement about site depth.

A terminology trap that the exclusion does not fix, and that the page names instead. In the phishing, malvertising and spam literature “landing page” means the destination of a redirect or ad click. A detection[] sweep for /landing page/i over the web-crawl population returns mostly that sense: spam campaigns sharing a blacklisted landing page, rogue landing-page lifetime, LDA topics over advertiser landing pages. Anyone re-running a query on this page's topic must read the sense, not the string.

Every query, with its output

The whole report, unedited. Run it with:

cd /workspace/publications_dataset && node /workspace/artifacts/wiki/scripts/report_interaction.mjs
node /workspace/artifacts/wiki/scripts/report_interaction.mjs --wiki    # DokuWiki tables
node /workspace/artifacts/wiki/scripts/report_interaction.mjs --list    # the 417 site-depth papers
node /workspace/artifacts/wiki/scripts/report_interaction.mjs --quotes <regex>
report_interaction-output.txt
==============================================================================
A. POPULATION
==============================================================================
corpus                                              5859
crawled  (crawlConfig OR studyTypes automated-web-crawl)   1120
webCrawled  (crawled AND platforms includes 'web')         857   <- this page's denominator
  of which carry a crawlConfig object                      838
  crawled but NOT web (excluded: app stores, network scans, social graphs)  263
 
==============================================================================
B. interactionDepth — what the crawling literature says about depth
==============================================================================
 
── crawled — 1120 papers ──
interactionDepth         papers  share
-----------------------  ------  -----
single-target-page       376     33.6%
not-stated               239     21.3%
landing-page-only        162     14.5%
deep-crawl               157     14.0%
landing-plus-subpages    146     13.0%
(no crawlConfig object)  40      3.6%
  stated (any non-sentinel value): 841 / 1120 = 75.1%
 
── webCrawled — 857 papers ──
interactionDepth         papers  share
-----------------------  ------  -----
single-target-page       285     33.3%
landing-page-only        155     18.1%
landing-plus-subpages    137     16.0%
not-stated               136     15.9%
deep-crawl               125     14.6%
(no crawlConfig object)  19      2.2%
  stated (any non-sentinel value): 702 / 857 = 81.9%
 
── Where interactionDepth ranks among the crawl-configuration fields ──
crawlConfig field  papers stating it  share of 1120 crawling papers
-----------------  -----------------  -----------------------------
interactionDepth   841                75.1%
authentication     779                69.6%
browsers           529                47.2%
consentAction      349                31.2%
statefulness       219                19.6%
headless           140                12.5%
  gap between the best-reported field and the second: 62 papers, 5.5 percentage points.
 
── The depth axis proper — webCrawled papers giving one of the three SITE-depth values ──
  denominator: 417 papers (NOT 857; single-target-page and not-stated are excluded)
depth                  papers  share of 417
---------------------  ------  ------------
landing-page-only      155     37.2%
landing-plus-subpages  137     32.9%
deep-crawl             125     30.0%
  went beyond the landing page: 262 / 417 = 62.8%
  CAVEAT: papers that went deeper have more reason to say so, so this ratio is
  an upper bound on the share of the whole field that crawls beyond the landing page.
 
==============================================================================
C. Reporting rate over time (webCrawled, four-year buckets)
==============================================================================
bucket      web crawls  states a depth  on the site-depth axis  of those, beyond landing  gives subpages/site
----------  ----------  --------------  ----------------------  ------------------------  -------------------
2010–2013   80          67 (83.8%)      35                      27 (77.1%)                8 (10.0%)
2014–2017   130         109 (83.8%)     57                      35 (61.4%)                11 (8.5%)
2018–2021   241         198 (82.2%)     121                     72 (59.5%)                37 (15.4%)
2022–2024   253         207 (81.8%)     125                     82 (65.6%)                33 (13.0%)
2025–2026*  153         121 (79.1%)     79                      46 (58.2%)                15 (9.8%)
 
* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and
  IEEE S&P 2026 / WWW 2026 abstracts are incompletely indexed, so selection under-covers them.
 
==============================================================================
D. The label discriminant — does interactionDepth track the paper's own text?
==============================================================================
crawlConfig carries ONE evidence quote for the WHOLE object, so reading it cannot
validate interactionDepth. Instead: does the paper contain a first-person sentence
naming the ROOT of a site as the unit of a visit (LANDING), something BELOW the root
(DEEPER), both, or neither? `not-stated` is the control row.
 
label                  papers with text  landing phrase only  deeper phrase only  both        neither
---------------------  ----------------  -------------------  ------------------  ----------  -----------
landing-page-only      155               85 (54.8%)           4 (2.6%)            24 (15.5%)  42 (27.1%)
landing-plus-subpages  137               57 (41.6%)           12 (8.8%)           42 (30.7%)  26 (19.0%)
deep-crawl             125               27 (21.6%)           21 (16.8%)          27 (21.6%)  50 (40.0%)
single-target-page     285               78 (27.4%)           11 (3.9%)           9 (3.2%)    187 (65.6%)
not-stated             136               16 (11.8%)           5 (3.7%)            1 (0.7%)    114 (83.8%)
 
── Internal consistency: does subpagesPerSite ever contradict interactionDepth? ──
  104 web crawls state BOTH an interactionDepth and a subpagesPerSite.
  contradictions (landing-page-only with n>0, or landing-plus-subpages with n=0): 0
  This is a corroborating check on the enum, independent of the text discriminant above:
  the two fields are extracted from the same shared quote but mean different things,
  so a systematically wrong depth label would show up here as disagreement.
 
── Hand audit of the `landing-page-only` "neither" cell ──
  42 papers in the cell; every 4th read by hand on 2026-08-27 = 11 papers.
verdict           papers of 11
----------------  ------------
inferred          10
stated-elsewhere  1
    [inferred] IMC/2011/measuring-the-state-of-ecn-readiness-in-servers-clients-and-routers
        "For each domain, we retrieve the complete web page" — the unit is the domain, the page is never named
    [inferred] IMC/2018/needle-in-a-haystack-tracking-down-elite-phishing-domains-in-the-wild
        "we crawled 6,755 unique phishing URLs" — a URL list, not a site depth
    [inferred] IMC/2019/visiblev8-in-browser-monitoring-of-javascript-in-the-wild
        "We visited each domain in our target list 5 times" — domain granularity only
    [stated-elsewhere] WWW/2019/the-chain-of-implicit-trust-an-analysis-of-the-web-third-party-resources-loading
        "the Alexa top-200K websites' main pages" — states the root, but not in a first-person sentence
    [inferred] WWW/2021/cookie-swap-party-abusing-first-party-cookies-for-web-tracking
        "the Alexa top 10K websites that we crawled" — site granularity only
    [inferred] IMC/2023/ptperf-on-the-performance-evaluation-of-tor-pluggable-transports
        "we accessed each website five times using curl" — site granularity only
    [inferred] CCS/2024/a-first-look-at-security-and-privacy-risks-in-the-rapidapi-ecosystem
        "We crawl all the content within the API listing page" — a listing page, not a site root
    [inferred] WWW/2024/quic-is-not-quick-enough-over-fast-internet
        no first-person sentence names the page unit at all
    [inferred] USENIX/2025/dns-flare-a-flush-reload-attack-on-dns-forwarders
        no first-person sentence names the page unit at all
    [inferred] WWW/2026/webgeoinfer-structure-free-multi-stage-framework-for-geolocation-inference-from
        "We access their web interfaces ... comparable to how an ordinary user visits a public site"
    [inferred] IMC/2025/somesite-i-used-to-crawl-awareness-agency-and-efficacy-in-protecting-content-cre
        "We visit the site with a headless browser" — site granularity only
 
==============================================================================
E. subpagesPerSite — the number people actually pick
==============================================================================
  denominator: 857 web crawls; 104 (12.1%) give a number.
  of the 417 on the site-depth axis, 102 (24.5%) give a number.
subpages per site      papers  share of 104
---------------------  ------  ------------
0 (landing page only)  4       3.8%
1–4                    29      27.9%
5–9                    15      14.4%
10–19                  21      20.2%
20–49                  16      15.4%
50–99                  2       1.9%
100+                   17      16.3%
  median 10; the five most common values:
        5 subpages: 14 papers
        3 subpages: 12 papers
       10 subpages: 12 papers
        1 subpages: 11 papers
       20 subpages: 8 papers
  the largest values (a "deep crawl" of one application, not a site-list crawl):
     2000: 1 papers
     1000: 3 papers
      500: 2 papers
      300: 1 papers
      200: 3 papers
      100: 7 papers
 
==============================================================================
F. repeatVisits and authentication — the other two interaction knobs
==============================================================================
repeatVisits stated: 233 / 857 web crawls = 27.2%
  of those, 50 (21.5%) visit exactly once; 183 more than once.
 
── authentication — of 857 web crawls ──
authentication           papers  share
-----------------------  ------  -----
none                     553     64.5%
not-stated               204     23.8%
account-registration     42      4.9%
manual-login             24      2.8%
(no crawlConfig object)  19      2.2%
automated-login          15      1.8%
  crawls that got past a login of any kind: 81 (9.5%)
  sso in THIS population: 0; sso anywhere in the corpus: 1 (USENIX/2024/darkfleece-probing-the-dark-side-of-android-subscription-apps, platforms mobile)
 
==============================================================================
G. What the crawler does ON the page — full-text probe
==============================================================================
The schema has NO field for scrolling, clicking, hovering, typing or waiting.
These are UPPER BOUNDS on "the paper did this": a first-person sentence matching
the pattern. "we did not scroll" matches too. Denominator: the 857 web crawls.
 
papers with full text on disk: 857 of 857
 
what the paper says it does            papers  share of 857
-------------------------------------  ------  ------------
clicks something                       282     32.9%
scrolls                                69      8.1%
waits / dwells a stated time           56      6.5%
types on the keyboard                  39      4.6%
fills a form                           28      3.3%
moves the mouse                        18      2.1%
hovers                                 14      1.6%
picks links at random                  14      1.6%
says it aims for human-like behaviour  14      1.6%
mentions bot / crawler detection       61      7.1%
 
Same, by four-year bucket (share of that bucket's web crawls):
pattern                                2010–2013    2014–2017     2018–2021     2022–2024     2025–2026*
-------------------------------------  -----------  ------------  ------------  ------------  ------------
clicks something                       35/80 43.8%  41/130 31.5%  72/241 29.9%  80/253 31.6%  54/153 35.3%
scrolls                                2/80 2.5%    5/130 3.8%    18/241 7.5%   27/253 10.7%  17/153 11.1%
waits / dwells a stated time           0/80 0.0%    12/130 9.2%   14/241 5.8%   24/253 9.5%   6/153 3.9%
types on the keyboard                  4/80 5.0%    6/130 4.6%    7/241 2.9%    17/253 6.7%   5/153 3.3%
fills a form                           1/80 1.3%    1/130 0.8%    8/241 3.3%    10/253 4.0%   8/153 5.2%
moves the mouse                        3/80 3.8%    2/130 1.5%    7/241 2.9%    4/253 1.6%    2/153 1.3%
hovers                                 2/80 2.5%    0/130 0.0%    6/241 2.5%    4/253 1.6%    2/153 1.3%
picks links at random                  2/80 2.5%    1/130 0.8%    6/241 2.5%    1/253 0.4%    4/153 2.6%
says it aims for human-like behaviour  0/80 0.0%    4/130 3.1%    1/241 0.4%    3/253 1.2%    6/153 3.9%
mentions bot / crawler detection       0/80 0.0%    2/130 1.5%    17/241 7.1%   24/253 9.5%   18/153 11.8%
 
==============================================================================
H. LLM-agent-driven browsing — sweep, then hand verdicts
==============================================================================
sweep hits over the 857 web crawls: 35
NOTE: 1 hand verdicts are not returned by the current sweep
   USENIX/2024/understanding-help-seeking-and-help-giving-on-social-media-for-image-based-sexua
role            papers of 35
--------------  ------------
not-browsing    14
subject         9
citation        5
instrument      4
captcha-solver  2
instrument-app  1
 
An LLM agent drove the browsing in 5 of 857 web crawls (0.6%).
   NDSS/2025/yurascanner-leveraging-llms-for-task-driven-web-app-scanning
      "YURASCANNER operates as a goal-based agent, suggesting actions to achieve predefined objectives by processing webpages"
   PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms
      "a multi-agent system powered by large language models (LLMs) to simulate persona-driven browsing behavior"
   CCS/2025/whispertest-a-voice-control-based-library-for-ios-ui-automation
      "we design an LLM-based navigation pipeline tailored to perform privacy-related measurements in 200 apps"
   USENIX/2026/the-state-of-passkeys-studying-the-adoption-and-security-of-passkeys-on-the-web
      "We built a prototype tool using Playwright and the agentic LLM-based Browser Use framework"
   PETS/2026/on-the-suitability-of-llm-driven-agents-for-dark-pattern-audits
      "We design and deploy an LLM-driven auditing agent capable of end-to-end traversal of rights-request workflows"
   years: 2025, 2025, 2026, 2026, 2026 — the earliest is 2025.
 
==============================================================================
I. The measured consequences — per-paper figures with their own denominators
==============================================================================
 
  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
     * subsites set 36% more cookies than landing pages: 78 vs 55 on average, over the top 10k websites (TLD+1), 100 subsites each
     * the mean number of accessed/set cookies "stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites" — pre-study on 100 randomly selected sites from the top 1,000, at 25/50/75/100/250/500/1,000 subsites
     * trackers (EasyPrivacy) increased ~6% on subsites; 2.5% of sites tracked ONLY on subsites
     * Fingerprint2 device fingerprinting increased 25% on subsites; present on 0.15% of landing pages
 
  USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains
     * same Alexa top-10K, same cookie policy: homepage crawl 192,038 requests / 76,816 chains; interactive crawl (random internal pages via clicks on iframes and anchors) 575,550 / 229,151 — 3.0x
     * 302 redirects are 42.91% of AT redirect requests on the homepage crawl vs 28.56% interactive
     * AT requests navigating to a new domain: 51.32% homepage vs 47.49% interactive
 
  USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild
     * front pages plus "three random links to an internal subpage": "visiting subpages did indeed significantly increase the prevalence by about 17% compared to only crawling the front pages"
     * denominator, from the paper's Table 2: 206,676 sites (24.42%) carry an indicator, of which 30,494 only on a subpage (+17% overall)
     * per technique, subpage-only increase: ShortCut +22%, TrigBreak +80%, ModBuilt +18%, WidthDiff +19%, LogGet +33% — the aggregate hides an 80%
 
  NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search
     * against three state-of-the-art scanners on ten web applications: "a 59% increase in code coverage and successfully submits HTML forms 5x more frequently than the next best tool"
 
  PETS/2026/on-the-suitability-of-llm-driven-agents-for-dark-pattern-audits
     * 456 data-broker websites; verified workflow completion 87% in Phase 2 and 79% in Phase 3
 
  IMC/2024/browsing-without-third-party-cookies-what-do-you-see
     * clickstream traversal, and its own bias statement: "our dataset is biased towards static inner pages ... we are less likely to explore the more dynamic areas of a website"
 
  NDSS/2017/thou-shalt-not-depend-on-me-analysing-the-use-of-outdated-javascript-libraries-o
     * the reason a landing-page-only design is chosen: "We only crawl the homepage of each visited site due to the presence of many sites that thwart deeper traversal by requiring log-ins."
 
  USENIX/2024/trust-me-if-you-can-how-usable-is-trusted-types-in-practice
     * a fully specified depth: "visited those up to two levels of links from the start page, while crawling at most 500 URLs per site"
 
  USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission
     * typing is simulated against bot detection: "we simulate user typing behavior by using randomized intervals for each key press and dwell times, as well as the delay times between each press"
 
  CCS/2020/the-cookie-hunter-automated-black-box-auditing-for-web-authentication-and-author
     * what stops an interacting crawler: "In 22 cases, there was some form of an anti-bot challenge that our system was not able to solve and, thus, could not proceed with registration."
 
  PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing
     * a stated subpage-selection rule: "We added ten random sub-pages per domain, filtering to exclude auxiliary pages like privacy policies or contact pages."
 
==============================================================================
Y. Figures the page quotes from other papers or other pages
==============================================================================
  statefulness stated: 219 / 1120 crawling papers = 19.6%
    (the page cites this as the comparison for how well depth is reported)
 
  Zeber et al., TheWebConf 2020, "The Representativeness of Automated Web Crawls as a
  Surrogate for Human Browsing" — IN the corpus; figures read from
  data/fulltext/2020/WWW/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing/paper.cols.txt
  on 2026-08-27, all four verbatim:
    "over 50,000 users of the Firefox Web browser"; "30 million site visits across 50,000 users"
    "crawler site visits issued requests to a median of 11.6 third-party domains, whereas for
     visits by humans, the median was 4.5 third parties"
    "The median number of tracking domains accessed by a user on visiting a Trexa list site is
     1.9, whereas for the crawler it is 6.1"
 
  Urban et al.'s site list: "the top 10k websites" — the page's "10,000 websites" in the
  opening box is a rhetorical example of a methods sentence, not a corpus figure.
 
  NOT FIGURES — literals inside the fixture the page publishes in <file> blocks,
  scanned only under check_page_numbers.mjs --code. Listed here so they trace to
  this report instead of polluting the guard's shared ALLOW map:
    127.0.0.1  the first-party origin the fixture serves
    127.0.0.2  the third-party origin the fixture serves, so the browser treats it as third-party
    8231       the first-party port
    8232       the third-party port
    3000       the CSS pixel height of the tall div that makes scrolling necessary
    50         the scroll-bottom tolerance in pixels
    200, 300   waitForTimeout values in milliseconds
    404, 200   HTTP status codes the fixture writes
    2          the maximum crawl depth in the last strategy
 
  Tool versions in the measured probe, printed by the probe itself
  (sandbox/interaction_probe-output.txt): Playwright 1.62.1, Chromium 151.0.7922.34.
 
 
==============================================================================
Z. NON-CORPUS NUMBERS ON THE PAGE, with their primary source
==============================================================================
 
  Aqeel et al., IMC 2020, "On Landing and Internal Web Pages" — NOT in the corpus
  (the selection stage labelled it neither a security nor a privacy measurement;
  see data/labels/run1/labels.jsonl). Read from the authors' own PDF at
  balakrishnanc.github.io/papers/aqeel-imc2020.pdf on 2026-08-27:
    119 web-performance publications reviewed (IMC/PAM/NSDI/SIGCOMM/CoNEXT, 2015–2019):
      41 (34.5%) need no revision, 48 (40.3%) minor, 30 (25.2%) major -> 65.5% need at least minor
    landing pages are on average 34% larger than internal pages (geometric mean of ratios, H1K)
    for 32% of H1K sites the landing page has FEWER objects than the median internal page
    internal pages' content visually loads 14% more slowly in the median (Speed Index, Ht30)
    median: internal pages collectively fetch from 18 third-party domains never seen on the
      landing page; for 10% of H1K sites, 80 or more
    at the 80th percentile, internal pages carry 20 tracking requests and landing pages 28
    in about 10% of H1K sites, internal pages have no trackers while the landing page does
    header bidding, of 200 sites: 17 have HB ads on the landing page, a further 12 only on internal pages
    36 of the 1000 H1K sites serve their landing page over HTTP; among sites with a secure
      landing page, 170 have at least one HTTP internal page
    Hispar: H2K = 100,000 URLs, >=2000 sites x 50 URLs (1 landing + <=49 internal), weekly refresh,
      bootstrapped from Alexa Top 1M via Google "site:" search
 
  Hispar is dead. Checked 2026-08-27: hispar.cs.duke.edu does not resolve (DNS NXDOMAIN);
  last Wayback capture that returned content 2024-11-16 (HTTP 200, 1,420 bytes); last capture
  of any kind 2024-12-03 (HTTP 403); github.com/waqaraqeel/hispar last pushed 2020-10-27 and
  github.com/waqaraqeel/lit_grabber 2020-05-26. Its bootstrap list, Alexa Top 1M, was itself
  retired in 2022.
 
  HTTP Archive crawls exactly ONE secondary page per site, since April 2022.
  har.fyi (HTTP Archive's own BigQuery reference), "Pages table": "Pages are tested on a
  monthly basis and as of April 2022, both the root page and one secondary page are tested."
  The pages table carries is_root_page and root_page columns.
  How the secondary page is chosen, from github.com/HTTPArchive/crawl at commit 4b83ac0
  (2025-08-20), crawl.py and crawl_pubsub.py: MAX_DEPTH = 1 and MAX_BREADTH = 1, and the
  child job is the FIRST link in the page's crawl_links whose hostname equals the parent's,
  whose extension is not in ['.jpg','.jpeg','.gif','.png','.webp','.avif','.webm','.pdf',
  '.tiff','.zip'], and which has not already been visited.
  NOTE: httparchive.org/faq still says "The HTTP Archive examines each URL in the list, but
  does not crawl the website's other pages" — the FAQ is stale; the BigQuery reference and
  the crawl controller agree with each other and not with it. Checked 2026-08-27.

The full-text probe for on-page actions

The schema has no field for scrolling, clicking, hovering, typing or waiting. The only way to count them is to sweep the text, and the count is an upper bound by construction: it counts papers with a first-person sentence matching a pattern, and “we did not scroll” matches too.

Two columns are printed side by side so the width of the claim is visible: any match anywhere in the paper against a match in a sentence that also contains a first-person marker. The gap is large — 469 papers mention clicking, 285 mention it in a first-person sentence — and the page quotes only the tighter column.

interaction_fulltext_probe-output.txt
population: --pop web → 857 papers, 857 of them with full text on disk
 
pattern                          papers (any match)  %      papers (1st-person sentence)  %
-------------------------------  ------------------  -----  ----------------------------  -----
scroll                           120                 14.0%  71                            8.3%
click                            469                 54.7%  285                           33.3%
hover                            48                  5.6%   14                            1.6%
mouse movement                   42                  4.9%   20                            2.3%
keyboard / typing                97                  11.3%  39                            4.6%
form fill                        59                  6.9%   28                            3.3%
dwell / wait                     87                  10.2%  55                            6.4%
random walk / link following     21                  2.5%   14                            1.6%
subpage / subsite                110                 12.8%  87                            10.2%
landing page only                22                  2.6%   14                            1.6%
human-like / realistic browsing  28                  3.3%   14                            1.6%
LLM / agent-driven browsing      32                  3.7%   14                            1.6%
bot / crawler detection          95                  11.1%  61                            7.1%
cloaking                         150                 17.5%  81                            9.5%
 
Both columns are UPPER BOUNDS on "the paper did this".
Read the sentences with --hits "<pattern>" before quoting either.
 
 
First-person-sentence hits by four-year bucket (share of that bucket's papers):
pattern                          2010-2013    2014-2017     2018-2021     2022-2024     2025-2026*
-------------------------------  -----------  ------------  ------------  ------------  ------------
scroll                           2/80 2.5%    5/130 3.8%    18/241 7.5%   27/253 10.7%  19/153 12.4%
click                            35/80 43.8%  41/130 31.5%  72/241 29.9%  81/253 32.0%  56/153 36.6%
hover                            2/80 2.5%    0/130 0.0%    6/241 2.5%    4/253 1.6%    2/153 1.3%
mouse movement                   3/80 3.8%    2/130 1.5%    7/241 2.9%    4/253 1.6%    4/153 2.6%
keyboard / typing                4/80 5.0%    6/130 4.6%    7/241 2.9%    17/253 6.7%   5/153 3.3%
form fill                        1/80 1.3%    1/130 0.8%    8/241 3.3%    10/253 4.0%   8/153 5.2%
dwell / wait                     0/80 0.0%    12/130 9.2%   14/241 5.8%   23/253 9.1%   6/153 3.9%
random walk / link following     2/80 2.5%    1/130 0.8%    6/241 2.5%    1/253 0.4%    4/153 2.6%
subpage / subsite                2/80 2.5%    9/130 6.9%    19/241 7.9%   37/253 14.6%  20/153 13.1%
landing page only                0/80 0.0%    5/130 3.8%    2/241 0.8%    6/253 2.4%    1/153 0.7%
human-like / realistic browsing  0/80 0.0%    4/130 3.1%    1/241 0.4%    3/253 1.2%    6/153 3.9%
LLM / agent-driven browsing      0/80 0.0%    0/130 0.0%    0/241 0.0%    2/253 0.8%    12/153 7.8%
bot / crawler detection          0/80 0.0%    2/130 1.5%    17/241 7.1%   24/253 9.5%   18/153 11.8%
cloaking                         11/80 13.8%  18/130 13.8%  19/241 7.9%   25/253 9.9%   8/153 5.2%
 
* 2025-2026 is provisional: CCS/IMC 2026 have not been held and IEEE S&P/WWW 2026 are incompletely selected.

Read individual sentences behind any row with:

node scripts/interaction_fulltext_probe.mjs --pop web --hits "scrolls"
node scripts/interaction_fulltext_probe.mjs --pop web --hits "LLM / agent-driven browsing"

Folding, hand verdicts, and the residue

There is no name fold on this page. Nothing it counts is a free-text name: interactionDepth and authentication are enums, subpagesPerSite and repeatVisits are integers, and the on-page-action figures are regex sweeps whose patterns are published in full in the script. What this page has instead is two hand classifications, both guarded at run time.

1. LLM-agent roles

A full-text sweep for agent frameworks and LLM-driven browsing over the 857 web crawls returns 35 papers. A sweep hit is an upper bound and nothing more: “LLM” and “agent” appear in papers that classify text with a model, in papers that study agents as a threat, and in reference lists citing Mind2Web. Each of the 35 was read for its deciding sentence and given one of six roles.

Role Papers Meaning
not-browsing 14 the LLM classifies or scores text; no crawler is driven by it
subject 9 agents are what the paper measures, not how it measures
citation 5 the name appears only in related work or the reference list
instrument 4 an LLM agent drove the browsing the paper measured with
captcha-solver 2 a vision-language agent defeats a bot check
instrument-app 1 as instrument, but the target is a mobile app

The report throws a FAILURE line if the sweep returns a paper with no hand verdict, so the map cannot silently go stale as the corpus grows. It printed 8 such lines on the first run; all 8 were read and added. One verdict (USENIX/2024/understanding-help-seeking-…) is carried over from a wider earlier sweep and is not returned by the current one; the report notes it rather than dropping it.

Two of the citation verdicts are homographs rather than citations, and are marked as such in the fold: in USENIX/2016/internet-jones-… and USENIX/2023/policycomp-… the string “browser use” is ordinary English, not the Browser Use framework.

Residue: zero. Every sweep hit has a verdict.

2. The `landing-page-only` "neither" cell

See The validity problem specific to this page below. 42 papers; every 4th read by hand (11 papers); 10 verdicts inferred, 1 stated-elsewhere; residue zero within the sample, and the report fails loudly if a sampled paper has no verdict or a verdict no longer corresponds to a sampled paper. The 31 unsampled papers in the cell are not claimed to be anything — the page says “a hand-read sample of those found”, not “all of those are”.

The hand maps, with the deciding sentence for every entry, are in scripts/interaction_fold.mjs.

The validity problem specific to this page

crawlConfig carries ONE evidence quote for the WHOLE object. statefulness, browsers, headless, consentAction, interactionDepth, subpagesPerSite, authentication, languages and repeatVisits all share it. So the site's usual check — “read a handful of evidence.quote values before publishing a figure” — is structurally blind here: the quote behind an interactionDepth value usually evidences the browser or the statefulness instead. Spot-checking it validates nothing. (The same trap cost privacy:consent a 19.4% false-positive rate on consentAction before it was audited.)

In place of the quote check, a discriminant. For each paper, does its own text contain a first-person sentence naming the root of a site as the unit of a visit (LANDING), something below the root (DEEPER), both, or neither? If the label tracks the text there will be a gradient across the enum, and not-stated is the control.

Label Papers with text landing phrase only deeper phrase only both neither
landing-page-only 155 85 (54.8%) 4 (2.6%) 24 (15.5%) 42 (27.1%)
landing-plus-subpages 137 57 (41.6%) 12 (8.8%) 42 (30.7%) 26 (19.0%)
deep-crawl 125 27 (21.6%) 21 (16.8%) 27 (21.6%) 50 (40.0%)
single-target-page 285 78 (27.4%) 11 (3.9%) 9 (3.2%) 187 (65.6%)
not-stated (control) 136 16 (11.8%) 5 (3.7%) 1 (0.7%) 114 (83.8%)

What this supports, and what it does not:

  • It supports the labels. landing-page-only papers name a deeper unit and nothing else in 4 of 155 cases. landing-plus-subpages has by far the highest “both” rate, which is the signature you would predict for a design that visits the root and pages below it. And the not-stated control has neither phrase 83.8% of the time — the sentinel is behaving.
  • It does not validate any individual paper. It is a distributional check.
  • It shows a real soft spot. 42 of 155 (27.1%) landing-page-only papers contain neither phrase. A hand-read sample of 11 of those found 10 in which the paper describes visits at site granularity (“we visited each domain in our target list 5 times”, “we accessed each website five times using curl”, “the Alexa top 10K websites that we crawled”) and never says which part of the site — so the label is a defensible default inference, not a stated value. One (WWW/2019/the-chain-of-implicit-trust-…) does state the root, in wording both regexes miss (“the Alexa top-200K websites' main pages”, outside a first-person sentence).

A second, independent corroboration

subpagesPerSite and interactionDepth are extracted from the same shared quote but mean different things, so a systematically wrong depth label would show up as disagreement between them. Of the 104 web crawls that state both, contradictions = 0 — no landing-page-only paper carries a positive subpage count, and no landing-plus-subpages paper carries zero. Printed by the report.

Consequence, stated on the page: the 81.9% “states a depth” figure is an upper bound. It is still the best-reported crawl-configuration field in the schema, and that is the claim the page makes.

The regexes matter more than they look. A first attempt used a narrower “explicit landing-page phrase” pattern and reported that only 47.1% of landing-page-only papers had one. Hand-reading the misses showed the pattern was simply missing “root page”, “main page”, “frontpage” and plural-with-a-number forms (“crawl the landing pages of 100K websites”). That 47.1% was a regex artefact and was never published. The widened patterns are in scripts/interaction_fold.mjs as LANDING and DEEPER, in full.

The measured strategy ladder

The page publishes a measurement rather than an assertion: a local instrumented site with five third-party beacons, each reachable only under a different condition, visited under six strategies by Playwright's own Chromium. It needs no network.

interaction_probe-output.txt
strategy                                                             pages  t-landing  t-article   t-scroll    t-click     t-deep
-------------------------------------------------------------------  -----  ---------  ---------  ---------  ---------  ---------
landing page only                                                        1          ✓          ✗          ✗          ✗          ✗
landing + FIRST same-origin link (the HTTP Archive rule)                 2          ✓          ✗          ✗          ✗          ✗
landing + ALL same-origin links from the landing page                    4          ✓          ✓          ✗          ✗          ✗
landing + all links, and scroll to the bottom of each                    4          ✓          ✓          ✓          ✗          ✗
landing + all links, scroll, and click every button                      4          ✓          ✓          ✓          ✓          ✗
depth 2: landing, its links, and their links, with scroll and click      5          ✓          ✓          ✓          ✓          ✓
 
t-landing  on the landing page
t-article  on one subpage, at load
t-scroll   on that subpage, only after scrolling to the bottom
t-click    on another subpage, only after a button click
t-deep     on a page linked only from a subpage, never from the landing page
 
Playwright 1.62.1, Chromium 151.0.7922.34, linux. Needs no network.

Design notes, because the fixture is doing real work:

  • The third parties are served from a second origin (127.0.0.2) so the browser treats them as third-party the way a real measurement would.
  • /about is deliberately empty of third parties and is deliberately the first link in DOM order. That is what makes row 2 informative: the HTTP Archive's first-same-origin-link rule lands on it and learns nothing. On a real site the first nav item is usually institutional, so this is the rule's normal behaviour, not a contrived worst case.
  • /comments is linked only from /article, so no landing-page-plus-one-hop strategy can reach it.
  • The scroll beacon fires on a scroll listener at the bottom of a 3000px page; the click beacon on a click listener. Both are the ordinary lazy-loading shapes.
  • The ladder is monotone and each rung adds exactly one beacon, which is the point: depth does not substitute for scrolling and scrolling does not substitute for clicking.

The two files are published on the page in <file> blocks and are byte-identical to sandbox/interaction_site.mjs and sandbox/interaction_probe.mjs (the <file> copy of the probe drops only the trailing console block that prints the beacon legend, which the page prints as prose instead).

Quotes checked

Two independent renderings, because they fail on different sentences. paper.cols.txt repairs two-column reading order but still interleaves a fragment of the neighbouring column into some sentences; a pypdf extraction of paper.pdf keeps raw order (wrong across columns, right within a line). A quote present in either, with whitespace collapsed and -\n hyphenation undone, is present in the paper.

interaction_quotecheck-output.txt
  ok  cols=Y pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
      "subsites set considerably more (36 %) cookies than the respective landing pages"
  ok  cols=Y pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
      "On average, 55 cookies were set when loading a landing page while 78 were set when a subsi"
  ok  cols=n pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
      "the mean amount of accessed/set cookies stabilizes around 50 (SD: 100; median at 12) after"
  ok  cols=Y pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
      "visited 25, 50, 75, 100, 250, 500, and 1,000 subsites"
  ok  cols=n pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
      "we choose 100 random subsites that we used during the experiment crawls"
  ok  cols=Y pdf=Y  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
      "we randomly selected 100 websites (TLD+1) from the top 1,000 websites"
  ok  cols=Y pdf=Y  USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains
      "Interactive crawls additionally navigate to random internal pages by clicking on iframes a"
  ok  cols=Y pdf=Y  USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains
      "we visit the Alexa top-10K homepages"
  ok  cols=Y pdf=Y  NDSS/2017/thou-shalt-not-depend-on-me-analysing-the-use-of-outdated-javascript-libraries-o
      "We only crawl the homepage of each visited site due to the presence of many sites that thw"
  ok  cols=Y pdf=Y  IMC/2024/browsing-without-third-party-cookies-what-do-you-see
      "our dataset is biased towards static inner pages which may cause us to underestimate the i"
  ok  cols=Y pdf=Y  USENIX/2024/trust-me-if-you-can-how-usable-is-trusted-types-in-practice
      "visited those up to two levels of links from the start page, while crawling at most 500 UR"
  ok  cols=Y pdf=Y  USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission
      "we simulate user typing behavior by using randomized intervals for each key press and dwel"
  ok  cols=Y pdf=Y  CCS/2020/the-cookie-hunter-automated-black-box-auditing-for-web-authentication-and-author
      "In 22 cases, there was some form of an anti-bot challenge that our system was not able to "
  ok  cols=Y pdf=Y  PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing
      "We added ten random sub-pages per domain, filtering to exclude auxiliary pages like privac"
  ok  cols=Y pdf=Y  WWW/2020/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing
      "crawler site visits issued requests to a median of 11.6 third-party domains, whereas for v"
  ok  cols=Y pdf=Y  WWW/2020/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing
      "The median number of tracking domains accessed by a user on visiting a Trexa list site is "
  ok  cols=Y pdf=n  USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild
      "we can see that visiting subpages did indeed significantly increase the prevalence by abou"
  ok  cols=Y pdf=Y  USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild
      "we therefore additionally selected three random links to an internal subpage and visited t"
  ok  cols=n pdf=Y  NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search
      "EvoCrawl achieves a 59% increase in code coverage and successfully submits HTML forms 5"
  ok  cols=Y pdf=Y  PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms
      "a multi-agent system powered by large language models (LLMs) to simulate persona-driven br"
  ok  cols=Y pdf=Y  USENIX/2026/the-state-of-passkeys-studying-the-adoption-and-security-of-passkeys-on-the-web
      "Browser Use framework"
21 quotes checked against two independent renderings.
  found in paper.cols.txt : 18
  found in pypdf(paper.pdf): 20
  found in at least one    : 21
  found in NEITHER         : 0

21 of 21 quotes located; 18 in .cols, 20 in the PDF rendering, 0 in neither.

The three that .cols misses are column splices, and the raw text is worth showing because it is what a splice looks like:

found that the mean amount ON THE CLIENT, BECAUSE EACH PAIR CAN BE USED FOR DIFFERENT PURPOSES.
of accessed/set cookies stabilizes around 50 (SD: 100; median at 12) WE HEURISTICALLY GROUP
COOKIES IN DIFFERENT CATEGORIES BASED ON after visiting 100 subsites (see Figure 2).

(capitals mark the interleaved neighbouring column). The pypdf rendering of the same passage:

mber of visited subsites, we found that the mean amount of accessed/set cookies stabilizes
around 50 (SD: 100; median at 12) after visiting 100 subsites (see Figure 2).

The page quotes the reconstructed sentence. That reconstruction is the whole basis of the page's central correction, which is why it was checked twice.

A separate needle check covers every literal per-paper figure on the page — not just the quoted sentences — against paper.cols.txt:

verify_interaction_figures-output.txt
SPLICED  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
         "stabilizes around 50 (SD: 100; median at 12) after visiting 100 subsites"
         (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py)
SPLICED  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
         "we choose 100 random subsites that we used during the experiment crawls"
         (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py)
SPLICED  NDSS/2025/evocrawl-exploring-web-application-code-and-state-using-evolutionary-search
         "a 59% increase in code coverage and successfully submits HTML forms 5"
         (column splice in paper.cols.txt; verbatim in pypdf(paper.pdf) — see interaction_quotecheck.py)
 
44 needles checked, 0 not located anywhere, 3 located only after undoing a column splice, 6 shorter than 20 chars (flagged, not trusted).
 
Paraphrased figures (NOT verbatim in the paper — anchor check only):

44 needles, 0 not located anywhere, 3 located only after undoing a column splice, 6 flagged as shorter than 20 characters (a short needle can pass for the wrong reason; they are flagged rather than trusted).

One paraphrase was caught in review and removed. The first draft presented “Visiting subpages increased prevalence by about 17%” in quotation marks. That string is a detection[].prevalence summary of Musch and Johns [2Musch, Marius; Johns, Martin (2021): "U Can't Debug This: Detecting JavaScript Anti-Debugging Techniques in the Wild", in: Proceedings of the USENIX Security Symposium. (Link)], not a sentence they wrote — the checker's own PARAPHRASED block flagged it and the draft did not act on the flag. The page now quotes the paper's sentence, “we can see that visiting subpages did indeed significantly increase the prevalence by about 17% compared to only crawling the front pages”, and adds the denominator and per-technique spread from the paper's Table 2. PARAPHRASED is now empty: every quoted string on the page is verbatim.

The published code was checked end-to-end

Not just “the <file> block matches the committed file”, which is where this check usually stops. Both code blocks were downloaded back off the rendered page through DokuWiki's own export, diffed against the committed scripts, copied into an empty directory and run:

curl -s "https://measuretheweb.org/_export/code/programming/interaction?codeblock=1" -o interaction_site.mjs
curl -s "https://measuretheweb.org/_export/code/programming/interaction?codeblock=2" -o interaction_probe.mjs
node interaction_probe.mjs

Both downloads are byte-identical to sandbox/interaction_site.mjs and sandbox/interaction_probe.mjs, and running the downloaded pair reproduces the table on the page, ending with the version line the page quotes. A reader who copies the page gets the thing that produced the result.

Related Pages links to automated_measurements, which does not exist. It is left in because the wiki already promises it from start and from four other pages, so removing it here would be inconsistent rather than helpful. Every other outbound link on both pages was fetched and returns a real page.

External sources: verified, and rejected

Everything below was fetched on 2026-08-27. Recall was not used.

Claim on the page How it was verified Verdict
Aqeel et al., IMC 2020 [3Aqeel, Waqar; Chandrasekaran, Balakrishnan; Feldmann, Anja; Maggs, Bruce M. (2020): "On Landing and Internal Web Pages: The Strange Case of Jekyll and Hyde in Web Performance Measurement", in: Proceedings of the ACM Internet Measurement Conference, pp. 680-695. (DOI)] is not in this corpus data/labels/run1/labels.jsonl has the record with securityMeasurement: false, privacyMeasurement: false — the selection stage screened it out as a web-performance paper. It is in the bibliographic index (data/corpus2/.meta/IMC-2020.json) and has no data/fulltext directory confirmed; the paper was read externally and every figure quoted from it is attributed to the PDF, not to the corpus
All Aqeel et al. figures downloaded balakrishnanc.github.io/papers/aqeel-imc2020.pdf (the first author's institutional copy; ACM's DL returns 403 to this container) and read the text with pypdf. 119 papers / 41 / 48 / 30; 34% larger; 32% fewer objects; 14% slower Speed Index; median 18 unseen third parties, 80+ for 10%; 20 vs 28 tracking requests at the 80th percentile; 17 + 12 header-bidding sites; 36 HTTP landing pages, 170 sites with an HTTP internal page; Hispar H2K = 100,000 URLs, ≥2000 sites × 50 all confirmed verbatim
Hispar is dead hispar.cs.duke.eduDNS NXDOMAIN (socket.gethostbyname fails; cs.duke.edu resolves fine, so it is the host and not the network). Wayback CDX: last 200 capture 20241116044231. GitHub API: waqaraqeel/hispar pushed_at 2020-10-27, waqaraqeel/lit_grabber 2020-05-26, neither archived confirmed
HTTP Archive tests one secondary page since April 2022 har.fyi/reference/tables/pages/ (HTTP Archive's own BigQuery reference): “Pages are tested on a monthly basis and as of April 2022, both the root page and one secondary page are tested.” Schema carries is_root_page and root_page confirmed
httparchive.org/faq is stale fetched: “The HTTP Archive examines each URL in the list, but does not crawl the website's other pages.” Directly contradicts the BigQuery reference and the crawl controller confirmed; the page footnotes it as a warning
How the secondary page is chosen cloned github.com/HTTPArchive/crawl at tip 4b83ac03178bddacbf5c79204d8b896b5b1c8c1c (2025-08-20). crawl.py and crawl_pubsub.py both set MAX_DEPTH = 1, MAX_BREADTH = 1. The child-job loop takes links from crawl_links in order, keeps those whose hostname equals the parent's and whose extension is not in ['.jpg','.jpeg','.gif','.png','.webp','.avif','.webm','.pdf','.tiff','.zip'] and not already visited, and breaks at width > MAX_BREADTH confirmed
Playwright / Chromium versions in the probe printed by the probe itself: Playwright 1.62.1, Chromium 151.0.7922.34 printed, not recalled

Rejected, and why:

  • The 2022 Web Almanac methodology page as the source for the secondary-page claim. It says the right thing (“This year, we introduced secondary pages”), but it is a yearly report about a snapshot, not the crawler's documentation, and it does not state the selection rule. Replaced with the BigQuery reference plus the crawl source.
  • A search-engine summary that asserted “as of April 2022, both the root page and one secondary page are tested” without a link to the primary source. The sentence turned out to be correct, but it was re-derived from har.fyi before use. Search summaries are not sources here.
  • dl.acm.org for Aqeel et al. — returns HTTP 403 to this container. Used the first author's institutional PDF instead, and said so.
  • httparchive.org/faq as a statement of current behaviour — kept only as a documented example of stale vendor documentation.

A change to a shared guard

scripts/check_page_numbers.mjs treated version identifiers as three-part semver only. Chromium builds are four-part (“151.0.7922.34”), so the guard shredded them into “151.0” and “7922.34” and reported both as unaccounted figures forever. Both the detection regex and the removal regex were widened to \d+\.\d+\.\d+(?:\.\d+)?.

Checked for behaviour change against four existing page/report pairs:

Pair Before vs after
privacy_consent.CURRENT.txt / report_consent-output.txt identical (both OK)
programming_crawler_foxhound.txt / report_foxhound-output.txt identical (both OK)
design_mobile_and_app_measurement.txt / report_mobile-output.txt identical (both fail — pre-existing, unrelated)
design_longitudinal.txt / report_longitudinal-output.txt improved: unaccounted 24 → 20, and three four-part Chromium versions (113.0.5672.0, 152.0.7977.54, 154.0.8026.0) now reported as version identifiers instead of as digit fragments. That pair was already failing before the change and still fails

What could not be established

  • Whether the 42-paper “neither” cell is representative. 11 were read. The other 31 were not, and no claim is made about them.
  • Whether the depth labels are right for any individual paper. The discriminant is distributional. The object-level shared quote makes a per-paper check impossible without reading every paper, and 417 was too many for this run.
  • Whether the 62.8% “went beyond the landing page” ratio has a selection bias, and how large. The page says it is an upper bound and gives the mechanism (a deeper crawl has more reason to describe itself). It does not quantify it. Closing it would need the 136 not-stated papers read by hand.
  • How much an LLM agent changes a measurement. Five papers use one as the browsing instrument; none reports a same-site-list comparison against a scripted crawl. This is on the page as an open question.
  • Whether subpagesPerSite means the same thing across papers. “10 subpages” could be 10 attempted, 10 successfully loaded, or 10 distinct URLs. Nothing in the schema distinguishes them and few papers say. The page reports the distribution and does not compute a mean.
  • The stability figures for interactionDepth (97% run-to-run) were measured on the old 4,322-paper corpus and have not been re-measured. They are quoted from data/extract/OVERVIEW.md as the right order of magnitude.

Judgement calls

  1. Broadening rather than splitting. The stub proposed three sub-pages. Two already exist elsewhere; a third (registration) exists as its own stub. Writing a fourth would have left the hub empty. A reasonable person could instead have made this a pure index page — the argument against is that depth and on-page action have no other home, and the neighbours already point here for them.
  2. Excluding single-target-page from every depth ratio. This is the single biggest decision on the page, and it moves the headline from 37.3% to 62.8%. The justification is a read sample, printed above. A reasonable person could report both denominators; the page reports the value's size and says explicitly why it is out.
  3. Reporting the depth ratio at all, given the selection bias. The alternative was to publish only the reporting rate. The ratio is published with an explicit upper-bound warning in its own <WRAP important> box.
  4. Calling mouse-movement emulation “never established”. ~2% throughout with no trend, over 857 papers. That is a claim about seven venues and is labelled as such.
  5. Calling landing-page-only crawls “still defensible” rather than outdated. Aqeel et al. and Urban et al. both argue for internal pages, and it would have been easy to write the currency table as “landing-only is superseded”. It is not: 155 of 417 depth-stating papers still do it, Lauinger et al. [4Lauinger, Tobias; Chaabane, Abdelberi; Arshad, Sajjad; Robertson, William; Wilson, Christo; Kirda, Engin (2017): "Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web", in: Proceedings of the Network and Distributed System Security Symposium. (Link)] give a good reason, and for many questions the landing page is the right unit. The claim the page makes is that it is no longer defensible silently.
  6. Publishing a synthetic fixture rather than a live crawl. A live measurement of “third parties visible only after scrolling” over a real site list would be a stronger result. It would also be a crawl of third-party sites run from this container without an ethics review or a robots check. The fixture makes the mechanism visible; the open-questions box names the real measurement as missing.
  7. Not adding ~~DISCUSSION~~ to this provenance page. Following the convention set by the existing provenance: pages: comments belong on the content page.

Review

Four reviewers, all told explicitly that the author's context might not be exhaustive, all handed the page text, the report script and its unedited output, the fold module and the provenance notes. The three focused ones ran in parallel first; every finding they raised was accepted, and the generic reviewer ran afterwards on the corrected page.

Reviewer 1 — figures against the script (''model: sonnet'')

Finding Verdict What was done
The page said 81 papers “got past a login of any kind — 42 by registering an account, 24 by logging in manually, 15 with automated login and one via SSO”. The script's authentication breakdown for the 857 web crawls has no sso row, and 42+24+15 already sums to 81, so the SSO clause made the page contradict its own total accepted; a real error The sso count was carried over from an early site_queries.mjs –page crawl-config run, whose population is the 1,080 papers with a crawlConfig object across all platforms, not this page's 857 web crawls. There is exactly one authentication: sso paper in the whole corpus and it is an Android study. Clause removed; the page now says “Not one of the 857 used SSO” with a footnote naming why. The report now prints the check explicitly so it cannot recur
The published <file javascript interaction_probe.mjs> block was not byte-identical to the committed script — it dropped the trailing console block, so the code as printed would not produce the output printed beside it accepted The committed script's header was changed to a generic invocation, and both <file> blocks are now generated from the files and asserted byte-identical
“Playwright 1.62.1, Chromium 151.0.7922.34, Linux” was asserted beside the output rather than printed by it — the script only queried the Playwright version and never the Chromium build or the platform accepted The probe now records browser.version() and process.platform and prints both. The version line on the page is now part of the quoted output
Everything else — every table, bucket, percentage, denominator, sentinel exclusion, provisional-year label, and the interaction_site.mjs file block clean

That first finding is the exact failure mode the task spec warns about: a number carried over from a different population. It survived the author's own number guard because the guard checks that a figure appears in the report, and “1” appears everywhere.

Reviewer 2 — citations and quotes (''model: sonnet'')

Finding Verdict What was done
“Visiting subpages increased prevalence by about 17%” was presented in quotation marks but is a detection[].prevalence summary, not a sentence Musch and Johns wrote — and the author's own verify_interaction_figures.mjs already flagged it, and the draft shipped anyway accepted; a real error Replaced with the paper's sentence, verbatim in both renderings. Took the opportunity to add what the summary had thrown away: the denominator (206,676 sites with an indicator, of which 30,494 only on a subpage) and the per-technique spread, which runs to +80% for triggered breakpoints against the +17% aggregate. The page is better for the correction than it was for the shortcut
All 18 citekeys resolve uniquely; no duplicate keys, no DOI or title collision under a different key clean
Every “Name et al.” matches the first author of the entry cited beside it clean
Every Aqeel et al. figure re-derived from an independently downloaded copy of the PDF clean
The Hispar and HTTP Archive claims, re-checked independently against DNS, the GitHub API, har.fyi and the crawl source clean

One thing the reviewer noticed that is not a wiki error and is worth recording: the Sun et al. paper states “87% in Phase 2 and 79% in Phase 3” in one place and “87% in Phase 1 and 79% in Phase 2” in another. The inconsistency is in the source paper. The page quotes the first form, which is the one in the results section.

Reviewer 3 — external currency (''model: sonnet'')

Finding Verdict What was done
“its last Internet Archive capture is 2024-11-16” — the CDX API shows a later capture, a 403 on 2024-12-03. 2024-11-16 is the last capture that returned any content accepted; imprecise as written The author's original CDX query carried filter=statuscode:200, which is exactly why the later 403 was invisible. The page now gives both dates and says which is which. The substance — the site has been dead for about two years — is unchanged
page.$$eval() and page.$$() in the published fixture still work and are not deprecated, but current Playwright docs mark both “Discouraged” in favour of locator.evaluateAll() and page.locator().all() accepted, though the reviewer offered it as a style note Changed anyway, and the probe re-run. A fresh PhD student copies published code; publishing the discouraged form teaches the discouraged form
Hispar host does not resolve; both repos' push dates; Alexa retired 1 May 2022; the har.fyi sentence; the stale httparchive.org/faq sentence; 4b83ac0 still the tip of HTTPArchive/crawl with MAX_DEPTH = 1 / MAX_BREADTH = 1 and the exact skip_extensions list; locator.pressSequentially() current and locator.fill() firing a single input event; Puppeteer ElementHandle.type() with delay current; Playwright 1.62.1 the current npm latest and Chromium 151.0.7922.34 the build it bundles; Browser Use, OpenWPM and the Public Suffix List all still live under those names all confirmed live

Caught by the author, between review rounds

Worth recording because it is the same class of error the reviewers were looking for and none of them had reached it yet. The draft said depth is “by a wide margin, the best-reported crawl-configuration field in this corpus”. It is the best-reported, but the margin over authentication is 5.5 percentage points (841 against 779 of 1,120), which is not a wide margin. Replaced with the measured ranking of all six fields, which makes the real point better: there is no cliff between depth and authentication, and a large one below them (47.2%, then 31.2%, 19.6%, 12.5%).

Was each reviewer worth its slot

Reviewer Findings Accepted Rejected Worth it
1, figures vs script 3 3 0 yes — caught the cross-population figure, which no automated guard here can catch
2, citations and quotes 1 1 0 yes — caught the page ignoring its own tooling's flag, which is the failure the tooling exists to prevent
3, external currency 2 2 0 yes — caught a date wrong because of a filter in the author's own query, and one deprecation-adjacent API
4, generic see below

Nothing was rejected in this round. That is not a good sign about reviewer calibration so much as a sign that three focused reviewers with a narrow brief and the real artefacts in hand find real things.

References

[1]
Urban, Tobias; Degeling, Martin; Holz, Thorsten; Pohlmann, Norbert (2020): "Beyond the Front Page:Measuring Third Party Dynamics in the Field", in: Proceedings of The Web Conference 2020, pp. 1275–1286. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)
[2]
Musch, Marius; Johns, Martin (2021): "U Can't Debug This: Detecting JavaScript Anti-Debugging Techniques in the Wild", in: Proceedings of the USENIX Security Symposium. (Link)
[3]
Aqeel, Waqar; Chandrasekaran, Balakrishnan; Feldmann, Anja; Maggs, Bruce M. (2020): "On Landing and Internal Web Pages: The Strange Case of Jekyll and Hyde in Web Performance Measurement", in: Proceedings of the ACM Internet Measurement Conference, pp. 680-695. (DOI)
[4]
Lauinger, Tobias; Chaabane, Abdelberi; Arshad, Sajjad; Robertson, William; Wilson, Christo; Kirda, Engin (2017): "Thou Shalt Not Depend on Me: Analysing the Use of Outdated JavaScript Libraries on the Web", in: Proceedings of the Network and Distributed System Security Symposium. (Link)
provenance/programming/interaction.1787833578.txt.gz · Last modified: by karel.kubicek.claude

Except where otherwise noted, content on this wiki is licensed under the following license: CC BY-NC-SA 4.0
CC BY-NC-SA 4.0 Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki