Working notes behind longitudinal — every query with its population and denominator, the report script and its unedited output, the fold and its full residue, the quotes that were checked, the probes that were hand-audited, the external sources that were verified or rejected, and what could not be established. Corpus-level caveats that apply to every page on this site are on corpus and are not restated here.

Contemporaneous. Written during the run that produced the content page, 2026-08-27, not reconstructed afterwards.

Four existing pages own a piece of this subject, and the judgement was that none of them owns the whole:

The gap none of them fills is the one between “I know how to draw a sample” and “wave two is comparable to wave one”: what has to be pinned, what cannot be pinned however hard you try, and what the literature actually reports. That is the page. Every overlapping claim links out rather than restating — the four papers whose variation figures appear in the noise-floor table are all already cited on statistics:biases, and the EasyList-anachronism figure is already on design:archives; both are re-used with a pointer because the longitudinal reading of them is different from the bias reading.

One consequence accepted deliberately: the content page's noise-floor table duplicates rows that appear on statistics:biases. A reader arriving from a search engine at either page should not have to visit the other to learn the number, and the framing differs (there: “your sample is not the population”; here: “this is your wave-to-wave error bar”).

Every figure on the content page is a share of one of these. crawled uses the same definition as data/extract/OVERVIEW.md so the numbers line up with the rest of the site.

The repeated rule is deliberately identical to the one biases uses, so the two pages agree on 250 rather than publishing two different counts of the same thing.

Sentinels. not-stated, not-applicable, none-mentioned, unknown are never counted as a stated value. This matters most for two rows: counting not-applicable as an answer would move consent action stated from 105 to 148 on the repeated population and statefulness stated from 149 to 166 on the single-shot one, which would have put this page out of step with OVERVIEW.md (219 and 349 over crawled respectively — reproduced exactly here as 70+149 and 105+244).

“Stated” is not “held fixed.” Every pin figure counts papers that mentioned the thing once. The extraction cannot see whether a paper used the same browser in wave one and wave two. Every pin figure is therefore a ceiling on comparability, and the content page says so in the same table.

cd /workspace/artifacts/wiki
node scripts/report_longitudinal.mjs > scripts/report_longitudinal-output.txt
node scripts/quotecheck_longitudinal.mjs > scripts/quotecheck_longitudinal-output.txt

Both default to /workspace/publications_dataset/data/extract/run1 and …/data/fulltext; –run and –fulltext override.

temporal.cadence is the only free-text field this page aggregates. 334 distinct strings fire on the 1,120 crawling papers. The fold is ordered fastest-interval-first, so “hourly and daily” folds to hourly — the fastest repeat a paper states is the one that bounds its time resolution.

The first rule is the interesting one. A large minority of the values are durations, not cadences: “one week”, “5-day experiment”, “month-long”, “single crawl lasting 12 days”. Those papers have not stated how often they repeated, and folding them into weekly or monthly would have converted a reporting gap into a finding about cadence. They get their own family and the content page reports them as silence.

After folding, 16 distinct strings (16 papers, 4.1% of the 389) remain unmapped. They are printed in full in the report output below, under UNMAPPED RESIDUE, and reproduced here so they are readable without running anything:

four increasing cumulative periods for threshold selection
four APKs per year per browser family
July and December
15-minute trend monitoring
one hour per website
two six-month intervals per year for 2009–2019
longitudinal measurement
eleven-minute time slots
Crawler B ran at the middle of each month
irregular confirmation crawls
collection cycles over six months
during the day, paused overnight
December 2023, May 2024, and December 2024
sequential experimental windows
four-month interval
before and after the second rollout

Most of those are genuinely ambiguous rather than mis-folded: “four-month interval” could be a cadence or a span, and the paper has to be read to tell. They are left unmapped rather than guessed at.

One value in the field is a sentinel. A single paper has cadence: “not-stated”. It is folded to sentinel and excluded from every cadence figure.

The double-counting trap, and how the published figure avoids it. The families are multi-valued: a paper stating both “daily” and “for three weeks” lands in two. Summing the non-cadence families (58 + 29 + 31 + 1 = 119) therefore over-counts. The figure published on the content page instead counts papers whose cadence values are all non-cadences: 88 of 389 (22.6%), with unmapped generously treated as a possible real cadence. Counting unmapped as a non-cadence would give a larger number; the smaller one is published.

Every quotation and every figure the content page takes from a cited paper is in scripts/quotecheck_longitudinal.mjs, whose unedited output is in §14. The checker reads four renderings of each paper: paper.cols.txt (two-column reading order repaired), paper.norm.txt, paper.txt, and paper.cols.txt with end-of-line hyphenation joined.

The verdicts mean:

No quote came back MISSING. Seven came back PARTIAL, every one for the same mechanical reason — a two-column interleave, a hyphen-split word, or both — and every one was read by hand against the source:

A correction the checker forced. An earlier draft of the lead said Demir et al. ran the twelve-day experiment “from the same place”. The paper's C4 paragraph says only “we visit the top 1k sites from our website corpus (18,377 distinct pages) daily (starting at midnight) throughout our experiment (from 07/08/2021 to 07/19/2021)”, and the study as a whole crawls from three geolocations. It does not say the temporal sub-analysis holds the vantage point fixed, so the claim was removed rather than assumed.

A figure the extraction offered and the page does not use. temporal.snapshots has nine papers reporting more than 1,000 snapshots — the largest is 550,000. Spot-reading them shows page counts and record counts mis-extracted into the field (an NDSS 2019 longitudinal browser study, a CCS 2025 robots.txt study). No maximum, mean or median of snapshots is published anywhere; only bands, and the band table carries the warning.

All probes collapse whitespace before matching, because a PDF line break inside a phrase would otherwise hide it. Every probe is a mention count and therefore an upper bound. Two probes carry a load-bearing claim on the content page and were therefore read in full.

The structured field crawlConfig.browsers contains a digit for 17 of 250 repeated-crawl papers (6.8%). That undercounts, because a paper can write “Chrome” in its setup table and “Chrome 88.0.4298.0” in a footnote. So a deliberately wide probe was run for any <browser> <number> string anywhere in the text: 58 hits (23.2%).

All 58 were read. 26 state the version of a browser the study's own measurement drove. The 32 rejected split into:

• bibliography entries — “Firefox 85 Cracks Down on Supercookies”, “About the security content of Safari 11.1”;
• ecosystem history — “Full support for this CSP feature first landed in browsers around March 2015 (with Chrome 43)”;
• the version of a browser being measured rather than driven — the NDSS 2019 mobile-browser study reports “Firefox v59.0 on Android 7.1” because those browsers are the subject;
• outright false positives — footnote markers (Chromium 1 and Firefox 2), citation markers (Firefox22), table cells whose numbers are accuracies or counts, and one “visited from a Chrome browser 100 times”.

So the honest range published on the content page is 17 (structured field) to 26 (text), of 250.

A wide probe for auto-update or browser-version drift returns 14 of 250. All 14 were read. Not one discusses pinning or reporting the crawler's own browser version across waves. The hits are: Windows Update traffic, phishing blacklist auto-updating, a cited paper title, mobile users auto-updating apps, WordPress auto-update, an ad-blocker list auto-update proposal, and Chromium's internal frame-process mapping being “updated”. The closest to the topic is a limitations note in a PETS 2020 paper: “Our measurement tools require constant maintenance to keep up with browser updates and interface changes.” That is about tool maintenance, not about wave comparability.

The content page states this as “not one of the 14 hits”, never as “nobody”, because a probe cannot support “nobody”.

These three are corpus-wide probes (all 5,855 readable papers), not restricted to repeated, because the question is whether the mechanism has any footprint in these seven venues at all. They live in §11 of the report script, and the script prints each regex beside its count — a different regex gives a different number, and hiding the regex would hide that.

• Chrome for Testing. The exact phrase returns 0. Widened to googlechromelabs, chrome labs, known-good-versions and the bare token CfT, it returns 10 — and reading all 10 shows every one is a false positive: Combating the Financing of Terrorism (three papers), Control Flow Trimming, Call Flow Tree, Crash-Fault Tolerant, a certificate organisation name O=CFT, a CFt variable in an algorithm listing, CFT-IDF, and a fitness-test app named CFT & PFT.
• Docker. /\bdocker\b/i gives 293 (5.0%); adding containerised/containerized gives 308 (5.3%). Both are printed, and the content page names which is which — an earlier draft published 308 as “mention Docker”, which is the wider figure under the narrower label.
• Image digest. /\bimage digest\b|\bsha256:[0-9a-f]{12}/i returns 0.
• WebREC. /\bWebREC\b|\bweb execution bundle/i returns 2, and reading both shows neither is a use: one is the WebREC paper, and the other cites it in its reference list only. An earlier draft said “two papers mention it” without that distinction, which reads as two adopters.

Every one of these is a mention count. A paper can pin a container digest in its repository and never say so in the text; these are reporting figures, not practice figures, and the content page says so.

Two wide probes were run over the 266 Tranco papers; their union was 37 candidates, and all 37 were read. 23 genuinely cite a Tranco list ID. The other 14 matched on a legal section number near the word Tranco (CAL. CIV. Code § 1798.120©), a reference-list code, or nothing identifiable. Two of the 23 were found only by the looser probe and would have been missed by the tighter one — mobileatlas writes “Tranco [25] list V78N” with no “ID” token, and autofr cites it only in the reference list. That is why the report script carries a written-out allowlist rather than recomputing the figure from a regex: no regex found all 23, and the script warns loudly if an allowlist entry stops matching a Tranco paper.

The 23, by year:

Everything below was fetched on 2026-08-27, from a primary source, not recalled.

• Whether a paper held a pin constant across waves. The extraction records what a paper stated, once. Every pin figure on the content page is a ceiling and is labelled as one. Closing this needs a hand-coded sample of the 250, reading each paper's per-wave methods description. That is a day of work and would turn every ceiling into an estimate; it is the single most valuable follow-up.
• Whether the 2025–2026 slice changes the trend. The repeat rate is flat at roughly one crawling paper in five across four buckets, and the last bucket is provisional by construction. Nothing in the 2025–2026 material suggests a change, but those venue-years are the thinnest in the corpus.
• Whether pinning actually changes a published answer. The whole case rests on single-wave variation studies ([1Demir, Nurullah; Große-Kampmann, Matteo; Urban, Tobias; Wressnegger, Christian; Holz, Thorsten; Pohlmann, Norbert (2022): "Reproducibility and Replicability of Web Measurement Studies", in: Proceedings of the ACM Web Conference. (DOI)], [2Demir, Nurullah; Hörnemann, Jan; Große-Kampmann, Matteo; Urban, Tobias; Pohlmann, Norbert; Holz, Thorsten; Wressnegger, Christian (2023): "On the Similarity of Web Measurements Under Different Experimental Setups", in: Proceedings of the ACM Internet Measurement Conference, pp. 356-369. ACM DOI 10.1145/3618257.3624795 is listed by DBLP but was not registered with the DOI resolver as of 2026-08-12; the DOI above resolves to the authors' institutional record of the same paper (DOI)], [3Jueckstock, Jordan; Sarker, Shaown; Snyder, Peter; Beggs, Aidan; Papadopoulos, Panagiotis; Varvello, Matteo; Livshits, Benjamin; Kapravelos, Alexandros (2021): "Towards Realistic and Reproducible Web Crawl Measurements", in: Proceedings of the ACM Web Conference. (DOI)], [4Ahmad, Syed Suleman; Dar, Muhammad Daniyal; Zaffar, Muhammad Fareed; Vallina-Rodriguez, Narseo; Nithyanand, Rishab (2020): "Apophanies or Epiphanies? How Crawlers Impact Our Understanding of the Web", in: Proceedings of The Web Conference, pp. 271-280. (DOI)]). No study repeats one measurement under pinned and unpinned conditions over a year. Listed as an open question on the content page.
• A denominator for “how many longitudinal studies got their trend wrong”. Unknowable from this data and not attempted.

Every figure on the content page comes from this script. It prints each population with its denominator, folds the one free-text field it aggregates, and prints that fold's unmapped residue in full.

report_longitudinal.mjs

#!/usr/bin/env node
// Every figure on measuretheweb.org/design:longitudinal, with its denominator.
//
//   node scripts/report_longitudinal.mjs [--run <dir>]
//
// Default run dir: /workspace/publications_dataset/data/extract/run1
//
// Rules this script obeys, from the site's own methodology:
//   * paper-counted, never tuple-counted;
//   * sentinels (`not-stated`, `not-applicable`, ...) are never an answer;
//   * every free-text field is folded before aggregating and the unmapped
//     residue is printed in full;
//   * every block names its denominator on the line above the numbers.
 
import fs from 'node:fs';
import path from 'node:path';
import { foldCadence } from './cadence_fold.mjs';
 
const argv = process.argv.slice(2);
const RUN = argv.includes('--run')
  ? argv[argv.indexOf('--run') + 1]
  : '/workspace/publications_dataset/data/extract/run1';
const FULLTEXT = argv.includes('--fulltext')
  ? argv[argv.indexOf('--fulltext') + 1]
  : '/workspace/publications_dataset/data/fulltext';
 
const P = fs
  .readFileSync(path.join(RUN, 'extractions.jsonl'), 'utf8')
  .split('\n')
  .filter(Boolean)
  .map((s) => JSON.parse(s));
 
const pct = (a, b) => (b ? ((100 * a) / b).toFixed(1) : '0.0');
const rule = (t) => console.log(`\n${'='.repeat(74)}\n${t}\n${'='.repeat(74)}`);
const head = (t, d) => console.log(`\n-- ${t}\n   population: ${d}`);
 
// -------------------------------------------------------------- populations
// `crawled` is the same definition extract_overview.mjs uses, so the counts on
// this page line up with the ones in OVERVIEW.md.
const crawled = P.filter((p) => p.crawlConfig != null || p.studyTypes.includes('automated-web-crawl'));
const empirical = P.filter((p) => p.isEmpirical);
 
// A paper is `repeated` when any temporal tuple states more than one snapshot.
// This is the same rule statistics:biases uses, so the two pages agree.
const snapOf = (p) => Math.max(0, ...p.temporal.map((t) => (t.snapshots == null ? 0 : t.snapshots)));
const repeated = crawled.filter((p) => snapOf(p) > 1);
const singleShot = crawled.filter((p) => snapOf(p) <= 1);
 
// ------------------------------------------------------------------- pins
// Four things a repeat crawl has to hold constant to be comparable with its own
// earlier waves. Each predicate asks only "did the paper state it at all" —
// the extraction cannot tell us whether the value was the SAME across waves, so
// every figure below is an upper bound on comparability, not a measure of it.
const statedListVersion = (p) =>
  p.population.some((x) => x.listVersion != null && !/^(?:not[- ]stated|unknown|n\/a)$/i.test(x.listVersion.trim()));
const namedBrowser = (p) => p.crawlConfig != null && p.crawlConfig.browsers.length > 0;
const browserWithVersion = (p) => p.crawlConfig != null && p.crawlConfig.browsers.some((b) => /\d/.test(b));
const statedVantage = (p) =>
  p.vantage.some((v) => v.locations.some((l) => !/^(?:not[- ]stated|unknown|n\/a)$/i.test(String(l).trim())));
const statedInfra = (p) => p.vantage.some((v) => v.infrastructure !== 'not-stated');
const ccStated = (p, f) => p.crawlConfig != null && !['not-stated', 'not-applicable'].includes(p.crawlConfig[f]);
const allFourPins = (p) => statedListVersion(p) && namedBrowser(p) && statedVantage(p) && ccStated(p, 'statefulness');
 
const PINS = [
  ['population list version stated', statedListVersion],
  ['browser named', namedBrowser],
  ['   ... with a version number in the name', browserWithVersion],
  ['vantage location stated', statedVantage],
  ['vantage infrastructure stated', statedInfra],
  ['statefulness stated', (p) => ccStated(p, 'statefulness')],
  ['consent action stated', (p) => ccStated(p, 'consentAction')],
  ['interaction depth stated', (p) => ccStated(p, 'interactionDepth')],
  ['headless stated', (p) => ccStated(p, 'headless')],
  ['authentication stated', (p) => ccStated(p, 'authentication')],
  ['span start AND end stated', (p) => p.temporal.some((t) => t.spanStart != null) && p.temporal.some((t) => t.spanEnd != null)],
  ['cadence stated', (p) => p.temporal.some((t) => t.cadence != null)],
];
 
// ---------------------------------------------------------------- span
// Longest stated span on any temporal tuple, in months. CAVEAT, and it matters:
// a paper's longest span is often a REUSED dataset arm, not the span of its own
// repeated crawl. The "multi-year" group below therefore mixes both, which is
// itself part of why it pins less — see the mode breakdown printed with it.
const MONTH_NAMES = ['jan', 'feb', 'mar', 'apr', 'may', 'jun', 'jul', 'aug', 'sep', 'oct', 'nov', 'dec'];
 
// Parse a stated date into {year, month}. spanStart/spanEnd are free-text as the
// paper wrote them, so BOTH forms occur and both must be handled:
//   numeric  "2019-03", "2019/03", "2019"
//   textual  "March 2019", "Mar. 2019", "26 March 2019", "April 17th, 2023"
// An earlier version of this function only understood the numeric form, so every
// textual month silently became January. That put a same-year span like
// "February 2011 -> July 2011" into the "<1 month" band and a cross-year span
// like "June 2010 -> January 2011" into "1-2 years". 129 stated dates in the
// corpus are textual, so this was not a rare edge case; it moved five bands and
// the multi-year/sub-year split. Caught by review, fixed, and both tables re-run.
//
// The year must be a standalone token, not a run of digits inside a larger
// number: "2000 domains, March 2020" would otherwise parse 2000 as the year, and
// 2000 sits inside the sanity window so nothing would catch it.
function parseDate(s) {
  if (!s) return null;
  const t = String(s).toLowerCase();
  const ym = /(?:^|[^\d])(\d{4})(?:[-/](0?[1-9]|1[0-2]))(?![\d])/.exec(t);
  if (ym) return { y: +ym[1], m: +ym[2] };
  const yOnly = /(?:^|[^\d])(\d{4})(?![\d])/.exec(t);
  if (!yOnly) return null;
  const y = +yOnly[1];
  if (y < 1990 || y > 2027) return null;
  const name = new RegExp(`\\b(${MONTH_NAMES.join('|')})[a-z]*\\b`).exec(t);
  return { y, m: name ? MONTH_NAMES.indexOf(name[1]) + 1 : 1 };
}
 
function spanMonths(a, b) {
  const pa = parseDate(a);
  const pb = parseDate(b);
  if (!pa || !pb) return null;
  if (pa.y < 1990 || pb.y < 1990 || pa.y > 2027 || pb.y > 2027) return null;
  const d = (pb.y - pa.y) * 12 + (pb.m - pa.m);
  return d < 0 ? null : d;
}
const longestSpan = (p) => {
  let best = null;
  for (const t of p.temporal) {
    const m = spanMonths(t.spanStart, t.spanEnd);
    if (m != null && (best == null || m > best)) best = m;
  }
  return best;
};
 
// ================================================================== output
console.log(`design:longitudinal — figures with denominators`);
console.log(`run: ${RUN}`);
console.log(`corpus: ${P.length.toLocaleString()} papers, 7 venues (CCS, IMC, NDSS, PETS, USENIX Sec, TheWebConf, IEEE S&P), 2010-2026`);
 
rule('1. POPULATIONS');
console.log(`all papers                                        ${P.length}`);
console.log(`empirical (measured the world as it is)           ${empirical.length}`);
console.log(`crawled  (crawlConfig != null OR studyType crawl) ${crawled.length}`);
console.log(`repeated (crawled AND temporal.snapshots > 1)     ${repeated.length}  = ${pct(repeated.length, crawled.length)}% of crawled`);
console.log(`single-shot or snapshot count unstated            ${singleShot.length}  = ${pct(singleShot.length, crawled.length)}% of crawled`);
const anyRepeat = P.filter((p) => snapOf(p) > 1);
console.log(`\nany paper (not only crawls) with snapshots > 1    ${anyRepeat.length}  = ${pct(anyRepeat.length, P.length)}% of all papers`);
 
rule('2. IS THE MEASUREMENT PERIOD EVEN STATED');
head('temporal reporting fields', `${empirical.length} empirical papers`);
for (const [label, f] of [
  ['spanStart stated', (p) => p.temporal.some((t) => t.spanStart != null)],
  ['spanEnd stated', (p) => p.temporal.some((t) => t.spanEnd != null)],
  ['both stated', (p) => p.temporal.some((t) => t.spanStart != null) && p.temporal.some((t) => t.spanEnd != null)],
  ['snapshots stated', (p) => p.temporal.some((t) => t.snapshots != null)],
  ['cadence stated', (p) => p.temporal.some((t) => t.cadence != null)],
]) {
  const n = empirical.filter(f).length;
  console.log(`   ${label.padEnd(22)} ${String(n).padStart(5)}  ${pct(n, empirical.length).padStart(5)}%`);
}
head('the same fields', `${crawled.length} crawling papers`);
for (const [label, f] of [
  ['spanStart stated', (p) => p.temporal.some((t) => t.spanStart != null)],
  ['spanEnd stated', (p) => p.temporal.some((t) => t.spanEnd != null)],
  ['both stated', (p) => p.temporal.some((t) => t.spanStart != null) && p.temporal.some((t) => t.spanEnd != null)],
  ['snapshots stated', (p) => p.temporal.some((t) => t.snapshots != null)],
  ['cadence stated', (p) => p.temporal.some((t) => t.cadence != null)],
]) {
  const n = crawled.filter(f).length;
  console.log(`   ${label.padEnd(22)} ${String(n).padStart(5)}  ${pct(n, crawled.length).padStart(5)}%`);
}
 
rule('3. HOW MANY WAVES, AND HOW FAR APART');
head('stated snapshot count, banded', `${crawled.length} crawling papers`);
{
  const bands = new Map();
  const order = ['not stated', '1', '2-3', '4-6', '7-12', '13-52', '>52'];
  for (const p of crawled) {
    const s = snapOf(p);
    const b = s === 0 ? 'not stated' : s === 1 ? '1' : s <= 3 ? '2-3' : s <= 6 ? '4-6' : s <= 12 ? '7-12' : s <= 52 ? '13-52' : '>52';
    bands.set(b, (bands.get(b) || 0) + 1);
  }
  for (const b of order) console.log(`   ${b.padEnd(12)} ${String(bands.get(b) || 0).padStart(5)}  ${pct(bands.get(b) || 0, crawled.length).padStart(5)}%`);
  const wild = crawled.filter((p) => snapOf(p) > 1000).length;
  console.log(`   NOTE: ${wild} papers carry a snapshot count above 1,000. Spot-checking these finds`);
  console.log(`         page counts and record counts mis-extracted as snapshots, so the top band is`);
  console.log(`         unreliable as a count of waves. Bands are used on the page, never the raw number.`);
}
head('longest stated span, banded', `${repeated.length} repeated crawling papers`);
{
  const bands = new Map();
  const order = ['<1 month', '1-2 months', '3-5 months', '6-11 months', '1-2 years', '2-5 years', '5+ years', 'no parsable end date'];
  for (const p of repeated) {
    const m = longestSpan(p);
    const b = m == null ? 'no parsable end date' : m < 1 ? '<1 month' : m < 3 ? '1-2 months' : m < 6 ? '3-5 months' : m < 12 ? '6-11 months' : m < 24 ? '1-2 years' : m < 60 ? '2-5 years' : '5+ years';
    bands.set(b, (bands.get(b) || 0) + 1);
  }
  for (const b of order) console.log(`   ${b.padEnd(22)} ${String(bands.get(b) || 0).padStart(5)}  ${pct(bands.get(b) || 0, repeated.length).padStart(5)}%`);
}
head('cadence, folded (scripts/cadence_fold.mjs)', `${crawled.filter((p) => p.temporal.some((t) => t.cadence != null)).length} crawling papers that state any cadence`);
{
  const denom = crawled.filter((p) => p.temporal.some((t) => t.cadence != null)).length;
  const fam = new Map();
  const residue = new Map();
  for (const p of crawled) {
    const seen = new Set();
    for (const t of p.temporal) {
      if (t.cadence == null) continue;
      const f = foldCadence(t.cadence);
      if (f === 'unmapped') residue.set(t.cadence.trim(), (residue.get(t.cadence.trim()) || 0) + 1);
      if (!seen.has(f)) { seen.add(f); fam.set(f, (fam.get(f) || 0) + 1); }
    }
  }
  for (const [f, n] of [...fam.entries()].sort((a, b) => b[1] - a[1])) {
    console.log(`   ${f.padEnd(26)} ${String(n).padStart(5)}  ${pct(n, denom).padStart(5)}%`);
  }
  // The families are multi-valued, so summing the non-cadence rows double-counts.
  // The publishable figure is the number of papers whose cadence values are ALL
  // non-cadences — a duration, a bare round count, or "periodically".
  const NOTACADENCE = new Set(['duration-not-cadence', 'n-rounds-no-interval', 'irregular-or-unspecified', 'sentinel']);
  let noRealCadence = 0;
  for (const p of crawled) {
    const fams = new Set(p.temporal.filter((t) => t.cadence != null).map((t) => foldCadence(t.cadence)));
    if (fams.size && [...fams].every((f) => NOTACADENCE.has(f))) noRealCadence++;
  }
  console.log(`\n   papers that fill the cadence slot but never state a cadence:`);
  console.log(`      ${noRealCadence} / ${denom} = ${pct(noRealCadence, denom)}%   (counting 'unmapped' as a possible cadence, i.e. conservative)`);
  console.log(`\n   A paper can land in several families (it may state more than one cadence), so`);
  console.log(`   the column does not sum to ${denom}. 'duration-not-cadence' is the field being used`);
  console.log(`   to state how long the study ran rather than how often it repeated — those papers`);
  console.log(`   have NOT stated a cadence and must not be counted as if they had.`);
  console.log(`\n   UNMAPPED RESIDUE — ${residue.size} distinct strings, printed in full:`);
  for (const [s, n] of [...residue.entries()].sort((a, b) => b[1] - a[1])) console.log(`      ${n} x  ${s}`);
}
 
rule('4. THE FOUR PINS: WHAT A REPEATED CRAWL HOLDS FIXED');
head('stated at all (an upper bound on holding it fixed)', `${repeated.length} repeated vs ${singleShot.length} other crawling papers`);
console.log(`   ${'pin'.padEnd(42)} ${'repeated'.padStart(14)} ${'other crawls'.padStart(14)}`);
for (const [label, f] of PINS) {
  const a = repeated.filter(f).length;
  const b = singleShot.filter(f).length;
  console.log(`   ${label.padEnd(42)} ${`${a} (${pct(a, repeated.length)}%)`.padStart(14)} ${`${b} (${pct(b, singleShot.length)}%)`.padStart(14)}`);
}
{
  const a = repeated.filter(allFourPins).length;
  const b = singleShot.filter(allFourPins).length;
  const none = repeated.filter((p) => !statedListVersion(p) && !namedBrowser(p) && !statedVantage(p) && !ccStated(p, 'statefulness')).length;
  const strict = repeated.filter((p) => statedListVersion(p) && browserWithVersion(p) && statedVantage(p) && ccStated(p, 'statefulness')).length;
  console.log(`\n   all four (list version + browser + vantage + statefulness)`);
  console.log(`      repeated      ${a} / ${repeated.length} = ${pct(a, repeated.length)}%`);
  console.log(`      other crawls  ${b} / ${singleShot.length} = ${pct(b, singleShot.length)}%`);
  console.log(`   all four, with a browser VERSION rather than a browser name`);
  console.log(`      repeated      ${strict} / ${repeated.length} = ${pct(strict, repeated.length)}%`);
  console.log(`   none of the four`);
  console.log(`      repeated      ${none} / ${repeated.length} = ${pct(none, repeated.length)}%`);
}
 
rule('5. THE LONGER THE PANEL, THE LESS IT PINS');
{
  const multi = repeated.filter((p) => { const m = longestSpan(p); return m != null && m >= 12; });
  const sub = repeated.filter((p) => { const m = longestSpan(p); return m != null && m < 12; });
  const unk = repeated.length - multi.length - sub.length;
  head('by longest stated span', `${repeated.length} repeated crawling papers (${unk} have no parsable end date and are excluded)`);
  for (const [name, g] of [['span >= 12 months', multi], ['span <  12 months', sub]]) {
    const f = g.filter(allFourPins).length;
    console.log(`   ${name}  n=${String(g.length).padStart(4)}   list ${String(g.filter(statedListVersion).length).padStart(3)}  browser ${String(g.filter(namedBrowser).length).padStart(3)}  vantage ${String(g.filter(statedVantage).length).padStart(3)}  statefulness ${String(g.filter((p) => ccStated(p, 'statefulness')).length).padStart(3)}   all four ${f} (${pct(f, g.length)}%)`);
  }
  console.log(`\n   Why the multi-year group pins less — its data provenance is different.`);
  console.log(`   temporal.mode, paper-counted, multi-valued:`);
  const modes = (g) => { const c = new Map(); for (const p of g) for (const m of new Set(p.temporal.map((t) => t.mode))) c.set(m, (c.get(m) || 0) + 1); return c; };
  for (const [name, g] of [['span >= 12 months', multi], ['span <  12 months', sub]]) {
    const c = modes(g);
    console.log(`   ${name}  ` + [...c.entries()].sort((a, b) => b[1] - a[1]).map(([m, n]) => `${m} ${n} (${pct(n, g.length)}%)`).join(', '));
  }
  const liveOnly = (g) => g.filter((p) => p.temporal.every((t) => ['live-crawl', 'active-probing', 'not-stated'].includes(t.mode)));
  console.log(`\n   Restricted to papers whose every temporal tuple is a live crawl or active probe`);
  console.log(`   (no archive, no reused dataset), so the pins were at least available:`);
  for (const [name, g] of [['span >= 12 months', liveOnly(multi)], ['span <  12 months', liveOnly(sub)]]) {
    const f = g.filter(allFourPins).length;
    console.log(`   ${name}  n=${String(g.length).padStart(4)}   all four ${f} (${pct(f, g.length)}%)   << small n, report as a count`);
  }
}
 
rule('6. IS REPEATING GETTING MORE COMMON');
head('share of crawling papers that repeat, by year', 'crawling papers in each year');
{
  const years = [...new Set(crawled.map((p) => p.year))].sort();
  for (const y of years) {
    const g = crawled.filter((p) => p.year === y);
    const r = g.filter((p) => snapOf(p) > 1).length;
    const flag = y >= 2025 ? '   << provisional venue-year, under-represented by construction' : '';
    console.log(`   ${y}   crawled ${String(g.length).padStart(4)}   repeated ${String(r).padStart(3)}  ${pct(r, g.length).padStart(5)}%${flag}`);
  }
  const bucket = (a, b) => { const g = crawled.filter((p) => p.year >= a && p.year <= b); const r = g.filter((p) => snapOf(p) > 1).length; return `${a}-${b}: ${r}/${g.length} = ${pct(r, g.length)}%`; };
  console.log(`\n   four-year buckets: ${bucket(2010, 2011)} | ${bucket(2012, 2015)} | ${bucket(2016, 2019)} | ${bucket(2020, 2023)} | ${bucket(2024, 2026)} (last bucket provisional)`);
}
 
rule('7. DATA PROVENANCE OF EMPIRICAL WORK');
head('temporal.mode', `${empirical.length} empirical papers, multi-valued, paper-counted`);
{
  const c = new Map();
  for (const p of empirical) for (const m of new Set(p.temporal.map((t) => t.mode))) c.set(m, (c.get(m) || 0) + 1);
  for (const [m, n] of [...c.entries()].sort((a, b) => b[1] - a[1])) console.log(`   ${m.padEnd(22)} ${String(n).padStart(5)}  ${pct(n, empirical.length).padStart(5)}%`);
}
 
rule('8. FULL-TEXT PROBES OVER THE REPEATED-CRAWL PAPERS');
console.log(`   Whitespace is collapsed before matching, because a PDF line break inside a`);
console.log(`   phrase would otherwise hide it. Every probe is a MENTION count and therefore`);
console.log(`   an upper bound; the two load-bearing ones were hand-read in full and the`);
console.log(`   audited counts are printed beside them.`);
head('probe hits', `${repeated.length} repeated crawling papers, all with readable full text`);
{
  const PROBES = [
    ['self-describes as longitudinal', /\blongitudinal\b/i, null],
    ['re-crawl / repeat crawl / second crawl', /\bre-?crawl|\brepeat(?:ed)? (?:the )?(?:crawl|measurement|experiment)|\bsecond crawl\b|\bfollow-?up crawl\b/i, null],
    ['wave / round / iteration terminology', /\b(?:each|every|first|second|third|per) (?:wave|round|run|iteration)\b|\bcrawl(?:ing)? (?:wave|round)s?\b/i, null],
    ['comparable / comparability', /\bcomparab(?:le|ility)\b/i, null],
    ['reproducib* / replicab* / replicat*', /\breproducib\w*|\breplicab\w*|\breplicat\w*/i, null],
    ['same version / list / snapshot / configuration', /\bsame (?:version|list|snapshot|configuration|setup|browser|profile|seed list|instance)\b/i, null],
    ['pinned / froze / fixed the version (wide)', /\bpinn?ed\b|\bfroze[nd]?\b|\bfixed\s+(?:the\s+)?(?:version|browser|list|snapshot|commit|configuration|seed)|\bversion[- ]controll?ed\b/i, null],
    ['Docker / container / VM image', /\bdocker\b|\bcontaineri[sz]ed\b|\bVM image\b|\bvirtual machine image\b/i, null],
    ['names a filter list or blocklist', /\bEasyList\b|\bEasyPrivacy\b|\bDisconnect\b|\bfilter\s+lists?\b|\bblock\s?lists?\b/i, null],
    ['filter list pinned to a version or date', /\b(?:EasyList|EasyPrivacy|Disconnect|filter list|blocklist|block list)\b.{0,80}\b(?:version|as of|snapshot|dated|commit|downloaded on)\b/i, null],
    ['attrition or survivorship', /\battrition\b|\bsurvivorship\b/i, null],
    ['churn', /\bchurn\b/i, null],
    ['<browser> <number> anywhere in the text', /\b(?:Chrome|Chromium|Firefox|Edge|Safari|Tor Browser|Brave)\s*(?:browser\s*)?(?:v(?:ersion)?\.?\s*)?\d{1,3}(?:\.\d+)*\b/i, 26],
    ['browser auto-update / version drift (wide)', /\bauto-?updat\w*|\bautomatic(?:ally)?\s+updat\w*|\bbrowser\s+(?:version\s+)?(?:updates?|upgrad\w*)|\bdisabl\w+\s+updates?/i, 0],
  ];
  const texts = repeated.map((p) => {
    const f = path.join(FULLTEXT, String(p.year), p.venue, p.slug, 'paper.cols.txt');
    return fs.existsSync(f) ? fs.readFileSync(f, 'utf8').replace(/\s+/g, ' ') : null;
  });
  const readable = texts.filter((t) => t != null).length;
  console.log(`   full text readable for ${readable} of ${repeated.length}\n`);
  for (const [label, re, audited] of PROBES) {
    const n = texts.filter((t) => t != null && re.test(t)).length;
    const aud = audited == null ? '' : `   -> hand-read all ${n}: ${audited} genuine (${pct(audited, readable)}% of the population)`;
    console.log(`   ${label.padEnd(46)} ${String(n).padStart(4)}  ${pct(n, readable).padStart(5)}%${aud}`);
  }
  console.log(`\n   The two audited probes, in words:`);
  console.log(`     * "<browser> <number>" is a candidate set, not an answer. Reading all of them,`);
  console.log(`       26 state the version of a browser the study's OWN measurement drove. The rest`);
  console.log(`       are bibliography entries, ecosystem history ("Chrome 43 shipped ..."), the`);
  console.log(`       version of a browser being measured rather than driven, and outright false`);
  console.log(`       positives (footnote markers, table cells, "Chrome browser 100 times").`);
  console.log(`     * The auto-update probe: reading all hits, NOT ONE discusses pinning or`);
  console.log(`       reporting the crawler's own browser version across waves. The closest is`);
  console.log(`       a note that measurement tools need maintenance as browsers change.`);
}
 
rule('9. PINNING THE LIST: DOES ANYONE CITE A PERMANENT TRANCO ID');
{
  // Tranco has published permanent, retrievable list IDs since 2019, and a list
  // ID is the only mechanism in this literature that makes a population frame
  // recoverable years later (verified 2026-08-27: the 2019 id GVWK still
  // resolves). So: of the papers that use Tranco at all, how many cite one?
  //
  // Two wide probes were run over the full text of every Tranco paper; their
  // union was 37 candidates; ALL 37 were read by hand. The 23 below are the ones
  // that genuinely cite an identifier. The rest matched legal section numbers
  // and reference codes sitting near the word "Tranco". The allowlist is written
  // out rather than recomputed because a probe cannot make this call.
  const CITES_TRANCO_ID = new Set([
    '2020/WWW/beyond-the-front-page-measuring-third-party-dynamics-in-the-field',
    '2021/NDSS/reining-in-the-webs-inconsistencies-with-site-policy',
    '2021/PETS/privacy-preference-signals-past-present-and-future',
    '2022/CCS/helping-or-hindering-how-browser-extensions-undermine-security',
    '2022/PETS/setting-the-bar-low-are-websites-complying-with-the-minimum-requirements-of-the',
    '2023/CCS/you-call-this-archaeology-evaluating-web-archives-for-reproducible-web-security',
    '2023/IEEE-SP/its-dom-clobbering-time-attack-techniques-prevalence-and-defenses',
    '2023/USENIX/autofr-automated-filter-rule-generation-for-adblocking',
    '2023/USENIX/mobileatlas-geographically-decoupled-measurements-in-cellular-networks-for-secur',
    '2023/WWW/the-hitchhikers-guide-to-facebook-web-tracking-with-invisible-pixels-and-click-i',
    '2024/IEEE-SP/to-auth-or-not-to-auth-a-comparative-analysis-of-the-pre-and-post-login-security',
    '2024/IEEE-SP/where-are-the-red-lines-towards-ethical-server-side-scans-in-security-and-privac',
    '2024/PETS/a-bilingual-longitudinal-analysis-of-privacy-policies-measuring-the-impacts-of-t',
    '2024/PETS/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework',
    '2025/CCS/in-the-dom-we-trust-exploring-the-hidden-dangers-of-reading-from-the-dom-on-the',
    '2025/IEEE-SP/a-wall-behind-a-wall-emerging-regional-censorship-in-china',
    '2025/IMC/chaos-in-the-chain-evaluate-deployment-and-construction-compliance-of-web-pki-ce',
    '2025/IMC/towards-a-non-binary-view-of-ipv6-adoption',
    '2025/PETS/johnny-can-t-revoke-consent-either-measuring-compliance-of-consent-revocation-on',
    '2025/PETS/understanding-privacy-norms-through-web-forms',
    '2025/USENIX/exposing-and-circumventing-sni-based-quic-censorship-of-the-great-firewall-of-ch',
    '2025/USENIX/navigating-cookie-consent-violations-across-the-globe',
    '2026/USENIX/inconsistent-incomplete-and-insecure-a-survey-of-account-security-interfaces',
  ]);
  const usesTranco = (p) => p.population.some((x) => /\btranco\b/i.test(x.sourceList || ''));
  const T = P.filter(usesTranco);
  head('Tranco users', `${T.length} papers naming Tranco as a population source`);
  const withVersion = T.filter((p) => p.population.some((x) => /\btranco\b/i.test(x.sourceList || '') && x.listVersion != null));
  console.log(`   states any list version or date (extraction field)   ${withVersion.length}  ${pct(withVersion.length, T.length)}%`);
  const cited = T.filter((p) => CITES_TRANCO_ID.has(`${p.year}/${p.venue}/${p.slug}`));
  console.log(`   cites a permanent Tranco list ID (hand-audited)      ${cited.length}  ${pct(cited.length, T.length)}%`);
  const years = [...new Set(T.map((p) => p.year))].sort();
  console.log(`\n   by year (Tranco users / of them, citing an ID):`);
  for (const y of years) {
    const g = T.filter((p) => p.year === y);
    const c = cited.filter((p) => p.year === y).length;
    console.log(`      ${y}   ${String(g.length).padStart(3)}   ${String(c).padStart(2)}  ${pct(c, g.length).padStart(5)}%`);
  }
  const stray = [...CITES_TRANCO_ID].filter((k) => !T.some((p) => `${p.year}/${p.venue}/${p.slug}` === k));
  if (stray.length) console.log(`\n   WARNING: ${stray.length} allowlist entries no longer match a Tranco paper: ${stray.join(', ')}`);
}
 
rule('10. EXEMPLARS — REPEATED CRAWLS THAT STATE ALL FOUR PINS');
console.log(`   ${repeated.filter(allFourPins).length} papers. Most recent first. These are the papers to copy a methods section from.`);
for (const p of repeated.filter(allFourPins).sort((a, b) => b.year - a.year || a.venue.localeCompare(b.venue))) {
  const cad = p.temporal.map((t) => t.cadence).filter(Boolean).join(' / ') || '-';
  console.log(`\n   ${p.year} ${p.venue}  ${p.slug}`);
  console.log(`        snapshots=${snapOf(p)}  cadence=${cad}  longest span=${longestSpan(p) == null ? 'n/a' : longestSpan(p) + 'mo'}`);
  console.log(`        browsers=${JSON.stringify(p.crawlConfig.browsers)}  statefulness=${p.crawlConfig.statefulness}  consent=${p.crawlConfig.consentAction}`);
  console.log(`        listVersion=${JSON.stringify(p.population.map((x) => x.listVersion).filter(Boolean))}`);
  console.log(`        vantage=${JSON.stringify([...new Set(p.vantage.flatMap((v) => v.locations))])}`);
}
// Compact form of the same 23 papers, in the column order the wiki table uses,
// so the page is copied from real output rather than retyped.
console.log(`\n   -- compact table (year | venue | slug | snapshots | distinct stated vantage locations)`);
for (const p of repeated.filter(allFourPins).sort((a, b) => b.year - a.year || a.venue.localeCompare(b.venue))) {
  const locs = new Set(p.vantage.flatMap((v) => v.locations).filter((l) => !/^(?:not[- ]stated|unknown|n\/a)$/i.test(String(l).trim())));
  console.log(`   ${p.year} | ${p.venue} | ${p.slug} | ${snapOf(p)} | ${locs.size}`);
}
console.log();
 
rule('11. CORPUS-WIDE ADOPTION PROBES FOR THE MODERN PINNING MECHANISMS');
{
  // These four probes are corpus-wide, not restricted to `repeated`, because the
  // question is "has this mechanism any footprint in these seven venues at all".
  // Each regex is printed with its count so a reader can re-run the exact query;
  // a different regex gives a different number, and that is the point of showing it.
  //
  // Every one of these is a MENTION count. A paper can pin a container digest in
  // its repository and never say so in the text. Read them as reporting figures.
  const PROBES = [
    ['Docker OR containerised', /\bdocker\b|\bcontaineri[sz]ed\b/i, null],
    ['   ...Docker alone', /\bdocker\b/i, null],
    ['image digest or sha256: image reference', /\bimage digest\b|\bsha256:[0-9a-f]{12}/i, null],
    ['"Chrome for Testing", exact phrase', /chrome[-\s]?for[-\s]?testing/i, null],
    ['Chrome for Testing, widened to the tooling and the acronym', /chrome[-\s]?for[-\s]?testing|googlechromelabs|chrome[-\s]?labs|known[-\s]good[-\s]versions|\bCfT\b/i, 0],
    ['WebREC or "web execution bundle"', /\bWebREC\b|\bweb execution bundle/i, 0],
  ];
  let readable = 0;
  const counts = PROBES.map(() => 0);
  for (const p of P) {
    const f = path.join(FULLTEXT, String(p.year), p.venue, p.slug, 'paper.cols.txt');
    if (!fs.existsSync(f)) continue;
    readable++;
    const t = fs.readFileSync(f, 'utf8').replace(/\s+/g, ' ');
    PROBES.forEach(([, re], i) => { if (re.test(t)) counts[i]++; });
  }
  head('mentions anywhere in the full text', `${readable} papers with readable full text, of ${P.length}`);
  PROBES.forEach(([label, re, audited], i) => {
    const aud = audited == null ? '' : `   -> hand-read all ${counts[i]}: ${audited} genuine`;
    console.log(`   ${label.padEnd(58)} ${String(counts[i]).padStart(4)}  ${pct(counts[i], readable).padStart(5)}%${aud}`);
    console.log(`   ${' '.repeat(58)} ${String(re)}`);
  });
  console.log(`\n   Hand audits:`);
  console.log(`     * The widened Chrome-for-Testing probe: every hit is a false positive —`);
  console.log(`       CFT as "Combating the Financing of Terrorism" (three papers), "Control Flow`);
  console.log(`       Trimming", "Call Flow Tree", "Crash-Fault Tolerant", a certificate subject`);
  console.log(`       O=CFT, a CFt variable in an algorithm listing, CFT-IDF, and a fitness app`);
  console.log(`       named "CFT & PFT". The exact-phrase probe is 0, and so is the audited count.`);
  console.log(`     * WebREC: one hit is the WebREC paper itself; the other cites it in its`);
  console.log(`       reference list only. No paper in the corpus USES it.`);
}

temporal.cadence is free text: 334 distinct strings fire on the 1,120 crawling papers. This is the fold, its rule order, and the reasoning behind the first rule.

cadence_fold.mjs

// Folds the free-text `temporal.cadence` string into a small family vocabulary.
//
// WHY THIS EXISTS. `temporal.cadence` is free text and 334 distinct strings fire
// on the 1,120 crawling papers alone. Counting exact strings would report
// "daily" at 66 papers and bury the other 268 values in a tail nobody reads.
// It is also *contaminated*: a large minority of the values state how LONG the
// measurement ran ("one week", "5-day experiment", "month-long") rather than how
// OFTEN it repeated. Those are not cadences and are folded into their own family
// so they cannot be counted as one.
//
// Rules, in order. The FIRST rule that matches wins, and rules are ordered from
// fastest interval to slowest, so "hourly and daily" folds to `hourly` — the
// fastest repeat the paper states is the one that bounds its time resolution.
//
// Anything no rule matches lands in `unmapped` and is PRINTED IN FULL by the
// report. A residue that only exists inside a script is a residue nobody reads.
 
const SENTINEL = /^(?:not[- ]stated|none[- ]mentioned|unclear|unknown|n\/?a)$/i;
 
const RULES = [
  // A duration is not a cadence. This rule runs FIRST for the bare forms
  // ("one week", "15 days", "month-long", "5-day experiment") because they
  // would otherwise be swept up by the weekly/monthly rules below. It is
  // deliberately anchored: "every 5 minutes for 8 days" states a real cadence
  // and must not land here, so a leading `every|per|once` disqualifies it.
  // "seven days", "over one month", "two-week measurement period", "single crawl
  // lasting 12 days", "24-hour runs", "one-week collection period" — the paper is
  // stating how long it ran, not how often it repeated.
  // A string containing "between" or "apart" states an INTERVAL, not a duration
  // ("15 days between batch starts"), so it must never reach the duration rules.
  // The interval rules below pick it up by unit.
  ['duration-not-cadence', /^(?!.*\b(?:between|apart)\b)(?:.*\b(?:collection|measurement|study|crawl(?:ing)?|campaign|experiment|observation)\s+(?:period|window|cycle)\b|\b(?:period|window)\s+of\s+\d+|\bsingle\s+(?:day|crawl|scrape|run|visit|snapshot)\b|\bpilot\s+study\b|\blasting\s+\d+|\b\d+-hour\s+runs?\b|^(?:about|approximately|around|over|nearly|spread\s+across|during)\s+(?:a\s+)?(?:couple\s+of\s+|few\s+|multiple\s+|one\s+|two\s+|three\s+|several\s+|\d+\s*)?(?:day|week|month|year)s?\b|^(?:several|multiple)\s+(?:day|week|month|year)s?\b|^\d+\s*(?:day|week|month|year)s?$|^(?:one|two|three|four|five|six|seven|eight|nine|ten|eleven|twelve)\s+(?:consecutive\s+)?(?:day|week|month|year)s?$|\b(?:one|two|three|four|five|six|\d+)-(?:day|week|month|year)\s+(?:campaign|collection|trace|slot|phishing))/i],
 
  ['duration-not-cadence', /^(?!.*\b(?:every|per|once|between|apart|times?\s+(?:a|per)|twice|daily|hourly|weekly|monthly|yearly|annual|continuous|periodic|regular)\b)(?:(?:about|over|nearly|one|two|three|four|five|six|\d+)[- ]?)?(?:\d+[- ])?(?:second|minute|hour|day|week|month|year)s?[- ]?(?:long|period|experiment|crawl|campaign|study|snapshot|window|trace)?$|^\d+\s*(?:second|minute|hour|day|week|month|year)s?\b(?!.*\b(?:every|between|apart)\b)|(?:^|\b)(?:one|two|three|four|five|a)?[- ]?(?:day|week|month|year)-long\b/i],
 
  ['sub-minute', /\bmillisecond|\b\d+\s*(?:ms|milliseconds)\b|every\s+\d+(?:\.\d+)?\s*seconds?\b|every\s+(?:few|five|ten|fifteen|thirty)\s+seconds?\b|per\s+second\b/i],
  ['minutes', /\bevery\s+(?:\d+|a|one|two|three|five|ten|twelve|fifteen|thirty|sixty)\s*minutes?\b|\b\d+-minute\s+(?:interval|snapshot)|\bminute\s+intervals?\b|every\s+minute\b|once\s+per\s+minute\b|\bper\s+minute\b/i],
  ['hourly', /\bhourly\b|\bevery\s+(?:\d+|a|an|one|two|three|four|six|eight|twelve|24)\s*hours?\b|\b\d+-hour(?:ly)?\s+(?:interval|window)|\bhours?\s+apart\b|\bfour-hourly\b|\bevery\s+few\s+hours\b|\b(?:twice|two|three)\s+(?:times\s+)?(?:a|per)\s+day\b|\bmultiple\s+(?:scrapes|observations)\s+per\s+day\b|\btwice\s+daily\b|\bfive\s+times\s+daily\b|\b12\s+hours\b/i],
  ['daily', /\bdaily\b|\bnightly\b|\bevery\s+day\b|\bper\s+day\b|\bday\b.*\bintervals?\b|\bat\s+least\s+24\s+hours\s+between\b|\bmultiple\s+times\s+a\s+day\b/i],
  ['every-few-days', /\bevery\s+(?:\d+|a\s+few|two|three|four|five|six|ten|other)\s*days?\b|\bevery\s+two\s+to\s+three\s+days\b|\b(?:two|three|\d+)-day\s+(?:retry\s+)?interval\b|\bonce\s+every\s+\d+\s*days?\b|\bevery\s+\d+(?:\.\d+)?\s*days?\b|\bone\s+day\s+between\b|\b\d+\s*days?\s+between\b/i],
  ['weekly', /\bweekly\b|\bevery\s+week\b|\bper\s+week\b|\bweekdays?\b|\bevery\s+seven\s+days\b|\bat\s+least\s+one\s+week\s+between\b|\bone\s+week\s+between\b/i],
  ['fortnightly', /\bfortnight|\bbi-?weekly\b|\bevery\s+(?:two\s+weeks|14\s*days|15\s*days)\b|\btwo-week\s+intervals?\b|\btwo\s+weeks\s+between\b/i],
  ['monthly', /\bmonthly\b|\bbimonthly\b|\bevery\s+(?:four|twelve)\s+(?:months|weeks)\b|\bevery\s+month\b|\bper\s+month\b|\bevery\s+30\s*days\b|\bmonths?\s+apart\b|\bat\s+least\s+one\s+month\s+between\b|\bmonths?\s+between\b/i],
  ['quarterly', /\bquarterly\b|\btrimester\b|\bevery\s+three\s+months\b|\bevery\s+quarter\b|\bthree-month\s+intervals?\b/i],
  ['yearly', /\byearly\b|\bannual\w*\b|\bevery\s+year\b|\bone\s+year\s+apart\b/i],
  ['continuous', /\bcontinuous\w*\b|\breal[- ]?time\b|\biteratively\s+without\s+sleep\b|\bstreaming\b/i],
  ['irregular-or-unspecified', /\bperiodic\w*\b|\bsporadic\b|\bsimultaneous\b|\btemporally\s+close\b|\bas\s+domains\s+appeared\b|\bincreasing\s+(?:historical\s+)?intervals?\b|\bregular\w*\b|\badaptive\b|\bon[- ]demand\b|\bas\s+needed\b|\bwhen(?:ever)?\b|\bvariable\b|\bad\s?hoc\b/i],
  ['n-rounds-no-interval', /\b(?:one|two|three|four|five|six|seven|eight|nine|ten|\d+)\s+(?:repeated\s+|sequential\s+|distinct\s+|measurement\s+|querying\s+|collection\s+)*(?:rounds?|repetitions?|iterations?|visits?|scans?|scrapes?|queries|campaigns?|snapshots?|periods?|timestamps?|crawls?|groups?|runs?|windows?|days?|weeks?|intervals?)\b|\bonce\b|\brepeated\s+(?:experiments?|collection\s+iterations?)\b|\bup\s+to\s+\w+\s+sequential\s+visits\b|\bconsecutive\s+crawls\b/i],
];
 
export function foldCadence(raw) {
  const s = String(raw).trim().toLowerCase().replace(/\s+/g, ' ');
  if (SENTINEL.test(s)) return 'sentinel';
  for (const [family, re] of RULES) if (re.test(s)) return family;
  return 'unmapped';
}
 
export const FAMILIES = RULES.map(([f]) => f).concat('unmapped');
export const REAL_FAMILIES = FAMILIES.filter((f) => f !== 'duration-not-cadence' && f !== 'unmapped');

quotecheck_longitudinal.mjs

#!/usr/bin/env node
// Verifies every quotation used on design:longitudinal against the paper text.
//
//   node scripts/quotecheck_longitudinal.mjs
//
// Two-column PDFs are repaired into paper.cols.txt but not perfectly, so a
// sentence can arrive spliced with text from the neighbouring column, and a word
// can be hyphen-split across a line break. Each quote is checked against FOUR
// renderings: paper.cols.txt, paper.norm.txt, paper.txt, and paper.cols.txt with
// end-of-line hyphenation joined. The verdicts:
//   EXACT     — the quote, whitespace-collapsed, appears verbatim
//   SPLICED   — every word is covered by contiguous runs of >= MIN_RUN words,
//               so only the reading order was broken
//   PARTIAL   — some words matched only in shorter runs. Read it by hand.
//   MISSING   — a word of the quote does not appear at all. A real failure.
import fs from 'node:fs';
import path from 'node:path';
const ROOT = '/workspace/publications_dataset/data/fulltext';
const norm = (s) => s.replace(/[‐-―−]/g, '-').replace(/[‘’]/g, "'").replace(/[“”]/g, '"').replace(/\s+/g, ' ').trim();
 
const QUOTES = [
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'Over twelve days, we saw a variation of up to 27% (max on day 3-80,274; min on day 9-58,951) in observed tracking requests.'],
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'the number of distinct tracking domains remains almost stable during the experiment (variation of 3.5%)'],
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'studies that analyze the ecosystem will find similar results, while studies that aim to analyze the extent of a tracking phenomenon might see different results based on the measurement day'],
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'we find that the identified trackers on pages can vary by 25% based on the used browser configuration'],
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'even the same setup measures different levels of tracking on different days, which might lead to different conclusions of a study'],
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'We use the popular Open-WPM Framework [21] (v0.15.0 - Firefox version 88)'],
  ['2023/IMC/on-the-similarity-of-web-measurements-under-different-experimental-setups',
    'when comparing two different profiles, 48% of the underlying data varies'],
  ['2026/PETS/overcoming-language-barriers-multilingual-analysis-of-the-2023-swiss-privacy-law',
    'we restrict the analysis to the balanced panel of websites with an observed policy in both snapshots'],
  ['2026/PETS/overcoming-language-barriers-multilingual-analysis-of-the-2023-swiss-privacy-law',
    'Standard errors are clustered at the website level to account for repeated observations of the same policy across snapshots.'],
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'The standard deviation of such requests is 8,203.'],
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'For our continuous measurement, we visit the top 1k sites from our website corpus (18,377 distinct pages) daily'],
  ['2022/WWW/reproducibility-and-replicability-of-web-measurement-studies',
    'privacy measurements and analyses can vary up to 65% depending on the region'],
  ['2023/IMC/on-the-similarity-of-web-measurements-under-different-experimental-setups',
    'only 32% of the cookies appear in all profiles and 42% only in one profile'],
  ['2021/WWW/towards-realistic-and-reproducibleweb-crawl-measurements',
    'Around 5% of content-providing domains show significant measurement bias across VP'],
  ['2020/WWW/apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web',
    'variation of over 16% in the number of successful page loads'],
  ['2026/PETS/overcoming-language-barriers-multilingual-analysis-of-the-2023-swiss-privacy-law',
    'The balanced panel comprises N = 11,800 websites observed in both August and October 2023'],
  ['2025/USENIX/web-execution-bundles-reproducible-accurate-and-archivable-web-measurements',
    'could be conducted using WebREC as is'],
  ['2026/PETS/the-empire-strikes-back-at-your-privacy-an-archaeology-of-tracking-on-government',
    'We therefore label trackers using the June 2025 EasyList and EasyPrivacy snapshot for all years, keeping the labeling rule fixed'],
];
 
// A two-column splice interleaves the neighbouring column every dozen words or
// so, which destroys a naive substring test and most n-gram tests too. The check
// that survives it: greedily segment the quote into the LONGEST contiguous word
// runs that each appear in the text. If every word of the quote is covered by a
// run of at least MIN_RUN words, the quote is present in the source and only the
// reading order was broken.
//
//   EXACT    the whole quote appears verbatim
//   SPLICED  every word covered by runs of >= MIN_RUN words
//   PARTIAL  some words only matched in short runs -> read it by hand
//   MISSING  a run of the quote does not appear at all
const MIN_RUN = 4;
 
function segment(words, text) {
  const runs = [];
  let i = 0;
  while (i < words.length) {
    let j = words.length;
    while (j > i && !text.includes(words.slice(i, j).join(' '))) j--;
    if (j === i) { runs.push({ len: 0, text: words[i] }); i++; }
    else { runs.push({ len: j - i, text: words.slice(i, j).join(' ') }); i = j; }
  }
  return runs;
}
 
let bad = 0;
for (const [slug, q] of QUOTES) {
  // Fourth rendering: paper.cols.txt with PDF end-of-line hyphenation repaired
  // ("varia- tion" -> "variation"). Without it a quote that straddles a line
  // break inside a word reports MISSING even though it is plainly in the paper.
  const files = ['paper.cols.txt', 'paper.norm.txt', 'paper.txt', 'paper.cols.txt (de-hyphenated)'].map((f) =>
    path.join(ROOT, slug, f.replace(' (de-hyphenated)', '')));
  const texts = files.map((f, i) => {
    if (!fs.existsSync(f)) return null;
    const raw = fs.readFileSync(f, 'utf8');
    return norm(i === 3 ? raw.replace(/(\w)-\s+(\w)/g, '$1$2') : raw);
  });
  const LABEL = ['paper.cols.txt', 'paper.norm.txt', 'paper.txt', 'paper.cols.txt (de-hyphenated)'];
  const nq = norm(q);
  const words = nq.split(' ');
  let verdict = 'MISSING', where = '', detail = '';
  for (let i = 0; i < files.length; i++) {
    if (texts[i] != null && texts[i].includes(nq)) { verdict = 'EXACT'; where = LABEL[i]; break; }
  }
  if (verdict !== 'EXACT') {
    let bestScore = -Infinity;
    for (let i = 0; i < files.length; i++) {
      if (texts[i] == null) continue;
      const runs = segment(words, texts[i]);
      const zero = runs.filter((r) => r.len === 0).length;
      const short = runs.filter((r) => r.len > 0 && r.len < MIN_RUN).length;
      const score = -zero * 100 - short;
      if (score > bestScore) {
        bestScore = score;
        where = LABEL[i];
        detail = `${runs.length} run(s): ${runs.map((r) => r.len).join('+')} words` + (zero ? `, ${zero} word(s) NOT FOUND` : '');
        verdict = zero > 0 ? 'MISSING' : short > 0 ? 'PARTIAL' : 'SPLICED';
      }
    }
  }
  if (verdict === 'MISSING' || verdict === 'PARTIAL') bad++;
  console.log(`${verdict.padEnd(8)} ${slug}\n         "${q.slice(0, 110)}${q.length > 110 ? '...' : ''}"\n         ${where}${detail ? ' — ' + detail : ''}\n`);
}
console.log(bad === 0 ? 'every quote located in its source (EXACT or SPLICED)' : `${bad} quote(s) need a hand read`);

node scripts/report_longitudinal.mjs, run on 2026-08-27 against data/extract/run1. Nothing below is edited.

design:longitudinal — figures with denominators
run: /workspace/publications_dataset/data/extract/run1
corpus: 5,859 papers, 7 venues (CCS, IMC, NDSS, PETS, USENIX Sec, TheWebConf, IEEE S&P), 2010-2026
 
==========================================================================
1. POPULATIONS
==========================================================================
all papers                                        5859
empirical (measured the world as it is)           5118
crawled  (crawlConfig != null OR studyType crawl) 1120
repeated (crawled AND temporal.snapshots > 1)     250  = 22.3% of crawled
single-shot or snapshot count unstated            870  = 77.7% of crawled
 
any paper (not only crawls) with snapshots > 1    818  = 14.0% of all papers
 
==========================================================================
2. IS THE MEASUREMENT PERIOD EVEN STATED
==========================================================================
 
-- temporal reporting fields
   population: 5118 empirical papers
   spanStart stated        2882   56.3%
   spanEnd stated          2888   56.4%
   both stated             2773   54.2%
   snapshots stated        1237   24.2%
   cadence stated          1304   25.5%
 
-- the same fields
   population: 1120 crawling papers
   spanStart stated         803   71.7%
   spanEnd stated           795   71.0%
   both stated              776   69.3%
   snapshots stated         409   36.5%
   cadence stated           389   34.7%
 
==========================================================================
3. HOW MANY WAVES, AND HOW FAR APART
==========================================================================
 
-- stated snapshot count, banded
   population: 1120 crawling papers
   not stated     711   63.5%
   1              159   14.2%
   2-3             93    8.3%
   4-6             41    3.7%
   7-12            33    2.9%
   13-52           46    4.1%
   >52             37    3.3%
   NOTE: 9 papers carry a snapshot count above 1,000. Spot-checking these finds
         page counts and record counts mis-extracted as snapshots, so the top band is
         unreliable as a count of waves. Bands are used on the page, never the raw number.
 
-- longest stated span, banded
   population: 250 repeated crawling papers
   <1 month                  35   14.0%
   1-2 months                35   14.0%
   3-5 months                41   16.4%
   6-11 months               26   10.4%
   1-2 years                 27   10.8%
   2-5 years                 23    9.2%
   5+ years                  18    7.2%
   no parsable end date      45   18.0%
 
-- cadence, folded (scripts/cadence_fold.mjs)
   population: 389 crawling papers that state any cadence
   daily                        102   26.2%
   duration-not-cadence          58   14.9%
   hourly                        46   11.8%
   monthly                       44   11.3%
   weekly                        43   11.1%
   irregular-or-unspecified      31    8.0%
   n-rounds-no-interval          29    7.5%
   minutes                       27    6.9%
   continuous                    27    6.9%
   every-few-days                20    5.1%
   unmapped                      16    4.1%
   sub-minute                     9    2.3%
   yearly                         9    2.3%
   quarterly                      4    1.0%
   fortnightly                    4    1.0%
   sentinel                       1    0.3%
 
   papers that fill the cadence slot but never state a cadence:
      88 / 389 = 22.6%   (counting 'unmapped' as a possible cadence, i.e. conservative)
 
   A paper can land in several families (it may state more than one cadence), so
   the column does not sum to 389. 'duration-not-cadence' is the field being used
   to state how long the study ran rather than how often it repeated — those papers
   have NOT stated a cadence and must not be counted as if they had.
 
   UNMAPPED RESIDUE — 16 distinct strings, printed in full:
      1 x  four increasing cumulative periods for threshold selection
      1 x  four APKs per year per browser family
      1 x  July and December
      1 x  15-minute trend monitoring
      1 x  one hour per website
      1 x  two six-month intervals per year for 2009–2019
      1 x  longitudinal measurement
      1 x  eleven-minute time slots
      1 x  Crawler B ran at the middle of each month
      1 x  irregular confirmation crawls
      1 x  collection cycles over six months
      1 x  during the day, paused overnight
      1 x  December 2023, May 2024, and December 2024
      1 x  sequential experimental windows
      1 x  four-month interval
      1 x  before and after the second rollout
 
==========================================================================
4. THE FOUR PINS: WHAT A REPEATED CRAWL HOLDS FIXED
==========================================================================
 
-- stated at all (an upper bound on holding it fixed)
   population: 250 repeated vs 870 other crawling papers
   pin                                              repeated   other crawls
   population list version stated                167 (66.8%)    493 (56.7%)
   browser named                                 143 (57.2%)    386 (44.4%)
      ... with a version number in the name        17 (6.8%)      49 (5.6%)
   vantage location stated                        89 (35.6%)    212 (24.4%)
   vantage infrastructure stated                 121 (48.4%)    324 (37.2%)
   statefulness stated                            70 (28.0%)    149 (17.1%)
   consent action stated                         105 (42.0%)    244 (28.0%)
   interaction depth stated                      208 (83.2%)    633 (72.8%)
   headless stated                                42 (16.8%)     98 (11.3%)
   authentication stated                         179 (71.6%)    600 (69.0%)
   span start AND end stated                     210 (84.0%)    566 (65.1%)
   cadence stated                                158 (63.2%)    231 (26.6%)
 
   all four (list version + browser + vantage + statefulness)
      repeated      23 / 250 = 9.2%
      other crawls  32 / 870 = 3.7%
   all four, with a browser VERSION rather than a browser name
      repeated      6 / 250 = 2.4%
   none of the four
      repeated      28 / 250 = 11.2%
 
==========================================================================
5. THE LONGER THE PANEL, THE LESS IT PINS
==========================================================================
 
-- by longest stated span
   population: 250 repeated crawling papers (45 have no parsable end date and are excluded)
   span >= 12 months  n=  68   list  53  browser  36  vantage  22  statefulness  14   all four 3 (4.4%)
   span <  12 months  n= 137   list  94  browser  84  vantage  55  statefulness  43   all four 18 (13.1%)
 
   Why the multi-year group pins less — its data provenance is different.
   temporal.mode, paper-counted, multi-valued:
   span >= 12 months  live-crawl 52 (76.5%), existing-dataset 27 (39.7%), web-archive 20 (29.4%), active-probing 15 (22.1%), passive-collection 12 (17.6%)
   span <  12 months  live-crawl 132 (96.4%), existing-dataset 35 (25.5%), active-probing 32 (23.4%), passive-collection 12 (8.8%), web-archive 3 (2.2%)
 
   Restricted to papers whose every temporal tuple is a live crawl or active probe
   (no archive, no reused dataset), so the pins were at least available:
   span >= 12 months  n=  16   all four 1 (6.3%)   << small n, report as a count
   span <  12 months  n=  94   all four 13 (13.8%)   << small n, report as a count
 
==========================================================================
6. IS REPEATING GETTING MORE COMMON
==========================================================================
 
-- share of crawling papers that repeat, by year
   population: crawling papers in each year
   2010   crawled   25   repeated   2    8.0%
   2011   crawled   22   repeated   2    9.1%
   2012   crawled   29   repeated   6   20.7%
   2013   crawled   26   repeated   6   23.1%
   2014   crawled   35   repeated   3    8.6%
   2015   crawled   41   repeated  12   29.3%
   2016   crawled   40   repeated  12   30.0%
   2017   crawled   51   repeated  16   31.4%
   2018   crawled   61   repeated  14   23.0%
   2019   crawled   97   repeated  23   23.7%
   2020   crawled   75   repeated  19   25.3%
   2021   crawled   75   repeated  17   22.7%
   2022   crawled  110   repeated  27   24.5%
   2023   crawled  125   repeated  20   16.0%
   2024   crawled  110   repeated  22   20.0%
   2025   crawled  129   repeated  34   26.4%   << provisional venue-year, under-represented by construction
   2026   crawled   69   repeated  15   21.7%   << provisional venue-year, under-represented by construction
 
   four-year buckets: 2010-2011: 4/47 = 8.5% | 2012-2015: 27/131 = 20.6% | 2016-2019: 65/249 = 26.1% | 2020-2023: 83/385 = 21.6% | 2024-2026: 71/308 = 23.1% (last bucket provisional)
 
==========================================================================
7. DATA PROVENANCE OF EMPIRICAL WORK
==========================================================================
 
-- temporal.mode
   population: 5118 empirical papers, multi-valued, paper-counted
   existing-dataset        2142   41.9%
   active-probing          1629   31.8%
   live-crawl              1236   24.2%
   passive-collection       988   19.3%
   not-stated               338    6.6%
   web-archive               65    1.3%
 
==========================================================================
8. FULL-TEXT PROBES OVER THE REPEATED-CRAWL PAPERS
==========================================================================
   Whitespace is collapsed before matching, because a PDF line break inside a
   phrase would otherwise hide it. Every probe is a MENTION count and therefore
   an upper bound; the two load-bearing ones were hand-read in full and the
   audited counts are printed beside them.
 
-- probe hits
   population: 250 repeated crawling papers, all with readable full text
   full text readable for 250 of 250
 
   self-describes as longitudinal                  123   49.2%
   re-crawl / repeat crawl / second crawl           42   16.8%
   wave / round / iteration terminology             54   21.6%
   comparable / comparability                       80   32.0%
   reproducib* / replicab* / replicat*              78   31.2%
   same version / list / snapshot / configuration   33   13.2%
   pinned / froze / fixed the version (wide)        13    5.2%
   Docker / container / VM image                    18    7.2%
   names a filter list or blocklist                 84   33.6%
   filter list pinned to a version or date           5    2.0%
   attrition or survivorship                         2    0.8%
   churn                                            19    7.6%
   <browser> <number> anywhere in the text          58   23.2%   -> hand-read all 58: 26 genuine (10.4% of the population)
   browser auto-update / version drift (wide)       14    5.6%   -> hand-read all 14: 0 genuine (0.0% of the population)
 
   The two audited probes, in words:
     * "<browser> <number>" is a candidate set, not an answer. Reading all of them,
       26 state the version of a browser the study's OWN measurement drove. The rest
       are bibliography entries, ecosystem history ("Chrome 43 shipped ..."), the
       version of a browser being measured rather than driven, and outright false
       positives (footnote markers, table cells, "Chrome browser 100 times").
     * The auto-update probe: reading all hits, NOT ONE discusses pinning or
       reporting the crawler's own browser version across waves. The closest is
       a note that measurement tools need maintenance as browsers change.
 
==========================================================================
9. PINNING THE LIST: DOES ANYONE CITE A PERMANENT TRANCO ID
==========================================================================
 
-- Tranco users
   population: 266 papers naming Tranco as a population source
   states any list version or date (extraction field)   159  59.8%
   cites a permanent Tranco list ID (hand-audited)      23  8.6%
 
   by year (Tranco users / of them, citing an ID):
      2020     7    1   14.3%
      2021    28    2    7.1%
      2022    29    2    6.9%
      2023    48    5   10.4%
      2024    56    4    7.1%
      2025    76    8   10.5%
      2026    22    1    4.5%
 
==========================================================================
10. EXEMPLARS — REPEATED CRAWLS THAT STATE ALL FOUR PINS
==========================================================================
   23 papers. Most recent first. These are the papers to copy a methods section from.
 
   2026 PETS  clicking-into-exposure-uncovering-privacy-risks-of-google-click-identifier-in-yo
        snapshots=5  cadence=five repeated iterations  longest span=0mo
        browsers=["Chrome","Firefox","Edge"]  statefulness=stateless  consent=accept-and-reject
        listVersion=["v3"]
        vantage=["Germany","Australia","India","Brazil","USA (California)","USA (New York)"]
 
   2026 PETS  privacy-vs-profit-the-impact-of-googles-manifest-version-3-mv3-update-on-ad-bloc
        snapshots=5  cadence=-  longest span=0mo
        browsers=["Google Chrome","Firefox"]  statefulness=stateless  consent=accept-all
        listVersion=["January 9, 2024","July 12, 2025"]
        vantage=["Europe"]
 
   2026 PETS  more-space-less-privacy-measuring-the-effectiveness-of-ip-based-website-fingerpr
        snapshots=5  cadence=15 days between batch starts / hourly refreshes  longest span=4mo
        browsers=["Google Chrome"]  statefulness=stateless  consent=no-interaction
        listVersion=["January 20, 2024","January 20, 2024","January 20, 2024","January 20, 2024"]
        vantage=["US"]
 
   2025 PETS  beyond-the-request-harnessing-http-response-headers-for-cross-browser-web-tracke
        snapshots=18  cadence=-  longest span=0mo
        browsers=["Chrome","Brave","Firefox"]  statefulness=stateful  consent=not-stated
        listVersion=["16th of May 2022","March 29th, 2023"]
        vantage=["Frankfurt, Germany"]
 
   2025 PETS  intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and
        snapshots=4  cadence=-  longest span=0mo
        browsers=["Firefox v121.0","Chrome"]  statefulness=both  consent=accept-and-reject
        listVersion=["ID K2NZW, generated 07 December 2023","ID K2NZW, generated 07 December 2023","ID K2NZW, generated 07 December 2023"]
        vantage=["EU"]
 
   2025 PETS  more-and-scammier-ads-the-perils-of-youtubes-ad-privacy-settings
        snapshots=6  cadence=at least one week between consecutive runs / at least 24 hours between runs  longest span=2mo
        browsers=["Google Chrome"]  statefulness=stateful  consent=not-stated
        listVersion=["data from September 9, 2022","March 14, 2024","November 2023","March 20, 2024 availability check"]
        vantage=["Sydney (Australia)","Toronto (Canada)","Dublin (Ireland)","London (the United Kingdom)","Oregon (the United States)"]
 
   2025 USENIX  navigating-cookie-consent-violations-across-the-globe
        snapshots=10  cadence=over one week  longest span=0mo
        browsers=["Google Chrome"]  statefulness=stateless  consent=reject-all
        listVersion=["November 2023, ID: 5Y3LN","July 2021, ID: 9QK2"]
        vantage=["Ireland","United Kingdom","California, USA","Michigan, USA","Canada","South Africa","Singapore","Australia"]
 
   2025 WWW  before-after-the-effect-of-eus-2022-code-of-practice-on-disinformation
        snapshots=2  cadence=-  longest span=24mo
        browsers=["Playwright-controlled browser"]  statefulness=stateful  consent=accept-all
        listVersion=["accessed October 2023","compiled October 2023 and earlier"]
        vantage=["Netherlands","Greece","USA"]
 
   2024 IEEE-SP  targeted-and-troublesome-tracking-and-advertising-on-childrens-websites
        snapshots=7  cadence=-  longest span=1mo
        browsers=["Puppeteer/TRC desktop browser","Puppeteer/TRC emulated mobile browser"]  statefulness=stateless  consent=accept-all
        listVersion=["May 2022","June-July 2022 crawl snapshot"]
        vantage=["Frankfurt","Amsterdam","London","San Francisco","New York City"]
 
   2024 PETS  the-devil-is-in-the-details-detection-measurement-and-lawfulness-of-server-side
        snapshots=3  cadence=-  longest span=26mo
        browsers=["Firefox"]  statefulness=stateless  consent=no-interaction
        listVersion=["2021"]
        vantage=["Europe","EU"]
 
   2024 PETS  a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users
        snapshots=3  cadence=temporally close repeated measurements  longest span=1mo
        browsers=["Firefox 100.0"]  statefulness=stateful  consent=accept-and-reject
        listVersion=["generated 29/08/2022; X568N","not-stated"]
        vantage=["Frankfurt am Main, DEU","Council Bluffs, IA, USA","Frankfurt, DEU","Paris, FRA","Stockholm, SWE"]
 
   2023 USENIX  a-study-of-multi-factor-and-risk-based-authentication-availability
        snapshots=2  cadence=two months between suspicious-login attempts  longest span=n/a
        browsers=["Chrome 89.0","Firefox 91.0","Safari 15.0"]  statefulness=stateless  consent=not-stated
        listVersion=["generated June 21, 2021"]
        vantage=["Boston, Massachusetts","Sofia, Bulgaria","San Francisco, California","Auckland, New Zealand","Dallas, Texas"]
 
   2022 IMC  respect-the-origin-a-best-case-evaluation-of-connection-coalescing-in-the-wild
        snapshots=2  cadence=-  longest span=5mo
        browsers=["Google Chrome v88.0"]  statefulness=stateless  consent=not-stated
        listVersion=["14 February 2021","14–18 February 2021","August 2021"]
        vantage=["East US","not-stated"]
 
   2022 USENIX  a-large-scale-investigation-into-geodifferences-in-mobile-apps
        snapshots=31  cadence=daily  longest span=0mo
        browsers=["Selenium"]  statefulness=stateful  consent=no-interaction
        listVersion=["June 2020"]
        vantage=["Canada","Germany","USA","UK","Australia","Japan","Hungary","Kenya","Colombia","South Korea","Tunisia","Mexico","Singapore","Ukraine","India","Zimbabwe","Turkey","Russia","Venezuela","Bahrain","UAE","Egypt","Iran","Hong Kong","Ireland","Israel","not-stated"]
 
   2022 WWW  reproducibility-and-replicability-of-web-measurement-studies
        snapshots=12  cadence=daily  longest span=60mo
        browsers=["Chrome","Chrome headless","Firefox","Firefox headless"]  statefulness=stateless  consent=no-interaction
        listVersion=["2016-2021","2016-2021","2021-06-23","2021-06-24"]
        vantage=["Germany (EU)","Japan (AS)","United States (NA)"]
 
   2020 NDSS  encrypted-dns-privacy-a-traffic-analysis-perspective
        snapshots=5  cadence=two-week intervals  longest span=3mo
        browsers=["Firefox","Chrome"]  statefulness=stateless  consent=not-stated
        listVersion=["26 March 2018","26 March 2018","accessed 2019-09-11"]
        vantage=["Lausanne","Leuven","Singapore"]
 
   2020 PETS  missed-by-filter-lists-detecting-unknown-third-party-trackers-with-invisible-pix
        snapshots=6  cadence=simultaneous crawls / simultaneous measurements  longest span=0mo
        browsers=["Firefox"]  statefulness=stateful  consent=no-interaction
        listVersion=["February 2019","November 2019","November 2019"]
        vantage=["France"]
 
   2020 WWW  beyond-the-front-page-measuring-third-party-dynamics-in-the-field
        snapshots=3  cadence=-  longest span=0mo
        browsers=["Firefox 52"]  statefulness=stateful  consent=no-interaction
        listVersion=["2019-03-26, ID W9L9"]
        vantage=["Europe (DE)","North America (US)","Asia (JP)"]
 
   2020 WWW  the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing
        snapshots=2  cadence=variable / two seven-day periods separated by seven days  longest span=3mo
        browsers=["Firefox 68","Firefox 69"]  statefulness=stateless  consent=not-stated
        listVersion=["2019; accessed 29-July-2019","top 10,000; accessed 29-July-2019","top 10,000","data collected July 16–August 13, 2019"]
        vantage=["not-stated","Brazil","Texas, USA"]
 
   2019 WWW  measuring-political-personalization-of-google-news-search
        snapshots=7  cadence=daily  longest span=n/a
        browsers=["Firefox"]  statefulness=stateful  consent=not-stated
        listVersion=["two weeks in March 2017"]
        vantage=["United States","Oregon"]
 
   2017 NDSS  thou-shalt-not-depend-on-me-analysing-the-use-of-outdated-javascript-libraries-o
        snapshots=2  cadence=-  longest span=0mo
        browsers=["Chromium"]  statefulness=stateless  consent=no-interaction
        listVersion=["May 2016"]
        vantage=["US"]
 
   2016 IEEE-SP  cloak-of-visibility-detecting-when-machines-browse-a-different-web
        snapshots=3  cadence=-  longest span=3mo
        browsers=["Chrome","basic HTTP fetcher"]  statefulness=stateless  consent=no-interaction
        listVersion=["collected February, 2015-May, 2015"]
        vantage=["not-stated","United States"]
 
   2016 USENIX  tracing-information-flows-between-ad-exchanges-using-retargeted-ads
        snapshots=9  cadence=-  longest span=0mo
        browsers=["instrumented Chromium"]  statefulness=stateful  consent=not-stated
        listVersion=["January 2015"]
        vantage=["Boston, United States"]
 
   -- compact table (year | venue | slug | snapshots | distinct stated vantage locations)
   2026 | PETS | clicking-into-exposure-uncovering-privacy-risks-of-google-click-identifier-in-yo | 5 | 6
   2026 | PETS | privacy-vs-profit-the-impact-of-googles-manifest-version-3-mv3-update-on-ad-bloc | 5 | 1
   2026 | PETS | more-space-less-privacy-measuring-the-effectiveness-of-ip-based-website-fingerpr | 5 | 1
   2025 | PETS | beyond-the-request-harnessing-http-response-headers-for-cross-browser-web-tracke | 18 | 1
   2025 | PETS | intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and | 4 | 1
   2025 | PETS | more-and-scammier-ads-the-perils-of-youtubes-ad-privacy-settings | 6 | 5
   2025 | USENIX | navigating-cookie-consent-violations-across-the-globe | 10 | 8
   2025 | WWW | before-after-the-effect-of-eus-2022-code-of-practice-on-disinformation | 2 | 3
   2024 | IEEE-SP | targeted-and-troublesome-tracking-and-advertising-on-childrens-websites | 7 | 5
   2024 | PETS | the-devil-is-in-the-details-detection-measurement-and-lawfulness-of-server-side | 3 | 2
   2024 | PETS | a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users | 3 | 5
   2023 | USENIX | a-study-of-multi-factor-and-risk-based-authentication-availability | 2 | 5
   2022 | IMC | respect-the-origin-a-best-case-evaluation-of-connection-coalescing-in-the-wild | 2 | 1
   2022 | USENIX | a-large-scale-investigation-into-geodifferences-in-mobile-apps | 31 | 26
   2022 | WWW | reproducibility-and-replicability-of-web-measurement-studies | 12 | 3
   2020 | NDSS | encrypted-dns-privacy-a-traffic-analysis-perspective | 5 | 3
   2020 | PETS | missed-by-filter-lists-detecting-unknown-third-party-trackers-with-invisible-pix | 6 | 1
   2020 | WWW | beyond-the-front-page-measuring-third-party-dynamics-in-the-field | 3 | 3
   2020 | WWW | the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing | 2 | 2
   2019 | WWW | measuring-political-personalization-of-google-news-search | 7 | 2
   2017 | NDSS | thou-shalt-not-depend-on-me-analysing-the-use-of-outdated-javascript-libraries-o | 2 | 1
   2016 | IEEE-SP | cloak-of-visibility-detecting-when-machines-browse-a-different-web | 3 | 1
   2016 | USENIX | tracing-information-flows-between-ad-exchanges-using-retargeted-ads | 9 | 1
 
 
==========================================================================
11. CORPUS-WIDE ADOPTION PROBES FOR THE MODERN PINNING MECHANISMS
==========================================================================
 
-- mentions anywhere in the full text
   population: 5855 papers with readable full text, of 5859
   Docker OR containerised                                     308    5.3%
                                                              /\bdocker\b|\bcontaineri[sz]ed\b/i
      ...Docker alone                                          293    5.0%
                                                              /\bdocker\b/i
   image digest or sha256: image reference                       0    0.0%
                                                              /\bimage digest\b|\bsha256:[0-9a-f]{12}/i
   "Chrome for Testing", exact phrase                            0    0.0%
                                                              /chrome[-\s]?for[-\s]?testing/i
   Chrome for Testing, widened to the tooling and the acronym   10    0.2%   -> hand-read all 10: 0 genuine
                                                              /chrome[-\s]?for[-\s]?testing|googlechromelabs|chrome[-\s]?labs|known[-\s]good[-\s]versions|\bCfT\b/i
   WebREC or "web execution bundle"                              2    0.0%   -> hand-read all 2: 0 genuine
                                                              /\bWebREC\b|\bweb execution bundle/i
 
   Hand audits:
     * The widened Chrome-for-Testing probe: every hit is a false positive —
       CFT as "Combating the Financing of Terrorism" (three papers), "Control Flow
       Trimming", "Call Flow Tree", "Crash-Fault Tolerant", a certificate subject
       O=CFT, a CFt variable in an algorithm listing, CFT-IDF, and a fitness app
       named "CFT & PFT". The exact-phrase probe is 0, and so is the audited count.
     * WebREC: one hit is the WebREC paper itself; the other cites it in its
       reference list only. No paper in the corpus USES it.

PARTIAL  2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "Over twelve days, we saw a variation of up to 27% (max on day 3-80,274; min on day 9-58,951) in observed track..."
         paper.cols.txt — 4 run(s): 6+2+12+3 words
 
EXACT    2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "the number of distinct tracking domains remains almost stable during the experiment (variation of 3.5%)"
         paper.cols.txt
 
PARTIAL  2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "studies that analyze the ecosystem will find similar results, while studies that aim to analyze the extent of ..."
         paper.cols.txt — 4 run(s): 6+11+10+3 words
 
EXACT    2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "we find that the identified trackers on pages can vary by 25% based on the used browser configuration"
         paper.cols.txt
 
SPLICED  2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "even the same setup measures different levels of tracking on different days, which might lead to different con..."
         paper.cols.txt — 3 run(s): 6+11+4 words
 
EXACT    2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "We use the popular Open-WPM Framework [21] (v0.15.0 - Firefox version 88)"
         paper.cols.txt
 
PARTIAL  2023/IMC/on-the-similarity-of-web-measurements-under-different-experimental-setups
         "when comparing two different profiles, 48% of the underlying data varies"
         paper.cols.txt — 2 run(s): 2+9 words
 
PARTIAL  2026/PETS/overcoming-language-barriers-multilingual-analysis-of-the-2023-swiss-privacy-law
         "we restrict the analysis to the balanced panel of websites with an observed policy in both snapshots"
         paper.cols.txt — 3 run(s): 6+10+1 words
 
SPLICED  2026/PETS/overcoming-language-barriers-multilingual-analysis-of-the-2023-swiss-privacy-law
         "Standard errors are clustered at the website level to account for repeated observations of the same policy acr..."
         paper.cols.txt — 2 run(s): 11+8 words
 
PARTIAL  2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "The standard deviation of such requests is 8,203."
         paper.cols.txt — 2 run(s): 7+1 words
 
EXACT    2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "For our continuous measurement, we visit the top 1k sites from our website corpus (18,377 distinct pages) dail..."
         paper.cols.txt
 
PARTIAL  2022/WWW/reproducibility-and-replicability-of-web-measurement-studies
         "privacy measurements and analyses can vary up to 65% depending on the region"
         paper.cols.txt — 3 run(s): 2+10+1 words
 
EXACT    2023/IMC/on-the-similarity-of-web-measurements-under-different-experimental-setups
         "only 32% of the cookies appear in all profiles and 42% only in one profile"
         paper.cols.txt
 
SPLICED  2021/WWW/towards-realistic-and-reproducibleweb-crawl-measurements
         "Around 5% of content-providing domains show significant measurement bias across VP"
         paper.cols.txt — 2 run(s): 6+5 words
 
EXACT    2020/WWW/apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web
         "variation of over 16% in the number of successful page loads"
         paper.cols.txt
 
PARTIAL  2026/PETS/overcoming-language-barriers-multilingual-analysis-of-the-2023-swiss-privacy-law
         "The balanced panel comprises N = 11,800 websites observed in both August and October 2023"
         paper.cols.txt — 3 run(s): 4+1+10 words
 
EXACT    2025/USENIX/web-execution-bundles-reproducible-accurate-and-archivable-web-measurements
         "could be conducted using WebREC as is"
         paper.cols.txt
 
EXACT    2026/PETS/the-empire-strikes-back-at-your-privacy-an-archaeology-of-tracking-on-government
         "We therefore label trackers using the June 2025 EasyList and EasyPrivacy snapshot for all years, keeping the l..."
         paper.cols.txt
 
7 quote(s) need a hand read

Four reviewers, all told explicitly that the author's context might not be exhaustive, all handed the page text, the report script and its unedited output, and these notes. The three focused passes ran in parallel first; the generic pass ran afterwards, on the corrected text.

← back to the content page · corpus-level provenance