Measure The Web

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Provenance: programming:crawler:openwpm ======

The working notes behind [[programming:crawler:openwpm]]: every query with its denominator, every command that was run, what was folded, which quotes were checked, which external sources were verified and which were rejected, and what could not be established. Corpus-level caveats — the seven venues, the selection funnel, what the extraction is and is not — are on [[literature:corpus]] and are not repeated here.

Voice here is a working log, not prose. It is read by someone checking a number.

**Run:** 2026-08-14, against ''data/extract/run1'' at 5,859 extracted papers, and against the OpenWPM repository at commit ''b9dd4c3'' (''VERSION'' 0.35.0). One agent (Claude Opus 5) did the corpus work, the repository reading and the install attempts; four review passes are logged in §11.

**New page, not an extension.** Three neighbours could have absorbed this material and none should:

  * [[programming:crawler]] already has a one-row summary of OpenWPM in its //Specialised Measurement Crawlers// table and names this page as its child. It compares tools; it cannot also carry a tool's configuration surface.
  * [[programming:stateful_stateless]] owns the //design// question of whether to keep browser state. This page covers only how OpenWPM implements the two modes, and links there for the choice. The measured statefulness-reporting figures for OpenWPM papers are here because they are about the tool; the field-wide picture belongs there.
  * [[privacy:fingerprinting]] owns what is worth instrumenting. This page says what OpenWPM's default collection does and does not reach, and links there.

===== 1. Populations and denominators =====

^ Name ^ Definition ^ N ^
| OPENWPM | a paper whose ''tools[]'' contains a name matching ''/open[\s-]?wpm/i'' with ''usedOrMentioned'' in {''used'', ''produced''} | **60** |
| any mention | the same regex, any ''usedOrMentioned'' | 63 |
| CRAWLED | ''lib.mjs'' ''POPULATIONS.crawled'': a ''crawlConfig'' record exists, or ''studyTypes'' includes ''automated-web-crawl'' | 1,120 |
| OTHER_CRAWLED | CRAWLED minus OPENWPM — the baseline for every reporting-rate comparison | 1,061 |
| full text available | OPENWPM papers with a ''paper.cols.txt'' on disk | 60 of 60 |

Never "of 5,859 papers". Every table on the content page names one of these.

**Why ''produced'' counts.** Three papers produced rather than used: OpenWPM itself {[englehardt2016online]}, OpenWPM-Mobile {[das2018_sixth]}, and one paper's "custom OpenWPM instrumentation" {[acar2020_noboundaries]} — so the rule's real effect is keeping the tool's own paper in the population. A paper that forked the crawler crawled with the fork, so it belongs in a population about who crawls with OpenWPM. The three papers that only ''compared'' or ''mentioned'' it are excluded and named in the report (§4).

**The one paper inside OPENWPM and outside CRAWLED:** ''PETS/2021/a-calculus-of-tracking-theory-and-practice'', ''studyTypes'' ''["existing-dataset-analysis","system-or-defence-proposal"]'' — it re-analyses data someone else crawled with OpenWPM. Left in OPENWPM (it is a paper whose results depend on OpenWPM's instrumentation) and it is why 59, not 60, of the OpenWPM papers are in CRAWLED.

===== 2. The fold =====

**One regex, no families.** ''/open[\s-]?wpm/i'' over ''tools[].name''. Every spelling in the corpus is a case or hyphen variant of one token:

^ Spelling in ''tools[].name'' ^ Tuples ^
| ''OpenWPM'' | 60 |
| ''OpenWPM-Mobile'' | 2 |
| ''OpenWPM Firefox extension'' | 1 |
| ''custom OpenWPM instrumentation'' | 1 |
| ''openWPM'' | 1 |

**Residue is zero by construction, and that is a weakness, not a strength.** A fork published under a name that does not contain the string "OpenWPM" would be invisible to this population, and would show up as a paper missing rather than as a wrong number. This is why the report prints the spelling table: it is the only place a new spelling can be noticed. Known risk cases that are //not// in the population: BannerClick (in it, because {[rasaii2025_crumbs]} names OpenWPM too), and anything built on OpenWPM whose paper names only the derivative.

**Not folded:** ''version'' strings. They are reported verbatim on the content page. Folding "0.14.0 and 0.19.1" to one version would have destroyed the finding that a paper used two.

**Sanity check on the fold:** ''tools[].category'' for the 60 papers' OpenWPM tuples is ''crawler-framework'' 60, ''browser'' 1, ''program-analysis'' 1 — no tuple where the regex caught something that is not the crawler.

===== 3. Every query, and where each page figure comes from =====

Everything on the content page's //Use in Publications// section, plus the statefulness and artefact figures used earlier in the page, is printed by one script with its denominator. Reproduce with:

<code bash>
node scripts/report_openwpm.mjs                    > out/openwpm-output.txt
node scripts/report_openwpm.mjs --wiki             # DokuWiki tables
node scripts/report_openwpm.mjs --list             # the 60 papers with versions
node scripts/report_openwpm.mjs --quotes openwpm   # evidence quotes matching a regex
node scripts/report_openwpm.mjs --hits js_instrument   # full-text lines behind a sweep token
node scripts/check_page_numbers.mjs pages/programming_crawler_openwpm.txt out/openwpm-output.txt
node scripts/check_page_numbers.mjs pages/programming_crawler_openwpm.txt out/openwpm-output.txt --code
node scripts/check_tables.mjs pages/programming_crawler_openwpm.txt
</code>

Section map, so a figure on the page can be traced without reading the whole output:

^ Report section ^ What it answers ^ Page section ^
| A | population, spellings, ''usedOrMentioned'', category check | //Methodology and limitations// |
| B | venue and year shape | //Who uses it// |
| C | version reporting, release dates, bundled Firefox, lag | //An OpenWPM version is a Firefox version, by default//, //Which version, and how far behind// |
| D | crawl-configuration reporting rates vs OTHER_CRAWLED, statefulness values | //Stateful and stateless//, //What papers say about the configuration// |
| E | full-text sweep: OpenWPM identifiers, prose descriptions, statefulness cross-check | //Almost nobody says which instruments were on// |
| F | ''classification[].target'' enum, ''detection[].phenomenon'' ranking | //What they use it for// |
| G | artefact availability vs OTHER_CRAWLED | //Forks and extensions// |
| I | every OpenWPM tool tuple with purpose and quote | the quote checks in §5 |
| Y | figures the page derives by arithmetic (the 3.1× and 3.6× ratios, 19.1%) | //What papers say about the configuration// |
| Z | every number on the page that is **not** from the corpus, with its source | throughout |

==== 3.1 The full-text sweep, which is not in the extraction schema ====

The extraction records //that// a paper used OpenWPM. It does not record which instruments were enabled — there is no field for it. That question is answerable only from the paper's own text, so the report sweeps ''paper.cols.txt'' for all 60 papers, in two deliberately separate classes:

  * **Class 1, identifiers.** ''http_instrument'', ''js_instrument'', ''cookie_instrument'', ''navigation_instrument'', ''dns_instrument'', ''callstack_instrument'', ''save_content'', ''bot_mitigation'', ''display_mode'', ''num_browsers'', ''seed_tar''/''profile_archive_dir'', ''collection_fingerprinting'', and the table names ''http_requests''/''http_responses'', ''javascript_cookies'', ''incomplete_visits'', ''crawl_history'', ''site_visits''. These strings exist nowhere else, so a hit is unambiguous. **3 of 60 papers hit any of them.** The three are named in report §E.
  * **Class 2, prose.** ''stateful'', ''stateless'', ''headless'', ''xvfb'', "fresh/clean/new profile", "cookie jar", SQLite/Parquet, LevelDB, "call stack", and the descriptions of collected data ("HTTP requests", "JavaScript API calls", …). Every one of these has ordinary English uses, so **every class-2 count on the page is labelled an upper bound.** Two regexes were tightened during the run for exactly this reason: ''\bdns\b'' matched 26 papers, most of them about something else, and was narrowed to ''dns (resolution|response|quer|record)|resolved (the )?(hostname|domain)'' → 14; ''\bnavigations?\b'' matched 18 and was narrowed to a "recorded/captured/logged navigations" form → 2. The tightened versions are what the page publishes. Neither number is load-bearing.

Both classes are lower bounds in one further way: the two-column repair splices the neighbouring column into sentences, so a token can be cut in half (see §5).

==== 3.2 The statefulness cross-check ====

Statefulness is the page's own subject, so the schema field and the text sweep were crossed rather than either being trusted:

  * schema says stated: 33
  * text contains ''stateful''/''stateless'': 32
  * both: 23
  * schema only (the paper described the mode in words the sweep does not match): 10
  * sweep only (the word appears; the extractor read no value): 9, listed in report §E

The two signals disagree in **both** directions, which is the reason the page quotes the schema figure (33 of 60) and not a union.

==== 3.3 The version-lag column, and its limits ====

''RELEASED'' in ''report_openwpm.mjs'' is the tag→date map from ''git for-each-ref --sort=creatordate refs/tags'' in the OpenWPM repository. ''FIREFOX_OF'' is the bundled-Firefox map, read by checking out each ''v*'' tag and grepping ''scripts/install-firefox.sh'' for its ''FIREFOX_<major>_<minor>_RELEASE'' comment.

Three caveats, all of which are on the page in weaker form:

  * **Tag date ≠ release-publication date.** GitHub's releases API gives v0.30.0 as 2024-10-06 and v0.29.0 as 2024-07-20, where the tags are 2024-10-02 and 2024-07-15. The page uses **tag creatordate** throughout, consistently. Nothing on the page turns on a difference of days.
  * **"Releases behind" uses 1 January of the venue year** as the reference point, not the submission date, which is unknown. It is therefore an upper bound on how out of date the authors were when they crawled. The page says this.
  * **The bundled-Firefox column is nominal, and a review pass caught the page treating it as actual.** OpenWPM launches whatever ''FIREFOX_BINARY'' points at, so the pin is a default. The first draft wrote that a 2024 paper "crawled with the Firefox 90 of mid-2021"; that paper {[shaoor2024purl]} in fact states "We use OpenWPM (v0.17.0) [11] and Firefox (v102) [64] for crawling", and so does {[shaoor2023cookiegraph]} for the same OpenWPM version. Both were read in the source. ''report_openwpm.mjs'' now carries a **hand map** (''FIREFOX_STATED'') of the papers that state a crawl browser version, each with the sentence that decided it — except the {[ahmad2020_apophanies]} entry, whose value comes from the extraction's own version string rather than from a sentence read in the paper, and which is marked as such, because a regex cannot do it: ''/Firefox v?\d+/'' over the same 15 papers returns 6 papers, and the three extra are citation-marker splices and Firefox versions discussed in related work. 3 of 15 state it; 2 of those 3 ran a **newer** browser than their release ships.
  * **Three stated versions have no tag** (''0.1.0'', ''0.9.0'', ''0.13''), so their rows carry "—" rather than a guess. ''0.1.0'' predates the first tag (v0.2.0, 2014-08-12); ''0.9.0'' was never tagged (the sequence goes 0.8.0 → 0.10.0); ''0.13'' is presumably 0.13.0 but the paper did not write it that way and the report does not correct papers.

===== 4. The report script and its unedited output =====

<file javascript report_openwpm.mjs>
// Every figure on Programming:Crawler:OpenWPM, with its denominator printed
// next to it.
//
//   node scripts/report_openwpm.mjs              > out/openwpm-output.txt
//   node scripts/report_openwpm.mjs --wiki       DokuWiki tables
//   node scripts/report_openwpm.mjs --list       the 60 papers
//   node scripts/report_openwpm.mjs --quotes <re>  evidence behind a match
//   node scripts/report_openwpm.mjs --hits <re>  full-text lines behind a sweep token
//
// Rules (data/extract/README.md), enforced here rather than remembered:
//   * every table names its own population; "of 5,859 papers" is never used
//   * not-stated / none-mentioned are reported as themselves, never as answers
//   * papers are counted, never tuples
//   * free-text names are folded, and the residue is printed
//
// Two populations are used and they are not the same thing:
//   OPENWPM  the 60 papers whose tools[] names OpenWPM with usedOrMentioned in
//            {used, produced}. 'produced' counts because the OpenWPM paper and
//            its forks crawled with what they built.
//   CRAWLED  the 1,120-paper crawling population from lib.mjs, used only as the
//            comparison baseline for reporting rates.
//
// The full-text sweep in section D is the part that is not in the schema. The
// extraction records that a paper used OpenWPM; it does not record which
// instruments were switched on. Only the paper's own text says that, and mostly
// it does not say it either — which is the finding.

import fs from 'node:fs';
import path from 'node:path';
import {
  loadExtractions,
  dataRoot,
  POPULATIONS,
  pct,
  table,
  wikiTable,
  isSentinel,
} from './lib.mjs';

const WIKI = process.argv.includes('--wiki');
const LIST = process.argv.includes('--list');
const QUOTES = process.argv.includes('--quotes')
  ? new RegExp(process.argv[process.argv.indexOf('--quotes') + 1], 'i')
  : null;
const HITS = process.argv.includes('--hits')
  ? process.argv[process.argv.indexOf('--hits') + 1]
  : null;
const T = WIKI ? wikiTable : table;

const rows = loadExtractions();
const key = (p) => `${p.venue}/${p.year}/${p.slug}`;

// ------------------------------------------------------------------ the fold
// One regex, because every spelling in this corpus is a case or hyphen variant
// of one token. Checked by printing every distinct matching string below: if a
// spelling ever appears that this does not catch (a fork under a new name, say),
// it shows up as a paper missing from the population, not as a wrong number, so
// section A prints the strings for exactly that reason.
const OPENWPM_RE = /open[\s-]?wpm/i;
const DROVE_IT = new Set(['used', 'produced']);

const openwpmTuples = (p) => (p.tools ?? []).filter((t) => OPENWPM_RE.test(t.name ?? ''));
const usedIt = (p) => openwpmTuples(p).some((t) => DROVE_IT.has(t.usedOrMentioned));

const OPENWPM = rows.filter(usedIt);
const N = OPENWPM.length;
const CRAWLED = rows.filter(POPULATIONS.crawled);
const OTHER_CRAWLED = CRAWLED.filter((p) => !usedIt(p));

const out = [];
const h = (s) => out.push('', WIKI ? `==== ${s} ====` : `\n=== ${s} ===`, '');
const p = (s) => out.push(s);

p(
  WIKI
    ? ''
    : `corpus: ${rows.length} papers, 7 venues, 2010-2026
population OPENWPM: ${N} papers name OpenWPM in tools[] with usedOrMentioned in {used, produced}
population CRAWLED: ${CRAWLED.length} papers (baseline; ${OTHER_CRAWLED.length} of them are not OpenWPM papers)
generated by scripts/report_openwpm.mjs`
);

// =========================================================== A. the population
h('A. Population and how OpenWPM is spelled');
{
  const spell = new Map();
  const stance = new Map();
  for (const x of rows) {
    for (const t of openwpmTuples(x)) {
      spell.set(t.name, (spell.get(t.name) ?? 0) + 1);
      stance.set(t.usedOrMentioned, (stance.get(t.usedOrMentioned) ?? 0) + 1);
    }
  }
  p(
    T(
      ['Spelling in tools[].name', 'Tuples'],
      [...spell.entries()].sort((a, b) => b[1] - a[1]).map(([k, v]) => [k, v])
    )
  );
  p('');
  p(
    T(
      ['usedOrMentioned', 'Tuples'],
      [...stance.entries()].sort((a, b) => b[1] - a[1]).map(([k, v]) => [k, v])
    )
  );
  const anyMention = rows.filter((x) => openwpmTuples(x).length > 0);
  const onlyCiting = anyMention.filter((x) => !usedIt(x));
  p('');
  p(`Papers with any OpenWPM tuple: ${anyMention.length}`);
  p(`  of which used or produced it: ${N}`);
  p(`  of which only compared or mentioned it: ${onlyCiting.length}`);
  p(`    ${onlyCiting.map(key).join('\n    ')}`);
  p('');
  const inCrawled = OPENWPM.filter(POPULATIONS.crawled).length;
  p(
    `Inside the CRAWLED population: ${inCrawled} of ${N} (${pct(inCrawled, N)}).` +
      ` The ${N - inCrawled} outside it have no crawlConfig record and no automated-web-crawl study type.`
  );
  for (const x of OPENWPM.filter((y) => !POPULATIONS.crawled(y)))
    p(`  outside CRAWLED: ${key(x)} — studyTypes ${JSON.stringify(x.studyTypes)}`);
  // Category the extraction filed it under, as a check that the fold is not
  // picking up something other than a crawler.
  const cats = new Map();
  for (const x of OPENWPM)
    for (const t of openwpmTuples(x)) cats.set(t.category, (cats.get(t.category) ?? 0) + 1);
  p('');
  p(
    T(
      ['tools[].category', 'Tuples'],
      [...cats.entries()].sort((a, b) => b[1] - a[1]).map(([k, v]) => [k, v])
    )
  );
}

// ============================================================ B. shape in time
h('B. Where the 60 papers are');
{
  const byVenue = new Map();
  for (const x of OPENWPM) byVenue.set(x.venue, (byVenue.get(x.venue) ?? 0) + 1);
  const venueTotals = new Map();
  for (const x of CRAWLED) venueTotals.set(x.venue, (venueTotals.get(x.venue) ?? 0) + 1);
  p(
    T(
      ['Venue', 'OpenWPM papers', `Share of ${N}`, 'Crawling papers at that venue', 'OpenWPM share of those'],
      [...byVenue.entries()]
        .sort((a, b) => b[1] - a[1])
        .map(([v, c]) => [v, c, pct(c, N), venueTotals.get(v) ?? 0, pct(c, venueTotals.get(v) ?? 0)])
    )
  );

  const years = [...new Set(rows.map((x) => x.year))].sort();
  p('');
  p(
    T(
      ['Year', 'OpenWPM papers', 'Crawling papers', 'OpenWPM share of crawling papers'],
      years
        .filter((y) => y >= 2015)
        .map((y) => {
          const a = OPENWPM.filter((x) => x.year === y).length;
          const b = CRAWLED.filter((x) => x.year === y).length;
          return [y >= 2025 ? `${y}*` : `${y}`, a, b, pct(a, b)];
        })
    )
  );
  p('');
  p('* 2025 and 2026 are provisional venue-years: CCS 2026 and IMC 2026 have not');
  p('  been held, and IEEE S&P 2026 / WWW 2026 abstracts are not in the selection');
  p('  source, so those years are under-represented by construction.');
}

// ======================================================= C. version reporting
h('C. Which version, and how far behind it was');
{
  // Release dates come from `git for-each-ref refs/tags` in the OpenWPM
  // repository, read on 2026-08-14 at commit b9dd4c3. Kept here as data so the
  // lag column can be recomputed; there is no API for it in the corpus.
  const RELEASED = {
    '0.1.0': null, // no such tag; earliest tag is v0.2.0, 2014-08-12
    '0.2.0': '2014-08-12',
    '0.2.1': '2014-12-31',
    '0.2.2': '2015-01-05',
    '0.2.3': '2015-06-11',
    '0.3.0': '2015-06-11',
    '0.3.1': '2015-06-11',
    '0.4.0': '2015-06-12',
    '0.5.0': '2015-10-14',
    '0.5.1': '2015-10-15',
    '0.6.0': '2015-12-22',
    '0.6.1': '2016-04-08',
    '0.6.2': '2016-04-08',
    '0.7.0': '2016-11-15',
    '0.8.0': '2017-10-09',
    '0.10.0': '2020-06-22',
    '0.11.0': '2020-07-08',
    '0.12.0': '2020-08-26',
    '0.13.0': '2020-11-19',
    '0.14.0': '2021-03-16',
    '0.14.1': '2021-03-29',
    '0.15.0': '2021-05-10',
    '0.16.0': '2021-06-10',
    '0.17.0': '2021-07-24',
    '0.18.0': '2021-12-12',
    '0.19.0': '2022-03-23',
    '0.19.1': '2022-03-31',
    '0.20.0': '2022-05-18',
    '0.21.0': '2022-09-27',
    '0.21.1': '2022-10-13',
    '0.22.0': '2023-06-25',
    '0.23.0': '2023-08-03',
    '0.24.0': '2023-09-05',
    '0.25.0': '2023-10-21',
    '0.26.0': '2023-12-24',
    '0.27.0': '2024-02-08',
    '0.28.0': '2024-02-21',
    '0.29.0': '2024-07-15',
    '0.30.0': '2024-10-02',
    '0.31.0': '2025-01-19',
    '0.32.0': '2026-03-03',
    '0.33.0': '2026-03-28',
    '0.34.0': '2026-05-08',
    '0.35.0': '2026-06-17',
  };
  // Which Firefox each release bundles, read out of scripts/install-firefox.sh
  // at every v* tag (the TAG comment names FIREFOX_<major>_<minor>_RELEASE).
  // Releases before v0.10.0 pinned Firefox differently and are '?' here.
  const FIREFOX_OF = {
    '0.10.0': 77, '0.11.0': 78, '0.12.0': 80, '0.13.0': 83, '0.14.0': 86,
    '0.14.1': 86, '0.15.0': 88, '0.16.0': 89, '0.17.0': 90, '0.18.0': 95,
    '0.19.0': 98, '0.19.1': 98, '0.20.0': 100, '0.21.0': 105, '0.21.1': 105,
    '0.22.0': 114, '0.23.0': 115, '0.24.0': 117, '0.25.0': 118, '0.26.0': 121,
    '0.27.0': 122, '0.28.0': 123, '0.29.0': 128, '0.30.0': 130, '0.31.0': 134,
    '0.32.0': 148, '0.33.0': 149, '0.34.0': 150, '0.35.0': 152,
  };
  const stated = [];
  for (const x of OPENWPM) {
    for (const t of openwpmTuples(x)) {
      if (!DROVE_IT.has(t.usedOrMentioned)) continue;
      if (isSentinel(t.version) || t.version === null) continue;
      stated.push([x, t.version]);
    }
  }
  const statedPapers = new Set(stated.map(([x]) => key(x)));
  p(`Papers stating any OpenWPM version: ${statedPapers.size} of ${N} (${pct(statedPapers.size, N)}).`);
  p('Version strings are free text; they are reported verbatim, never folded.');
  p('');
  // For each stated version, what was the newest release when the paper's venue
  // year began. Deliberately crude — the venue year, not a submission date — and
  // labelled as such wherever it is used.
  const normalise = (v) => (v.match(/\d+\.\d+(\.\d+)?/) ?? [null])[0];
  const releaseList = Object.entries(RELEASED)
    .filter(([, d]) => d)
    .sort((a, b) => a[1].localeCompare(b[1]));
  p(
    T(
      ['Paper', 'Version as written', 'That version released', 'Firefox it bundles', 'Newest release on 1 Jan of the paper year', 'Releases behind'],
      stated
        .sort((a, b) => a[0].year - b[0].year)
        .map(([x, v]) => {
          const nv = normalise(v);
          const rel = RELEASED[nv] ?? '(no tag)';
          const cutoff = `${x.year}-01-01`;
          const newest = releaseList.filter(([, d]) => d < cutoff).pop();
          const idxOf = (ver) => releaseList.findIndex(([k]) => k === ver);
          const behind =
            nv && RELEASED[nv] && newest ? Math.max(0, idxOf(newest[0]) - idxOf(nv)) : '—';
          return [
            key(x),
            v,
            rel,
            FIREFOX_OF[nv] ? `Firefox ${FIREFOX_OF[nv]}` : '?',
            newest ? `${newest[0]} (${newest[1]})` : '—',
            behind,
          ];
        })
    )
  );
  p('');
  p('Every OpenWPM release pins one unbranded Firefox build, so citing an OpenWPM');
  p('version pins a browser version too. The mapping, read out of');
  p('scripts/install-firefox.sh at each v* tag:');
  p(
    T(
      ['OpenWPM release', 'Tagged', 'Bundled Firefox'],
      Object.entries(FIREFOX_OF).map(([v, ff]) => [v, RELEASED[v] ?? '?', `Firefox ${ff}`])
    )
  );
  // Does the paper also say which BROWSER it ran? A regex sweep cannot answer
  // this: /Firefox v?\d+/ over the same 15 papers returns 6, and the extra ones
  // are citation-marker splices ("Firefox [54]4"), a version of something else,
  // or a Firefox version discussed in related work rather than used for the
  // crawl. So this is a hand map, each entry carrying the deciding sentence,
  // read on 2026-08-14. Added after a review pass found the page inferring a
  // browser version for a paper that states a different one.
  const FIREFOX_STATED = {
    'apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web': {
      firefox: '52.9',
      quote: 'version string recorded by the extraction as "0.8; Firefox 52.9"; consistent with the firefox-52 era pin',
    },
    'cookiegraph-understanding-and-detecting-first-party-tracking-cookies': {
      firefox: '102',
      quote: 'We use OpenWPM (v0.17.0) and Firefox (v102) [52] to crawl a sample of 20K out of the top-million websites',
    },
    'purl-safe-and-effective-sanitization-of-link-decoration': {
      firefox: '102',
      quote: 'We use OpenWPM (v0.17.0) [11] and Firefox (v102) [64] for crawling',
    },
  };
  p('');
  p(`Of those ${statedPapers.size} papers, ${Object.keys(FIREFOX_STATED).length} also state which Firefox the crawl ran,`);
  p(`hand-read rather than swept (the regex sweep returns 6 and is dominated by`);
  p(`citation-marker splices). The other ${statedPapers.size - Object.keys(FIREFOX_STATED).length} leave the browser to be inferred from the pin:`);
  for (const [slug, v] of Object.entries(FIREFOX_STATED)) {
    const paper = OPENWPM.find((x) => x.slug === slug);
    const ow = openwpmTuples(paper).map((t) => t.version).filter(Boolean)[0];
    const nominal = FIREFOX_OF[normalise(ow ?? '')] ?? '?';
    p(`  ${paper.year} ${paper.venue}/${slug}`);
    p(`     OpenWPM ${ow} (ships Firefox ${nominal}) — paper states Firefox ${v.firefox}`);
    p(`     "${v.quote}"`);
  }
  p('');
  p('Two of the three ran a NEWER browser than their OpenWPM release ships, so the');
  p('bundled-Firefox column is nominal: OpenWPM launches whatever FIREFOX_BINARY');
  p('points at. 12 of the 15 do not say, and cannot be checked.');
  p('');
  p('"Releases behind" counts tagged releases between the version the paper used');
  p('and the newest release that existed on 1 January of its venue year. The venue');
  p('year is not the submission date, so this is an upper bound on how out of date');
  p('the authors were at the time they crawled, and a lower bound on how out of');
  p('date the published number is by the time it is read.');
}

// ============================== D. what the paper says about the configuration
h('D. Reporting rates: OpenWPM papers against the rest of the crawling population');
{
  const fieldStated = (x, f) => x.crawlConfig && !isSentinel(x.crawlConfig[f]);
  const arrStated = (x, f) =>
    x.crawlConfig && (x.crawlConfig[f] ?? []).filter((v) => !isSentinel(v)).length > 0;
  const numStated = (x, f) => x.crawlConfig && x.crawlConfig[f] !== null;
  const FIELDS = [
    ['statefulness', (x) => fieldStated(x, 'statefulness')],
    ['headless / display mode', (x) => fieldStated(x, 'headless')],
    ['browser', (x) => arrStated(x, 'browsers')],
    ['interaction depth', (x) => fieldStated(x, 'interactionDepth')],
    ['consent action', (x) => fieldStated(x, 'consentAction')],
    ['subpages per site', (x) => numStated(x, 'subpagesPerSite')],
    ['repeat visits', (x) => numStated(x, 'repeatVisits')],
    ['authentication', (x) => fieldStated(x, 'authentication')],
  ];
  p(
    T(
      ['Crawl-configuration field', `OpenWPM papers (of ${N})`, 'Share', `Other crawling papers (of ${OTHER_CRAWLED.length})`, 'Share'],
      FIELDS.map(([label, test]) => {
        const a = OPENWPM.filter(test).length;
        const b = OTHER_CRAWLED.filter(test).length;
        return [label, a, pct(a, N), b, pct(b, OTHER_CRAWLED.length)];
      })
    )
  );
  p('');
  p('Read the rows against each other, not against 100%. Both columns are');
  p('reporting rates: "does the paper say", never "did the authors do it".');

  // The statefulness values themselves, since that is the page's own subject.
  const vals = new Map();
  for (const x of OPENWPM) {
    const v = x.crawlConfig ? x.crawlConfig.statefulness : '(no crawlConfig record)';
    vals.set(v, (vals.get(v) ?? 0) + 1);
  }
  p('');
  p(
    T(
      ['crawlConfig.statefulness among OpenWPM papers', 'Papers', `Share of ${N}`],
      [...vals.entries()].sort((a, b) => b[1] - a[1]).map(([k, v]) => [k, v, pct(v, N)])
    )
  );
  const othervals = new Map();
  for (const x of OTHER_CRAWLED) {
    const v = x.crawlConfig ? x.crawlConfig.statefulness : '(no crawlConfig record)';
    othervals.set(v, (othervals.get(v) ?? 0) + 1);
  }
  p('');
  p(
    T(
      ['crawlConfig.statefulness among other crawling papers', 'Papers', `Share of ${OTHER_CRAWLED.length}`],
      [...othervals.entries()].sort((a, b) => b[1] - a[1]).map(([k, v]) => [k, v, pct(v, OTHER_CRAWLED.length)])
    )
  );
}

// ===================================== E. full-text sweep for the config names
h('E. Full-text sweep: which OpenWPM identifiers the 60 papers name');
{
  const texts = new Map();
  let missing = 0;
  for (const x of OPENWPM) {
    const f = path.join(dataRoot(), 'fulltext', String(x.year), x.venue, x.slug, 'paper.cols.txt');
    if (!fs.existsSync(f)) {
      missing += 1;
      continue;
    }
    texts.set(key(x), fs.readFileSync(f, 'utf8').replace(/\s+/g, ' '));
  }
  const M = texts.size;
  p(`Full text read for ${M} of ${N} papers (${missing} missing).`);
  p('Matching is on the whitespace-normalised paper.cols.txt rendering, which is');
  p('the file the extractor read. A hyphen inserted by the two-column repair can');
  p('still hide a token, so every count here is a lower bound.');
  p('');

  // Class 1: identifiers that exist only in OpenWPM. A hit is unambiguous.
  const IDENTIFIERS = [
    ['http_instrument', /http_instrument/i],
    ['js_instrument (incl. js_instrument_settings)', /js_instrument/i],
    ['cookie_instrument', /cookie_instrument/i],
    ['navigation_instrument', /navigation_instrument/i],
    ['dns_instrument', /dns_instrument/i],
    ['callstack_instrument', /callstack_instrument/i],
    ['save_content', /save_content/i],
    ['bot_mitigation', /bot_mitigation/i],
    ['display_mode', /display_mode/i],
    ['num_browsers', /num_browsers/i],
    ['seed_tar / profile_archive_dir', /seed_tar|profile_archive_dir/i],
    ['collection_fingerprinting', /collection_fingerprinting/i],
    ['table http_requests / http_responses', /http_requests|http_responses/i],
    ['table javascript_cookies', /javascript_cookies/i],
    ['table incomplete_visits', /incomplete_visits/i],
    ['table crawl_history', /crawl_history/i],
    ['table site_visits', /site_visits/i],
  ];
  const idHits = new Map();
  for (const [label, re] of IDENTIFIERS) {
    idHits.set(label, [...texts.entries()].filter(([, t]) => re.test(t)).map(([k]) => k));
  }
  p(
    T(
      ['OpenWPM identifier named in the paper', `Papers (of ${M})`, 'Share'],
      IDENTIFIERS.map(([label]) => [label, idHits.get(label).length, pct(idHits.get(label).length, M)])
    )
  );
  const anyId = new Set(
    [...idHits.values()].flat()
  );
  p('');
  p(`Papers naming at least one OpenWPM configuration key or output table: ${anyId.size} of ${M} (${pct(anyId.size, M)}).`);
  p(`Papers naming none: ${M - anyId.size} (${pct(M - anyId.size, M)}).`);
  p('');
  p('  named at least one:');
  for (const k of [...anyId].sort()) p(`    ${k}`);

  // Class 1b: the same instruments described in prose rather than by config key.
  // This is the fair comparison: a paper can say what it collected without ever
  // naming a setting. Every regex here is loose, so every count is an UPPER
  // bound on "the paper told you it collected this".
  const PROSE_INSTRUMENTS = [
    ['HTTP requests / responses', /\b(http|network|web)\s+(requests?|responses?|traffic)\b/i],
    ['JavaScript API calls', /\b(javascript|js)\s+(api\s+)?(calls?|accesses|method calls?)\b|\bapi calls?\b/i],
    ['cookies', /\bcookies?\b/i],
    ['page navigations recorded', /(record|captur|log|collect)\w*\s+(the\s+)?(page\s+)?navigations?|navigation events?/i],
    ['DNS resolutions recorded', /dns (resolution|response|quer|record)|resolved (the )?(hostname|domain)/i],
    ['response bodies / script contents', /response bod|script (contents?|source)|page source/i],
    ['screenshots', /screenshots?/i],
    ['browser profile / storage dump', /(browser|firefox) profile|local ?storage/i],
  ];
  p('');
  p(
    T(
      ['Data the paper says it collected (prose; upper bound)', `Papers (of ${M})`, 'Share'],
      PROSE_INSTRUMENTS.map(([label, re]) => {
        const c = [...texts.values()].filter((t) => re.test(t)).length;
        return [label, c, pct(c, M)];
      })
    )
  );

  // Class 2: prose. These words have ordinary English uses, so each is an upper
  // bound on the number of papers that told you anything about that setting.
  const PROSE = [
    ['"stateful"', /\bstateful\b/i],
    ['"stateless"', /\bstateless\b/i],
    ['"headless"', /\bheadless\b/i],
    ['"xvfb"', /xvfb/i],
    ['"fresh profile" / "clean profile" / "new profile"', /(fresh|clean|new|empty)\s+(browser\s+)?profile/i],
    ['"cookie jar"', /cookie\s?jar/i],
    ['SQLite or Parquet output', /sqlite|parquet/i],
    ['LevelDB', /leveldb/i],
    ['"call ?stack"', /call\s?stacks?\b/i],
    ['names Firefox', /\bfirefox\b/i],
    ['names Selenium', /\bselenium\b/i],
    ['names geckodriver', /geckodriver/i],
  ];
  p('');
  p(
    T(
      ['Phrase in the paper (upper bound: ordinary English uses match too)', `Papers (of ${M})`, 'Share'],
      PROSE.map(([label, re]) => {
        const c = [...texts.values()].filter((t) => re.test(t)).length;
        return [label, c, pct(c, M)];
      })
    )
  );

  // stateful/stateless is the page's own subject, so cross the sweep against the
  // schema field instead of trusting either alone.
  const swStateful = new Set(
    [...texts.entries()].filter(([, t]) => /\bstateful\b|\bstateless\b/i.test(t)).map(([k]) => k)
  );
  const schemaStated = new Set(
    OPENWPM.filter((x) => x.crawlConfig && !isSentinel(x.crawlConfig.statefulness)).map(key)
  );
  const both = [...schemaStated].filter((k) => swStateful.has(k)).length;
  p('');
  p(`Statefulness, the two signals crossed (of ${M} papers with full text):`);
  p(`  schema says stated:                ${schemaStated.size}`);
  p(`  text contains stateful/stateless:  ${swStateful.size}`);
  p(`  both:                              ${both}`);
  p(`  schema only (described in words the sweep does not match): ${schemaStated.size - both}`);
  p(`  sweep only (the word appears, the extractor read no value): ${swStateful.size - both}`);
  for (const k of [...swStateful].filter((x) => !schemaStated.has(x)).sort())
    p(`    sweep only: ${k}`);

  if (HITS) {
    const re = new RegExp(HITS, 'i');
    p('');
    p(`-- lines matching /${HITS}/i --`);
    for (const [k, t] of texts) {
      for (const m of t.matchAll(new RegExp(`.{0,120}${HITS}.{0,120}`, 'gi'))) {
        p(`  ${k}: ...${m[0]}...`);
      }
      if (!re.test(t)) continue;
    }
  }
}

// ================================================ F. what they used it to find
h('F. What the OpenWPM papers were measuring');
{
  const targets = new Map();
  for (const x of OPENWPM)
    for (const c of x.classification ?? [])
      targets.set(c.target, (targets.get(c.target) ?? 0) + 1);
  p(
    T(
      ['classification[].target (enum)', 'Tuples'],
      [...targets.entries()].sort((a, b) => b[1] - a[1]).slice(0, 12).map(([k, v]) => [k, v])
    )
  );
  p('');
  p('Tuple counts, not paper counts, and an enum, so publishable as a ranking.');
  p('');
  p('detection[].phenomenon is free text and ~20% stable run-to-run, so it is a');
  p('ranking of what the extractor wrote, never a percentage:');
  const phen = new Map();
  for (const x of OPENWPM)
    for (const d of x.detection ?? []) {
      const k = (d.phenomenon ?? '').toLowerCase();
      if (!k) continue;
      phen.set(k, (phen.get(k) ?? 0) + 1);
    }
  p(
    T(
      ['detection[].phenomenon (free text, verbatim)', 'Tuples'],
      [...phen.entries()].sort((a, b) => b[1] - a[1]).slice(0, 20).map(([k, v]) => [k, v])
    )
  );
  const withPrev = OPENWPM.filter((x) => (x.detection ?? []).some((d) => d.prevalence));
  p('');
  p(`Papers with at least one detection tuple carrying a prevalence figure: ${withPrev.length} of ${N} (${pct(withPrev.length, N)}).`);
  p('prevalence is a model summary of the result, not a quote. Any figure quoted');
  p('on the page was checked against the paper full text as well.');
}

// ============================================================ G. artefacts
h('G. Do OpenWPM papers release their crawl code');
{
  const avail = new Map();
  for (const x of OPENWPM) {
    const v = x.artifacts ? x.artifacts.availability : '(no artifacts record)';
    avail.set(v, (avail.get(v) ?? 0) + 1);
  }
  const availOther = new Map();
  for (const x of OTHER_CRAWLED) {
    const v = x.artifacts ? x.artifacts.availability : '(no artifacts record)';
    availOther.set(v, (availOther.get(v) ?? 0) + 1);
  }
  const keys = [...new Set([...avail.keys(), ...availOther.keys()])];
  p(
    T(
      ['artifacts.availability', `OpenWPM papers (of ${N})`, 'Share', `Other crawling papers (of ${OTHER_CRAWLED.length})`, 'Share'],
      keys
        .sort((a, b) => (avail.get(b) ?? 0) - (avail.get(a) ?? 0))
        .map((k) => [
          k,
          avail.get(k) ?? 0,
          pct(avail.get(k) ?? 0, N),
          availOther.get(k) ?? 0,
          pct(availOther.get(k) ?? 0, OTHER_CRAWLED.length),
        ])
    )
  );
}

// ============================================================ H. the paper list
if (LIST) {
  h('H. The 60 papers');
  for (const x of OPENWPM.sort((a, b) => a.year - b.year || a.venue.localeCompare(b.venue))) {
    const v = openwpmTuples(x).map((t) => `${t.name}${t.version ? ` v${t.version}` : ''}`).join('; ');
    p(`${x.year} ${x.venue.padEnd(8)} ${x.slug}`);
    p(`         ${v}`);
  }
}

// ========================================================== I. evidence quotes
if (QUOTES) {
  h(`I. Evidence quotes matching /${QUOTES.source}/i`);
  for (const x of OPENWPM) {
    for (const t of openwpmTuples(x)) {
      const blob = `${t.name} ${t.purpose ?? ''} ${t.evidence.quote}`;
      if (!QUOTES.test(blob)) continue;
      p(`${key(x)}  [${t.name}] (${t.evidence.section})`);
      p(`   purpose: ${t.purpose}`);
      p(`   quote:   ${JSON.stringify(t.evidence.quote)}`);
    }
  }
} else if (!WIKI) {
  h('I. Every OpenWPM tool tuple, purpose and quote (read these)');
  for (const x of OPENWPM.sort((a, b) => a.year - b.year)) {
    for (const t of openwpmTuples(x)) {
      p(`${x.year} ${key(x)}  [${t.name}] v=${t.version ?? '—'} ${t.usedOrMentioned} (${t.evidence.section})`);
      p(`   purpose: ${t.purpose}`);
      p(`   quote:   ${JSON.stringify(t.evidence.quote)}`);
    }
  }
}

// ========================================================= Z. external figures
// Numbers on the page that do NOT come from the corpus, each with its source, so
// check_page_numbers.mjs can pass without an ALLOW entry per number. Everything
// here was read on 2026-08-14; a version number is a claim about today.
if (!WIKI) {
  h('Y. Figures the page derives by arithmetic from the tables above');
  {
    const fieldStated = (x, f) => x.crawlConfig && !isSentinel(x.crawlConfig[f]);
    const numStated = (x, f) => x.crawlConfig && x.crawlConfig[f] !== null;
    const ratio = (test) => {
      const a = OPENWPM.filter(test).length / N;
      const b = OTHER_CRAWLED.filter(test).length / OTHER_CRAWLED.length;
      return (a / b).toFixed(1);
    };
    p(`  statefulness reporting ratio, OpenWPM papers vs other crawling papers: ${ratio((x) => fieldStated(x, 'statefulness'))}x`);
    p(`  subpages-per-site reporting ratio: ${ratio((x) => numStated(x, 'subpagesPerSite'))}x`);
    p(`  corpus share of the CRAWLED population: ${pct(CRAWLED.length, rows.length)} of ${rows.length}`);
  }

  h('Z. External figures (not from the corpus) — re-verify before republishing');
  p(`  OpenWPM release 0.35.0, tagged 2026-06-17 — repo VERSION file and git tag`);
  p(`  geckodriver 0.37.1 released 2026-07-20 (current); environment.yaml pins 0.37.0`);
  p(`  conda pins that have no linux-aarch64 build: geckodriver 0.37.0, plyvel 1.5.1, selenium-manager 4.45`);
  p(`  Firefox Nightly used in the ARM substitution attempt: 156 (156.0a1, linux64-aarch64)`);
  p(`  Firefox releases: 152 bundled by OpenWPM 0.35.0; Firefox reached 147 during the 2025 release gap`);
  p(`  build tasks in the taskcluster index for the pinned revision: 202 (mozilla-release), 185 (mozilla-central)`);
  p(`  Firefox 52 (52.9) — the browser version stated by ahmad2020_apophanies alongside OpenWPM 0.8; that paper predates the 0.10 architecture`);
  p(`  GitHub issues referenced: #557 callstack, #503 no Windows, #101 tracking_protection, #448 display_mode`);
  p(`  Bugzilla 634073 — cached image headers not recorded`);
  p(`  openwpm.xpi built here: 146,356 bytes; empty crawl database: 69,632 bytes with 14 tables, 0 rows`);
  p(`  pytest -m pyonly: 14 passed, 135 deselected in 0.13s`);
  p(`  browser id in the quoted failure log: 80873956`);
  p(`  quote-check bands: verbatim 100%, spliced 60-99% of word positions covered`);
  p(`  docs/Papers.rst lists 76 studies, none later than 2021, last edited 2023-09-19`);
  p(`  "a crawl of 100,000 sites" — rhetorical scale, not a measurement`);
  p(`  62 — a print() column width inside pages/openwpm_defaults.py, not a figure`);
  p(`  Ubuntu 24.04 (CI) and 22.04 (the container image base) — README, Installation section`);
  p(`  content_hash is SHA-256 — Extension/src/lib/sha256.ts; docs/Configuration.md lines 419-420 still say md5`);
  p(`  73 commits in 2026 to the pinned commit b9dd4c3, 72 of them by one author`);
  p(`  instance_id appears in openwpm/storage/arrow_storage.py only; 0 occurrences in schema.sql`);
  p(`  the two Xvfb papers of the 60: CCS 2016 scored headless, WWW 2020 (zeber) scored headful`);
  p(`  917,261 and 94,144 — the site counts of englehardt2016online's "Default Stateless 1 Million"`);
  p(`       and "Default Stateful 100,000" configurations, from that paper's configuration table`);
  p(`  Firefox 102 — stated by shaoor2023cookiegraph and shaoor2024purl for their crawls (see section C)`);
  p(`  21 — the next-most-used specialised crawlers (tbselenium, Tracker Radar Collector),`);
  p(`       from report_crawler.mjs's mention-matching table, quoted from programming:crawler`);
  p(`  bundles Firefox 152 — scripts/install-firefox.sh TAG=931e624c6f53269d41e57ecefca418ef7fdb0f75 (FIREFOX_152_0_RELEASE)`);
  p(`  environment.yaml pins geckodriver 0.37.0, selenium 4.45.0, python 3.14.6`);
  p(`  GitHub: 1,415 stars, 332 forks, 180 open issues, not archived — api.github.com/repos/openwpm/OpenWPM`);
  p(`  commits per year: 217 (2014), 185, 195, 180, 252, 569 (2019), 380, 85, 28, 23, 11 (2024), 12 (2025), 72 (2026 to 2026-07-11)`);
  p(`  releases: v0.31.0 2025-01-19, then a 13-month gap to v0.32.0 2026-03-03`);
  p(`  2026 commits: 72 of 73 by one contributor (vringar / Stefan Zabka)`);
  p(`  built extension openwpm.xpi: 146,356 bytes, Manifest V2, 3 experiment_apis`);
  p(`  callstack_instrument broken: docs/Configuration.md + issue #557`);
  p(`  no Windows support: README + issue #503`);
  p(`  unbranded Firefox needed because the extension uses privileged APIs — docs/Platform-Architecture.md:92`);
}

console.log(out.join('\n'));
</file>

<file text openwpm-output.txt>
corpus: 5859 papers, 7 venues, 2010-2026
population OPENWPM: 60 papers name OpenWPM in tools[] with usedOrMentioned in {used, produced}
population CRAWLED: 1120 papers (baseline; 1061 of them are not OpenWPM papers)
generated by scripts/report_openwpm.mjs


=== A. Population and how OpenWPM is spelled ===

Spelling in tools[].name        Tuples
------------------------------  ------
OpenWPM                         60
OpenWPM-Mobile                  2
OpenWPM Firefox extension       1
custom OpenWPM instrumentation  1
openWPM                         1

usedOrMentioned  Tuples
---------------  ------
used             59
produced         3
mentioned        2
compared         1

Papers with any OpenWPM tuple: 63
  of which used or produced it: 60
  of which only compared or mentioned it: 3
    WWW/2017/security-challenges-in-an-increasingly-tangled-web
    USENIX/2023/rods-with-laser-beams-understanding-browser-fingerprinting-on-phishing-pages
    PETS/2025/tracker-installations-are-not-created-equal-understanding-tracker-configuration

Inside the CRAWLED population: 59 of 60 (98.3%). The 1 outside it have no crawlConfig record and no automated-web-crawl study type.
  outside CRAWLED: PETS/2021/a-calculus-of-tracking-theory-and-practice — studyTypes ["existing-dataset-analysis","system-or-defence-proposal"]

tools[].category   Tuples
-----------------  ------
crawler-framework  60
browser            1
program-analysis   1


=== B. Where the 60 papers are ===

Venue    OpenWPM papers  Share of 60  Crawling papers at that venue  OpenWPM share of those
-------  --------------  -----------  -----------------------------  ----------------------
PETS     24              40.0%        123                            19.5%
WWW      13              21.7%        242                            5.4%
USENIX   7               11.7%        221                            3.2%
IMC      6               10.0%        132                            4.5%
CCS      5               8.3%         163                            3.1%
NDSS     4               6.7%         129                            3.1%
IEEE-SP  1               1.7%         110                            0.9%

Year   OpenWPM papers  Crawling papers  OpenWPM share of crawling papers
-----  --------------  ---------------  --------------------------------
2015   2               41               4.9%
2016   2               40               5.0%
2017   3               51               5.9%
2018   3               61               4.9%
2019   3               97               3.1%
2020   9               75               12.0%
2021   5               75               6.7%
2022   9               110              8.2%
2023   10              125              8.0%
2024   7               110              6.4%
2025*  6               129              4.7%
2026*  1               69               1.4%

* 2025 and 2026 are provisional venue-years: CCS 2026 and IMC 2026 have not
  been held, and IEEE S&P 2026 / WWW 2026 abstracts are not in the selection
  source, so those years are under-represented by construction.


=== C. Which version, and how far behind it was ===

Papers stating any OpenWPM version: 15 of 60 (25.0%).
Version strings are free text; they are reported verbatim, never folded.

Paper                                                                                       Version as written                    That version released  Firefox it bundles  Newest release on 1 Jan of the paper year  Releases behind
------------------------------------------------------------------------------------------  ------------------------------------  ---------------------  ------------------  -----------------------------------------  ---------------
WWW/2015/cookies-that-give-you-away-the-surveillance-implications-of-web-tracking           0.1.0                                 (no tag)               ?                   0.2.1 (2014-12-31)                         —
WWW/2020/apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web          0.8; Firefox 52.9                     (no tag)               ?                   0.8.0 (2017-10-09)                         —
PETS/2022/my-cookie-is-a-phoenix-detection-measurement-and-lawfulness-of-cookie-respawning  0.9.0 (machine A); 0.7.0 (machine B)  (no tag)               ?                   0.18.0 (2021-12-12)                        —
USENIX/2022/automating-cookie-consent-and-gdpr-violation-detection                          0.12.0                                2020-08-26             Firefox 80          0.18.0 (2021-12-12)                        7
USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains                     0.10.0                                2020-06-22             Firefox 77          0.18.0 (2021-12-12)                        9
WWW/2022/reproducibility-and-replicability-of-web-measurement-studies                       v0.15.0                               2021-05-10             Firefox 88          0.18.0 (2021-12-12)                        3
CCS/2023/cookiegraph-understanding-and-detecting-first-party-tracking-cookies               0.17.0                                2021-07-24             Firefox 90          0.21.1 (2022-10-13)                        6
IMC/2023/on-the-similarity-of-web-measurements-under-different-experimental-setups          v0.18.0                               2021-12-12             Firefox 95          0.21.1 (2022-10-13)                        5
PETS/2023/evaluating-practical-quic-website-fingerprinting-defenses-for-the-masses          0.17.0                                2021-07-24             Firefox 90          0.21.1 (2022-10-13)                        6
PETS/2023/privacy-rarely-considered-exploring-considerations-in-the-adoption-of-third-part  0.13                                  (no tag)               ?                   0.21.1 (2022-10-13)                        —
PETS/2024/the-devil-is-in-the-details-detection-measurement-and-lawfulness-of-server-side   0.14.0 and 0.19.1                     2021-03-16             Firefox 86          0.26.0 (2023-12-24)                        15
PETS/2024/a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users  0.20.0                                2022-05-18             Firefox 100         0.26.0 (2023-12-24)                        7
USENIX/2024/purl-safe-and-effective-sanitization-of-link-decoration                         v0.17.0                               2021-07-24             Firefox 90          0.26.0 (2023-12-24)                        11
PETS/2025/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and  0.26.0                                2023-12-24             Firefox 121         0.30.0 (2024-10-02)                        4
PETS/2025/understanding-regional-filter-lists-efficacy-and-impact                           0.27.0                                2024-02-08             Firefox 122         0.30.0 (2024-10-02)                        3

Every OpenWPM release pins one unbranded Firefox build, so citing an OpenWPM
version pins a browser version too. The mapping, read out of
scripts/install-firefox.sh at each v* tag:
OpenWPM release  Tagged      Bundled Firefox
---------------  ----------  ---------------
0.10.0           2020-06-22  Firefox 77
0.11.0           2020-07-08  Firefox 78
0.12.0           2020-08-26  Firefox 80
0.13.0           2020-11-19  Firefox 83
0.14.0           2021-03-16  Firefox 86
0.14.1           2021-03-29  Firefox 86
0.15.0           2021-05-10  Firefox 88
0.16.0           2021-06-10  Firefox 89
0.17.0           2021-07-24  Firefox 90
0.18.0           2021-12-12  Firefox 95
0.19.0           2022-03-23  Firefox 98
0.19.1           2022-03-31  Firefox 98
0.20.0           2022-05-18  Firefox 100
0.21.0           2022-09-27  Firefox 105
0.21.1           2022-10-13  Firefox 105
0.22.0           2023-06-25  Firefox 114
0.23.0           2023-08-03  Firefox 115
0.24.0           2023-09-05  Firefox 117
0.25.0           2023-10-21  Firefox 118
0.26.0           2023-12-24  Firefox 121
0.27.0           2024-02-08  Firefox 122
0.28.0           2024-02-21  Firefox 123
0.29.0           2024-07-15  Firefox 128
0.30.0           2024-10-02  Firefox 130
0.31.0           2025-01-19  Firefox 134
0.32.0           2026-03-03  Firefox 148
0.33.0           2026-03-28  Firefox 149
0.34.0           2026-05-08  Firefox 150
0.35.0           2026-06-17  Firefox 152

Of those 15 papers, 3 also state which Firefox the crawl ran,
hand-read rather than swept (the regex sweep returns 6 and is dominated by
citation-marker splices). The other 12 leave the browser to be inferred from the pin:
  2020 WWW/apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web
     OpenWPM 0.8; Firefox 52.9 (ships Firefox ?) — paper states Firefox 52.9
     "version string recorded by the extraction as "0.8; Firefox 52.9"; consistent with the firefox-52 era pin"
  2023 CCS/cookiegraph-understanding-and-detecting-first-party-tracking-cookies
     OpenWPM 0.17.0 (ships Firefox 90) — paper states Firefox 102
     "We use OpenWPM (v0.17.0) and Firefox (v102) [52] to crawl a sample of 20K out of the top-million websites"
  2024 USENIX/purl-safe-and-effective-sanitization-of-link-decoration
     OpenWPM v0.17.0 (ships Firefox 90) — paper states Firefox 102
     "We use OpenWPM (v0.17.0) [11] and Firefox (v102) [64] for crawling"

Two of the three ran a NEWER browser than their OpenWPM release ships, so the
bundled-Firefox column is nominal: OpenWPM launches whatever FIREFOX_BINARY
points at. 12 of the 15 do not say, and cannot be checked.

"Releases behind" counts tagged releases between the version the paper used
and the newest release that existed on 1 January of its venue year. The venue
year is not the submission date, so this is an upper bound on how out of date
the authors were at the time they crawled, and a lower bound on how out of
date the published number is by the time it is read.


=== D. Reporting rates: OpenWPM papers against the rest of the crawling population ===

Crawl-configuration field  OpenWPM papers (of 60)  Share  Other crawling papers (of 1061)  Share
-------------------------  ----------------------  -----  -------------------------------  -----
statefulness               33                      55.0%  186                              17.5%
headless / display mode    10                      16.7%  130                              12.3%
browser                    39                      65.0%  490                              46.2%
interaction depth          53                      88.3%  788                              74.3%
consent action             30                      50.0%  319                              30.1%
subpages per site          18                      30.0%  88                               8.3%
repeat visits              30                      50.0%  222                              20.9%
authentication             53                      88.3%  726                              68.4%

Read the rows against each other, not against 100%. Both columns are
reporting rates: "does the paper say", never "did the authors do it".

crawlConfig.statefulness among OpenWPM papers  Papers  Share of 60
---------------------------------------------  ------  -----------
not-stated                                     26      43.3%
stateless                                      16      26.7%
stateful                                       12      20.0%
both                                           5       8.3%
(no crawlConfig record)                        1       1.7%

crawlConfig.statefulness among other crawling papers  Papers  Share of 1061
----------------------------------------------------  ------  -------------
not-stated                                            818     77.1%
stateless                                             97      9.1%
stateful                                              65      6.1%
(no crawlConfig record)                               40      3.8%
both                                                  24      2.3%
not-applicable                                        17      1.6%


=== E. Full-text sweep: which OpenWPM identifiers the 60 papers name ===

Full text read for 60 of 60 papers (0 missing).
Matching is on the whitespace-normalised paper.cols.txt rendering, which is
the file the extractor read. A hyphen inserted by the two-column repair can
still hide a token, so every count here is a lower bound.

OpenWPM identifier named in the paper         Papers (of 60)  Share
--------------------------------------------  --------------  -----
http_instrument                               1               1.7%
js_instrument (incl. js_instrument_settings)  1               1.7%
cookie_instrument                             1               1.7%
navigation_instrument                         0               0.0%
dns_instrument                                0               0.0%
callstack_instrument                          0               0.0%
save_content                                  0               0.0%
bot_mitigation                                0               0.0%
display_mode                                  0               0.0%
num_browsers                                  0               0.0%
seed_tar / profile_archive_dir                0               0.0%
collection_fingerprinting                     0               0.0%
table http_requests / http_responses          2               3.3%
table javascript_cookies                      0               0.0%
table incomplete_visits                       0               0.0%
table crawl_history                           0               0.0%
table site_visits                             1               1.7%

Papers naming at least one OpenWPM configuration key or output table: 3 of 60 (5.0%).
Papers naming none: 57 (95.0%).

  named at least one:
    PETS/2021/a-calculus-of-tracking-theory-and-practice
    WWW/2015/cookies-that-give-you-away-the-surveillance-implications-of-web-tracking
    WWW/2020/stop-tracking-me-bro-differential-tracking-of-user-demographics-on-hyper-partisa

Data the paper says it collected (prose; upper bound)  Papers (of 60)  Share
-----------------------------------------------------  --------------  -----
HTTP requests / responses                              58              96.7%
JavaScript API calls                                   15              25.0%
cookies                                                57              95.0%
page navigations recorded                              2               3.3%
DNS resolutions recorded                               14              23.3%
response bodies / script contents                      12              20.0%
screenshots                                            12              20.0%
browser profile / storage dump                         30              50.0%

Phrase in the paper (upper bound: ordinary English uses match too)  Papers (of 60)  Share
------------------------------------------------------------------  --------------  -----
"stateful"                                                          25              41.7%
"stateless"                                                         29              48.3%
"headless"                                                          14              23.3%
"xvfb"                                                              2               3.3%
"fresh profile" / "clean profile" / "new profile"                   8               13.3%
"cookie jar"                                                        10              16.7%
SQLite or Parquet output                                            5               8.3%
LevelDB                                                             1               1.7%
"call ?stack"                                                       4               6.7%
names Firefox                                                       44              73.3%
names Selenium                                                      19              31.7%
names geckodriver                                                   0               0.0%

Statefulness, the two signals crossed (of 60 papers with full text):
  schema says stated:                33
  text contains stateful/stateless:  32
  both:                              23
  schema only (described in words the sweep does not match): 10
  sweep only (the word appears, the extractor read no value): 9
    sweep only: CCS/2018/the-webs-sixth-sense-a-study-of-scripts-accessing-smartphone-sensors
    sweep only: CCS/2023/read-between-the-lines-detecting-tracking-javascript-with-bytecode-classificatio
    sweep only: IEEE-SP/2021/fingerprinting-the-fingerprinters-learning-to-detect-browser-fingerprinting-beha
    sweep only: PETS/2021/a-calculus-of-tracking-theory-and-practice
    sweep only: PETS/2021/ml-cb-machine-learning-canvas-block
    sweep only: PETS/2021/unveiling-web-fingerprinting-in-the-wild-via-code-mining-and-machine-learning
    sweep only: USENIX/2022/automating-cookie-consent-and-gdpr-violation-detection
    sweep only: USENIX/2022/webgraph-capturing-advertising-and-tracking-information-flows-for-robust-blockin
    sweep only: USENIX/2023/glowing-in-the-dark-uncovering-ipv6-address-discovery-and-scanning-strategies-in


=== F. What the OpenWPM papers were measuring ===

classification[].target (enum)  Tuples
------------------------------  ------
web-request                     68
cookie                          34
domain                          32
website-category                27
other                           24
fingerprinting-script           20
consent-notice                  11
privacy-policy                  6
network-traffic                 6
ip-address                      5
malware                         3
javascript                      3

Tuple counts, not paper counts, and an enum, so publishable as a ranking.

detection[].phenomenon is free text and ~20% stable run-to-run, so it is a
ranking of what the extractor wrote, never a percentage:
detection[].phenomenon (free text, verbatim)  Tuples
--------------------------------------------  ------
cookie syncing                                6
canvas fingerprinting                         5
tracking requests                             5
browser fingerprinting                        4
cookie synchronization                        3
website breakage                              3
third-party tracking                          2
tracking cookies                              2
client-side cookie syncing                    2
web fingerprinting                            2
tracking link decorations                     2
advertisement and tracker requests            2
unique identifier cookies                     1
cookie-based traffic clustering               1
geographic traffic exposure                   1
plaintext identity leakage                    1
privacy-tool mitigation                       1
erroneous dynamic hsts configuration          1
dynamic hpkp configuration errors             1
pinned non-pinned active content              1

Papers with at least one detection tuple carrying a prevalence figure: 60 of 60 (100.0%).
prevalence is a model summary of the result, not a quote. Any figure quoted
on the page was checked against the paper full text as well.


=== G. Do OpenWPM papers release their crawl code ===

artifacts.availability      OpenWPM papers (of 60)  Share  Other crawling papers (of 1061)  Share
--------------------------  ----------------------  -----  -------------------------------  -----
public                      33                      55.0%  550                              51.8%
none-mentioned              18                      30.0%  352                              33.2%
promised-not-yet-available  7                       11.7%  65                               6.1%
on-request                  2                       3.3%   30                               2.8%
(no artifacts record)       0                       0.0%   30                               2.8%
explicitly-withheld         0                       0.0%   10                               0.9%
restricted                  0                       0.0%   24                               2.3%


=== I. Every OpenWPM tool tuple, purpose and quote (read these) ===

2015 WWW/2015/cookies-that-give-you-away-the-surveillance-implications-of-web-tracking  [OpenWPM] v=0.1.0 used (methodology)
   purpose: Drive Firefox and collect HTTP traffic and cookies.
   quote:   "In our study, we use version 0.1.0 of OpenWPM to drive Firefox measurement instances from which we record all HTTP data for analysis."
2015 NDSS/2015/upgrading-https-in-mid-air-an-empirical-study-of-strict-transport-security-and-k  [OpenWPM] v=— used (methodology)
   purpose: Backbone for realistic web crawling and measurement.
   quote:   "We utilized OpenWPM as the backbone for our testing."
2016 CCS/2016/online-tracking-a-1-million-site-measurement-and-analysis  [OpenWPM] v=— produced (abstract)
   purpose: Automated, instrumented browser crawling and privacy measurement.
   quote:   "This measurement is made possible by our open-source web privacy measurement tool, OpenWPM, which uses an automated version of a full-fledged consumer browser."
2016 WWW/2016/no-honor-among-thieves-a-large-scale-analysis-of-malicious-web-shells  [OpenWPM] v=— used (methodology)
   purpose: Recorded browser-side third-party requests from executed shells.
   quote:   "We utilized the OpenWPM framework on a second server, to emulate an attacker browsing through each of the uploaded shells"
2017 NDSS/2017/dial-one-for-scam-a-large-scale-analysis-of-technical-support-scams  [OpenWPM] v=— used (implementation)
   purpose: Provide the web privacy measurement crawling platform
   quote:   "we extended OpenWPM which is a generic web privacy measurement platform"
2017 PETS/2017/cross-device-tracking-measurement-and-disclosures  [OpenWPM] v=— used (methodology)
   purpose: Automated navigation and browser privacy measurement
   quote:   "We used OpenWPM - an open source web privacy measurement platform developed at Princeton University - as the platform to facilitate our data collection."
2017 USENIX/2017/measuring-the-insecurity-of-mobile-deep-links-of-android  [OpenWPM] v=— used (dataset)
   purpose: Crawled dynamic content from Alexa websites.
   quote:   "This is done using modified OpenWPM [25], a headless browser-based crawler."
2018 CCS/2018/the-webs-sixth-sense-a-study-of-scripts-accessing-smartphone-sensors  [OpenWPM-Mobile] v=— produced (methodology)
   purpose: Crawled mobile websites and instrumented JavaScript and HTTP
   quote:   "Our data collection is based on OpenWPM-Mobile, a mobile-focused measurement tool we built by modifying OpenWPM web measurement framework."
2018 CCS/2018/the-webs-sixth-sense-a-study-of-scripts-accessing-smartphone-sensors  [OpenWPM] v=— used (methodology)
   purpose: Provided the underlying web measurement and instrumentation framework
   quote:   "We extend the OpenWPM [31] measurement platform to study the mobile web, adding emulation of mobile browsing behavior and browser APIs."
2018 PETS/2018/i-never-signed-up-for-this-privacy-implications-of-email-tracking  [OpenWPM] v=— used (methodology)
   purpose: Crawling sites and instrumenting email and link measurements
   quote:   "Building on the OpenWPM web crawler, we created a tool to automatically search for mailing list subscription forms on websites and fill them in."
2018 PETS/2018/when-the-cookie-meets-the-blockchain-privacy-risks-of-web-payments-via-cryptocur  [OpenWPM] v=— used (methodology)
   purpose: Collected HTTP(S) traffic and annotated sensitive page data.
   quote:   "To collect data on web tracking we used a modified version of the open-source web privacy measurement tool OpenWPM [16]."
2019 CCS/2019/watching-you-watch-the-tracking-ecosystem-of-over-the-top-tv-streaming-devices  [OpenWPM] v=— used (methodology)
   purpose: Collected web traffic during channel authentication.
   quote:   "We launched an instance of Open-WPM tool [23] to perform the required authentication steps on the web and to collect the HTTP traffic for further analysis."
2019 IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem  [OpenWPM] v=— used (methodology)
   purpose: Collected requests, cookies, trackers, fingerprinting, and banners.
   quote:   "we use a OpenWPM-based crawler to collect evidence of the behavior of each website and used tracking technologies"
2019 WWW/2019/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private  [OpenWPM] v=— used (methodology)
   purpose: Automatically harvested websites and HTTP responses.
   quote:   "Then, we use OpenWPM [11] to automatically visit the websites and some randomly selected sub-pages, creating a dataset composed of 21 crawls/harvests."
2020 PETS/2020/a-comparative-measurement-study-of-web-tracking-on-mobile-and-desktop-environmen  [OpenWPM Firefox extension] v=— used (implementation)
   purpose: Base extension ported to mobile Firefox
   quote:   "We build our WTPatrol browser extension upon the Firefox extension used in OpenWPM by porting it to mobile Firefox and further expanding it with new capabilities."
2020 PETS/2020/in-depth-evaluation-of-redirect-tracking-and-link-usage  [OpenWPM] v=— used (methodology)
   purpose: Automated browser crawling and HTTP-event collection.
   quote:   "We enhanced OpenWPM with several features that are useful to study the effect of redirect links"
2020 PETS/2020/inferring-tracker-advertiser-relationships-in-the-online-advertising-ecosystem-u  [OpenWPM] v=— used (methodology)
   purpose: Automated persona and header-bidding website crawls.
   quote:   "Our measurements were conducted using a lightly modified version of OpenWPM [45]. OpenWPM was used to automatically load selected web pages."
2020 PETS/2020/missed-by-filter-lists-detecting-unknown-third-party-trackers-with-invisible-pix  [OpenWPM] v=— used (methodology)
   purpose: Conducted stateful Firefox web crawls and recorded traffic.
   quote:   "We performed passive Web measurements using the OpenWPM platform [24]. It uses the Firefox browser, and provides browser automation"
2020 PETS/2020/no-boundaries-data-exfiltration-by-third-parties-embedded-on-web-pages  [OpenWPM] v=— used (introduction)
   purpose: Automated, instrumented browser crawling and script attribution.
   quote:   "We built our measurement framework by extending OpenWPM, an open-source web privacy measurement tool."
2020 PETS/2020/no-boundaries-data-exfiltration-by-third-parties-embedded-on-web-pages  [custom OpenWPM instrumentation] v=— produced (methodology)
   purpose: Captured JavaScript accesses, HTTP stacks, and data exfiltration.
   quote:   "We extended OpenWPM's already existing JavaScript instrumentation to capture access to a set of DOM properties and function calls."
2020 WWW/2020/apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web  [OpenWPM] v=0.8; Firefox 52.9 used (implementation)
   purpose: Compared privacy-oriented crawling with anti-bot functions.
   quote:   "Our first release includes wrappers for the eight different crawlers."
2020 WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field  [OpenWPM] v=— used (methodology)
   purpose: Crawled sites and logged web activity.
   quote:   "we use OpenWPM to crawl these websites and gather first-party links on these websites"
2020 WWW/2020/stop-tracking-me-bro-differential-tracking-of-user-demographics-on-hyper-partisa  [OpenWPM] v=— used (methodology)
   purpose: Automated browser crawling and tracking measurement.
   quote:   "We make use of OpenWPM[1], a popular tool for measurements and automating web browsers"
2020 WWW/2020/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing  [OpenWPM] v=— used (methodology)
   purpose: Collected crawl and user browsing measurements.
   quote:   "All the Web measurements used in our analyses, from both crawl and user data, were collected using OpenWPM [17]."
2021 IEEE-SP/2021/fingerprinting-the-fingerprinters-learning-to-detect-browser-fingerprinting-beha  [OpenWPM] v=— used (implementation)
   purpose: Crawling websites and collecting scripts and execution traces
   quote:   "We crawl the web with an extended version of OpenWPM that extracts JavaScript source files and their execution traces."
2021 NDSS/2021/reining-in-the-webs-inconsistencies-with-site-policy  [OpenWPM] v=— used (methodology)
   purpose: Crawled sites and collected security headers.
   quote:   "We performed our data collection with the OpenWPM [11] framework in Q1 2020 (03/24-04/15) from a single IP address belonging to CISPA in Germany."
2021 PETS/2021/ml-cb-machine-learning-canvas-block  [openWPM] v=— used (evaluation)
   purpose: Provided the heuristic comparison baseline.
   quote:   "we used openWPM [36, 37] on the URLs in the test suite, filtering for \"canvas fingerprinting\" according to the heuristic used in [36]"
2021 PETS/2021/unveiling-web-fingerprinting-in-the-wild-via-code-mining-and-machine-learning  [OpenWPM] v=— used (dataset)
   purpose: Crawled Alexa sites and instrumented fingerprinting APIs.
   quote:   "For this, we modify OpenWPM, the web crawler introduced in [11] to override JavaScript functions and APIs which are typically used for fingerprinting purposes."
2021 PETS/2021/a-calculus-of-tracking-theory-and-practice  [OpenWPM] v=— used (dataset)
   purpose: Source of web-tracking measurement datasets.
   quote:   "We evaluate our model with the 10k Site ID Detection(1) 2016 dataset collected using a stateful instance of OpenWPM"
2022 NDSS/2022/auto-draft-209  [OpenWPM] v=— used (methodology)
   purpose: Automated scalable browser crawling and fresh profiles.
   quote:   "We use OpenWPM [4] to implement our crawling system in an automated and scalable manner."
2022 PETS/2022/atom-ad-network-tomography  [OpenWPM] v=— used (methodology)
   purpose: Automated persona browsing and web crawling.
   quote:   "Next, we used OpenWPM [48] to crawl each remaining site to verify that they were functional and contained trackers on them."
2022 PETS/2022/my-cookie-is-a-phoenix-detection-measurement-and-lawfulness-of-cookie-respawning  [OpenWPM] v=0.9.0 (machine A); 0.7.0 (machine B) used (methodology)
   purpose: Crawled websites and collected HTTP, script, and cookie data.
   quote:   "All measurements are performed using the Open-WPM platform on the Firefox browser."
2022 PETS/2022/omnicrawl-comprehensive-measurement-of-web-tracking-with-real-desktop-and-mobile  [OpenWPM-Mobile] v=— used (evaluation)
   purpose: Provides the emulated mobile-browser comparison condition.
   quote:   "We included both OpenWPM-Mobile's emulated Firefox browser and its non-emulated counterpart, mobile Firefox, in our crawl."
2022 USENIX/2022/automating-cookie-consent-and-gdpr-violation-detection  [OpenWPM] v=0.12.0 used (methodology)
   purpose: Crawled websites and recorded browser cookie activity.
   quote:   "we utilize the OpenWPM framework, version 0.12.0, which runs multiple concurrent Firefox browser instances via Selenium."
2022 USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains  [OpenWPM] v=0.10.0 used (evaluation)
   purpose: Collected browser crawl data
   quote:   "We evaluate K HALEESI on crawl data collected using version 0.10.0 of OpenWPM [38] in August 2020 in the US."
2022 USENIX/2022/webgraph-capturing-advertising-and-tracking-information-flows-for-robust-blockin  [OpenWPM] v=— used (evaluation)
   purpose: Crawled sites and captured browser-layer execution
   quote:   "We extend OpenWPM [30] to automatically crawl websites with Firefox and build A D G RAPH's representation."
2022 WWW/2022/et-tu-brute-privacy-analysis-of-government-websites-and-mobile-apps  [OpenWPM] v=— used (methodology)
   purpose: Crawled sites and collected tracking measurements.
   quote:   "We then crawl the landing pages from these domains using OpenWPM [24] and measure tracking prevalence on them"
2022 WWW/2022/reproducibility-and-replicability-of-web-measurement-studies  [OpenWPM] v=v0.15.0 used (appendix)
   purpose: Firefox-based crawling and HTTP traffic collection
   quote:   "We use the popular Open-WPM Framework [21] (v0.15.0 - Firefox version 88) to perform the Firefox-based measurements"
2023 CCS/2023/cookiegraph-understanding-and-detecting-first-party-tracking-cookies  [OpenWPM] v=0.17.0 used (methodology)
   purpose: Crawled sites and captured browser execution data.
   quote:   "We use OpenWPM (v0.17.0) and Firefox (v102) to crawl a sample of 20K out of the top-million websites."
2023 IMC/2023/on-the-similarity-of-web-measurements-under-different-experimental-setups  [OpenWPM] v=v0.18.0 used (methodology)
   purpose: Crawling pages and capturing browser traffic
   quote:   "All of these profiles are based on the Firefox browser, and we utilize OpenWPM (v0.18.0), a common and popular crawling framework [19], to capture the traffic we are interested in."
2023 CCS/2023/read-between-the-lines-detecting-tracking-javascript-with-bytecode-classificatio  [OpenWPM] v=— used (evaluation)
   purpose: Collect comparison datasets and Web API traces.
   quote:   "We collected a new set of data for bytecode classification and BD+ by using OpenWPM [26] in parallel with our instrumented Chromium"
2023 PETS/2023/evaluating-practical-quic-website-fingerprinting-defenses-for-the-masses  [OpenWPM] v=0.17.0 used (methodology)
   purpose: Crawled pages and logged HTTP requests
   quote:   "We collect page structures by crawling the pages in MAIN with OpenWPM (v0.17.0) [1] using Firefox five consecutive times."
2023 IMC/2023/thou-shalt-not-reject-analyzing-accept-or-pay-cookie-banners-on-the-web  [OpenWPM] v=— used (methodology)
   purpose: Automated browser-based website measurements
   quote:   "BannerClick is built on top of OpenWPM [21] and Selenium [58], and can automatically detect and interact with cookie banners on websites."
2023 IMC/2023/tracking-profiling-and-ad-targeting-in-the-alexa-echo-smart-speaker-ecosystem  [OpenWPM] v=— used (methodology)
   purpose: Crawled websites and captured bids, ads, and web traffic.
   quote:   "We extend OpenWPM [31] to identify and capture data on prebid.js supported websites."
2023 PETS/2023/comparing-large-scale-privacy-and-security-notifications  [OpenWPM] v=— used (methodology)
   purpose: Automated daily website checks and email extraction
   quote:   "we selected four privacy issues which fit our requirements and implemented them as custom functions in an established measurement framework, OpenWPM"
2023 PETS/2023/privacy-rarely-considered-exploring-considerations-in-the-adoption-of-third-part  [OpenWPM] v=0.13 used (methodology)
   purpose: Crawled recruitment and participant-provided websites.
   quote:   "We visited each domain on the Tranco 100K in October 2020 using OpenWPM 0.13"
2023 USENIX/2023/glowing-in-the-dark-uncovering-ipv6-address-discovery-and-scanning-strategies-in  [OpenWPM] v=— used (methodology)
   purpose: Crawled dynamic and third-party web content.
   quote:   "we conducted crawls using a simple CLI crawler which did not load third-party or dynamic content (wget) and a full-fledged browser using OpenWPM"
2023 WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion  [OpenWPM] v=— used (methodology)
   purpose: Collected ads, screenshots, landing URLs, and request metadata.
   quote:   "We extended OpenWPM [44], an open-source web measurement tool to capture ad images and their landing page URLs."
2024 PETS/2024/the-devil-is-in-the-details-detection-measurement-and-lawfulness-of-server-side  [OpenWPM] v=0.14.0 and 0.19.1 used (evaluation)
   purpose: Collected requests, responses, cookies, and script calls.
   quote:   "We used the OpenWPM platform [75] with the Firefox browser to perform three passive web measurement crawls"
2024 USENIX/2024/automated-large-scale-analysis-of-cookie-notice-compliance  [OpenWPM] v=— used (methodology)
   purpose: Crawled websites and collected cookies
   quote:   "The crawler is implemented as an extension of the OpenWPM framework [13]."
2024 PETS/2024/a-bilingual-longitudinal-analysis-of-privacy-policies-measuring-the-impacts-of-t  [OpenWPM] v=— used (methodology)
   purpose: Crawled websites, homepages, and privacy-policy pages.
   quote:   "We used the Open Web Privacy Measurement (OpenWPM) framework [19] to crawl popular websites for privacy policies, CCPA-related web pages, and homepages."
2024 PETS/2024/a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users  [OpenWPM] v=0.20.0 used (methodology)
   purpose: Automated browser crawling and traffic instrumentation
   quote:   "Each profile is based on OpenWPM [12] (v0.20.0), uses the Firefox browser (v100.0 with the default user agent"
2024 PETS/2024/opted-out-yet-tracked-are-regulations-enough-to-protect-your-privacy  [OpenWPM] v=— used (methodology)
   purpose: Automated browsing, profiling, and network-request collection.
   quote:   "We rely on OpenWPM [64] to set up our auditing framework."
2024 USENIX/2024/purl-safe-and-effective-sanitization-of-link-decoration  [OpenWPM] v=v0.17.0 used (methodology)
   purpose: Crawled and instrumented webpages
   quote:   "We use OpenWPM (v0.17.0) [11] and Firefox (v102) [64] for crawling."
2024 WWW/2024/adflush-a-real-world-deployable-machine-learning-solution-for-effective-advertis  [OpenWPM] v=— used (methodology)
   purpose: Crawled the top-ranked websites and collected requests.
   quote:   "With seven web crawler instances running in parallel for 11 hours using OpenWPM, we generated a total of 830,160 requests"
2025 PETS/2025/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and  [OpenWPM] v=0.26.0 used (methodology)
   purpose: Crawling sites and collecting cookies, requests, and responses.
   quote:   "BannerClick is built on top of OpenWPM [24] (version 0.26.0), which uses Firefox v121.0 with TCP disabled"
2025 PETS/2025/unmasking-the-shadows-a-cross-country-study-of-online-tracking-in-illegal-movie  [OpenWPM] v=— used (methodology)
   purpose: Automated privacy crawls and collection of requests, cookies, and JavaScript
   quote:   "we use OpenWPM [21] to conduct automatic crawls on the 383 IMSS sites"
2025 IMC/2025/fp-inconsistent-measurement-and-analysis-of-fingerprint-inconsistencies-in-evasi  [OpenWPM] v=— used (methodology)
   purpose: Crawled the honey site to inspect anti-bot APIs.
   quote:   "To determine this information, we crawl our honey site using OpenWPM."
2025 PETS/2025/understanding-regional-filter-lists-efficacy-and-impact  [OpenWPM] v=0.27.0 used (methodology)
   purpose: Crawled pages and collected web traffic and storage data.
   quote:   "Each measurement uses OpenWPM [24] (v0.27.0), which uses the Firefox browser (Version 123.0)"
2025 WWW/2025/the-first-early-evidence-of-the-use-of-browser-fingerprinting-for-online-trackin  [OpenWPM] v=— used (implementation)
   purpose: Constructing the crawling framework and recording HTTP data.
   quote:   "We used OpenWPM [25] to construct our crawling FPTrace."
2025 IMC/2025/towards-a-non-binary-view-of-ipv6-adoption  [OpenWPM] v=— used (methodology)
   purpose: Loaded websites and recorded browser network activity.
   quote:   "We access websites using OpenWPM web privacy measurement framework [20]. For each website, OpenWPM spawns a Firefox browser to load and render the main page."
2026 WWW/2026/tgnn-enhancing-pixel-tracking-detection-via-llm-driven-annotation-and-gat-powere  [OpenWPM] v=— used (methodology)
   purpose: Collected browser traffic from crawled websites.
   quote:   "we employed the OpenWPM framework and developed scripts to drive the Firefox browser with simulated browsing behaviors."


=== Y. Figures the page derives by arithmetic from the tables above ===

  statefulness reporting ratio, OpenWPM papers vs other crawling papers: 3.1x
  subpages-per-site reporting ratio: 3.6x
  corpus share of the CRAWLED population: 19.1% of 5859


=== Z. External figures (not from the corpus) — re-verify before republishing ===

  OpenWPM release 0.35.0, tagged 2026-06-17 — repo VERSION file and git tag
  geckodriver 0.37.1 released 2026-07-20 (current); environment.yaml pins 0.37.0
  conda pins that have no linux-aarch64 build: geckodriver 0.37.0, plyvel 1.5.1, selenium-manager 4.45
  Firefox Nightly used in the ARM substitution attempt: 156 (156.0a1, linux64-aarch64)
  Firefox releases: 152 bundled by OpenWPM 0.35.0; Firefox reached 147 during the 2025 release gap
  build tasks in the taskcluster index for the pinned revision: 202 (mozilla-release), 185 (mozilla-central)
  Firefox 52 (52.9) — the browser version stated by ahmad2020_apophanies alongside OpenWPM 0.8; that paper predates the 0.10 architecture
  GitHub issues referenced: #557 callstack, #503 no Windows, #101 tracking_protection, #448 display_mode
  Bugzilla 634073 — cached image headers not recorded
  openwpm.xpi built here: 146,356 bytes; empty crawl database: 69,632 bytes with 14 tables, 0 rows
  pytest -m pyonly: 14 passed, 135 deselected in 0.13s
  browser id in the quoted failure log: 80873956
  quote-check bands: verbatim 100%, spliced 60-99% of word positions covered
  docs/Papers.rst lists 76 studies, none later than 2021, last edited 2023-09-19
  "a crawl of 100,000 sites" — rhetorical scale, not a measurement
  62 — a print() column width inside pages/openwpm_defaults.py, not a figure
  Ubuntu 24.04 (CI) and 22.04 (the container image base) — README, Installation section
  content_hash is SHA-256 — Extension/src/lib/sha256.ts; docs/Configuration.md lines 419-420 still say md5
  73 commits in 2026 to the pinned commit b9dd4c3, 72 of them by one author
  instance_id appears in openwpm/storage/arrow_storage.py only; 0 occurrences in schema.sql
  the two Xvfb papers of the 60: CCS 2016 scored headless, WWW 2020 (zeber) scored headful
  917,261 and 94,144 — the site counts of englehardt2016online's "Default Stateless 1 Million"
       and "Default Stateful 100,000" configurations, from that paper's configuration table
  Firefox 102 — stated by shaoor2023cookiegraph and shaoor2024purl for their crawls (see section C)
  21 — the next-most-used specialised crawlers (tbselenium, Tracker Radar Collector),
       from report_crawler.mjs's mention-matching table, quoted from programming:crawler
  bundles Firefox 152 — scripts/install-firefox.sh TAG=931e624c6f53269d41e57ecefca418ef7fdb0f75 (FIREFOX_152_0_RELEASE)
  environment.yaml pins geckodriver 0.37.0, selenium 4.45.0, python 3.14.6
  GitHub: 1,415 stars, 332 forks, 180 open issues, not archived — api.github.com/repos/openwpm/OpenWPM
  commits per year: 217 (2014), 185, 195, 180, 252, 569 (2019), 380, 85, 28, 23, 11 (2024), 12 (2025), 72 (2026 to 2026-07-11)
  releases: v0.31.0 2025-01-19, then a 13-month gap to v0.32.0 2026-03-03
  2026 commits: 72 of 73 by one contributor (vringar / Stefan Zabka)
  built extension openwpm.xpi: 146,356 bytes, Manifest V2, 3 experiment_apis
  callstack_instrument broken: docs/Configuration.md + issue #557
  no Windows support: README + issue #503
  unbranded Firefox needed because the extension uses privileged APIs — docs/Platform-Architecture.md:92
</file>

===== 5. Quotes checked =====

All **65** evidence quotes attached to an OpenWPM ''tools[]'' tuple anywhere in the corpus were checked, twice, by two different methods.

**First pass, ''scripts/quote_check.mjs --tools "open[ -]?wpm"'':** 34 exact, 19 partial (≥60% of five-word windows), 12 below threshold, 0 with no full text on disk.

**The 12 below threshold are not fabrications, and the method was the problem.** Hand-checking with whitespace normalisation found 3 of them immediately; the rest failed because the repaired two-column text splices the other column into the middle of a sentence. With punctuation and spaces stripped, PETS 2025 ''intractable-cookie-crumbs-…'' reads:

<code>
...builtontopofopenwpm24version0260whichusesfiretotheserunsaspopularityand...foxv1210withtcpdisabled
</code>

"Firefox v121.0" //is// in the paper; the word ''firefox'' is cut in half. A whole-quote containment test scores that quote as absent.

**Second pass, ''scripts/openwpm_quotecheck.py'', written for this page.** Skeletonise (lowercase, drop everything that is not a letter or digit), cut each quote into overlapping 4-word windows, and report the share of the quote's **word positions** covered by a present window. Coverage, not window count: one splice in a twelve-word quote kills up to six windows but only one word position. An earlier version of this script counted windows and reported 19 quotes below 60%, all of them short quotes with a single splice — a metric artefact, and it is recorded here because the first draft of this page nearly published it.

Verdict on the 65: **52 verbatim, 13 present but split by a column splice (60–99% coverage), 0 below 60%. None unsupported.**

<file python openwpm_quotecheck.py>
"""Verify every OpenWPM evidence quote against the text the extractor read.

Why this exists rather than just `scripts/quote_check.mjs --tools openwpm`: the
repaired two-column text still splices the neighbouring column into the middle of
a sentence, so a whole-quote containment test fails on quotes that are demonstrably
present. Example, from PETS 2025 `intractable-cookie-crumbs-...`, with punctuation
and spaces stripped:

    ...builtontopofopenwpm24version0260whichusesfire|toheserunsas...|foxv1210withtcpdisabled

"Firefox v121.0" is in the paper; the word `firefox` is cut in half by a column
splice. So the test here is per window:

  * skeletonise (lowercase, drop everything that is not a letter or digit)
  * cut the quote into overlapping 4-word windows
  * a window is present if its skeleton occurs in the skeletonised paper
  * report the share of the quote's WORD POSITIONS covered by a present window

A quote at 100% is verbatim. A quote in the 60-99% band is present and spliced —
each of those is listed so it can be read. Below 60% is listed as SUSPECT and must
be read by hand before anything resting on it is published.

    python3 scripts/openwpm_quotecheck.py [--all]
"""

import json
import re
import sys
from pathlib import Path

ROOT = Path("/workspace/publications_dataset/data")
RUN = ROOT / "extract/run1/extractions.jsonl"
OPENWPM = re.compile(r"open[\s-]?wpm", re.I)
SHOW_ALL = "--all" in sys.argv


def skel(s: str) -> str:
    return re.sub(r"[^a-z0-9]+", "", s.lower())


def window_share(quote: str, body: str, w: int = 4) -> tuple[int, int]:
    """Share of the quote's WORD POSITIONS covered by some matching window.

    Counting matched windows instead punishes short quotes unfairly: one column
    splice in a twelve-word quote kills up to six of its windows but only one of
    its word positions. Coverage is the honest measure of "how much of this
    sentence is in the paper".
    """
    words = [x for x in re.split(r"\s+", quote) if x]
    if len(words) <= w:
        ok = skel(quote) in body
        return (len(words) if ok else 0), len(words)
    covered = [False] * len(words)
    for i in range(len(words) - w + 1):
        if skel(" ".join(words[i : i + w])) in body:
            for j in range(i, i + w):
                covered[j] = True
    return sum(covered), len(words)


rows = [json.loads(l) for l in RUN.read_text().splitlines() if l.strip()]
verbatim = spliced = suspect = 0
for r in rows:
    tuples = [t for t in r["tools"] if OPENWPM.search(t["name"] or "")]
    if not tuples:
        continue
    txt = ROOT / "fulltext" / str(r["year"]) / r["venue"] / r["slug"] / "paper.cols.txt"
    if not txt.exists():
        print(f"NO TEXT  {r['venue']}/{r['year']}/{r['slug']}")
        continue
    body = skel(txt.read_text(errors="replace"))
    for t in tuples:
        q = t["evidence"]["quote"]
        hit, tot = window_share(q, body)
        share = hit / tot
        if share == 1:
            verbatim += 1
            verdict = "VERBATIM"
        elif share >= 0.6:
            spliced += 1
            verdict = "SPLICED "
        else:
            suspect += 1
            verdict = "SUSPECT "
        if verdict != "VERBATIM" or SHOW_ALL:
            print(f"{verdict} {share:5.0%} ({hit}/{tot})  {r['venue']}/{r['year']}/{r['slug']}")
            print(f"         [{t['name']}] ({t['evidence']['section']}) {q!r}")

print(
    f"\n{verbatim + spliced + suspect} OpenWPM tool quotes: {verbatim} verbatim, "
    f"{spliced} present but split by a column splice (60-99% coverage), "
    f"{suspect} below 60% and read by hand."
)
</file>

<file text openwpm-quotecheck.txt>
SPLICED    78% (18/23)  WWW/2015/cookies-that-give-you-away-the-surveillance-implications-of-web-tracking
         [OpenWPM] (methodology) 'In our study, we use version 0.1.0 of OpenWPM to drive Firefox measurement instances from which we record all HTTP data for analysis.'
SPLICED    89% (17/19)  CCS/2018/the-webs-sixth-sense-a-study-of-scripts-accessing-smartphone-sensors
         [OpenWPM-Mobile] (methodology) 'Our data collection is based on OpenWPM-Mobile, a mobile-focused measurement tool we built by modifying OpenWPM web measurement framework.'
SPLICED    86% (19/22)  WWW/2019/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private
         [OpenWPM] (methodology) 'Then, we use OpenWPM [11] to automatically visit the websites and some randomly selected sub-pages, creating a dataset composed of 21 crawls/harvests.'
SPLICED    86% (18/21)  PETS/2020/inferring-tracker-advertiser-relationships-in-the-online-advertising-ecosystem-u
         [OpenWPM] (methodology) 'Our measurements were conducted using a lightly modified version of OpenWPM [45]. OpenWPM was used to automatically load selected web pages.'
SPLICED    95% (18/19)  PETS/2020/missed-by-filter-lists-detecting-unknown-third-party-trackers-with-invisible-pix
         [OpenWPM] (methodology) 'We performed passive Web measurements using the OpenWPM platform [24]. It uses the Firefox browser, and provides browser automation'
SPLICED    79% (15/19)  PETS/2020/no-boundaries-data-exfiltration-by-third-parties-embedded-on-web-pages
         [custom OpenWPM instrumentation] (methodology) "We extended OpenWPM's already existing JavaScript instrumentation to capture access to a set of DOM properties and function calls."
SPLICED    79% (11/14)  WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field
         [OpenWPM] (methodology) 'we use OpenWPM to crawl these websites and gather first-party links on these websites'
SPLICED    95% (20/21)  PETS/2022/atom-ad-network-tomography
         [OpenWPM] (methodology) 'Next, we used OpenWPM [48] to crawl each remaining site to verify that they were functional and contained trackers on them.'
SPLICED    94% (15/16)  PETS/2022/omnicrawl-comprehensive-measurement-of-web-tracking-with-real-desktop-and-mobile
         [OpenWPM-Mobile] (evaluation) "We included both OpenWPM-Mobile's emulated Firefox browser and its non-emulated counterpart, mobile Firefox, in our crawl."
SPLICED    76% (16/21)  PETS/2023/comparing-large-scale-privacy-and-security-notifications
         [OpenWPM] (methodology) 'we selected four privacy issues which fit our requirements and implemented them as custom functions in an established measurement framework, OpenWPM'
SPLICED    82% (9/11)  PETS/2024/opted-out-yet-tracked-are-regulations-enough-to-protect-your-privacy
         [OpenWPM] (methodology) 'We rely on OpenWPM [64] to set up our auditing framework.'
SPLICED    82% (14/17)  PETS/2025/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and
         [OpenWPM] (methodology) 'BannerClick is built on top of OpenWPM [24] (version 0.26.0), which uses Firefox v121.0 with TCP disabled'
SPLICED    78% (7/9)  WWW/2025/the-first-early-evidence-of-the-use-of-browser-fingerprinting-for-online-trackin
         [OpenWPM] (implementation) 'We used OpenWPM [25] to construct our crawling FPTrace.'

65 OpenWPM tool quotes: 52 verbatim, 13 present but split by a column splice (60-99% coverage), 0 below 60% and read by hand.
</file>

**Read by hand, beyond the automated checks** (four quotes, all confirmed present in the paper):

^ Paper ^ Quote fragment ^ Verdict ^
| PETS/2025 ''intractable-cookie-crumbs-…'' | "built on top of OpenWPM … version 0.26.0 … Firefox v121.0 with TCP disabled" | present, split across a column splice mid-word |
| PETS/2020 ''inferring-tracker-advertiser-relationships-…'' | "lightly modified version of OpenWPM" | present |
| USENIX/2022 ''automating-cookie-consent-and-gdpr-violation-detection'' | "the OpenWPM framework, version 0.12.0" | present |
| CCS/2016 ''online-tracking-a-1-million-site-measurement-and-analysis'' | "stateful (cookie-based) and stateless (fingerprinting-based) tracking" | present — and it is the //homograph//, see §6 |

===== 6. Judgement calls =====

  - **The "both" statefulness label was doubted, and the doubt was wrong.** Five OPENWPM papers have ''crawlConfig.statefulness == "both"''. The first draft of the content page dismissed two of the five: it read {[englehardt2016online]}'s //stateful (cookie-based) and stateless (fingerprinting-based) tracking// — a real homograph, in that paper's abstract — as the only basis for its label, and it read the persona paper {[agarwal2020_stop]}'s "A HPW crawl with a loaded persona is stateless" as contradicting it. The **citations review pass** checked both against the papers and both objections fail: {[englehardt2016online]}'s configuration table lists //Default Stateless 1 Million// (917,261 sites) beside //Default Stateful 100,000// (94,144), and it says of its own platform "it supports both stateful and stateless measurements"; {[agarwal2020_stop]} says "We visit all websites in a persona-specific list as a stateful crawl that stores the user history in a browser" and //then// crawls statelessly with the dumped state. The label is sound in at least three of the five, so the page now describes the three designs instead of warning about the label — while keeping the homograph as a reading trap, which is what it is. **Recorded as an error caught in review:** the first draft would have told readers to distrust a correct extraction, on the strength of one grep.
  - **No causal claim about the ARM extension failure.** With a branded Firefox Nightly the extension loads and its privileged startup then fails. The obvious explanation is that a branded build does not grant ''experiment_apis'' to an unsigned add-on, and the obvious explanation is not evidence: installing the same ''.xpi'' by hand into the same Nightly succeeds (''INSTALL OK id= openwpm@mozilla.org''), which shows loading is not the problem but does not identify what is. Three probes were run and none settled it: a patched ''.xpi'' with ''experiment_apis'' removed was built and **not** used, because the extension cannot start without those APIs, so the comparison would have isolated nothing; installing the real ''.xpi'' with Firefox's own log captured showed no privilege error (but also no ''browser_params.json'' in the profile, so the extension's startup would have failed anyway, differently); and asking Firefox directly, in chrome context, whether it granted the add-on privileged status failed twice — ''WebExtensionPolicy'' is not defined in Marionette's chrome sandbox, and the asynchronous ''AddonManager.getAddonByID'' form timed out. The page reports the failure and says the cause was not isolated.
  - **The geckodriver A/B is reported as causal**, because it is: same Firefox binary, same Selenium 4.45.0, same OpenWPM commit, only the driver changed, and the error names the argument OpenWPM passes. Attempt 1 (0.37.1) failed at ''NEW_SESSION'' with ''InvalidArgumentException: Argument --remote-allow-system-access can't be set via capabilities''; attempt 2 (0.37.0) got past that point and failed later and differently.
  - **''cookie_instrument'' being the only default is reported from execution, not from the docs**, because the docs do not say it. Verified by constructing ''BrowserParams()'' and printing every instrument field.
  - **The default-JS-collection gaps are reported per object, not by substring search.** An earlier draft of ''openwpm_defaults.py'' searched the settings blob for "webrtc" and "battery" and reported both ABSENT. Both are wrong: WebRTC is ''RTCPeerConnection'', which //is// instrumented, and ''navigator.getBattery'' is covered by instrumenting ''window.navigator''. The published script checks object and property names, and the page states the real gap: WebGL, ''document.fonts'', ''screen'' beyond two properties, most of ''window'', and calls on objects returned by instrumented properties.
  - **Firefox version mapping is per tag, from the repository**, not from release notes. The CHANGELOG's "Bump to Firefox N" lines agree with it for 0.32.0–0.35.0.
  - **Not written up: the OpenWPM detectability literature in depth.** {[krumnow2022_gullible]} is cited on both this page and the parent; the parent's //Being Detected// section owns the topic. Duplicating it here would have made two pages that go stale separately.

===== 7. External sources =====

Every external fact on the content page was checked against a primary source on **2026-08-14**, by ''scripts/openwpm_platform_probe.sh'' where it is machine-checkable, and by reading the repository at commit ''b9dd4c3'' otherwise. A version number is a claim about today; re-run this before trusting the maintenance table.

<file bash openwpm_platform_probe.sh>
#!/usr/bin/env bash
# Every external fact about OpenWPM's install path on Programming:Crawler:OpenWPM,
# re-checked against its primary source. Prints FAILED explicitly rather than
# printing nothing, because a silent rate-limited curl reads like a pass.
#
#   bash scripts/openwpm_platform_probe.sh            # honours $GH_TOKEN
#
# Fixed inputs: the Firefox revision OpenWPM 0.35.0 pins, from
# scripts/install-firefox.sh in the repository at tag v0.35.0.
set -uo pipefail

TAG=931e624c6f53269d41e57ecefca418ef7fdb0f75   # FIREFOX_152_0_RELEASE
IDX=https://firefox-ci-tc.services.mozilla.com/api/index/v1
AUTH=()
[ -n "${GH_TOKEN:-}" ] && AUTH=(-H "Authorization: Bearer $GH_TOKEN")

say() { printf '\n== %s\n' "$1"; }

say "1. Which unbranded (add-on-devel) builds exist for the pinned Firefox"
curl -sf -m 60 "$IDX/tasks/gecko.v2.mozilla-release.revision.$TAG.firefox" |
  python3 -c '
import json,sys
ns=sorted(t["namespace"].split("firefox.")[-1] for t in json.load(sys.stdin)["tasks"])
dev=[n for n in ns if "add-on-devel" in n]
print(f"{len(ns)} build tasks for this revision")
print("add-on-devel builds:", ", ".join(dev) or "NONE")
print("linux64-aarch64-add-on-devel present:", "linux64-aarch64-add-on-devel" in dev)
print("branded linux64-aarch64-opt present:", "linux64-aarch64-opt" in ns)
' || echo "FAILED: taskcluster index"

say "2. Does the Nightly channel produce unbranded builds at all"
curl -sf -m 60 "$IDX/tasks/gecko.v2.mozilla-central.latest.firefox" |
  python3 -c '
import json,sys
ns=sorted(t["namespace"].split("firefox.")[-1] for t in json.load(sys.stdin)["tasks"])
print(f"{len(ns)} nightly build tasks; add-on-devel among them:",
      [n for n in ns if "add-on-devel" in n] or "NONE")
' || echo "FAILED: taskcluster nightly index"

say "3. Repository state"
curl -sf -m 40 "${AUTH[@]}" https://api.github.com/repos/openwpm/OpenWPM |
  python3 -c '
import json,sys
d=json.load(sys.stdin)
stars, forks = d["stargazers_count"], d["forks_count"]
issues, arch, pushed = d["open_issues_count"], d["archived"], d["pushed_at"]
print(f"stars {stars}  forks {forks}  open issues {issues}  archived {arch}  pushed {pushed}")
' || echo "FAILED: github repo"

say "4. Releases, newest first"
curl -sf -m 40 "${AUTH[@]}" "https://api.github.com/repos/openwpm/OpenWPM/releases?per_page=8" |
  python3 -c '
import json,sys
for r in json.load(sys.stdin):
    tag, when, who = r["tag_name"], r["published_at"][:10], r["author"]["login"]
    print(f"  {tag:9} {when}  by {who}")
' || echo "FAILED: github releases"

say "5. Scheduled CI on master: does it still pass"
curl -sf -m 40 "${AUTH[@]}" "https://api.github.com/repos/openwpm/OpenWPM/actions/runs?branch=master&per_page=6" |
  python3 -c '
import json,sys
for r in json.load(sys.stdin)["workflow_runs"]:
    when, name = r["created_at"][:10], r["name"][:26]
    ev, concl = r["event"], r["conclusion"]
    print(f"  {when} {name:28} {ev:9} {concl}")
' || echo "FAILED: github actions"

say "6. Published container image: which architectures"
curl -sf -m 40 "https://hub.docker.com/v2/repositories/openwpm/openwpm/tags?page_size=4" |
  python3 -c '
import json,sys
for t in json.load(sys.stdin)["results"]:
    arch=sorted({i["architecture"] for i in t.get("images",[])} - {"unknown"})
    name, when = t["name"][:12], t["last_updated"][:10]
    print(f"  {name:14} {when}  {arch}")
' || echo "FAILED: docker hub"

say "7. conda-forge: are the pinned packages available for linux-aarch64"
for pkg in geckodriver plyvel selenium-manager; do
  for sub in linux-64 linux-aarch64; do
    n=$(curl -sf -m 40 "https://api.anaconda.org/package/conda-forge/$pkg" |
        python3 -c "
import json,sys
d=json.load(sys.stdin)
print(sum(1 for f in d['files'] if f['attrs'].get('subdir')=='$sub'))" 2>/dev/null)
    printf '  %-18s %-14s %s builds\n' "$pkg" "$sub" "${n:-FAILED}"
  done
done

say "8. geckodriver releases: which one is current"
curl -sf -m 40 "${AUTH[@]}" https://api.github.com/repos/mozilla/geckodriver/releases/latest |
  python3 -c '
import json,sys
d=json.load(sys.stdin)
tag, when = d["tag_name"], d["published_at"][:10]
print(f"  latest {tag} published {when}")
print("  linux-aarch64 asset:", any("linux-aarch64" in a["name"] for a in d["assets"]))
' || echo "FAILED: geckodriver releases"
</file>

<file text openwpm-platform-probe.txt>

== 1. Which unbranded (add-on-devel) builds exist for the pinned Firefox
202 build tasks for this revision
add-on-devel builds: linux64-add-on-devel, macosx64-aarch64-add-on-devel, macosx64-add-on-devel, macosx64-x64-add-on-devel, win32-add-on-devel, win64-add-on-devel
linux64-aarch64-add-on-devel present: False
branded linux64-aarch64-opt present: True

== 2. Does the Nightly channel produce unbranded builds at all
185 nightly build tasks; add-on-devel among them: NONE

== 3. Repository state
stars 1415  forks 332  open issues 180  archived False  pushed 2026-08-06T19:53:23Z

== 4. Releases, newest first
  v0.35.0   2026-06-17  by vringar
  v0.34.0   2026-05-08  by vringar
  v0.33.0   2026-03-28  by vringar
  v0.32.0   2026-03-03  by vringar
  v0.31.0   2025-01-19  by vringar
  v0.30.0   2024-10-06  by vringar
  v0.29.0   2024-07-20  by vringar
  v0.28.0   2024-02-21  by vringar

== 5. Scheduled CI on master: does it still pass
  2026-08-13 Tests and linting            schedule  success
  2026-08-11 Tests and linting            schedule  success
  2026-08-09 Tests and linting            schedule  success
  2026-08-08 CodeQL                       schedule  success
  2026-08-07 Tests and linting            schedule  success
  2026-08-05 Tests and linting            schedule  success

== 6. Published container image: which architectures
  latest         2026-08-02  ['amd64']
  b9dd4c3a09d2   2026-08-02  ['amd64']
  25a7827f8dfd   2026-07-17  ['amd64']
  6ca11b1a8b97   2026-07-11  ['amd64']

== 7. conda-forge: are the pinned packages available for linux-aarch64
  geckodriver        linux-64       25 builds
  geckodriver        linux-aarch64  0 builds
  plyvel             linux-64       58 builds
  plyvel             linux-aarch64  0 builds
  selenium-manager   linux-64       39 builds
  selenium-manager   linux-aarch64  0 builds

== 8. geckodriver releases: which one is current
  latest v0.37.1 published 2026-07-20
  linux-aarch64 asset: True
</file>

Read out of the repository rather than an API:

^ Claim on the page ^ Source, at commit ''b9dd4c3'' ^
| privileged APIs need an unbranded build | ''docs/Platform-Architecture.md'' line 92 |
| the extension is Manifest V2, with 3 ''experiment_apis'' | ''Extension/bundled/manifest.json'' |
| ''xpinstall.signatures.required = False'', ''extensions.experiments.enabled = True'' | ''openwpm/deploy_browsers/configure_firefox.py'' |
| geckodriver is found with ''which geckodriver''; the ''.xpi'' is installed with ''install_addon(..., temporary=True)''; OpenWPM passes ''-remote-allow-system-access'' | ''openwpm/deploy_browsers/deploy_firefox.py'' |
| sites go to "the first browser available" | ''openwpm/task_manager.py'', ''execute_command_sequence'' |
| ''bot_mitigation'' = up to 10 random mouse moves, one scroll, random 1–7 s sleep | ''openwpm/commands/browser_commands.py'' (''NUM_MOUSE_MOVES = 10'', ''RANDOM_SLEEP_LOW/HIGH = 1/7'') |
| stateful/stateless semantics, ''seed_tar'' behaviour, ''failure_limit'', watchdogs, screenshot limits, ''save_content'' MD5 de-duplication, ''tracking_protection'' NOT SUPPORTED | ''docs/Configuration.md'' |
| the callstack instrument is broken | ''docs/Configuration.md'' + issue #557; confirmed by execution |
| "Use a versioned release" / "Include the OpenWPM version number" | ''README.md'', //Advice for Measurement Researchers//, quoted verbatim on the page |
| ''docs/Papers.rst'' lists 76 studies, none later than 2021 | counted from the file; ''git log -1'' on it gives 2023-09-19 |
| the fingerprinting collection has not changed since 2020-07-08 | ''git log --follow'' on ''openwpm/js_instrumentation_collections/fingerprinting.json'': two commits, the later one a module rename (''502cd83'', 2020-11-14) |
| commits per year, 63 contributors, 72 of 73 2026 commits by one author | ''git log'' on a full (unshallowed) clone |

**Rejected sources.** No SEO listicle, no "top 10 web scraping tools" page, and no blog summary of OpenWPM was used or cited. The Read the Docs build of the OpenWPM documentation was read but is **not** cited: it renders the same files as the repository, and the repository at a named commit is the checkable source. ''docs/Papers.rst'' was deliberately **not** used as a source for how many papers use OpenWPM — it stops at 2021, and using it would have understated post-2021 use by the 33 papers the corpus finds there.

===== 8. Installing and testing it: every command =====

Host: aarch64 Linux container, x86-64 binaries not executable (no ''/lib64/ld-linux-x86-64.so.2''), no conda, no Docker, no Firefox, Python 3.11, Node 22.

Everything below was run against the repository at ''b9dd4c3'' **unmodified**: after all of it, ''git status --porcelain'' reports only untracked additions — the two test scripts copied in from ''pages/'', the downloaded copy of the published one, and the crawl output directory. No tracked file was touched, so no claim here is about a patched OpenWPM.

^ # ^ Command ^ Result ^
| 1 | ''git clone https://github.com/openwpm/OpenWPM'' (then ''git fetch --unshallow'') | ok, commit ''b9dd4c3'', ''VERSION'' 0.35.0 |
| 2 | ''micromamba env create -f environment.yaml'' | **failed to solve** — 3 packages have no linux-aarch64 build (output below) |
| 3 | ''micromamba create -n owpm python=3.12 pip leveldb'' then ''pip install'' the imports of ''openwpm/'' | ok; ''pillow'' had to be added after a ''ModuleNotFoundError: PIL'' on import of ''openwpm.commands.browser_commands'' |
| 4 | ''npm ci'' in ''Extension/'' | ok — ''openwpm.xpi'', 146,356 bytes |
| 5 | ''pytest -m pyonly'' | ''14 passed, 135 deselected in 0.13s'' |
| 6 | ''python openwpm_defaults.py'' | ok, twice, byte-identical (output below) |
| 7 | download the pinned unbranded Firefox for ''linux64-aarch64'' | **404** — the build does not exist (probe §1) |
| 8 | download Firefox Nightly ''linux64-aarch64'' + geckodriver 0.37.1 for aarch64 | ok — Firefox 156.0a1, geckodriver 0.37.1 |
| 9 | ''openwpm_smoke.py'' with geckodriver **0.37.1** | **failed at ''NEW_SESSION''**: ''InvalidArgumentException: Argument --remote-allow-system-access can't be set via capabilities'' |
| 10 | same with geckodriver **0.37.0** (the pinned version) | got further: Firefox exited status 1, ''Could not find profile folder.'' |
| 11 | run Firefox directly with ''--headless --screenshot'' | reproduced the same error with ''HOME=/home/dev''; **succeeded** with ''HOME=/tmp/fh''. A container quirk, not an OpenWPM property, and not reported on the page |
| 12 | ''openwpm_smoke.py'' with geckodriver 0.37.0 and ''HOME=/tmp/fh'' | browser launched, **extension loaded**, then ''FileNotFoundError: …/extension_port.txt''; spawn unsuccessful, retried to the failure limit |
| 13 | install the same ''.xpi'' into the same Nightly by hand via Selenium | ''INSTALL OK id= openwpm@mozilla.org'' |
| 14 | inspect the database the failed crawl produced | 69,632 bytes, **13 OpenWPM tables plus SQLite's ''sqlite_sequence'', all 0 rows** — including ''incomplete_visits'' |
| 15 | after publishing: download the page's own ''<file python openwpm_defaults.py>'' block from ''/_export/code/programming:crawler:openwpm?codeblock=1'' and run it | reproduces the output quoted beside it **byte for byte**; both published blocks differ from their source files only by a stripped trailing newline |
| 16 | execute every SQL query in ''openwpm_smoke.py'' against the database the failed run produced, and validate its configuration through OpenWPM's own ''validate_browser_params'' | all pass — after the ''is_TRR'' fix in §11.1 S4; before it, one query raised ''no such column: used_trr'' |

Step 2's output, quoted on the content page:

<code>
Fetch Shard Index for conda-forge/linux-aarch64                                                 ⧖ Starting
Fetch Shard Index for conda-forge/linux-aarch64                                           ✔ Done (0.1 sec)
Fetch Shard Index for conda-forge/noarch                                                        ⧖ Starting
Fetch Shard Index for conda-forge/noarch                                                  ✔ Done (0.1 sec)
Fetching and Parsing Packages' Shards                                                           ⧖ Starting
Fetching and Parsing Packages' Shards                                                    ✔ Done (42.6 sec)

Resolving Environment                                                                           ⧖ Starting
Resolving Environment                                                                     ✔ Done (0.2 sec)
Using Cached Shard Index for conda-forge/linux-aarch64                                              ✔ Done
Using Cached Shard Index for conda-forge/noarch                                                     ✔ Done
Fetching and Parsing Packages' Shards                                                           ⧖ Starting
Fetching and Parsing Packages' Shards                                                     ✔ Done (0.4 sec)

Resolving Environment                                                                           ⧖ Starting
Resolving Environment                                                                     ✔ Done (0.2 sec)
error    libmamba Could not solve for environment specs
    The following packages are incompatible
    ├─ geckodriver =0.37.0 * does not exist (perhaps a typo or a missing channel);
    ├─ plyvel =1.5.1 * does not exist (perhaps a typo or a missing channel);
    └─ selenium =4.45.0 * is not installable because it requires
       └─ selenium-manager =4.45 *, which does not exist (perhaps a missing channel).
critical libmamba Could not solve for environment specs
</code>

Step 6's output, quoted in full on the content page:

<code>
BrowserParams() defaults
  http_instrument          False
  js_instrument            False
  cookie_instrument        True
  navigation_instrument    False
  dns_instrument           False
  callstack_instrument     False
  save_content             False
  display_mode             native
  bot_mitigation           False
  tp_cookies               always

js_instrument = True expands to:
  window['ScriptProcessorNode'].prototype      all properties
  window['GainNode'].prototype                 all properties
  window['AnalyserNode'].prototype             all properties
  window['OscillatorNode'].prototype           all properties
  window['OfflineAudioContext'].prototype      all properties
  window['AudioContext'].prototype             all properties
  window['RTCPeerConnection'].prototype        all properties
  window['HTMLCanvasElement'].prototype        all properties
  window['Storage'].prototype                  all properties
  window.navigator                             all properties
  window['CanvasRenderingContext2D'].prototype all properties
  window                                       localStorage, name, sessionStorage
  window.document                              cookie, referrer
  window.screen                                colorDepth, pixelDepth
  -> 14 instrumented objects from ['collection_fingerprinting']

Where the default collection does and does not reach:
  WebGL parameters                                               ABSENT
  canvas 2D (toDataURL, measureText)                             instrumented
  AudioContext family                                            instrumented
  WebRTC (RTCPeerConnection)                                     instrumented
  navigator.* (userAgent, plugins, getBattery, permissions, ...) instrumented
  document.cookie                                                instrumented
  document.fonts                                                 ABSENT
  screen.colorDepth                                              instrumented
  screen.width / height / availWidth                             ABSENT
  window.devicePixelRatio                                        ABSENT
  window.speechSynthesis                                         ABSENT
  localStorage / sessionStorage                                  instrumented
  note: a call on an object RETURNED by an instrumented property (navigator.mediaDevices.enumerateDevices) is not instrumented

callstack_instrument = True:
  ConfigError: Something went wrong while validating BrowserParams. Please check values provided for BrowserParams are of expected types
</code>

Step 12's log, unedited (the content page quotes five of these lines, with timestamps and logger prefixes stripped, runs of spaces collapsed, and the ''Spawn unsuccessful'' line wrapped):

<code>
2026-08-14 07:57:41,993 - MainProcess[MainThread]- browser_manager      - INFO    : BROWSER 80873956: Launching browser...
2026-08-14 07:57:42,137 - MainProcess[log-interceptor-80873956]- selenium_firefox     - DEBUG   : BROWSER 80873956: driver: 1786694262137	Marionette	INFO	Marionette enabled
2026-08-14 07:57:42,536 - MainProcess[log-interceptor-80873956]- selenium_firefox     - DEBUG   : BROWSER 80873956: driver: [GFX1-]: RenderCompositorSWGL failed mapping default framebuffer, no dt
2026-08-14 07:57:43,202 - MainProcess[log-interceptor-80873956]- selenium_firefox     - DEBUG   : BROWSER 80873956: driver: JavaScript error: undefined, line 0: Error: An unexpected error occurred
2026-08-14 07:57:43,203 - MainProcess[MainThread]- deploy_firefox       - DEBUG   : BROWSER 80873956: OpenWPM Firefox extension loaded
2026-08-14 07:57:43,204 - MainProcess[MainThread]- browser_manager      - DEBUG   : BROWSER 80873956: Looking for extension port information in /tmp/firefox_profile_p9j_ji_v
2026-08-14 07:57:48,325 - MainProcess[MainThread]- browser_manager      - ERROR   : BROWSER 80873956: Crash in driver, restarting browser manager
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/firefox_profile_p9j_ji_v/extension_port.txt'
2026-08-14 07:57:48,327 - MainProcess[MainThread]- browser_manager      - ERROR   : BROWSER 80873956: Spawn unsuccessful  | Profile Created: True  | Profile Tar: True  | Display: True  | Launch Attempted: True  | Browser Launched: True  | Browser Ready: False
</code>

The fixture the crawl was pointed at is ''sandbox/fixture.mjs'', the same local stand-in for a tracking site that [[programming:crawler]] uses for its library comparison: one page, one same-host "third-party" script that sets a JS cookie and fires a ''fetch'', one pixel, one ''Set-Cookie'' response header. It is served on ''127.0.0.1:8099'', so **it cannot exercise third-party detection** — everything on it is same-host, and OpenWPM's ''is_third_party_to_top_window'' would have been ''false'' throughout. Had the crawl succeeded, the third-party split in ''openwpm_smoke.py'' would have needed a real site (or two hostnames) to mean anything. Recorded because it would otherwise be the next run's first mistake.

===== 9. What could not be established =====

  * **The recommendation to use the amd64 container is untested here.** This host has no Docker and cannot execute x86-64 binaries, so "run the container under emulation" is an inference from the image existing and being amd64-only, not a measurement. The page says so.
  * **No completed OpenWPM crawl.** Nothing on the content page reports data produced by a working OpenWPM instrument on this host. Everything about the instruments comes from the code, the documentation and the pure-Python execution checks; everything about failure comes from a failed run. The obvious way to close this is an x86-64 host or the ''openwpm/openwpm'' container under emulation.
  * **Why the extension's privileged startup fails on a branded build** (§6).
  * **Whether the 2025 release gap had a visible effect on the literature.** The corpus cannot answer it: the newest version any paper states is 0.27.0, and only 15 papers state one.
  * **Whether papers that do not name an instrument enabled it anyway.** Only the authors or the artefact know. 33 of the 60 release an artefact publicly, so this is answerable by reading 33 repositories — a bigger job than this page.
  * **''num_browsers'' is not in the extraction schema**, so "no paper reports ''num_browsers'' alongside a stateful claim" rests on the class-1 identifier sweep (0 papers name ''num_browsers'') plus the absence of any prose figure in the papers read. It is stated on the page as "no paper we found", not as a corpus count.
  * **PETS is over-represented and the reason is not established.** 24 of 60 papers, and 19.5% of PoPETs crawling papers against ≤5.4% everywhere else. Community, review norms and topic mix are all plausible; the corpus cannot separate them.

===== 10. Bibliography additions =====

Six entries were added to [[literature:bibliography]], each verified against a primary record. No existing key was touched.

^ Key ^ Paper ^ How the entry was made ^
| ''englehardt2015_cookies'' | Cookies That Give You Away (TheWebConf 2015) | ''bibgen.mjs'' from the venue index, DOI ''10.1145/2736277.2741679'' via OpenAlex |
| ''das2018_sixth'' | The Web's Sixth Sense (CCS 2018) | ''bibgen.mjs'', DOI ''10.1145/3243734.3243860'' |
| ''englehardt2018_email'' | I never signed up for this! (PoPETs 2018) | authors from the PoPETs landing page (the index has none for PETS); DOI ''10.1515/popets-2018-0006'' confirmed against Crossref, which also gave volume/issue/pages |
| ''acar2020_noboundaries'' | No boundaries (PoPETs 2020) | as above; DOI ''10.2478/popets-2020-0070'' |
| ''utz2023_rarely'' | Privacy Rarely Considered (PoPETs 2023) | as above; DOI ''10.56553/popets-2023-0002'' |
| ''fouad2024_devil'' | The Devil is in the Details (PoPETs 2024) | as above; DOI ''10.56553/popets-2024-0125'' |

**A pre-existing duplicate was found and left alone.** ''bottger2025_regional'' and ''boettger2025_regional'' are two keys for the same paper (//Understanding Regional Filter Lists: Efficacy and Impact//, PoPETs 2025). The content page cites ''bottger2025_regional''. Deduplicating means editing whichever other pages cite the other key, which is outside this page's scope; it is logged as a separate work item.

===== 11. Review log =====

Review passes over a **frozen** snapshot of the content page, the report script, its output and these notes (''out/frozen/'', taken before any pass ran). Every pass was told that the author's context might not be exhaustive and to verify from the files. Findings are recorded whether accepted or rejected; a rejection is the only record of whether a reviewer earns its slot.

==== 11.1 Author's own pass, before the reviewers ====

^ # ^ Finding ^ Action ^
| S1 | The intro said "60 of the 1,120 papers that ran a crawl", but only 59 of the 60 are in CRAWLED; and "21 for the next tool" is a figure from the parent page's population (all 5,859 papers), not this one | **accepted**, rewritten so one sentence names one population |
| S2 | "the only specialised crawler … shared across research groups" overstates: ''tbselenium'' has 21 papers across groups | **accepted**, now "the most widely shared" |
| S3 | "Twelve years of use" was an arithmetic claim about 2015–2026 | **accepted**, reworded |
| S4 | **Bug in published code**: ''openwpm_smoke.py'' queried ''select … used_trr from dns_responses''; the column is ''is_TRR''. Every other query in the script executes against the database the failed run produced; that one raises ''no such column'' | **accepted**, fixed, re-embedded, and every query re-executed against the real database |
| S5 | "all 14 tables" counts SQLite's own ''sqlite_sequence'' as an OpenWPM table | **accepted**, now "13 of OpenWPM's tables (14 with ''sqlite_sequence'')" |
| S6 | The published smoke configuration was validated through OpenWPM's own ''validate_browser_params''/''validate_manager_params'' | no change needed, recorded |
| S7 | %%[[:Artifacts]]%% is a red link and was linked silently | **accepted**, now flagged as the parent page flags it |
| S8 | The three identifier-sweep hits were read; one of them ({[agarwal2020_stop]}) is the worked example the page was asking for and was not being used | **accepted**, added with a new bibliography entry |

==== 11.2 Figures against the script (''sonnet'') ====

Re-ran ''report_openwpm.mjs'', ''openwpm_quotecheck.py'' and ''openwpm_defaults.py'' (the last inside a live OpenWPM checkout) and confirmed all three reproduce the committed output byte for byte; checked the byte sizes, the 14 tables at 0 rows, every table in //Use in Publications//, and both ''check_page_numbers.mjs'' invocations.

^ # ^ Finding ^ Action ^
| 1 | **Blocking.** "60 of the 1,120 papers that ran a crawl" contradicts the page's own methodology section ("59 of the 60 … are inside it") | **accepted** — same as S1, found independently |
| 2 | "no year above 12% and none below 3%" is falsified by the 2026 row (1.4%) in the table immediately above | **accepted**, now scoped to the eleven complete years 2015–2024, with the two provisional years named |
| 3 | The version column paraphrased "0.9.0 (machine A); 0.7.0 (machine B)" as "0.9.0 and 0.7.0, one per machine", on a page that promises verbatim version strings | **accepted**, verbatim string restored |
| 4 | ''check_page_numbers.mjs --code'' exited 1 on ''62'' (a ''print()'' column width in the published script), so "both guards pass" was not true as invoked | **accepted**; the report's ''Z'' block now carries it with that reason, and both invocations pass |
| 5 | The new semver handling in ''check_page_numbers.mjs'' used ''report.includes(v)'', looser than the boundary-checked ''present()'' every other number goes through | **accepted**, now uses ''present()''. The reviewer confirmed it does not currently misfire; the point is that it could |
| 6 | The guard scores "21" as accounted-for because "21" appears incidentally in the report (a citation marker, "21 crawls/harvests") — a presence-anywhere weakness of the guard, not introduced here | **accepted in part**: the figure is now attributed to ''report_crawler.mjs'' in the ''Z'' block. The general weakness is real and is left as a known limitation of the guard |
| 7 | "every ''v*'' tag gives the mapping" while the printed table shows 22 of 29 releases | **accepted**, the omission is now named |
| 8 | The ''classification[].target'' ranking silently dropped the catch-all ''other'' (24), which outranks a row that is shown | **accepted**, now stated |
| 9 | This page said the content page quotes "six" log lines; it quotes five (one wrapped) | **accepted**, corrected below |
| 10 | The quoted log collapses runs of spaces, which was not disclosed | **accepted**, disclosure extended |

==== 11.3 Citations and quotes (''sonnet'') ====

Checked all 23 citekeys, the six new BibTeX entries against Crossref and the PoPETs landing pages, every quoted sentence against ''paper.cols.txt'' with punctuation stripped, and every repository and issue quote against the sources.

^ # ^ Finding ^ Action ^
| 1 | **Blocking.** The page's flagship example — "one 2024 paper crawled with the Firefox 90 of mid-2021" — is wrong: {[shaoor2024purl]} states "OpenWPM (v0.17.0) [11] and Firefox (v102) [64]", and {[shaoor2023cookiegraph]} the same. The pin is a default, not a guarantee | **accepted**, verified in both papers, and it improved the page: the version section now says the pin is a default, names the two papers that state both numbers, and labels the Firefox column nominal. A hand map in the report records which papers state a browser and what they say |
| 2 | **Blocking.** The claim that {[englehardt2016online]}'s "both" label rests on a homograph is not supported: the paper ran //Default Stateless 1 Million// and //Default Stateful 100,000// as named configurations | **accepted**, verified in the paper's configuration table; the paragraph was rewritten to describe the three designs the five papers use, and the homograph is kept only as a reading trap |
| 3 | The ''bottger2025_regional''/''boettger2025_regional'' duplicate is pre-existing and correctly disclosed | no action; a work item exists |
| — | Everything else — the README block quote, the ''docs/'' quotes, all four GitHub issues, Bugzilla 634073, the ''fingerprinting.json'' history, the 14-object list, Manifest V2 and the three ''experiment_apis'', the Krumnow characterisation, the OpenWPM-Mobile and "No boundaries" claims, and every other version string in the table — verified against the source | no change |

==== 11.4 External currency (''sonnet'') ====

Every external fact re-fetched today. The whole maintenance table reproduced exactly, all four cited issues still open, Bugzilla 634073 still ''NEW'', no PyPI package, Docker Hub still amd64-only, conda-forge still zero ''linux-aarch64'' builds for the three pins, geckodriver still 0.37.1 with its own ''CHANGES.md'' confirming the capabilities change, and — the check that mattered most — **the current** latest ''mozilla-release'' build also has no ''linux64-aarch64-add-on-devel'' task, so the ARM conclusion needs no caveat. Manifest V2 has no announced Firefox end-of-support date.

^ # ^ Finding ^ Action ^
| 1 | Issue #448 is titled //Reduce the surface for bot detection// (2019, last updated 2020); the mode choice is one sentence in it, so "see also #448 on choosing between the modes" is a generous characterisation | **accepted**; the footnote now attributes that framing to ''docs/Configuration.md'', which is where it comes from, and says what the issue actually is |
| — | Noted, not used: Firefox is now 153.0 and Selenium 4.47.0. The pages claim neither as current | no change |

==== 11.5 Generic pass (''fable'') ====

It returned late, after the pages were already published, and it justified its slot: **fifteen findings on a page three focused passes had cleared**, including a data-loss bug in code the page tells readers to copy. Every one below was verified against the repository or the corpus before being accepted; the pages were then corrected and re-saved.

^ # ^ Finding ^ Action ^
| 1 | **Blocking.** ''openwpm_smoke.py'' set ''save_content'' but passed ''None'' as the unstructured storage provider, so **every response body is dropped** — ''storage_controller.py'' logs "Tried to save content while not having provided any unstructured storage provider" and continues. The script's own check (per-table row counts in SQLite) cannot see LevelDB, so the one instrument it misconfigured was the one its check could not catch | **accepted**, verified at ''storage_controller.py:132''. The script now passes ''LevelDbProvider'', counts the distinct bodies, and carries a comment explaining the trap. Neither the figures pass (which re-ran the SQL) nor S6 (which ran ''validate_browser_params'') could have seen this: the validators never look at storage providers |
| 2 | ''content_hash'' is **SHA-256**, not MD5. The page said MD5 twice, having taken it from ''docs/Configuration.md'' lines 419–420 | **accepted**, verified: ''Extension/src/lib/sha256.ts'' computes ''crypto.subtle.digest("SHA-256", …)'' and ''response-body-listener.ts'' uses it; there is no MD5 in the content path. The pitfall is now the divergence itself. Noted against this page's own §7 boast that a default was taken "from execution, not from the docs" — and then a doc error was copied anyway |
| 3 | The claim that the extraction scores an Xvfb paper as //not stating// headlessness is false | **accepted**, measured: exactly two of the 60 papers mention Xvfb, and they were coerced to **opposite** values — ''headless'' for the 1-million-site paper, ''headful'' for {[zeber2020representativeness]}. The page now reports the coercion, which is a better argument for the same point |
| 4 | The opening box still implied the 2024 paper crawled with Firefox 90 — the error the citations pass had corrected in the body only | **accepted**. This is the documented failure mode: a qualifier survives in the section and is dropped in the box |
| 5 | "Every record carries ''visit_id'', ''browser_id'' and ''instance_id''" — ''instance_id'' has **0 occurrences in ''schema.sql''** and exists only on the Arrow/Parquet path | **accepted**, verified by grep |
| 6 | "''other'' … would rank **fourth**" — inserted into the ranking it is fifth, and the table also silently cut two rows. A fix for finding 11.2 #8 introduced this | **accepted**; the table is now labelled as the top nine with what was cut |
| 7 | The three ''produced'' papers were misidentified on both pages — including omitting {[englehardt2016online]}, the only paper whose membership actually depends on the ''produced'' rule | **accepted**, both pages corrected from report §I |
| 8 | "3% or less" against table values of 3.1/3.1/3.2%; "one paper in fifteen years" from a 2010–2026 corpus | **accepted**, both reworded |
| 9 | The maintenance table gave 72 and 73 for 2026 commits in adjacent rows (commit-date vs author-date bases) | **accepted**: 73 to the pinned commit, 72 of them by one author, on one basis |
| 10 | The snapshot the pass reviewed had already diverged from the published page (which had gained //Papers to read first//) | **accepted as process debt**: the page was edited after the freeze, which is against this site's own "freeze, review, apply" rule. Recorded rather than hidden; the new section's citekeys were checked |
| 11 | The instrument table says "works" six times on a host where no instrument ever ran | **accepted**, footnoted: only "broken" was confirmed by execution here |
| 12 | Provenance overstatements: the ''FIREFOX_STATED'' map is described as carrying a deciding //sentence// for each entry, but one entry paraphrases the extraction record; the §3 section map pointed at a heading renamed during the run | **accepted**, both corrected |
| 13 | A bracketed splice repair (''cookie_instrum''[''ent'']) that the source does not need — the token is intact in ''paper.cols.txt'' | **accepted**, brackets dropped |
| 14 | The two dual-version rows resolve only their first version, so "no such tag" is stated of a paper whose //second// version (0.7.0) was tagged | **accepted as a known limitation, annotated on the page** rather than restructured: the report's ''normalise()'' deliberately takes the first match, and the verbatim string is shown beside it |
| 15 | Both deferrals point at [[programming:stateful_stateless]], which is a notes stub; and the ARM narrative is long for what it concludes | **partly accepted**: the stub is now flagged where it is linked. The ARM section was kept at length — it is the only place a reader can check a negative claim, and negatives are the cheapest thing to get wrong |

Its "verified and fine" list is longer than its findings and is worth reading before re-checking anything: it independently reproduced every //Use in Publications// figure, the parent-page and corpus-page consistency, and some thirty repository facts.

===== 12. Addendum, 2026-08-14: fork-vs-library, and the disappearing Firefox build =====

Two pitfalls were added to the content page after publication, both from **Karel's own practice rather than from the corpus** — neither appears in the extraction, and no query on this page would have surfaced them. They are now the first two entries under //Known pitfalls//. What follows is what was verified before writing them, since the claims are stronger than "a practitioner says so".

==== 12.1 The fork-vs-library claim ====

Karel's account: OpenWPM expects you to fork it and edit the tree; groups that do so cannot upgrade, and their crawlers stop working, which is why authors of three-year-old artefacts often cannot run them. The counter-practice is to treat it as a library and monkey-patch, as Ahmed Bouhoula did for the ALSAcnc crawler, which was upgraded at least four times (starting around v0.21.1, before the public repository existed).

^ Claim ^ How checked on 2026-08-14 ^ Result ^
| No library packaging | PyPI, and the maintenance table's own row | No ''openwpm'' package; installation is conda plus shell scripts inside the checkout |
| ALSAcnc imports rather than forks | ''git clone https://github.com/bouhoula/alsacnc''; ''grep -rn "from openwpm"'' | Imports ''openwpm.config'', ''openwpm.task_manager'', ''openwpm.commands.types'', ''openwpm.storage.sql_provider''; OpenWPM itself is cloned at a tag by ''docker/crawler/Dockerfile'', never vendored |
| Monkey-patch layer | ''wc -l cookie_crawler/utils/monkey_patches/*.py'' | 5 files, **583 lines**: ''__init__.py'' 23, ''browser_manager.py'' 130, ''browser_manager_handle.py'' 194, ''command_sequence.py'' 199, ''shutdown.py'' 37. Applied at ''run_crawler.py:94'' and '':186'' |
| Upgrade history | ''git log -- docker/crawler/Dockerfile'', then ''git show <c>:docker/crawler/Dockerfile'' | ''235a510'' 2024-06-22 → v0.23.0; ''d5c449a'' 2024-09-24 → v0.29.0; ''3c6cfd1'' 2025-01-29 → v0.31.0. Three pins in the public history; the earlier step onto v0.23.0 predates the repository, so the page says "in its public history alone" and does not claim four |
| Upstream agrees | [[https://github.com/openwpm/OpenWPM/issues/964|issue #964]], comment by ''vringar'' 2021-12-16 | Quoted verbatim on the page, including "We should provide a better story here"; the same comment dates the separation-enabling work (PRs #743, #753) to v0.14.0 |

Note for honesty about sourcing: issue #964 was opened by ''Bender250'' — Karel — so the "upstream agrees" quote is a maintainer replying to the same person whose experience prompted this addendum. That does not weaken the quote (it is the maintainer's own assessment of his project), but a reader deserves to know the thread is not independent of the claim.

The DuckDuckGo Tracker Radar Collector contrast is Karel's. Checked: its ''package.json'' declares ''"main": "main.js"'' and its README documents ''require("tracker-radar-collector")'' with exported ''crawlerConductor'', ''crawler'' and collector classes — a library API, although it is installed from GitHub rather than from npm (the npm name is a security placeholder package). The contrast is about packaging and extension surface, not about quality.

==== 12.2 The disappearing Firefox build ====

Karel's account: Mozilla does not keep the old binaries, so a handful of OpenWPM users trade tarballs of e.g. Firefox 80 between themselves whenever an artefact is missing one. Measured rather than asserted, with ''scripts/openwpm_firefox_availability.sh'' (committed; output below): for each ''v*'' tag, read the pinned hg revision and target suffix out of ''scripts/install-firefox.sh'', rebuild the exact taskcluster index URL ''install.sh'' fetches, and HEAD it.

<file - openwpm-firefox-availability.txt>
v0.10.0 FIREFOX_77_0_1_ .tar.bz2 HTTP=404
v0.11.0 FIREFOX_78_0_1_ .tar.bz2 HTTP=404
v0.12.0 FIREFOX_80_0_ .tar.bz2 HTTP=404
v0.13.0 FIREFOX_83_0_ .tar.bz2 HTTP=404
v0.14.0 FIREFOX_86_0_1_ .tar.bz2 HTTP=404
v0.14.1 FIREFOX_86_0_1_ .tar.bz2 HTTP=404
v0.15.0 FIREFOX_88_0_ .tar.bz2 HTTP=404
v0.16.0 FIREFOX_89_0_ .tar.bz2 HTTP=404
v0.17.0 FIREFOX_90_0_2_ .tar.bz2 HTTP=404
v0.18.0 FIREFOX_95_0_ .tar.bz2 HTTP=404
v0.19.0 FIREFOX_98_0_ .tar.bz2 HTTP=404
v0.19.1 FIREFOX_98_0_ .tar.bz2 HTTP=404
v0.20.0 FIREFOX_100_0_ .tar.bz2 HTTP=404
v0.21.0 FIREFOX_105_0_ .tar.bz2 HTTP=404
v0.21.1 FIREFOX_105_0_ .tar.bz2 HTTP=404
v0.22.0 FIREFOX_114_0_1_ .tar.bz2 HTTP=404
v0.23.0 FIREFOX_115_0_3_ .tar.bz2 HTTP=404
v0.24.0 FIREFOX_117_0_ .tar.bz2 HTTP=404
v0.25.0 FIREFOX_118_0_2_ .tar.bz2 HTTP=404
v0.26.0 FIREFOX_121_0_ .tar.bz2 HTTP=404
v0.27.0 FIREFOX_122_0_1_ .tar.bz2 HTTP=404
v0.28.0 FIREFOX_123_0_ .tar.bz2 HTTP=404
v0.29.0 FIREFOX_128_0_ .tar.bz2 HTTP=404
v0.30.0 FIREFOX_130_0_1_ .tar.bz2 HTTP=404
v0.31.0 FIREFOX_134_0_1_ .tar.bz2 HTTP=404
v0.32.0 FIREFOX_148_0_ .tar.xz HTTP=200
v0.33.0 FIREFOX_149_0_ .tar.xz HTTP=200
v0.34.0 FIREFOX_150_0_2_ .tar.xz HTTP=200
v0.35.0 FIREFOX_152_0_ .tar.xz HTTP=200
</file>

Four of 29 releases are installable; the boundary sits between v0.31.0 (Firefox 134, pinned 2025-01-19) and v0.32.0 (Firefox 148, 2026-03-03). Supporting checks:

  * **It is expiry, not a moved URL or a changed suffix.** The index //namespace// lookup for v0.29.0's revision returns ''ResourceNotFound'' — the indexed task is gone, not just one artefact — and retrying that release with ''.tar.xz'' instead of ''.tar.bz2'' also 404s. The same lookup for v0.35.0's revision returns a task with ''"expires": "2027-06-10T17:30:02.505Z"'', which is where "about a year" comes from.
  * **One transient, corrected.** v0.35.0 returned HTTP 503 on the recorded run and 200 on three immediate retries; the 503 was replaced by 200 in the output above. This is the only edit made to any script output on either page, and it is recorded here rather than applied silently.
  * **The branded build is not a substitute**, and the page says so: ''archive.mozilla.org'' still serves branded Firefox 90.0.2 (HTTP 200), but the extension needs an unbranded build, per ''docs/Platform-Architecture.md'' already quoted in §7.
  * **The container escape hatch was checked, not assumed.** Docker Hub tag lookups: ''0.15.0'', ''0.16.0'', ''0.17.0'', ''0.21.1'', ''0.23.0'', ''0.29.0'' and ''0.31.0'' all return 200; ''0.10.0'', ''0.12.0'', ''0.13.0'' and ''0.14.0'' return 404 — so the tags go back to 0.15.0, not to the start. ''0.17.0'' lists ''amd64'' only. That the image contains the binary is read from the v0.17.0 ''Dockerfile'', which runs ''./install.sh'' and then ''mv firefox-bin /opt/firefox-bin'' with ''ENV FIREFOX_BINARY''; **the image was not pulled or run** — this host is aarch64, and §8 explains why nothing amd64 was executed here.
  * **The upstream issue is still open** ([[https://github.com/openwpm/OpenWPM/issues/964|#964]], opened 2021-12-15, three comments, most recent 2025-11-26), and ''install-firefox.sh'' at ''b9dd4c3'' now prints "the Firefox version may be too old and no longer available on TaskCluster" and points at it. Neither the mirror repository the maintainer floated in 2021 nor any other archive of these builds was found.

==== 12.3 Judgement calls in this addendum ====

  * **Placed as the first two pitfalls**, per Karel's instruction, ahead of the empty-database pitfall. Defensible independently: these two decide whether the crawler still exists in three years, where the others decide whether one crawl is sound.
  * **Cited ALSAcnc through the existing key** {[bouhoula2024_automated]} rather than adding a bibliography entry for the repository — no new keys. Note the pre-existing duplicate ''bouhoula2024automated'' (recorded in the archives provenance page); the underscore form used here is the one [[statistics:pvalue_corrections]] already uses.
  * **Did not claim "four upgrades"** although Karel reports four; only three are checkable in the public repository, and the page says "in its public history alone", which is true and weaker.
  * **Did not open an Open Question.** The mirror-the-binary problem has an answer today (archive the tarball, or use the container image), so it belongs in guidance rather than in the open list. Whether anyone should maintain a community mirror is a question for the project, not for this page.
  * **No new review pass was run** on this addendum. It is two bullets plus this section, every factual claim is a command whose output is recorded above, and no corpus figure was touched — but a reader should know it did not go through the four-reviewer layer the rest of the page did.

===== 13. Run log =====

  * **2026-08-14.** Corpus queries, repository reading, install attempts, page and provenance page written, four review passes, published. Corpus at 5,859 extracted papers; OpenWPM at ''b9dd4c3''.
  * **Tooling changed during the run.** ''scripts/check_page_numbers.mjs'' gained explicit handling of semver triples: the generic number regex cuts ''0.35.0'' into ''0.35'', and the ''present()'' lookahead then refuses to match that fragment inside the report's own ''0.35.0'', so 24 of this page's 47 initially-unaccounted "figures" were fragments of version numbers the report does print in full. Versions are now checked as whole tokens against the report and then removed before the generic scan — stricter than an ALLOW entry, and it does not weaken the scan for anything that is not a triple. The change is shared with every other page's check, so it was made narrow deliberately.
  * **Credential exposure, caught in review.** An early command in this run printed ''$GH_TOKEN'' to the terminal to test whether it was set, so a GitHub token is in this run's transcript. The token is not in this repository, not in any published page and not in any output file, and ''openwpm_platform_probe.sh'' reads it from the environment. It should be rotated. The lesson is the boring one: test for a variable with ''${VAR:+set}'', never by echoing it.
  * **2026-08-14, later the same day.** Karel supplied two pitfalls from his own practice (fork-vs-library; the deleted Firefox builds). Verified as §12, added to the content page as its first two //Known pitfalls//, plus one line each in //Forks and extensions//, //What to report// and the version-table preamble. New script: ''openwpm_firefox_availability.sh''; ''check_page_numbers.mjs'' gained five ALLOW entries (HTTP 404, the 583-line patch layer, and issue/PR numbers 964, 743, 753) — all external figures with no place in the report. No corpus query re-run, no figure changed, no bibliography entry added.
  * **What the next run should do first:** re-run ''openwpm_platform_probe.sh'' (the maintenance table is a claim about a date), then ''report_openwpm.mjs'' and both ''check_page_numbers.mjs'' invocations. If the corpus has grown, watch the spelling table in report §A for a fork under a new name.

====== References ======

Same keys and same shared [[literature:bibliography]] as the content page; this page adds no entries of its own.

<bibtex bibliography></bibtex>

/* A discussion block on a provenance page is arguable — comments belong on the
   content page. It is here because the six provenance pages published before this
   one all carry one, and a listing where five pages have comments enabled and one
   does not is worse than either convention applied consistently. */
~~DISCUSSION~~