docs/Platform-Architecture.md: “The Extension makes heavy use of privileged APIs and can only be installed on unbranded or custom builds of Firefox with add-on security disabled.” Repository read at commit b9dd4c3, 2026-08-14.This is an old revision of the document!
OpenWPM [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] is the closest thing web privacy measurement has to a standard instrument: a Python platform that drives an unbranded Firefox through Selenium, and records what the browser did through a privileged WebExtension rather than through the automation protocol. It is the most widely shared specialised crawler in our corpus: 60 papers used or extended it, against 21 each for the next two, on the mention-matching count over all 5,859 papers in the comparison page. 59 of those 60 are inside the 1,120 papers that ran a crawl; the exception re-analysed someone else's OpenWPM data.
This page is about running it and reading papers that ran it: which instrumentation surfaces exist and which are silent by default, what stateful and stateless mean in OpenWPM's implementation, what the version number commits you to, and what its maintenance looks like today. For the choice between OpenWPM and Playwright, Tracker Radar Collector or a patched browser, see Crawler. For the design question of whether to keep browser state at all, see Stateful stateless — currently a notes stub, so treat it as a reading list rather than an answer.
Two findings from our corpus that should change how you write your methodology section:
OpenWPM v0.17.0 ships Firefox 90 — and a 2024 paper still crawled with that 2021 release. The pin is only a default, though: that paper is one of just three that name a browser too, and it ran Firefox 102 (see An OpenWPM version is a Firefox version, by default).cookie_instrument is the only instrument enabled by default; everything else is off until you set it.
A Selenium or Playwright crawl sees the network layer and the DOM from outside. OpenWPM instruments the browser from inside, and the difference is attribution: not “a request to doubleclick.net happened” but “this frame, on this visit, with this triggering origin, in this tab, at this time”.
webRequest, webNavigation, cookies and dns, plus three experimental APIs of its own (sockets, profileDirIO, stackDump). Because it uses privileged APIs, it can only be loaded by an unbranded or custom Firefox build with add-on security disabled — a documented requirement1) and the reason scripts/install-firefox.sh downloads a specific build from Mozilla's CI rather than using whatever Firefox you have. The extension is still Manifest V2.TaskManager supervises N BrowserManager processes, each owning one Firefox and one geckodriver; a separate storage controller process serialises records. A browser that crashes is restarted and its profile restored, and the visit is recorded as incomplete rather than lost silently — in principle (see Known pitfalls).visit_id and browser_id, which is what makes a multi-browser crawl analysable at all; the Parquet path adds an instance_id per run for partitioning, and the SQLite schema has no such column.CommandSequence per site, built from GetCommand, BrowseCommand (follow a random internal link), screenshot, page-source and profile-dump commands, with per-command timeouts and retries — plus your own BaseCommand subclasses, which is how most papers extend it.
Each instrument is a boolean on BrowserParams and writes to its own table(s). The defaults matter more than the table: a BrowserParams() with nothing set records cookies and nothing else.
| Instrument | What it records | Tables | Default | Status2) |
|---|---|---|---|---|
http_instrument | request and response headers, redirect chains, POST bodies, resource_type, third-party flags, triggering and loading origin | http_requests, http_responses, http_redirects | off | works; headers for cached content are saved except for images3) |
js_instrument | method calls with arguments and property accesses, for the APIs you configure | javascript | off | works; the default configuration is narrower than it looks — see below |
cookie_instrument | cookie changes from both JavaScript and HTTP responses | javascript_cookies | on | works |
navigation_instrument | onBeforeNavigate / onCommitted with transition types and qualifiers, plus tab and window metadata | navigations | off | works |
dns_instrument | hostname, resolved addresses, CNAME, whether DoH/TRR was used; since v0.34.0 also failed resolutions | dns_responses | off | works |
callstack_instrument | JavaScript call stacks for HTTP requests | callstacks | off | broken. Enabling it raises ConfigError4) |
save_content | response bodies, all of them or filtered by resource type (“script”, “script,image”, …) | LevelDB, joined to http_responses.content_hash | off | works; needs plyvel and disk |
Not instruments, but collected on request by a command: viewport and stitched full-page screenshots, top-level or recursive rendered page source, and a tar of the browser profile. Full-page screenshots are stitched by scrolling and taking viewport shots, so an infinite-scroll page is captured only to its original height.
The following is real output of the script below, run against OpenWPM at commit b9dd4c3 (0.35.0) on 2026-08-14. Two runs are byte-identical.
BrowserParams() defaults http_instrument False js_instrument False cookie_instrument True navigation_instrument False dns_instrument False callstack_instrument False save_content False display_mode native bot_mitigation False tp_cookies always js_instrument = True expands to: window['ScriptProcessorNode'].prototype all properties window['GainNode'].prototype all properties window['AnalyserNode'].prototype all properties window['OscillatorNode'].prototype all properties window['OfflineAudioContext'].prototype all properties window['AudioContext'].prototype all properties window['RTCPeerConnection'].prototype all properties window['HTMLCanvasElement'].prototype all properties window['Storage'].prototype all properties window.navigator all properties window['CanvasRenderingContext2D'].prototype all properties window localStorage, name, sessionStorage window.document cookie, referrer window.screen colorDepth, pixelDepth -> 14 instrumented objects from ['collection_fingerprinting'] Where the default collection does and does not reach: WebGL parameters ABSENT canvas 2D (toDataURL, measureText) instrumented AudioContext family instrumented WebRTC (RTCPeerConnection) instrumented navigator.* (userAgent, plugins, getBattery, permissions, ...) instrumented document.cookie instrumented document.fonts ABSENT screen.colorDepth instrumented screen.width / height / availWidth ABSENT window.devicePixelRatio ABSENT window.speechSynthesis ABSENT localStorage / sessionStorage instrumented note: a call on an object RETURNED by an instrumented property (navigator.mediaDevices.enumerateDevices) is not instrumented callstack_instrument = True: ConfigError: Something went wrong while validating BrowserParams. Please check values provided for BrowserParams are of expected types
Three things in that output are worth a sentence each.
js_instrument = True does not mean “instrument JavaScript”. It means collection_fingerprinting, a fixed list of 14 objects. The list is a good list — it is essentially the fingerprinting surface of the 2016 paper — but it is not today's surface: no WebGLRenderingContext, no document.fonts, and window.screen only for colorDepth and pixelDepth, so a script reading screen.width leaves no row in the javascript table. The file that defines it, openwpm/js_instrumentation_collections/fingerprinting.json, has had its content unchanged since it was introduced on 2020-07-08; the only commit touching it since was a module rename. If your paper is about fingerprinting, write your own js_instrument_settings and say in the paper which APIs it covered. Fingerprinting has the list of what is worth covering.
Instrumentation is per object, not per reachable path. window.navigator with all properties catches navigator.userAgent, navigator.plugins, a call to navigator.getBattery and a read of navigator.mediaDevices — but a call to navigator.mediaDevices.enumerateDevices() happens on a different object and is not recorded. Reads of the property, yes; the call it leads to, no.
tp_cookies defaults to always. The crawler accepts all third-party cookies by default, which is not what a stock Firefox has done for years. That is the right default for measuring what trackers try to do, and the wrong one for measuring what a Firefox user experiences; either way it is a choice your paper should state.
"""Print what an OpenWPM crawl records if you do not configure it. Run inside an OpenWPM checkout, in its environment: python openwpm_defaults.py Answers three questions the documentation answers only in prose: 1. which instruments are on when you construct BrowserParams() and touch nothing 2. what `js_instrument = True` actually instruments, after the `collection_fingerprinting` shortcut is expanded 3. whether the broken callstack instrument fails loudly Nothing here starts a browser, so it runs anywhere OpenWPM imports. """ from openwpm.config import BrowserParams, ConfigError, validate_browser_params from openwpm.js_instrumentation import clean_js_instrumentation_settings INSTRUMENTS = [ "http_instrument", "js_instrument", "cookie_instrument", "navigation_instrument", "dns_instrument", "callstack_instrument", "save_content", ] bp = BrowserParams() print("BrowserParams() defaults") for name in INSTRUMENTS: print(f" {name:24} {getattr(bp, name)}") print(f" {'display_mode':24} {bp.display_mode}") print(f" {'bot_mitigation':24} {bp.bot_mitigation}") print(f" {'tp_cookies':24} {bp.tp_cookies}") print("\njs_instrument = True expands to:") bp.js_instrument = True settings = clean_js_instrumentation_settings(bp.js_instrument_settings) for s in settings: props = s["logSettings"]["propertiesToInstrument"] which = "all properties" if not props else ", ".join(sorted(props)) print(f" {s['object']:44} {which}") print(f" -> {len(settings)} instrumented objects from " f"{bp.js_instrument_settings}") # Which fingerprinting surfaces the default collection reaches. Two things make # this less obvious than a substring search: instrumenting `window.navigator` # with all properties covers every navigator-based probe, but only the ACCESS on # navigator — a call on the object a navigator property returns # (navigator.mediaDevices.enumerateDevices()) is a call on a different object and # is not instrumented. And `window` and `window.screen` are instrumented for # three and two named properties only, so everything else on them is invisible. objects = {s["object"] for s in settings} props = { s["object"]: set(s["logSettings"]["propertiesToInstrument"]) for s in settings } # sets, but every display of them is sorted so the output is reproducible def reached(obj, prop=None): if obj not in objects: return "ABSENT" if prop is None: return "instrumented" p = props[obj] return "instrumented" if not p or prop in p else "ABSENT" print("\nWhere the default collection does and does not reach:") CHECKS = [ ("WebGL parameters", "window['WebGLRenderingContext'].prototype", None), ("canvas 2D (toDataURL, measureText)", "window['HTMLCanvasElement'].prototype", None), ("AudioContext family", "window['AudioContext'].prototype", None), ("WebRTC (RTCPeerConnection)", "window['RTCPeerConnection'].prototype", None), ("navigator.* (userAgent, plugins, getBattery, permissions, ...)", "window.navigator", None), ("document.cookie", "window.document", "cookie"), ("document.fonts", "window.document", "fonts"), ("screen.colorDepth", "window.screen", "colorDepth"), ("screen.width / height / availWidth", "window.screen", "width"), ("window.devicePixelRatio", "window", "devicePixelRatio"), ("window.speechSynthesis", "window", "speechSynthesis"), ("localStorage / sessionStorage", "window", "localStorage"), ] for label, obj, prop in CHECKS: print(f" {label:62} {reached(obj, prop)}") print(" note: a call on an object RETURNED by an instrumented property" " (navigator.mediaDevices.enumerateDevices) is not instrumented") print("\ncallstack_instrument = True:") bp.callstack_instrument = True try: validate_browser_params(bp) print(" accepted — the instrument was repaired, check issue #557") except ConfigError as e: print(f" ConfigError: {e}")
The general design question — whether to carry browser state at all — is Stateful stateless. What follows is only how OpenWPM implements it, because the implementation has consequences the design question does not.
reset=True when constructing each CommandSequence; the browser is restarted with a fresh profile afterwards. There is no stateless=True in ManagerParams, which is why papers describe this in prose and reviewers cannot check it.ManagerParams(num_browsers=N) runs N browsers, and execute_command_sequence assigns each site to whichever browser is free first. With N browsers your stateful crawl of M sites is N independent cookie jars over an arbitrary, run-dependent partition of the site list — not one user browsing M sites. If the claim you want to make is about accumulated state (cookie syncing, respawning, retargeting), N is part of your design and belongs in the paper; if you want one user, num_browsers=1 and accept the wall-clock cost.seed_tar is loaded on the first launch of a stateful crawl but not on crash recovery, where the most recent pre-crash profile is used instead; in a stateless crawl it is re-loaded for every visit, and OpenWPM's own documentation warns the result is “very likely incomplete”, because cookies and storage set during the visit are not written back into the seed. A consent-cookie seed that you believe is being reapplied while it is quietly being overwritten by crash recovery is a hard bug to see in the data.profile_archive_dir is saved by the platform on close and on a platform-level crash; the dump_profile command runs only if the browser gets that far. For long crawls, use the parameter.In the corpus, OpenWPM papers state which mode they used far more often than other crawling papers — 33 of 60 (55.0%) against 186 of 1,061 (17.5%) — which is what you would expect of a tool whose documentation names the choice. Of the 60: 16 stateless, 12 stateful, 5 labelled as doing both, 26 not stated, 1 with no crawl-configuration record at all.
Those five are the ones to read if you are choosing, and they use the two modes in three different ways. The 1-million-site paper [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] ran them as separate crawls of different sizes — its configuration table lists Default Stateless 1 Million (917,261 sites) beside Default Stateful 100,000 (94,144) — because a stateful crawl of a million sites is not affordable. The email-tracking study [2Englehardt, Steven; Han, Jeffrey; Narayanan, Arvind (2018): "I never signed up for this! Privacy implications of email tracking", Proceedings on Privacy Enhancing Technologies 2018(1):109-126. (DOI)] runs them as a matched pair, loading each email “twice in its own measurement instance: once with a fresh profile, and then again keeping the same browser profile”. The persona study [3Agarwal, Pushkal; Joglekar, Sagar; Papadopoulos, Panagiotis; Sastry, Nishanth; Kourtellis, Nicolas (2020): "Stop tracking me Bro! Differential Tracking of User Demographics on Hyper-Partisan Websites", in: Proceedings of the ACM Web Conference. (DOI)] uses them in sequence: stateful browsing to build a demographic profile, whose state is then dumped and re-loaded for stateless measurement crawls. Note the trap on the way in, though: “stateful” and “stateless” are also used in this literature for types of tracking — cookies versus fingerprinting — and [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)]'s abstract uses them in that second sense in the same paper that runs both crawl modes. Read the sentence, not the word.
Every OpenWPM release pins one unbranded Firefox build, in one line of scripts/install-firefox.sh. Reading that line out of every v* tag gives the mapping. Note before you read it as an installation recipe: for every release below v0.32.0 the build that line points at no longer exists — see Known pitfalls. The table below omits seven point releases for width (0.11.0, 0.14.1, 0.16.0, 0.19.0, 0.21.0, 0.24.0, 0.33.0); the complete map is in report_openwpm.mjs. It is the reason the README asks for the version number:
| OpenWPM | Tagged | Firefox | OpenWPM | Tagged | Firefox | |
|---|---|---|---|---|---|---|
| 0.10.0 | 2020-06-22 | 77 | 0.23.0 | 2023-08-03 | 115 | |
| 0.12.0 | 2020-08-26 | 80 | 0.25.0 | 2023-10-21 | 118 | |
| 0.13.0 | 2020-11-19 | 83 | 0.26.0 | 2023-12-24 | 121 | |
| 0.14.0 | 2021-03-16 | 86 | 0.27.0 | 2024-02-08 | 122 | |
| 0.15.0 | 2021-05-10 | 88 | 0.28.0 | 2024-02-21 | 123 | |
| 0.17.0 | 2021-07-24 | 90 | 0.29.0 | 2024-07-15 | 128 | |
| 0.18.0 | 2021-12-12 | 95 | 0.30.0 | 2024-10-02 | 130 | |
| 0.19.1 | 2022-03-31 | 98 | 0.31.0 | 2025-01-19 | 134 | |
| 0.20.0 | 2022-05-18 | 100 | 0.32.0 | 2026-03-03 | 148 | |
| 0.21.1 | 2022-10-13 | 105 | 0.34.0 | 2026-05-08 | 150 | |
| 0.22.0 | 2023-06-25 | 114 | 0.35.0 | 2026-06-17 | 152 |
The project is explicit about what it wants from researchers, and about why:
Use a versioned release. […] Versions more than a few months out of date will use unsupported versions of Firefox, which are likely to have known security vulnerabilities. Versions less than v0.10.0 are from a previous architecture and should not be used.
Include the OpenWPM version number in your publication. As of v0.10.0 OpenWPM pins all python, npm, and system dependencies. Including this information alongside your work will allow other researchers to contextualize the results, and can be helpful if future versions of OpenWPM have instrumentation bugs that impact results.5)
Both halves are routinely ignored. 15 of the 60 papers state a version, and the versions they state are old: a 2024 paper crawled with v0.17.0 of mid-2021 [4Munir, Shaoor; Lee, Patrick; Iqbal, Umar; Shafiq, Zubair; Siby, Sandra (2024): "PURL: Safe and Effective Sanitization of Link Decoration", in: 33rd USENIX Security Symposium (USENIX Security 24), pp. 4103-4120. USENIX Association, Philadelphia, PA. (Link)]. The full table with each paper's lag is on Which version, and how far behind.
The pin is a default, not a guarantee — check before you infer the browser. OpenWPM launches whatever FIREFOX_BINARY points at, so a paper can pair an old OpenWPM with a newer browser, and two of them do. CookieGraph [5Munir, Shaoor; Siby, Sandra; Iqbal, Umar; Englehardt, Steven; Shafiq, Zubair; Troncoso, Carmela (2023): "CookieGraph: Understanding and Detecting First-Party Tracking Cookies", pp. 3490–3504. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] and PURL [4Munir, Shaoor; Lee, Patrick; Iqbal, Umar; Shafiq, Zubair; Siby, Sandra (2024): "PURL: Safe and Effective Sanitization of Link Decoration", in: 33rd USENIX Security Symposium (USENIX Security 24), pp. 4103-4120. USENIX Association, Philadelphia, PA. (Link)] both write “OpenWPM (v0.17.0) and Firefox (v102)”, where v0.17.0 ships Firefox 90. They are also the only two papers of the fifteen that state both numbers for the crawl; a third [6Ahmad, Syed Suleman; Dar, Muhammad Daniyal; Zaffar, Muhammad Fareed; Vallina-Rodriguez, Narseo; Nithyanand, Rishab (2020): "Apophanies or Epiphanies? How Crawlers Impact Our Understanding of the Web", in: Proceedings of The Web Conference, pp. 271-280. (DOI)] states “0.8; Firefox 52.9”, which is consistent with the pin of that era. So the Firefox column in the table below is nominal — what that release ships — and for twelve of the fifteen papers it is the best a reader can do, which is the whole argument for reporting both.
openwpm package on PyPI and no stable public API surface: the documented path clones the repository, builds a conda environment inside it, and expects your crawl script to live in that tree. The obvious thing to do next is to edit the tree, and it is the mistake. Every subsequent upgrade is then a merge against a codebase that has repeatedly rewritten the parts you edited (the whole architecture at v0.10.0, the command and storage interfaces at v0.14.0), so in practice the upgrade never happens: a crawler built by editing OpenWPM stops being buildable roughly when its pinned Firefox stops being downloadable (next bullet), which is why a three-year-old artefact so often cannot be run by its own authors. Contrast DuckDuckGo's Tracker Radar Collector, which is a library you depend on.git clone –branch v0.xx.0 in your Dockerfile, keep every line you write in your own repository, and express changes as BaseCommand subclasses where the API allows and monkey patches where it does not — so an upgrade is a version bump plus whatever patches broke, and the patches are a short, reviewable list of exactly where you diverge from upstream. A worked example is the ALSAcnc crawler [7Bouhoula, Ahmed; Kubicek, Karel; Zac, Amit; Cotrini, Carlos; Basin, David (2024): "Automated Large-Scale Analysis of Cookie Notice Compliance", in: Proceedings of the USENIX Security Symposium. (Link)], whose cookie_crawler/utils/monkey_patches/ is five files and 583 lines rebinding BrowserManager, BrowserManagerHandle, CommandSequence, the storage-controller shutdown and the default screen resolution; on that footing it tracked upstream through v0.23.0 → v0.29.0 → v0.31.0 in its public history alone6) — while a fork of any of those releases would by now be unbuildable. Upstream agrees this is the right shape and says the project has not made it easy: “Iirc other people keep the OpenWPM code and their code separated by using a git submodule. Tbh I always assumed that people would just check out one version and build on top of it and never upgrade. […] We should provide a better story here.”7)scripts/install-firefox.sh fetches the unbranded build from Mozilla's CI index, not from a release archive, and those artefacts expire: the index entry for the build 0.35.0 pins reports expires: 2027-06-10. We resolved the pinned URL for all 29 releases from v0.10.0 on 2026-08-14; only the four 2026 releases (Firefox 148–152) still download. Everything up to v0.31.0 — including the Firefox 134 pinned as recently as January 2025 — returns HTTP 404. install.sh therefore fails on any older release, which is the state most published artefacts are in; the failure has been open since 2021, and the current script at least names it in the error message. The branded build of the same version is still on archive.mozilla.org and is not a substitute — the instrumenting extension needs privileged APIs that only an unbranded build will load.target.tar.* you actually crawled with, alongside your data, and give its version and checksum in the paper. If you are reviving someone else's crawl, the practical escape hatch is the container image — docker pull openwpm/openwpm:0.17.0 still works today (tags go back to 0.15.0, linux/amd64 only) and the image bakes the Firefox binary in at /opt/firefox-bin. Failing both, ask around: a handful of groups keep private copies of these tarballs and pass them between each other, which is not a reproducibility story anyone should be content with.crawl-data.sqlite containing all 13 of OpenWPM's tables (14 with SQLite's own sqlite_sequence), all of them empty — including incomplete_visits, because a visit that never started is never recorded as incomplete. We produced exactly that file (see Installing it, and what we could and could not run). Print row counts per table and check crawl_history.command_status before you analyse anything. A crawl that half-failed is worse than one that failed: it looks like a low-tracking result.bot_mitigation = True is not an answer to this: it performs up to ten random mouse moves, one scroll, and a random 1–7 second sleep, and its own documentation says these “aren't comprehensive and automated interaction with the site will still appear very bot-like”.headless and xvfb are not the same browser. OpenWPM's display_mode takes native, headless and xvfb; xvfb runs a full Firefox inside a virtual display, while headless is Firefox's own headless mode, in which WebGL is not supported8). If you are measuring fingerprinting, headless mode changes what the page can do and what your crawl looks like [9Vastel, Antoine; Laperdrix, Pierre; Rudametkin, Walter; Rouvoy, Romain (2018): "Fp-Scanner: The Privacy Implications of Browser Fingerprint Inconsistencies", in: Proceedings of the USENIX Security Symposium. (Link)] — and note that the default JS collection would not have recorded WebGL calls anyway. Say which of the three you used; “headless” alone does not distinguish headless from xvfb, and the extraction behind this page has no slot for it: the two papers of the 60 whose text mentions Xvfb were coerced to opposite values, headless for the 1-million-site paper and headful for [10Zeber, David; Bird, Sarah; Oliveira, Camila; Rudametkin, Walter; Segall, Ilana; Wolls´en, Fredrik; Lopatka, Martin (2020): "The Representativeness of Automated Web Crawls as a Surrogate for Human Browsing", in: Proceedings of The Web Conference 2020, pp. 167–178. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)]. Neither is wrong exactly; the dichotomy is.BrowserParams.browser accepts firefox and nothing else. A result about Chrome's behaviour cannot be obtained here, and a result about “the web” measured only in Firefox inherits Firefox's cookie policy, its tracking protection defaults and its API surface. The instrumenting extension is Manifest V2, which is itself part of why this is a Firefox tool.environment.yaml pins geckodriver=0.37.0. With the current geckodriver release, 0.37.1 (2026-07-20), browser launch fails immediately: OpenWPM passes -remote-allow-system-access (it needs system access for the privileged profile APIs) and 0.37.1 refuses it — InvalidArgumentException: Argument –remote-allow-system-access can't be set via capabilities. We measured this as a clean A/B: same Firefox, same Selenium 4.45.0, only the driver changed. Install from environment.yaml; if you install geckodriver by hand, pin it.tracking_protection does not work. It is documented as NOT SUPPORTED (issue #101). To measure with protections on you set the underlying about:config preferences through BrowserParams.prefs and report exactly which ones.memory_watchdog and process_watchdog (which kills orphaned geckodriver and Xvfb processes) exist for cloud-scale crawls; maximum_profile_size recycles a browser whose profile has grown too large — relevant precisely in a stateful crawl, where the profile is what you are accumulating. failure_limit defaults to 2 × num_browsers + 10 consecutive command failures before the crawl aborts, which on a large list is a threshold you should choose deliberately.save_content stores each distinct body once in LevelDB keyed by that hash, and http_responses.content_hash is the join — so count responses in http_responses, never rows in LevelDB. docs/Configuration.md calls it an md5 hash in two places; the extension computes crypto.subtle.digest(“SHA-256”, …)9). Also: save_content does nothing unless you pass an unstructured storage provider to TaskManager — with None every body is dropped with one logged error and no other trace.num_browsers is not free parallelism. Each browser is a full Firefox with its own profile and its own state; see the per-browser cookie jar point above, and remember that concurrency changes what the sites see (N simultaneous connections from one address) and therefore what they serve [11Jueckstock, Jordan; Sarker, Shaown; Snyder, Peter; Beggs, Aidan; Papadopoulos, Panagiotis; Varvello, Matteo; Livshits, Benjamin; Kapravelos, Alexandros (2021): "Towards Realistic and Reproducible Web Crawl Measurements", in: Proceedings of the ACM Web Conference. (DOI)].
Checked 2026-08-14, against the repository and the GitHub, Docker Hub and conda-forge APIs (the script is openwpm_platform_probe.sh, on the provenance page).
| Signal | State on 2026-08-14 |
|---|---|
| Latest release | 0.35.0, tagged 2026-06-17, bundling Firefox 152 |
| Repository | github.com/openwpm/OpenWPM, not archived; 1,415 stars, 332 forks, 180 open issues; last push 2026-08-06 |
| Release cadence, 2026 | four releases: 0.32.0 (Mar), 0.33.0 (Mar), 0.34.0 (May), 0.35.0 (Jun) — tracking Firefox 148→152 |
| Release cadence, 2025 | one release (0.31.0, 19 January), then a 13-month gap; the newest release spent 2025 pinned to Firefox 134 while Firefox reached 147 |
| Commits per year | 569 (2019), 380 (2020), 85, 28, 23, 11 (2024), 12 (2025), 73 (2026, to the pinned commit b9dd4c3) |
| Contributors | 63 all-time; of the 73 commits in 2026 up to the pinned commit, 72 are by one person (vringar, Stefan Zabka) |
| CI | “Tests and linting” runs on a two-day cron and passed on 2026-08-13; CodeQL passing |
| Container image | openwpm/openwpm on Docker Hub, rebuilt per commit (latest 2026-08-02) — linux/amd64 only |
| Python packaging | none. There is no openwpm package on PyPI; installation is conda plus two shell scripts, or the container |
| Platforms | Linux and macOS. No Windows10); no ARM Linux (see below) |
| Project's own study list | docs/Papers.rst lists 76 studies, none later than 2021, last edited 2023-09-19 — do not use it as a current index |
Read that table as one story rather than a scorecard. OpenWPM is maintained and current — a release stream that follows Firefox within weeks, green scheduled CI, a rebuilt container — and it is maintained by one person, after a 2024–2025 in which it nearly stopped. For a fresh PhD student, two practical consequences: the tool is safe to build a thesis on today, and a bug you find is your bug to fix and upstream, not a ticket someone else will take.
The documented path is ./install.sh: create the openwpm conda environment from environment.yaml, download the pinned unbranded Firefox, build the extension with npm ci. On x86-64 Linux or macOS that works and the container image is easier still. We ran it on aarch64 Linux, and it cannot work there — for three independent reasons, each of which is worth knowing before you provision a machine or a CI runner.
1. Mozilla does not build an unbranded Firefox for ARM Linux. The unbranded builds are the add-on-devel tasks in Mozilla's CI index. For the revision OpenWPM 0.35.0 pins there are six of them, and linux64-aarch64 is not among them, although a branded ARM Linux build of the same revision exists. Nightly produces no unbranded builds at all:
== 1. Which unbranded (add-on-devel) builds exist for the pinned Firefox 202 build tasks for this revision add-on-devel builds: linux64-add-on-devel, macosx64-aarch64-add-on-devel, macosx64-add-on-devel, macosx64-x64-add-on-devel, win32-add-on-devel, win64-add-on-devel linux64-aarch64-add-on-devel present: False branded linux64-aarch64-opt present: True == 2. Does the Nightly channel produce unbranded builds at all 185 nightly build tasks; add-on-devel among them: NONE
Note the third row: an unbranded Apple Silicon build does exist (macosx64-aarch64-add-on-devel), but install.sh asks for macosx64 and sets CONDA_SUBDIR=osx-64, so on an M-series Mac OpenWPM installs the x86-64 build and runs under Rosetta. That works; it is worth knowing that it is what is happening.
2. Three of the pinned conda packages have no ARM Linux build. The environment does not solve, and it is not a version-range problem — below, the solver's own output, then the conda-forge build counts it reflects:
error libmamba Could not solve for environment specs
The following packages are incompatible
├─ geckodriver =0.37.0 * does not exist (perhaps a typo or a missing channel);
├─ plyvel =1.5.1 * does not exist (perhaps a typo or a missing channel);
└─ selenium =4.45.0 * is not installable because it requires
└─ selenium-manager =4.45 *, which does not exist (perhaps a missing channel).
== 7. conda-forge: are the pinned packages available for linux-aarch64
geckodriver linux-64 25 builds
geckodriver linux-aarch64 0 builds
plyvel linux-64 58 builds
plyvel linux-aarch64 0 builds
selenium-manager linux-64 39 builds
selenium-manager linux-aarch64 0 builds
3. Substituting a branded ARM Firefox gets further than expected and still fails. With the dependencies installed by pip instead, geckodriver 0.37.0 from Mozilla's own ARM release, and Firefox Nightly 156.0a1 for linux64-aarch64 pointed at by FIREFOX_BINARY, OpenWPM launches the browser and loads the extension — and then the extension's privileged startup fails and no crawl begins. From openwpm.log, with each line's timestamp and logger prefix stripped, runs of spaces collapsed, and the last line wrapped11):
BROWSER 80873956: driver: JavaScript error: undefined, line 0: Error: An unexpected error occurred
BROWSER 80873956: OpenWPM Firefox extension loaded
BROWSER 80873956: Looking for extension port information in /tmp/firefox_profile_p9j_ji_v
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/firefox_profile_p9j_ji_v/extension_port.txt'
BROWSER 80873956: Spawn unsuccessful | Profile Created: True | Profile Tar: True | Display: True |
Launch Attempted: True | Browser Launched: True | Browser Ready: False
We did not isolate the cause and do not claim one: installing the same .xpi into the same Nightly by hand succeeds (INSTALL OK id= openwpm@mozilla.org), so the failure is in what the extension is allowed to do once loaded, not in loading it. The practical answer on ARM is the amd64 container under emulation, or an x86-64 machine. Neither could be tested here — this host has no Docker and cannot execute x86-64 binaries at all — so treat the first as what it is: the project's own recommended path (its README says OpenWPM “is commonly used via the docker container that this repo builds”, and is tested on Ubuntu 24.04 in CI), on an image that exists, is rebuilt per commit and is amd64-only.
What we could run on this host, and what it establishes:
| What | Result |
|---|---|
npm ci in Extension/ | builds; openwpm.xpi, 146,356 bytes, Manifest V2, three experiment_apis |
pytest -m pyonly (the tests that need no browser or server) | 14 passed, 135 deselected in 0.13s |
openwpm_defaults.py (above) | the defaults, the expanded JS collection, and the callstack_instrument ConfigError |
| A one-site crawl of a local fixture | failed at browser startup, three times, for the reasons above; produced a 69,632-byte SQLite with all 13 OpenWPM tables present and 0 rows |
The crawl script is below. It is the smallest thing that turns on every working instrument and then checks that each one recorded something, which is the step a crawl of 100,000 sites needs and rarely has.
"""Smallest OpenWPM crawl that exercises every working instrument, then reports what each one actually recorded. Run from an OpenWPM checkout (it imports `openwpm`), with FIREFOX_BINARY pointing at a Firefox that will load an unsigned privileged extension and `geckodriver` on PATH: FIREFOX_BINARY=/path/to/firefox \ PATH=/path/to/geckodriver-dir:$PATH \ python openwpm_smoke.py http://127.0.0.1:8099/ Prints one row per SQLite table, which is the check that matters: an instrument that is enabled and silent looks exactly like an instrument that is off. """ import sqlite3 import sys from pathlib import Path from openwpm.command_sequence import CommandSequence from openwpm.commands.browser_commands import GetCommand from openwpm.config import BrowserParams, ManagerParams from openwpm.storage.leveldb import LevelDbProvider from openwpm.storage.sql_provider import SQLiteStorageProvider from openwpm.task_manager import TaskManager SITES = sys.argv[1:] or ["http://127.0.0.1:8099/"] DATA = Path("./smokedir") manager_params = ManagerParams(num_browsers=1) manager_params.data_directory = DATA manager_params.log_path = DATA / "openwpm.log" bp = BrowserParams(display_mode="headless") bp.http_instrument = True bp.cookie_instrument = True bp.navigation_instrument = True bp.js_instrument = True bp.dns_instrument = True # callstack_instrument is deliberately left off: it raises ConfigError (issue #557). bp.save_content = "script" # response bodies for scripts only, into LevelDB browser_params = [bp] # save_content needs an UNSTRUCTURED storage provider as well. Pass None and # OpenWPM logs one error per body and drops it (storage_controller.py: "Tried to # save content while not having provided any unstructured storage provider"), # which no per-table row count can see — exactly the silent-instrument failure # this script exists to catch. CONTENT = DATA / "content.ldb" with TaskManager( manager_params, browser_params, SQLiteStorageProvider(DATA / "crawl-data.sqlite"), LevelDbProvider(CONTENT), ) as manager: for i, site in enumerate(SITES): cs = CommandSequence(site, site_rank=i, callback=lambda ok, u=site: print( f"{'ok ' if ok else 'FAIL'} {u}")) cs.append_command(GetCommand(url=site, sleep=3), timeout=60) manager.execute_command_sequence(cs) db = sqlite3.connect(DATA / "crawl-data.sqlite") print(f"\n{'table':22} rows") for (t,) in db.execute( "select name from sqlite_master where type='table' order by name" ): print(f"{t:22} {db.execute(f'select count(*) from {t}').fetchone()[0]}") print("\njavascript: top symbols") for row in db.execute( "select symbol, operation, count(*) c from javascript group by 1,2 order by c desc limit 12" ): print(" %-46s %-8s %d" % row) print("\nhttp_requests: third-party split") for row in db.execute( "select is_third_party_to_top_window, count(*) from http_requests group by 1" ): print(" is_third_party_to_top_window=%s %d" % row) print("\njavascript_cookies:") for row in db.execute( "select record_type, host, name, is_http_only from javascript_cookies" ): print(" %-8s %-24s %-14s http_only=%s" % row) print("\ndns_responses:") # The column is is_TRR, not used_trr: the schema documentation calls it "whether # Trusted Recursive Resolver (TRR/DoH) was used" and the obvious guess is wrong. for row in db.execute("select hostname, addresses, is_TRR from dns_responses"): print(" %-24s %-30s trr=%s" % row) # The bodies do not live in SQLite, so a per-table row count cannot see them. print("\nresponse bodies in LevelDB:") try: import plyvel db2 = plyvel.DB(str(CONTENT)) print(f" {sum(1 for _ in db2.iterator(include_value=False))} distinct bodies" f" (keyed by content_hash)") except Exception as e: # no plyvel, or nothing was written print(f" could not read {CONTENT}: {e}")
If you are provisioning for OpenWPM: x86-64 Linux, or the openwpm/openwpm container. Not ARM Linux, not Windows. On Apple Silicon expect Rosetta.
The figures below come from a structured extraction over 5,859 full-text papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. The population is the 60 papers whose tool list names OpenWPM as used or produced; where a comparison is drawn, the baseline is the 1,061 other papers that ran a crawl. Sentinel values (not-stated) are reported as themselves and never as answers. The 2025 and 2026 venue-years are provisional — CCS and IMC 2026 have not been held, and IEEE S&P and WWW 2026 abstracts are not yet in the selection source — so any row reaching them is under-represented by construction. Methodology and limitations are at the end of this section.
| Venue | OpenWPM papers | Share of 60 | Crawling papers at that venue | OpenWPM's share of those |
|---|---|---|---|---|
| PETS | 24 | 40.0% | 123 | 19.5% |
| TheWebConf | 13 | 21.7% | 242 | 5.4% |
| USENIX Security | 7 | 11.7% | 221 | 3.2% |
| IMC | 6 | 10.0% | 132 | 4.5% |
| CCS | 5 | 8.3% | 163 | 3.1% |
| NDSS | 4 | 6.7% | 129 | 3.1% |
| IEEE S&P | 1 | 1.7% | 110 | 0.9% |
OpenWPM is a PETS instrument first. Nearly a fifth of PoPETs crawling papers use it, against about 3% at CCS, NDSS and USENIX Security, and one paper in the whole corpus at IEEE S&P. If your reviewers come from the security venues, they will not treat “we used OpenWPM” as self-explanatory; if they come from PETS, they may.
| Year | OpenWPM papers | Crawling papers | OpenWPM's share |
|---|---|---|---|
| 2015 | 2 | 41 | 4.9% |
| 2016 | 2 | 40 | 5.0% |
| 2017 | 3 | 51 | 5.9% |
| 2018 | 3 | 61 | 4.9% |
| 2019 | 3 | 97 | 3.1% |
| 2020 | 9 | 75 | 12.0% |
| 2021 | 5 | 75 | 6.7% |
| 2022 | 9 | 110 | 8.2% |
| 2023 | 10 | 125 | 8.0% |
| 2024 | 7 | 110 | 6.4% |
| 2025* | 6 | 129 | 4.7% |
| 2026* | 1 | 69 | 1.4% |
Use in every year from 2015 on, and in the eleven complete years 2015–2024 no year is above 12% or below 3%: OpenWPM is a steady minority instrument, not a rising or falling one. The 2025 and 2026 rows are lower (4.7% and 1.4%) and should not be read as a decline — those are the provisional venue-years, and 2026 is one paper.
15 of 60 papers (25.0%) state a version. Version strings are free text and are reported verbatim, never folded. “Releases behind” counts tagged releases between the version used and the newest release existing on 1 January of the paper's venue year — an upper bound on how out of date the authors were when they crawled, and a lower bound on how out of date the published measurement is when you read it.
| Paper | Version as written | Released | Firefox the release ships | Newest release that January | Behind |
|---|---|---|---|---|---|
| [14Englehardt, Steven; Reisman, Dillon; Eubank, Christian; Zimmerman, Peter; Mayer, Jonathan R.; Narayanan, Arvind; Felten, Edward W. (2015): "Cookies That Give You Away: The Surveillance Implications of Web Tracking", in: Proceedings of the ACM Web Conference. (DOI)] Cookies that give you away (2015) | 0.1.0 | pre-tag | ? | 0.2.1 (2014-12-31) | — |
| [6Ahmad, Syed Suleman; Dar, Muhammad Daniyal; Zaffar, Muhammad Fareed; Vallina-Rodriguez, Narseo; Nithyanand, Rishab (2020): "Apophanies or Epiphanies? How Crawlers Impact Our Understanding of the Web", in: Proceedings of The Web Conference, pp. 271-280. (DOI)] Apophanies or Epiphanies (2020) | 0.8; Firefox 52.9 | pre-0.10 architecture | ? | 0.8.0 (2017-10-09) | — |
| [15Fouad, Imane; Santos, Cristiana; Legout, Arnaud; Bielova, Nataliia (2022): "My Cookie is a phoenix: Detection, measurement, and lawfulness of cookie respawning with browser fingerprinting", in: PETS 2022-22nd Privacy Enhancing Technologies Symposium. (DOI) (Link)] My Cookie is a Phoenix (2022) | 0.9.0 (machine A); 0.7.0 (machine B) | no such tag | ? | 0.18.0 (2021-12-12) | — |
| [16Bollinger, Dino; Kubicek, Karel; Cotrini, Carlos; Basin, David (2022): "Automating Cookie Consent and GDPR Violation Detection", in: 31st USENIX Security Symposium (USENIX Security 22), pp. 2893-2910. USENIX Association, Boston, MA. (Link)] Automating consent and violation detection (2022) | 0.12.0 | 2020-08-26 | 80 | 0.18.0 (2021-12-12) | 7 |
| [17Iqbal, Umar; Wolfe, Charlie; Nguyen, Charles; Englehardt, Steven; Shafiq, Zubair (2022): "Khaleesi: Breaker of Advertising and Tracking Request Chains", in: 31st USENIX Security Symposium (USENIX Security 22), pp. 2911-2928. USENIX Association, Boston, MA. (Link)] Khaleesi (2022) | 0.10.0 | 2020-06-22 | 77 | 0.18.0 (2021-12-12) | 9 |
| [12Demir, Nurullah; Große-Kampmann, Matteo; Urban, Tobias; Wressnegger, Christian; Holz, Thorsten; Pohlmann, Norbert (2022): "Reproducibility and Replicability of Web Measurement Studies", in: Proceedings of the ACM Web Conference. (DOI)] Reproducibility and replicability (2022) | v0.15.0 | 2021-05-10 | 88 | 0.18.0 (2021-12-12) | 3 |
| [5Munir, Shaoor; Siby, Sandra; Iqbal, Umar; Englehardt, Steven; Shafiq, Zubair; Troncoso, Carmela (2023): "CookieGraph: Understanding and Detecting First-Party Tracking Cookies", pp. 3490–3504. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] CookieGraph (2023) | 0.17.0 | 2021-07-24 | 90 | 0.21.1 (2022-10-13) | 6 |
| [13Demir, Nurullah; Hörnemann, Jan; Große-Kampmann, Matteo; Urban, Tobias; Pohlmann, Norbert; Holz, Thorsten; Wressnegger, Christian (2023): "On the Similarity of Web Measurements Under Different Experimental Setups", in: Proceedings of the ACM Internet Measurement Conference, pp. 356-369. ACM DOI 10.1145/3618257.3624795 is listed by DBLP but was not registered with the DOI resolver as of 2026-08-12; the DOI above resolves to the authors' institutional record of the same paper (DOI)] Similarity under different setups (2023) | v0.18.0 | 2021-12-12 | 95 | 0.21.1 (2022-10-13) | 5 |
| [18Siby, Sandra; Barman, Ludovic; Wood, Christopher; Fayed, Marwan; Sullivan, Nick; Troncoso, Carmela (2023): "Evaluating Practical QUIC Website Fingerprinting Defenses for the Masses", Proceedings on Privacy Enhancing Technologies 2023(4):79-95. (DOI)] Practical QUIC website-fingerprinting defenses (2023) | 0.17.0 | 2021-07-24 | 90 | 0.21.1 (2022-10-13) | 6 |
| [19Utz, Christine; Amft, Sabrina; Degeling, Martin; Holz, Thorsten; Fahl, Sascha; Schaub, Florian (2023): "Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites", Proceedings on Privacy Enhancing Technologies 2023(1):5-28. (DOI)] Privacy rarely considered (2023) | 0.13 | no such tag | ? | 0.21.1 (2022-10-13) | — |
| [20Fouad, Imane; Santos, Cristiana; Laperdrix, Pierre (2024): "The Devil is in the Details: Detection, Measurement and Lawfulness of Server-Side Tracking on the Web", Proceedings on Privacy Enhancing Technologies 2024(4):450-465. (DOI)] Server-side tracking (2024) | 0.14.0 and 0.19.1 | 2021-03-16 | 86 | 0.26.0 (2023-12-24) | 15 |
| [21Demir, Nurullah; Urban, Tobias; Pohlmann, Norbert; Wressnegger, Christian (2024): "A Large-Scale Study of Cookie Banner Interaction Tools and their Impact on Users' Privacy", in: Proceedings on Privacy Enhancing Technologies, pp. 5-20. (DOI)] Cookie banner interaction tools (2024) | 0.20.0 | 2022-05-18 | 100 | 0.26.0 (2023-12-24) | 7 |
| [4Munir, Shaoor; Lee, Patrick; Iqbal, Umar; Shafiq, Zubair; Siby, Sandra (2024): "PURL: Safe and Effective Sanitization of Link Decoration", in: 33rd USENIX Security Symposium (USENIX Security 24), pp. 4103-4120. USENIX Association, Philadelphia, PA. (Link)] PURL (2024) | v0.17.0 | 2021-07-24 | 90 | 0.26.0 (2023-12-24) | 11 |
| [22Rasaii, Ali; Dao, Ha; Feldmann, Anja; Javid, Mohammadmahdi; Gasser, Oliver; Gosain, Devashish (2025): "Intractable Cookie Crumbs: Unveiling the Nexus of Stateful Banner Interaction and Tracking Cookies", in: Proceedings on Privacy Enhancing Technologies, pp. 429-445. (DOI)] Intractable Cookie Crumbs (2025) | 0.26.0 | 2023-12-24 | 121 | 0.30.0 (2024-10-02) | 4 |
| [23Böttger, Christian; Demir, Nurullah; Hörnemann, Jan; Acharya, Bhupendra; Pohlmann, Norbert; Holz, Thorsten; Grosse-Kampmann, Matteo; Urban, Tobias (2025): "Understanding Regional Filter Lists: Efficacy and Impact", in: Proceedings on Privacy Enhancing Technologies, pp. 309-325. (DOI)] Regional filter lists (2025) | 0.27.0 | 2024-02-08 | 122 | 0.30.0 (2024-10-02) | 3 |
Three of the fifteen name a version that was never tagged (0.9.0, 0.13, and the 0.1.0 of 2015), so a reader cannot even resolve them to a browser. The best-documented row is arguably [22Rasaii, Ali; Dao, Ha; Feldmann, Anja; Javid, Mohammadmahdi; Gasser, Oliver; Gosain, Devashish (2025): "Intractable Cookie Crumbs: Unveiling the Nexus of Stateful Banner Interaction and Tracking Cookies", in: Proceedings on Privacy Enhancing Technologies, pp. 429-445. (DOI)], which states the OpenWPM version and the Firefox version it implies: “BannerClick is built on top of OpenWPM [24] (version 0.26.0), which uses Firefox v121.0”.
| Crawl-configuration field | OpenWPM papers (of 60) | Share | Other crawling papers (of 1,061) | Share |
|---|---|---|---|---|
| statefulness | 33 | 55.0% | 186 | 17.5% |
| headless / display mode | 10 | 16.7% | 130 | 12.3% |
| browser | 39 | 65.0% | 490 | 46.2% |
| interaction depth | 53 | 88.3% | 788 | 74.3% |
| consent action | 30 | 50.0% | 319 | 30.1% |
| subpages per site | 18 | 30.0% | 88 | 8.3% |
| repeat visits | 30 | 50.0% | 222 | 20.9% |
| authentication | 53 | 88.3% | 726 | 68.4% |
Read the rows against each other, not against 100%: both columns are reporting rates. OpenWPM papers report better on every field, most sharply on statefulness (3.1×) and subpages per site (3.6×) — the tool's configuration names the choice, so the paper names it. The exception is headless mode, 16.7%, and OpenWPM's own design explains it: with three display modes, a paper that says “we ran under Xvfb” has said something the headless/headful dichotomy has no slot for.
This is the gap that matters most for reading the literature, and it is not in the extraction schema at all — the schema records that a paper used OpenWPM, not how it was configured. So we swept the full text of all 60 papers for OpenWPM's own identifiers: its configuration keys and its output table names. A hit is unambiguous, because these strings exist nowhere else.
| OpenWPM identifier named in the paper | Papers (of 60) |
|---|---|
http_requests / http_responses (tables) | 2 |
http_instrument | 1 |
js_instrument (incl. js_instrument_settings) | 1 |
cookie_instrument | 1 |
site_visits (table) | 1 |
navigation_instrument, dns_instrument, callstack_instrument, save_content, bot_mitigation, display_mode, num_browsers, seed_tar, profile_archive_dir, collection_fingerprinting, javascript_cookies, incomplete_visits, crawl_history | 0 |
Three papers of 60 (5.0%) name any configuration key or output table. What papers do instead is describe the data in prose, and there the coverage is wide: 58 of 60 mention HTTP requests or responses, 57 mention cookies — but only 15 (25.0%) mention JavaScript API calls, and 12 (20.0%) response bodies or script contents. Those prose counts are upper bounds, because the regexes match ordinary English uses too. Two consequences:
javascript table was populated. If a paper reports “no fingerprinting on X% of sites” without saying that js_instrument was on and with what settings, the absence may be the instrument's.http_instrument, js_instrument with collection_fingerprinting plus WebGL, cookie_instrument, display_mode='xvfb', num_browsers=8.” That is fully reproducible and costs a line. One paper in the corpus does close to this and is worth copying: [3Agarwal, Pushkal; Joglekar, Sagar; Papadopoulos, Panagiotis; Sastry, Nishanth; Kourtellis, Nicolas (2020): "Stop tracking me Bro! Differential Tracking of User Demographics on Hyper-Partisan Websites", in: Proceedings of the ACM Web Conference. (DOI)] writes that its “updates include enabling the http_instrument which logs HTTP responses, requests and redirects, using the selenium headless browser to perform crawling and, setting js_instrument, cookie_instrument…”, and names the extensions it turned off as well.
Enum counts over classification[].target tuples in the 60 papers, publishable as a ranking. This is the top nine; the catch-all other (24 tuples) would rank fifth and is omitted, as are malware and javascript (3 each) below the cut:
| What the paper classified | Tuples |
|---|---|
| web-request | 68 |
| cookie | 34 |
| domain | 32 |
| website-category | 27 |
| fingerprinting-script | 20 |
| consent-notice | 11 |
| privacy-policy | 6 |
| network-traffic | 6 |
| ip-address | 5 |
The free-text detection[].phenomenon values behind those, as a ranking and never as percentages (free-text names agree run-to-run on only about a fifth of exact strings), are led by cookie syncing, canvas fingerprinting, tracking requests and browser fingerprinting. That is OpenWPM's own agenda from 2016 [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)], still being worked a decade later, with consent notices and privacy policies added since the GDPR.
Three of the 60 papers produced rather than used: OpenWPM itself [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)], OpenWPM-Mobile [24Das, Anupam; Acar, Gunes; Borisov, Nikita; Pradeep, Amogh (2018): "The Web's Sixth Sense: A Study of Scripts Accessing Smartphone Sensors", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)], and one paper's own extended instrumentation [25Acar, Gunes; Englehardt, Steven; Narayanan, Arvind (2020): "No boundaries: data exfiltration by third parties embedded on web pages", Proceedings on Privacy Enhancing Technologies 2020(4):220-238. (DOI)]. Several more describe local modifications:
collection_fingerprinting is not enough, and the one to copy.BaseCommand subclasses and monkey patches over a pinned release can be, and have been. This is the first entry under Known pitfalls, with a worked example.33 of the 60 (55.0%) release an artefact publicly, against 51.8% of other crawling papers — no better than the field, for a tool whose whole value is shared instrumentation.
/open[\s-]?wpm/i, over tools[].name with usedOrMentioned in {used, produced} — 60 papers. produced counts because a paper that built or forked the crawler crawled with it; the 3 papers that only compared or mentioned it are excluded and listed on the provenance page. Every figure, with its denominator, is printed by report_openwpm.mjs; that script's unedited output, the full-text sweep, the quote checks and the external checks are on openwpm, and corpus-level caveats are on corpus.OpenWPM, openWPM, OpenWPM-Mobile, OpenWPM Firefox extension, custom OpenWPM instrumentation) are variants of one token that the regex catches. The provenance page prints them with counts, which is how a future fork under a new name would show up as missing rather than as a wrong number.paper.cols.txt rendering, and a hyphen inserted by the two-column repair can still hide a token.crawled is defined as a paper whose crawl configuration was recorded or whose study types include an automated web crawl (1,120 papers, 19.1% of the corpus). 59 of the 60 OpenWPM papers are inside it; the one outside re-analysed a dataset someone else crawled.Six, in the order that is useful if you are about to run an OpenWPM crawl rather than write about one:
collection_fingerprinting, which is what you will need if your question is about a specific API.0.35.0 (Firefox 152). Not “the latest version”, not “OpenWPM”.js_instrument_settings, give the API list or point to it in your artefact.num_browsers. Together, not separately: they jointly determine how many cookie jars your “user” had.display_mode: native, headless or xvfb.tp_cookies, donottrack, and any prefs you set. The defaults are a policy choice, and tp_cookies defaults to accepting everything.incomplete_visits, how many commands have a non-ok command_status, and what you did about them. This is the number nobody publishes and every reviewer should ask for.collection_fingerprinting misses on a modern fingerprinting corpus. The list has not changed since 2020 and the surface has; measuring the gap is a small paper.num_browsers > 1 has this property; no paper we found reports num_browsers alongside a stateful claim.add-on-devel build matrix, or an OpenWPM-side path that does not need an unbranded build.docs/Platform-Architecture.md: “The Extension makes heavy use of privileged APIs and can only be installed on unbranded or custom builds of Firefox with add-on security disabled.” Repository read at commit b9dd4c3, 2026-08-14.docs/Configuration.md, citing Bugzilla 634073. OCSP POST bodies are also not recorded.docs/Configuration.md and issue #557: “The callstack instrument requires intricate machinery that broke in a previous Firefox version.” We confirmed by execution — see What is on by default — and note that the exception message does not say why, it says the parameters are of unexpected types.README.md, section Advice for Measurement Researchers, read 2026-08-14.bouhoula/alsacnc, docker/crawler/Dockerfile, commits 235a510 (2024-06-22, v0.23.0), d5c449a (2024-09-24, v0.29.0) and 3c6cfd1 (2025-01-29, v0.31.0); the project began on v0.21.x before that repository was published. Read 2026-08-14.vringar in issue #964, 2021-12-16, which also notes that PRs #743 and #753, released in v0.14.0, made keeping the two codebases separate easier. Read 2026-08-14.docs/Configuration.md, which points at issue #448 for “additional factors to consider when picking a display_mode”. That issue is in fact a 2019 thread titled Reduce the surface for bot detection, still open and last updated in 2020, in which the mode choice is one sentence; checked 2026-08-14.Extension/src/lib/sha256.ts and response-body-listener.ts at commit b9dd4c3; docs/Configuration.md lines 419–420 still say md5. Hashing a body with MD5 to look it up will simply miss.OpenWPM does not support windows — README, and issue #503.