| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:programming:crawler:openwpm [2026/09/17 07:24] – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude | provenance:programming:crawler:openwpm [2026/09/21 13:36] (current) – Quote-check refresh 2026-09-21: re-ran quote_check.mjs --tools open[ -]?wpm with the pypdf fallback; 12 below threshold -> 10 rescued + 2 below in both; new provenance section. Authored by Claude karel.kubicek.claude |
|---|
| All **65** evidence quotes attached to an OpenWPM ''tools[]'' tuple anywhere in the corpus were checked, twice, by two different methods. | All **65** evidence quotes attached to an OpenWPM ''tools[]'' tuple anywhere in the corpus were checked, twice, by two different methods. |
| |
| **First pass, ''scripts/quote_check.mjs %%--tools%% "open[ -]?wpm"'':** 34 exact, 19 partial (≥60% of five-word windows), 12 below threshold, 0 with no full text on disk. | **First pass, ''scripts/quote_check.mjs %%--tools%% "open[ -]?wpm"'', re-run 2026-09-21 with the PDF fallback:** 34 exact, 19 partial (≥60% of five-word windows), **10 rescued from the PDF**, **2 below threshold in both renderings**, 0 with no full text on disk. See //Quote-check refresh, 2026-09-21// at the foot of this page. |
| |
| **The 12 below threshold are not fabrications, and the method was the problem.** Hand-checking with whitespace normalisation found 3 of them immediately; the rest failed because the repaired two-column text splices the other column into the middle of a sentence. With punctuation and spaces stripped, PETS 2025 ''intractable-cookie-crumbs-…'' reads: | **The old figure was 12 below threshold, and the script now agrees with what this page worked out by hand.** 10 of the 12 are located in an independent ''pypdf'' rendering of the same ''paper.pdf'' — the stored text is what was wrong — and only **2** are below threshold in both, which is 3.1% of the 65. ''exact'' (34) and ''partial'' (19) did not move. |
| | |
| | **The 12 below threshold were not fabrications, and the method was the problem.** Hand-checking with whitespace normalisation found 3 of them immediately; the rest failed because the repaired two-column text splices the other column into the middle of a sentence. With punctuation and spaces stripped, PETS 2025 ''intractable-cookie-crumbs-…'' reads: |
| |
| <code> | <code> |
| </code> | </code> |
| |
| "Firefox v121.0" //is// in the paper; the word ''firefox'' is cut in half. A whole-quote containment test scores that quote as absent. | "Firefox v121.0" //is// in the paper; the word ''firefox'' is cut in half. A whole-quote containment test scores that quote as absent. **That quote is one of the 10 the PDF fallback now rescues**: 46% of its five-word windows in the stored rendering, 62% in ''pypdf'''s. The two still below threshold in both are ''PETS/2022/my-cookie-is-a-phoenix…'' (50%) and ''USENIX/2022/automating-cookie-consent-and-gdpr-violation-detection'' (58%), and in both the mismatch is inside the extraction's own string — a dropped citation marker — which no rendering of the PDF can repair. |
| |
| **Second pass, ''scripts/openwpm_quotecheck.py'', written for this page.** Skeletonise (lowercase, drop everything that is not a letter or digit), cut each quote into overlapping 4-word windows, and report the share of the quote's **word positions** covered by a present window. Coverage, not window count: one splice in a twelve-word quote kills up to six windows but only one word position. An earlier version of this script counted windows and reported 19 quotes below 60%, all of them short quotes with a single splice — a metric artefact, and it is recorded here because the first draft of this page nearly published it. | **Second pass, ''scripts/openwpm_quotecheck.py'', written for this page.** Skeletonise (lowercase, drop everything that is not a letter or digit), cut each quote into overlapping 4-word windows, and report the share of the quote's **word positions** covered by a present window. Coverage, not window count: one splice in a twelve-word quote kills up to six windows but only one word position. An earlier version of this script counted windows and reported 19 quotes below 60%, all of them short quotes with a single splice — a metric artefact, and it is recorded here because the first draft of this page nearly published it. |
| - **The "both" statefulness label was doubted, and the doubt was wrong.** Five OPENWPM papers have ''crawlConfig.statefulness == "both"''. The first draft of the content page dismissed two of the five: it read {[englehardt2016online]}'s //stateful (cookie-based) and stateless (fingerprinting-based) tracking// — a real homograph, in that paper's abstract — as the only basis for its label, and it read the persona paper {[agarwal2020_stop]}'s "A HPW crawl with a loaded persona is stateless" as contradicting it. The **citations review pass** checked both against the papers and both objections fail: {[englehardt2016online]}'s configuration table lists //Default Stateless 1 Million// (917,261 sites) beside //Default Stateful 100,000// (94,144), and it says of its own platform "it supports both stateful and stateless measurements"; {[agarwal2020_stop]} says "We visit all websites in a persona-specific list as a stateful crawl that stores the user history in a browser" and //then// crawls statelessly with the dumped state. The label is sound in at least three of the five, so the page now describes the three designs instead of warning about the label — while keeping the homograph as a reading trap, which is what it is. **Recorded as an error caught in review:** the first draft would have told readers to distrust a correct extraction, on the strength of one grep. | - **The "both" statefulness label was doubted, and the doubt was wrong.** Five OPENWPM papers have ''crawlConfig.statefulness == "both"''. The first draft of the content page dismissed two of the five: it read {[englehardt2016online]}'s //stateful (cookie-based) and stateless (fingerprinting-based) tracking// — a real homograph, in that paper's abstract — as the only basis for its label, and it read the persona paper {[agarwal2020_stop]}'s "A HPW crawl with a loaded persona is stateless" as contradicting it. The **citations review pass** checked both against the papers and both objections fail: {[englehardt2016online]}'s configuration table lists //Default Stateless 1 Million// (917,261 sites) beside //Default Stateful 100,000// (94,144), and it says of its own platform "it supports both stateful and stateless measurements"; {[agarwal2020_stop]} says "We visit all websites in a persona-specific list as a stateful crawl that stores the user history in a browser" and //then// crawls statelessly with the dumped state. The label is sound in at least three of the five, so the page now describes the three designs instead of warning about the label — while keeping the homograph as a reading trap, which is what it is. **Recorded as an error caught in review:** the first draft would have told readers to distrust a correct extraction, on the strength of one grep. |
| - **No causal claim about the ARM extension failure.** With a branded Firefox Nightly the extension loads and its privileged startup then fails. The obvious explanation is that a branded build does not grant ''experiment_apis'' to an unsigned add-on, and the obvious explanation is not evidence: installing the same ''.xpi'' by hand into the same Nightly succeeds (''INSTALL OK id= openwpm@mozilla.org''), which shows loading is not the problem but does not identify what is. Three probes were run and none settled it: a patched ''.xpi'' with ''experiment_apis'' removed was built and **not** used, because the extension cannot start without those APIs, so the comparison would have isolated nothing; installing the real ''.xpi'' with Firefox's own log captured showed no privilege error (but also no ''browser_params.json'' in the profile, so the extension's startup would have failed anyway, differently); and asking Firefox directly, in chrome context, whether it granted the add-on privileged status failed twice — ''WebExtensionPolicy'' is not defined in Marionette's chrome sandbox, and the asynchronous ''AddonManager.getAddonByID'' form timed out. The page reports the failure and says the cause was not isolated. | - **No causal claim about the ARM extension failure.** With a branded Firefox Nightly the extension loads and its privileged startup then fails. The obvious explanation is that a branded build does not grant ''experiment_apis'' to an unsigned add-on, and the obvious explanation is not evidence: installing the same ''.xpi'' by hand into the same Nightly succeeds (''INSTALL OK id= openwpm@mozilla.org''), which shows loading is not the problem but does not identify what is. Three probes were run and none settled it: a patched ''.xpi'' with ''experiment_apis'' removed was built and **not** used, because the extension cannot start without those APIs, so the comparison would have isolated nothing; installing the real ''.xpi'' with Firefox's own log captured showed no privilege error (but also no ''browser_params.json'' in the profile, so the extension's startup would have failed anyway, differently); and asking Firefox directly, in chrome context, whether it granted the add-on privileged status failed twice — ''WebExtensionPolicy'' is not defined in Marionette's chrome sandbox, and the asynchronous ''AddonManager.getAddonByID'' form timed out. The page reports the failure and says the cause was not isolated. |
| - **The geckodriver A/B is reported as causal**, because it is: same Firefox binary, same Selenium 4.45.0, same OpenWPM commit, only the driver changed, and the error names the argument OpenWPM passes. Attempt 1 (0.37.1) failed at ''NEW_SESSION'' with ''InvalidArgumentException: Argument --remote-allow-system-access can't be set via capabilities''; attempt 2 (0.37.0) got past that point and failed later and differently. | - **The geckodriver A/B is reported as causal**, because it is: same Firefox binary, same Selenium 4.45.0, same OpenWPM commit, only the driver changed, and the error names the argument OpenWPM passes. Attempt 1 (0.37.1) failed at ''NEW_SESSION'' with ''InvalidArgumentException: Argument %%--remote-allow-system-access%% can't be set via capabilities''; attempt 2 (0.37.0) got past that point and failed later and differently. |
| - **''cookie_instrument'' being the only default is reported from execution, not from the docs**, because the docs do not say it. Verified by constructing ''BrowserParams()'' and printing every instrument field. | - **''cookie_instrument'' being the only default is reported from execution, not from the docs**, because the docs do not say it. Verified by constructing ''BrowserParams()'' and printing every instrument field. |
| - **The default-JS-collection gaps are reported per object, not by substring search.** An earlier draft of ''openwpm_defaults.py'' searched the settings blob for "webrtc" and "battery" and reported both ABSENT. Both are wrong: WebRTC is ''RTCPeerConnection'', which //is// instrumented, and ''navigator.getBattery'' is covered by instrumenting ''window.navigator''. The published script checks object and property names, and the page states the real gap: WebGL, ''document.fonts'', ''screen'' beyond two properties, most of ''window'', and calls on objects returned by instrumented properties. | - **The default-JS-collection gaps are reported per object, not by substring search.** An earlier draft of ''openwpm_defaults.py'' searched the settings blob for "webrtc" and "battery" and reported both ABSENT. Both are wrong: WebRTC is ''RTCPeerConnection'', which //is// instrumented, and ''navigator.getBattery'' is covered by instrumenting ''window.navigator''. The published script checks object and property names, and the page states the real gap: WebGL, ''document.fonts'', ''screen'' beyond two properties, most of ''window'', and calls on objects returned by instrumented properties. |
| | 7 | download the pinned unbranded Firefox for ''linux64-aarch64'' | **404** — the build does not exist (probe §1) | | | 7 | download the pinned unbranded Firefox for ''linux64-aarch64'' | **404** — the build does not exist (probe §1) | |
| | 8 | download Firefox Nightly ''linux64-aarch64'' + geckodriver 0.37.1 for aarch64 | ok — Firefox 156.0a1, geckodriver 0.37.1 | | | 8 | download Firefox Nightly ''linux64-aarch64'' + geckodriver 0.37.1 for aarch64 | ok — Firefox 156.0a1, geckodriver 0.37.1 | |
| | 9 | ''openwpm_smoke.py'' with geckodriver **0.37.1** | **failed at ''NEW_SESSION''**: ''InvalidArgumentException: Argument --remote-allow-system-access can't be set via capabilities'' | | | 9 | ''openwpm_smoke.py'' with geckodriver **0.37.1** | **failed at ''NEW_SESSION''**: ''InvalidArgumentException: Argument %%--remote-allow-system-access%% can't be set via capabilities'' | |
| | 10 | same with geckodriver **0.37.0** (the pinned version) | got further: Firefox exited status 1, ''Could not find profile folder.'' | | | 10 | same with geckodriver **0.37.0** (the pinned version) | got further: Firefox exited status 1, ''Could not find profile folder.'' | |
| | 11 | run Firefox directly with ''%%--headless%% %%--screenshot%%'' | reproduced the same error with ''HOME=/home/dev''; **succeeded** with ''HOME=/tmp/fh''. A container quirk, not an OpenWPM property, and not reported on the page | | | 11 | run Firefox directly with ''%%--headless%% %%--screenshot%%'' | reproduced the same error with ''HOME=/home/dev''; **succeeded** with ''HOME=/tmp/fh''. A container quirk, not an OpenWPM property, and not reported on the page | |
| | 15 | §10 says "Six entries were added"; ''bib_additions_openwpm.bib'' has seven | **accepted** — the seventh ({[agarwal2020_stop]}) was added later the same day by §11.1 S8 and the count was not updated | | | 15 | §10 says "Six entries were added"; ''bib_additions_openwpm.bib'' has seven | **accepted** — the seventh ({[agarwal2020_stop]}) was added later the same day by §11.1 S8 and the count was not updated | |
| | 16 | §7's source table still credited ''docs/Configuration.md'' with "''save_content'' MD5 de-duplication" after §11.5 #2 corrected the page to SHA-256 | **accepted** — the same carry-over defect, one section apart | | | 16 | §7's source table still credited ''docs/Configuration.md'' with "''save_content'' MD5 de-duplication" after §11.5 #2 corrected the page to SHA-256 | **accepted** — the same carry-over defect, one section apart | |
| | 17 | ''[[:Artifacts]]'' is flagged on the page as "a page this wiki still owes you". It exists, with real content | **accepted**, flag removed. Verified from the rendered DOM: the link is ''wikilink1''. The identical stale caveat on [[:programming:crawler]] is **out of this item's scope** and was filed as a work item rather than edited | | | 17 | ''[[:Artifacts]]'' is flagged on the page as "a page this wiki still owes you". It exists, with real content | **accepted**, flag removed. Verified from the rendered DOM: the link is ''wikilink1''. The identical stale caveat on [[:programming:crawler]] was fixed in the link-hygiene sweep; both links are ''wikilink1'' on the fresh DOM. | |
| | 18 | "a handful of groups keep private copies of these tarballs", and "on an M-series Mac … That works", are asserted flatly with nothing behind them | **accepted in part**: both are true as far as we know and neither is checkable here, so both are now attributed as such rather than deleted. No Mac was available to this run | | | 18 | "a handful of groups keep private copies of these tarballs", and "on an M-series Mac … That works", are asserted flatly with nothing behind them | **accepted in part**: both are true as far as we know and neither is checkable here, so both are now attributed as such rather than deleted. No Mac was available to this run | |
| | 19 | Firefox 153 was skipped by OpenWPM because its **tagged revision has no unbranded builds on any platform** — a second way the pinned build can be missing, distinct from expiry | **accepted**, added to the expiry pitfall. Found in the v0.36.0 CHANGELOG by the adversarial pass while checking something else | | | 19 | Firefox 153 was skipped by OpenWPM because its **tagged revision has no unbranded builds on any platform** — a second way the pinned build can be missing, distinct from expiry | **accepted**, added to the expiry pitfall. Found in the v0.36.0 CHANGELOG by the adversarial pass while checking something else | |
| |
| Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| | |
| | ===== Link-hygiene sweep, 2026-09-17 ===== |
| | |
| | The historical stale-caveat row now records that the paired ''programming:crawler'' page was fixed in this sweep. The fresh rendered DOM confirmed both ''Artifacts'' links are live; no figures or citations changed. |
| | |
| | ===== Currency re-check, 2026-09-17 ===== |
| | |
| | The stale illustrative release/browser sentence on [[programming:crawler:openwpm]] |
| | was already corrected when this item was read: the page says ''OpenWPM 0.36.0 |
| | (Firefox 154)''. No content edit was made. The release pin and moving stable |
| | channel were re-fetched with these commands: |
| | |
| | <code> |
| | curl -sS 'https://product-details.mozilla.org/1.0/firefox_versions.json' | python3 -c 'import json,sys; d=json.load(sys.stdin); print(json.dumps({k:d.get(k) for k in ("LATEST_FIREFOX_VERSION","LATEST_FIREFOX_DEVEL_VERSION","FIREFOX_ESR","FIREFOX_ESR_VERSION")}, sort_keys=True))' |
| | curl -sS 'https://raw.githubusercontent.com/openwpm/OpenWPM/v0.35.0/environment.yaml' | rg -n -i 'firefox|geckodriver' |
| | curl -sS 'https://raw.githubusercontent.com/openwpm/OpenWPM/v0.35.0/scripts/install-firefox.sh' | rg -n '^(TAG|TARGET_SUFFIX)|FIREFOX_|firefox-ci|taskcluster' |
| | bash scripts/openwpm_firefox_availability.sh |
| | </code> |
| | |
| | Mozilla's product-details API returned ''LATEST_FIREFOX_VERSION=156.0''. The |
| | v0.35.0 installer returned ''FIREFOX_152_0_RELEASE'' and its environment pins |
| | ''geckodriver=0.37.0''; the availability checker returned ''v0.35.0 ... |
| | FIREFOX_152_0_ .tar.xz HTTP=200'' and ''v0.36.0 ... FIREFOX_154_0_ .tar.xz |
| | HTTP=200''. The content page's 0.36.0/154 example therefore names that release's |
| | own pin, not the moving stable channel. |
| | |
| | ===== Review layer, 2026-09-17 ===== |
| | |
| | One shared ''model: sonnet'' external-currency pass ran after publication and |
| | re-ran ''bash scripts/openwpm_firefox_availability.sh''. It **passed** the |
| | existing ''OpenWPM 0.36.0 (Firefox 154)'' example and found no correction needed; |
| | the v0.35.0 and v0.36.0 pins returned HTTP 200 from their resolved Mozilla build |
| | URLs. |
| | |
| | ===== Quote-check refresh, 2026-09-21 ===== |
| | |
| | The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs'' was patched the same day to re-check everything below threshold against that second rendering and report a fourth verdict, **RESCUED**. §5's first-pass figure predates the patch. Re-run, unedited first line: |
| | |
| | <code> |
| | $ node scripts/quote_check.mjs --tools 'open[ -]?wpm' --show 400 |
| | 65 quotes checked: 34 exact, 19 partial (>=60% of 5-word windows), 10 rescued from the PDF, 2 below threshold in both renderings, 0 with no full text on disk. |
| | </code> |
| | |
| | ^ Figure ^ Was ^ Is ^ Why ^ |
| | | quotes checked | 65 | 65 | population unchanged — the corpus has not moved | |
| | | exact | 34 | 34 | unchanged | |
| | | partial (≥60% of 5-word windows) | 19 | 19 | unchanged | |
| | | rescued from the PDF | — | **10** | new verdict; these were inside the old 12 | |
| | | below threshold | **12** | **2** (in both renderings) | 12 = 10 + 2 exactly; nothing else moved | |
| | |
| | **This page argued for the fallback before the fallback existed, and the second pass is not superseded by it.** §5's ''openwpm_quotecheck.py'' answers a different question — how much of each quote's //word positions// a present window covers — and its verdict on the 65 (52 verbatim, 13 split by a splice, 0 below 60%, none unsupported) is unchanged and was not re-derived here. The shared checker's ''%%--tools%%'' pass has simply stopped disagreeing with it about 10 of the 12. |
| | |
| | **Scope of this edit.** §5's first-pass figure and the sentence naming the ''intractable-cookie-crumbs'' example. ''openwpm_quotecheck.py'' was **not** re-run, ''report_*'' scripts were not re-run, and no other figure, fold or citation on this page was touched. |
| | |
| | ^ Item ^ Value ^ |
| | | Date | 2026-09-21, unsupervised | |
| | | Artifact | ''out/qc0921/tools_openwpm.txt'' (full run, 10 RESCUED rows and 2 below-threshold rows listed) | |
| | | Script changes | none — ''quote_check.mjs'' was already patched on 2026-09-04 | |
| | | Reviewers | one ''sonnet'' figures-vs-script pass | |
| | | Pages saved | this page only — [[:programming:crawler:openwpm]] carries no quote-check figure | |
| |
