Differences

This shows you the differences between two versions of the page.

--- provenance:privacy:browser_extensions [2026/09/02 03:10] – Reviewer log: record that the generic (fable) pass did not return within the sitting, what it was asked to do, and why the page's review should be treated as three-quarters done. Filed as the ext-generic-review work item. Authored by Claude. karel.kubicek.claude
+++ provenance:privacy:browser_extensions [2026/09/22 19:06] (current) – Reviewer log: correct the ext-external-assertions helper count (eight) and mutation count (46/46 after pinning the timeline footer date). Authored by Claude karel.kubicek.claude
@@ Line 79: / Line 79: @@
 ==== How the verdicts were made, and the ones a reasonable person would decide otherwise ====
-Every verdict was made by reading the paper's own first-person sentences about extensions, printed by a helper over ''paper.cols.txt'' (the query is in the report's ''--quotes'' path; the raw dossier is ''out/ext/firstperson.txt'', not published because it is 98 KB of quotations). The distinction that did the most work is between a paper whose //results// include a statement about extensions and one where extensions appear only in the evaluation of the paper's own system.
+Every verdict was made by reading the paper's own first-person sentences about extensions, printed by a helper over ''paper.cols.txt'' (the query is in the report's ''%%--quotes%%'' path; the raw dossier is ''out/ext/firstperson.txt'', not published because it is 98 KB of quotations). The distinction that did the most work is between a paper whose //results// include a statement about extensions and one where extensions appear only in the evaluation of the paper's own system.
 ^ Paper ^ Verdict ^ Why, and the case against ^
@@ Line 156: / Line 156: @@
 Papers outside the candidate set whose full text names Chrome Web Store,
 .crx, moz-extension, chrome-extension://, manifest.json, MV3 or a
-web-accessible resource: 32. All 32 were read (2026-09-02); every one is a
+web-accessible resource: 32.
-citation, a reference-list URL or a different sense of the word. In-scope
+All 32 were read by hand on 2026-09-02; every one is a citation, a
-papers added by the widening: 0.
+reference-list URL or a different sense of the word.
+In-scope papers added by the widening: 0 — true by construction, not a
+measurement: the verdict map is keyed on candidates, so a non-candidate
+cannot carry a verdict. The hand reading is what rules them out.
 Bare-mention control: /browser extensions?/i alone fires on 535 papers.
@@ Line 375: / Line 378: @@
 Snapshot date / list version stated:   38 of 74 extension-corpus tuples (51.4%)
+  of those, carrying an actual year:    37 of 74 (50.0%)
+  stated but with NO year in the value: 1
+CCS n=2790 :: "at the time of analysis"
   counted by paper:                    21 of 38 papers (55.3%)
   corpus-wide BASELINE, a DIFFERENT population: population.listVersion stated by
@@ Line 521: / Line 527: @@
       MV3 and MV2 means were 1.42 and 1.46 blocked ads.
       technique: Subtracted ad-blocker display-ad counts from no-blocker baseline.
+==============================================================================
+H2. THE PEOPLE HALF — the guard for "all but one are from 2021 onwards"
+==============================================================================
+CCS      Let's Go in for a Closer Look: Observing Passwords in Their Natural Habitat.
+IEEE-SP  They Would do Better if They Worked Together: The Case of Interaction Problems Betwe
+PETS     How Can and Would People Protect From Online Tracking?
+USENIX   Defining "Broken": User Experiences and Remediation Tactics When Ad-Blocking or Trac
+PETS     Block Cookies, Not Websites: Analysing Mental Models and Usability of the Privacy-Pr
+USENIX   "I have no idea how to make it safer": Studying Security and Privacy Mindsets of Bro
+WWW      Broken Access: On the Challenges of Screen Reader Assisted Two-Factor and Passwordle
+  people papers: 7; from 2021 onwards: 6; before 2021: 1
 ==============================================================================
@@ Line 547: / Line 566: @@
 ===== Quotes and per-paper figures =====
-''node scripts/verify_extension_figures.mjs --show''. **60 of 60** needles located. Every needle is a phrase the page prints, checked against two renderings — ''paper.cols.txt'' with whitespace collapsed, and an independent ''pypdf'' extraction of ''paper.pdf'' — because the two fail on different sentences. **On this run all 60 were found in ''.cols'' and none needed the PDF route**, which is worth recording: the fallback exists and was not exercised, so it is not evidence that it works here.
+''node scripts/verify_extension_figures.mjs %%--show%%''. **66 of 66** needles located as of 2026-09-22((The needle set has grown since first publication and this sentence did not follow it. It was written at 60; the citations pass added needles for papers the page began citing; the self-audit added the ''"2,523"'' Opera needle; the run of 2026-09-22 added two with the corrected Carnus row. The count reached 64 at some point before 2026-09-21 and is 66 now — but the exact increments cannot be reconstructed from this log, which is the point: **the count is only trustworthy from the script.** The //Quote-check refresh, 2026-09-21// section below said //70-needle//, which the check has never been. Re-run the script rather than quoting any figure in prose, including this one.)). Every needle is a phrase from the paper, checked against two renderings — ''paper.cols.txt'' with whitespace collapsed, and an independent ''pypdf'' extraction of ''paper.pdf'' — because the two fail on different sentences. **On this run all 66 were found in ''.cols'' and none needed the PDF route**, which is worth recording: the fallback exists and was not exercised, so it is not evidence that it works here.
+Needles were deliberately rewritten from bare numbers into **verbatim phrases** after the first run: the first version had 53 of 58 needles under 20 characters, which is the length at which a match can happen by accident and //"0 not located"// starts passing for the wrong reason. **How much of the page the 66 actually cover is now measured, not asserted.** The script prints it: **40** of the 66 guard a figure the page prints, **9** guard a figure it does not, and **17** carry no number distinctive enough to test either way — and it lists all 26 of the latter two groups so they can be read. Until 2026-09-22 this section said //every needle is a phrase the page prints//, which was an assertion and was false: a needle is the //paper's// sentence, and the page usually states the figure in its own words. A needle in the 9 still does real work — it proves the figure exists in the paper — it just is not evidence about this page. Quote the 40, not the 66, when you mean page coverage.
-Needles were deliberately rewritten from bare numbers into **verbatim phrases** after the first run: the first version had 53 of 58 needles under 20 characters, which is the length at which a match can happen by accident and //"0 not located"// starts passing for the wrong reason. After the rewrite only two needles are short (''"= 1.46 "'' and ''"= 1.42 "'', from a figure caption in the PoPETs 2026 MV3 paper), and both were read in context by hand.
+After the rewrite **three** needles are short: ''"= 1.46 "'' and ''"= 1.42 "'' from a figure caption in the PoPETs 2026 MV3 paper, and ''"2,523"'' — the Opera corpus of {[some2019_empoweb]}, added when the self-audit corrected the //nothing measures Opera// claim. All three were read in context by hand. The script prints the count, so the figure in this sentence is checkable against it.
 Three needles failed on the first run and were **not** fabrications; all three were the page paraphrasing where the paper had different wording, and all three were corrected on the page rather than in the checker:
@@ Line 570: / Line 591: @@
 | exact | 54 |
 | partial (≥60% of 5-word windows present) | 15 |
-| below threshold | 5 |
+| rescued from the PDF | 4 |
+| below threshold in both renderings | 1 |
 | **total extension-corpus tuples with a quote** | **74** |
+**Re-run 2026-09-21 with the PDF fallback.** The old figure was **5 below threshold**; 4 of the 5 reach threshold against an independent ''pypdf'' rendering of the same ''paper.pdf'' and are reported as //rescued//. ''exact'' (54) and ''partial'' (15) did not move. See //Quote-check refresh, 2026-09-21// at the foot of this page.
+**The one still below threshold in both renderings is the IBEX 2011 ''n = 1,139'' tuple**, at 56% coverage — the first row of the table below, and a column splice. By elimination the other four are the remaining rows: //Ad Injection at Scale// 2015, //Experimental Security Analysis// 2024 and both //Privilege Separation in HTML5// 2012 tuples.
+**Two of the four rescued are the two the table calls paraphrases, and that is a caution about what //rescued// means.** It means ≥60% of the quote's five-word windows are present in the second rendering — not that the extraction is verbatim. A paraphrase that keeps most of the paper's wording can clear the threshold in a cleaner rendering. The hand verdicts below are the stronger evidence and they are unchanged.
 **Below threshold is not "unsupported".** All five were read by hand against
@@ Line 596: / Line 624: @@
 ===== External sources =====
-Everything below was fetched on **2026-09-02** and is re-checkable with ''bash scripts/external_checks_browser_extensions.sh''. That script prints ''FAILED'' explicitly per check, so a blocked or rate-limited call cannot read as a pass.
+Everything below was fetched on **2026-09-02**, and every row was re-checked on **2026-09-22**, with ''bash scripts/external_checks_browser_extensions.sh''. **All 26 of that script's checks now assert**: each names the needle, status code or floor it requires, a non-200 fetch is itself a failure, and the run exits non-zero when any of them is gone. Until 2026-09-22 only three did, and the sentence that stood here — //"that script prints FAILED explicitly per check, so a blocked or rate-limited call cannot read as a pass"// — was false for the other twenty-three. The conversion and its mutation test are in the reviewer log under ''ext-external-assertions''.
 ==== Verified, and used ====
@@ Line 611: / Line 639: @@
 | ''data_collection_permissions'' coverage | the same API, 400 most-installed | 314 empty (78.5%), 63 ''["none"]'' (15.8%), 23 named (5.8%); ''sandbox/amo_data_collection.py'' |
 | Edge has no bulk listing; Opera has none | Microsoft Learn; ''addons.opera.com'' | Edge's REST API covers your own submissions; Opera's ''robots.txt'' and ''sitemap.xml'' both 404 |
-| Manifest V2 timeline, all six milestones | ''developer.chrome.com/docs/extensions/develop/migrate/mv2-deprecation-timeline'' | page footer //"Last updated 2026-07-08 UTC"//; every quoted phrase on the page is verbatim from it |
+| Manifest V2 timeline, all six milestones | ''developer.chrome.com/docs/extensions/develop/migrate/mv2-deprecation-timeline'' | page footer //"Last updated 2026-09-09 UTC"// on the 2026-09-22 re-check, //"Last updated 2026-07-08 UTC"// on 2026-09-02; every quoted phrase on the page is verbatim from it. **Google reworded the 2026-08-31 entry in between** and the page's quotes were updated to match — see the reviewer log. Check 8 now asserts each quoted phrase and the footer date |
 | Edge has set no MV2 date | ''learn.microsoft.com'' ''ExtensionManifestV2Availability'' | //"More information about and the timeline of the migration hasn't been established."// |
 | Firefox keeps blocking ''webRequest'' | MDN ''webRequest'' vs Chrome ''webRequest'' | MDN documents ''webRequestBlocking'' as an ordinary permission with no MV3 note; Chrome's says //"only available to policy installed extensions"// |
@@ Line 637: / Line 665: @@
 ==== Could not be established ====
-  * **No authoritative count of Chrome Web Store extensions exists.** Google publishes none. The page reports the sitemap's 356,984 //items// with the unit stated and a 40-item sample of what those items are, and declines to convert it into an extension count with a confidence interval a 40-item sample cannot support.
+  * **No authoritative count of Chrome Web Store extensions exists.** Google publishes none. The page reports the sitemap's 356,984 //items// with the unit stated, and converts it with the **200-item** CRX sample (146 extensions of 198 served, 73.7%) to //"on the order of 260,000 extensions"//, saying in the same sentence that a 200-item sample can only bound that loosely. The earlier 40-item sample is superseded for the composition question and supports nothing narrower than //roughly three-quarters extensions//. Until 2026-09-22 this bullet said the page //declined// to convert at all, and cited the 40-item sample for it — a description of a decision the page had already stopped making.
   * **Nothing.** The Manifest V2 question that looked like a discrepancy was resolved by measurement rather than left open: 17 of the 198 CRXs served are ''manifest_version: 2'', which two days after the MV2 removal looks like the sitemap lagging the store. Reading the manifests shows it is not — **0 of the 17 are extensions**; 9 are themes and 8 are legacy Chrome apps, neither of which the MV2 phase-out touches. The page states this, because a researcher filtering a corpus on ''manifest_version'' will otherwise conclude that MV2 extensions survived.
   * **Apple's current position on Manifest V2 in Safari.** ''developer.apple.com'' serves client-rendered pages with no fetchable text, and the only Apple primary source found is a 2022 WWDC transcript. The page therefore says only that the standalone Safari extension gallery no longer exists, which is checkable, and makes no claim about Safari's manifest support.
@@ Line 645: / Line 673: @@
 ===== Original measurements on this page =====
-Four numbers on the page are not in any paper and not in any vendor document. Each is a single-day snapshot and the page says so.
+Five numbers on the page are not in any paper and not in any vendor document. Each is a single-day snapshot and the page says so.
 ^ Measurement ^ Script ^ Result ^ Weakness ^
@@ Line 654: / Line 682: @@
 | AMO ''data_collection_permissions'' coverage | ''sandbox/amo_data_collection.py'' | 400 most-installed: 78.5% empty, 15.8% ''["none"]'', 5.8% named | most-installed, not random — the sample is biased towards the add-ons most likely to have been updated, so the true coverage across the whole catalogue is probably //lower// |
-The sampling design in all three samples is **every-k-th over a sorted list**, not random, so a re-run on the same sitemap reproduces the same items exactly. That is deliberate: a random sample would be unreproducible, and the store changes daily anyway.
+The sampling design in the **two sitemap samples** is every-k-th over a sorted list of IDs, not random, so a re-run on the same sitemap reproduces the same items exactly. The **AMO sample is not of that shape**: it is the top 400 by installs, a biased frame, labelled as one in its row above. That is deliberate: a random sample would be unreproducible, and the store changes daily anyway.
 ===== The published script =====
-''pages/ext_corpus.py'' is embedded on the page as a downloadable %%<file>%% block and is byte-identical to the committed file apart from the docstring, which is all the page shows.
+''pages/ext_corpus.py'' is embedded on the page as a downloadable %%<file>%% block and carries **the whole file, not an abridgement**; ''scripts/build_ext_codeblock.py %%--check%%'' fails if the embedded block stops being byte-identical to the committed one. An earlier draft published only the docstring, which is the failure that guard exists to prevent.
 It was **mutation-tested**, not read:
@@ Line 681: / Line 709: @@
 ===== Reviewer log =====
-Four review passes, all told explicitly that the reviewer's own context may not be
+Five review passes, all told explicitly that the reviewer's own context may not be
-exhaustive and all handed the same **frozen** snapshot of the page, the report
+exhaustive and all handed a **frozen** snapshot of the page, the report script and
-script and its output, the external-check output, and this log. The freeze is
+its output, the external-check output, and this log. Corrections were applied
-''out/ext/freeze/'' in the workdir. Corrections were applied //after// each pass
+//after// each pass returned, not while it was running, so each finding can be
-returned, not while it was running, so each finding can be attributed to what the
+attributed to what the reviewer actually saw.
-reviewer actually saw.
+There is more than one freeze, and saying so matters, because a finding is only
+attributable if you know which text produced it. The three focused passes saw
+''out/ext/freeze/''; the generic pass of 2026-09-02 saw ''out/ext/freeze2/'', the
+re-frozen copy taken after those passes' fixes were applied; the generic pass of
+-09-22 saw ''out/ext/freeze3/'', taken after the deferred findings below were
+closed. Until 2026-09-22 this paragraph said //the same// freeze and named only
+the first — which the 2026-09-02 pass flagged about itself, and which stood
+uncorrected for three weeks.
+''freeze3'' was **not a clean freeze either**: the external-check re-run
+overwrote ''external.txt'' inside it while pass 5 was reading it, and pass 5
+noticed and said so. Nothing in its findings depends on that file's live values,
+but it is the second time this page has made the same mistake, so it is recorded
+here rather than quietly fixed.
 Rejections are recorded as carefully as fixes: they are the only evidence of
@@ Line 719: / Line 761: @@
 | **The Safari row asserted "since Safari 13"** with no check in the external script | **accepted.** The claim is dropped, Apple's own distribution sentence is quoted, and a Safari check is now check 24 in the external script |
 | **The reviewer could not reproduce the frozen outputs byte-for-byte**, because both scripts had been edited after the freeze while it worked | **recorded as a process failure, not a page defect.** It is the reason the freeze exists and the reason it has to be a real freeze. The reviewer flagged it rather than assuming, which is the behaviour the brief asked for; four of its eight findings had in fact already been fixed in the unfrozen copy, and it said so each time |
-| Verified clean: the 218/68/47/21 splits and every percentage and sum, the year buckets closed at both ends, the %%<file>%% block byte-identical to the committed script, ''ext_corpus.py --self-test'' passing, and — by **mutation-testing both directions** — that the report really does exit 1 when a verdict has no candidate or a candidate has no verdict | no change needed |
+| Verified clean: the 218/68/47/21 splits and every percentage and sum, the year buckets closed at both ends, the %%<file>%% block byte-identical to the committed script, ''ext_corpus.py %%--self-test%%'' passing, and — by **mutation-testing both directions** — that the report really does exit 1 when a verdict has no candidate or a candidate has no verdict | no change needed |
 ==== Pass 2 — citations, quotes and attribution (sonnet) ====
@@ Line 739: / Line 781: @@
 ==== Pass 4 — generic, no checklist (fable) ====
-**This pass did not return within the sitting.** It was launched over the
+It returned after the pages were first published; its findings were applied in a
-corrected snapshot in ''out/ext/freeze2/'' after the three focused passes and the
+follow-up save. Twelve findings, of which **four were already fixed** by the
-self-audit below, and the pages were published without it. That is recorded rather
+self-audit below (Edge/Opera, "Nobody does", "Nobody has published it", and two
-than omitted: a review log that quietly drops a pass is worse than one that says
+placeholder slots that were filled between the freeze it saw and publication) —
-which pass is missing.
+it said so where it could tell. The rest:
-What it was asked to do, with no checklist: look for the failure modes the three
+^ Finding ^ Disposition ^
-focused passes are structurally blind to — a claim that is true in one section and
+| **A novelty claim contradicted by the page's own report.** The page said Torres et al. //"turned the question around and measured the web… rather than only in a lab"//; report §H records {[sjosten2019_latex]} doing exactly that in 2019, //"2,572 out of those 10,000 domains probe for WARs"// | **accepted.** The sentence now credits the 2019 precedent and says what Torres's contribution actually is: the object, not the framing |
-contradicted by another, confidence outrunning evidence, an unmeasured framing
+| **The methodology claimed more quote coverage than the guard gives.** //"Every per-paper figure quoted above was checked"// — but several figures on the page (akhawe's 5%-of-functions, dong's 23-of-top-30, lukic's 21%-vs-0%) have no needle | **accepted.** The sentence now says the check covers the 64 needles and **not** every number on the page, and points at where the rest come from |
-sentence, a page that does not answer its own question, structural problems, and
+| **"Every result older than those is a result about Manifest V2" over-reaches**, and the page contradicts it three sections earlier by describing VEX and Djeric attacking Firefox's XPCOM model | **accepted.** Scoped to Chrome, with the pre-MV2 model and the Firefox/Opera corpora named |
-anything in //this log// that claims more rigour than the run had.
+| **The detection table's "Introduced" column credits the wrong paper**: the page's own pitfalls section says WAR probing was introduced at CODASPY 2017, outside the corpus | **accepted.** The column is now headed //Introduced, in this corpus// |
+| **"They agree to within half a percentage point"** — 2.68 − 2.13 = 0.55 | **accepted**, and the difference is now stated as a number |
+| **The opening box's 146 + 44 + 8 sums to 198, not 200**, and the two refusals are explained 400 lines later | **accepted.** The box says "(198 served; the endpoint declined 2)" |
+| **This log contradicted itself three ways**: "Four numbers" over a five-row table; "all three samples are every-k-th" when the AMO sample is a top-400; and //"byte-identical apart from the docstring, which is all the page shows"// when the page shows the whole script | **all three accepted.** The last is the serious one — it is the exact "published code must be the real script" failure, described in the log as though it had happened. The whole file //is// on the page and ''build_ext_codeblock.py %%--check%%'' proves it |
+| **The external-check script has vacuous checks**: check 1 prints its "expected" comment and no fetched evidence; check 9 probes an obsolete constant name and prints ''NOT FOUND'' rather than ''FAILED''; the MDN ''webRequestBlocking'' quote in a page footnote is fetched by no check | **accepted, and closed on 2026-09-22** (see //Deferred findings, closed// below). **Originally deferred with this note:** it is the trap this log lectures about, found in this log's own script. Recorded here and carried by the ''ext-generic-review'' work item |
+| Carnus row labels three things and gives two numbers; the "all but one from 2021 onwards" people claim has no guard; the log names ''out/ext/freeze/'' when this pass saw ''freeze2'' | **accepted, and all three closed on 2026-09-22** — see //Deferred findings, closed// below. They were recorded here and carried by an ''ext-generic-review'' work item that, it turned out, was never actually filed: the only record they existed was this row |
+| "The page answers its own question well… the What-to-Report box is the best section", structure sound, all re-checked arithmetic correct, the published script reads correctly as code and as prose, and its self-test genuinely asserts negative cases | recorded |
+**The lesson this pass repeats.** Three of its findings — the Torres novelty claim,
+the Manifest V2 over-reach, and the WAR-probing attribution — are each true in the
+sentence where they appear and false against a different section of the same page.
+That is the third run on this site where the generic pass earned its slot by
+reading the joins that no focused brief owns.
+==== Pass 5 — generic, no checklist (fable), 2026-09-22 ====
+Run over ''out/ext/freeze3/'' — the page, the provenance, the three scripts and
+their fresh outputs — **after** the deferred findings below were closed, so that
+the newest and least-reviewed material was the material under review. Twelve
+findings confirmed, five raised as suspected. It opened with a process note that
+is itself a finding: ''external.txt'' inside the //frozen// directory changed
+while it was reading it, because the external-check re-run overwrote it. That is
+pass 1's "the freeze was not a freeze" failure, committed again, by me.
+^ Finding ^ Disposition ^
+| **A fingerprintability paper was illustrating the eleven blocking-efficacy papers.** The //Does the Protective Extension Protect?// section says //"Eleven of the 68 papers ask this"// and then closes with //"{[nguyen2025_breaking]} and {[elhajjchehade2025_double]}, above"//. The report tags {[elhajjchehade2025_double]} ''fingerprintability'', and the page's own detection table treats it as a fingerprinting paper — //"fingerprints the filter list rather than the blocker"// | **accepted, and it is the best finding of the pass.** The bullet now keeps {[nguyen2025_breaking]}, says in terms that {[elhajjchehade2025_double]} is **not** one of the eleven, and gives the reason. This is the two-sections-meet failure again: both sentences are right where they sit |
+| **The needle fix left the stale number on the public page.** The count moved to 66 and the provenance was corrected; the content page still said //"64 of 64 located"// and //"a check on those 64"// | **accepted, fixed.** Mine, from earlier the same day, and exactly the defect the //residue table does not fix the prose// rule exists for: I corrected the section that computes the number and not the sentence that quotes it |
+| **//Every needle is a phrase the page prints// was false.** At least six needles guard figures that appear nowhere on the page: ''699'', ''29,428'', ''854'', ''18,937'', ''190 extensions'', ''5.5%'' | **accepted, and resolved by measuring instead of asserting.** ''verify_extension_figures.mjs'' now computes page coverage and prints the residue: **40** of the 66 guard a figure the page prints, **9** guard one it does not, **17** carry no number distinctive enough to test. The prose gives all three and says to quote the 40 when you mean page coverage. A needle in the 9 still proves its figure exists in the paper — it is just not evidence about this page |
+| **The new failure mechanism covers 3 of 26 checks, and the closing line claimed all of them.** ''fail()''/''expect()'' are called in checks 1, 9 and 26 only. Five checks use ''%%|| echo FAILED%%'', which prints the word without incrementing ''FAILS''. The remaining eighteen are regex printers that emit **nothing** on a 403, a 429 or a reworded source. Yet the run ended //"all checks produced fetched evidence"// | **accepted in full, and it is the sharpest thing said about my own work today.** Closing the vacuous-check finding made the summary line claim //more// than before the fix. The header now sets out which checks are asserted, which are reported and which are unasserted, and the closing line says //the 3 asserted checks passed; the other 23 are printers — an empty section is not a pass//. Converting the other 23 is the ''ext-external-assertions'' work item, filed this time |
+| **The report printed a hand verdict as though the run computed it.** //"All ${extra.length} were read (2026-09-02)"// — the count is computed, the reading and the date are string literals. If the corpus moved, the report would claim 40 papers were read on a day 32 were. Same shape at //the 429 papers below it// | **accepted, fixed, and mutation-tested.** The hand-read set is pinned at 32; a different count now prints ''HAND-READ SET IS STALE'' and exits non-zero (verified by setting it to 33). The 429 is computed from the distribution. The //In-scope papers added: 0// line now says it is true **by construction** — the verdict map is keyed on candidates, so a non-candidate cannot carry a verdict — rather than posing as a measurement |
+| **The provenance contradicted the page on the store composition.** //Could not be established// said the page //"declines to convert [the sitemap count] into an extension count"//, citing the 40-item sample; the page converts, from the 200-item sample, to //"on the order of 260,000 extensions"//. The provenance's own tools table says the 200-item sample supersedes the 40 | **accepted, fixed.** The bullet describes what the page actually does and its stated caveat; the TODO that rested on the 40-item sample is rewritten |
+| **//Only 51.4% state a snapshot date at all// counted a non-date.** The filter excludes only ''not-stated'' and ''null'', so the Mystique Opera tuple's //"at the time of analysis"// passed as a stated date | **accepted, fixed, and made computable.** The report now prints the dated/undated split and the offending value: **37 of 74 (50.0%)** carry an actual year, and the one exception is named. The per-paper 21 of 38 is unaffected — the same paper's Chrome tuple gives a real range. A non-sentinel value is not automatically a value |
+| **A top-400 sample generalised to the catalogue.** //"currently a signal about 5.8% of the catalogue"//, where the provenance says the sample is most-installed, not random, so true coverage is probably lower | **accepted, fixed.** 5.8% is now stated as an **upper bound** from the head of the catalogue, not an estimate of it |
+| **A bullet contradicted itself two clauses later.** //"Every whole-store paper rebuilt one"// against //"Corpora //do// get passed between groups"// and two ''pre-existing-dataset'' tuples in the report | **accepted, fixed** — //either rebuilt one or inherited one privately// |
+| **A dangling reference.** The page promised the provenance says //"which one it is still missing"//; no pass is missing | **accepted, fixed** — it now promises what each pass changed |
+| **The rewritten Carnus row carried its own changelog.** //"two novel vectors, not three"// contrasts with a version the reader has never seen, and sat under a bullet calling Carnus a comparison of //four// vectors | **accepted, fixed.** The row names the two vectors Carnus introduces, says the other two are the rows above, and drops the //not three// |
+| **The needle ledger does not close.** The footnote's 60 → 64 → 66 story cannot accommodate the ''2,523'' Opera needle, which the next paragraph attributes to the self-audit | **accepted, and fixed by giving up the reconstruction.** The footnote now says the increments cannot be reconstructed from this log and that the count is only trustworthy from the script — which is the honest version and the one that stops the next person quoting a number out of prose |
+| It also flagged five **suspected** items it could not decide: the ''people'' verdict on the CCS 2017 password study (which, if it is ''instrument'' rather than ''people'', makes the sentence //all seven//); the Firefox WAR result of 2017 sitting beside //Firefox: closed to this vector// with no reconciliation; four quantified claims the report does not print; and //none needed the PDF route// resting on a run without ''%%--show%%'' | **recorded.** The last is settled here: re-run with ''%%--show%%'', **66 of 66 located in ''.cols'', 0 via the PDF** — the claim is true and now has a run behind it. The other four are carried by the ''ext-suspected-four'' work item. The CCS 2017 verdict is the one worth doing first: the guard added today formally depends on it, and it has no row in the borderline-verdicts table |
+| Verified clean, by re-derivation: the role table, venue split, question table by row and by every period column, the 27-row sampling-word split, 146+44+8=198, 73.7%, 0.55 pp, the 49 citekeys — and that the H2 guard does what the page says and fails on both halves | no change needed |
+**What this pass says about the previous four.** Six of its twelve findings are in
+material written **the same day**, by me, while closing the previous pass's
+deferred list: the stale 64, the overstated needle claim, the overstated
+external-check summary, the Carnus changelog, and two more. A correction is
+new, unreviewed text. **The pass that reviews a fix is worth as much as the pass
+that found the defect**, and this log had not been run that way before today.
+==== Deferred findings, closed 2026-09-22 ====
+The generic pass of 2026-09-02 accepted findings it did not have time to fix and
+recorded them as carried by an ''ext-generic-review'' work item. **That item was
+never filed.** They survived only because the two rows above recorded them, which
+is the argument for writing a deferred finding down in the same detail as a fixed
+one: for three weeks this log was the entire backing store.
+They are **two rows but six findings**, which is itself worth noting — a deferred
+row that bundles three things is three things that can be lost together. All six
+reproduced exactly as described, and all six are now closed.
+^ Deferred finding ^ What it was ^ How it is closed ^
+| Check 1 of ''external_checks_browser_extensions.sh'' was vacuous | It grepped the **raw HTML** for a sentence, so the tag attributes broke its ''[^.]'' run and it matched nothing; it then printed its own ''%%-- expected: the Web Store API publishes/manages YOUR OWN item%%'' comment. The re-run on 2026-09-22 reproduced it: the whole of check 1 was two lines, one of them the comment. **An assertion with no fetched evidence under it, in the script whose header promises the opposite** | The check strips tags before matching, and an ''expect'' helper prints the located sentence or records a failure. It now prints Google's own //"The Chrome Web Store API lets you programmatically publish and manage your Chrome Web Store items"//, and separately probes for a listing endpoint rather than asserting there is none |
+| Check 9 probed an obsolete constant and printed ''NOT FOUND'' | ''MAX_NUMBER_OF_DYNAMIC_AND_SESSION_RULES'' is the **pre-Chrome-120** constant and is gone from the reference page. The check printed ''NOT FOUND'' as an ordinary output line inside a run that exited 0 | The two constants that replaced it, ''MAX_NUMBER_OF_DYNAMIC_RULES'' and ''MAX_NUMBER_OF_SESSION_RULES'', are probed instead, plus ''MAX_GETMATCHEDRULES_CALLS_PER_INTERVAL'' for the ''getMatchedRules'' quota the page publishes. A missing constant now prints ''FAILED'' and exits non-zero. **The page's figures were right**; only the script's name for one of them was stale |
+| The MDN ''webRequestBlocking'' quote was fetched by no check | The page's footnote at //"Firefox supports both, and keeps blocking ''webRequest''"// quotes MDN verbatim. Chrome's side of that comparison was check 21; the Firefox side was checked by nothing, so the whole Firefox half of the MV2/MV3 divergence rested on an unverified quote | New **check 26** fetches the MDN page and locates the sentence. It is there verbatim: //"To use the 'blocking' feature, the extension must also have the 'webRequestBlocking' API permission"//. The check also reports whether MDN has begun mentioning Manifest V3 on that page, which is the way this claim would go stale |
+| The Carnus row labelled three things and gave two numbers | //"Intra- and inter-browser messaging, and outgoing HTTP"// against //"450 and 859 extensions respectively"//. Reading the paper makes it a **worse** error than a missing number: Carnus's //inter//-communication **is** the outgoing-HTTP vector — //"two novel fingerprinting techniques … namely outgoing HTTP requests and intra-browser message exchanges"// — so the row invented a third vector the paper does not have | The row names the two vectors, says in terms that they are two and not three, and adds Table I's //Unique// column (105 and 181) with the paper's own gloss, //"unique to each technique (i.e., cannot be detected by any other technique)"// — flagged explicitly as **not** the sense of //unique// used in the identifiability rows above it. Two needles were added to ''verify_extension_figures.mjs'' so both the caption and the two-not-three claim are checked against the paper |
+| The //people// claim had no guard | //"Seven of the 68 papers study the humans rather than the code, and all but one are from 2021 onwards."// Nothing computed either half | New report section **H2** prints the seven papers with their years and asserts both halves. Mutation-tested in both directions: moving the cut-off to 2022 fails and **names the two papers**, changing the expected count fails. The claim is **true** — 2017, 2021, 2022, 2023, 2024, 2025, 2025 |
+| This log named one freeze when there were two | The opening paragraph said all four passes were //handed the same frozen snapshot// and named ''out/ext/freeze/'', when the generic pass had been handed ''freeze2''. A finding is only attributable to a reviewer if you know which text it saw | The paragraph names all three freezes, says which pass saw which, and dates them. This pass's own snapshot is ''out/ext/freeze3/'' |
+**And one thing the deferred list did not contain.** Closing the Carnus row and
+the people guard changed the needle count, and that exposed a number this page had
+been publishing three different ways for the same quantity: //"60 of 60"// in
+//Quotes and per-paper figures//, //"the 64 needles"// in the Pass 2 row below,
+and //"the 70-needle per-paper check"// in the 2026-09-21 quote-check refresh. The
+check has never been 70. **60 went stale the moment the citations pass added four
+needles, and nothing pointed at it**, because the sentence that has to change is
+in a different section from the fix that changes it. All three are corrected, the
+figure is 66, and the sentence now says to re-run the script rather than quote it.
+**A process failure worth more than any of the findings.** The prose file this
+provenance page is built from, ''pages/provenance_privacy_browser_extensions.txt'',
+was **three weeks stale**: the markup sweep of 2026-09-17 and the quote-check
+refresh of 2026-09-21 were both edited straight onto the live page and never back-
+ported. Rebuilding with ''build_provenance_browser_extensions.py'' would have
+deleted both sections silently — the builder checks that no slot is empty, but it
+cannot check for content that only exists downstream of it. The prose file was
+reconstructed from the live page and the rebuild was then proved byte-identical to
+what was published before any of today's edits were applied. **If you edit one of
+these pages in the browser, back-port it the same day, or the next build eats it.**
+==== ext-external-assertions, closed 2026-09-22 ====
+Pass 5's sharpest finding was that the external-check script could not fail: three
+of its 26 checks called ''fail()'', five printed the word FAILED without
+incrementing anything, and eighteen were regex printers that emit **nothing** on a
+, a 429 or a reworded source. The header was corrected to say so and the
+conversion was filed as the ''ext-external-assertions'' item. This is that item.
+All 26 checks now assert. Each names the needle, status code or floor it requires
+and calls ''fail()'' when it is gone. The mechanism is eight new helpers alongside the
+existing ''expect'': ''fetch'' (anything but HTTP 200 is itself a failure, which
+is the direct fix for //a 403 on the AdGuard store page yields an empty section//),
+''expect_text'' for a phrase the source renders as a list item, ''expect_flat''
+for a phrase that wraps across lines in a source file, ''expect_re'' for a shape,
+''expect_code'' and ''expect_final'' for a status or a redirect the page states as
+a fact, ''expect_dead'' for a host the page asserts is gone, and ''expect_ge'' for
+a count that must not collapse. An ''ONLY="5 13"'' filter runs a subset, which is
+what makes the mutation test affordable.
+**It was mutation-tested, not read.** ''scripts/mutate_external_checks_browser_extensions.py''
+corrupts one needle, constant, status or floor at a time and re-runs only the
+affected check: **46 mutations, 46 killed the run**, at least one per check, log at
+''out/ext/mutation_browser_extensions.txt''. A conversion that cannot be shown to
+fail is not a conversion.
-That brief is not idle: on ''programming:traffic_files'' the three focused passes
+^ What the conversion found ^ Disposition ^
-returned ten findings and looked comprehensive, and the generic pass then found
+| **Check 16 had been fetching a 404.** Its source, ''%%extensionworkshop.com/documentation/develop/best-practices-for-collecting-user-data-consent/%%'', is gone — the live documentation for the ''data_collection_permissions'' key is at ''%%.../develop/firefox-builtin-data-consent/%%''. Because the check was a printer, it emitted an empty section inside a run that exited 0, for an unknown length of time | **accepted, fixed.** The check fetches the live page and asserts two phrases: //"data_collection_permissions key in the manifest"// and the //"none"// value the page's 16.5% bucket counts. This is the predicted failure caught in the act, in the one check nobody had singled out. **No figure on either page depended on the dead URL** — the 78.5/15.8/5.8 split comes from the AMO API, not from that doc — so nothing published was wrong; it was simply unchecked |
-fifteen more on the already-corrected page, including the two worst. **Treat this
+| **Chrome reworded the Manifest V2 timeline and the page went on quoting the old words.** The footer moved from //"Last updated 2026-07-08 UTC"// to //2026-09-09//; //"All remaining Manifest V2 extensions **are** removed"// became //"**have been** removed"//, and //"will remain installed, but will be unable to receive any updates and cannot be reinstalled"// was rewritten to //"Existing installs on Chrome 138 or earlier will continue to run, but they can no longer receive updates or be reinstalled if removed."// The footnote claims every quoted phrase in the table is verbatim, and two were not | **accepted, fixed, and the date is now pinned.** The page's quotes were **verbatim when fetched** — the Internet Archive captures of 2026-08-26, 2026-09-01 and 2026-09-06 all carry the old wording and the old footer — so this is currency, not misquotation. Both quotes and the third instance in the opening WRAP are updated, the footnote gives the new footer date and records the rewording, and check 8 now asserts the footer string itself. Pinning a footer date will fail the run on an edit that leaves the quotes intact; that is deliberate, because the quotes then have to be re-read |
-page's review as three-quarters done.** The work item ''ext-generic-review''
+| **This provenance page carried the same overclaim the script header did.** //"That script prints FAILED explicitly per check, so a blocked or rate-limited call cannot read as a pass"// was false for 23 of 26 checks when it was written, and stayed false after the script's own header was corrected on 2026-09-02 | **accepted, fixed.** A correction applied to one artefact does not propagate to the page that describes it — the //residue table does not fix the prose// failure, one level up |
-carries what is needed to finish it, and this section is where its findings go.
+| **Eight of the eighteen ''%%external_checks_*.sh%%'' scripts cannot fail either, and four of them carry the same header sentence.** ''crawler_detection'', ''dns'', ''filter_lists'', ''internet_scanning'', ''llm_agents'', ''preregistration'', ''requests'' and ''traffic_files'' have no failure counter, no exit gate, or both; ''dns'' says //"Every check prints FAILED explicitly, so a 403 or a rate limit cannot read as a pass"//, which is word-for-word the claim this file made and did not meet | **recorded, not fixed — out of this item's scope.** The audit, with the verdict read out of each file's fail helper and exit gate rather than grepped for the word FAILED, is at ''out/ext/sibling_assertion_audit.txt''. Ten of the eighteen do assert, including ''email_tracking'', which was fixed by an earlier review of exactly this kind |
+| **The ''expect'' helper cannot see a phrase in a bullet list.** It requires a sentence-terminating period within 200 characters, and the Cyberhaven post renders //"The malicious code was active between 1:32 AM UTC on December 25 and 2:50 AM UTC on December 26"// as a list item. The check failed on a page that carries the quote perfectly well | **accepted, and fixed by a second helper rather than a wider window.** ''expect_text'' drops the sentence requirement and keeps everything else. Widening the window in ''expect'' instead would have let every other needle match across a sentence boundary — a fallback that rescues one case by weakening the check that caught it |
+| **Pinning the Chrome Web Store's live counts was considered and rejected.** 43 sitemap shards, 8,338 ''%%<loc>%%'' entries in shard 0, 103,979 AMO extensions, 83,246 EasyList lines — all drift daily, and all are published on the page as dated snapshots | **rejected, with the reason in the file header.** These go through ''expect_ge'' against a floor far below the quoted figure, so the check fires on an empty parse or a collapsed response and not on ordinary store churn. A check that fails every morning is a check that gets ignored |
+| **What check 5 does not establish.** It asserts that the install count the store //shows// is rounded to a thousand. The page's claim is stronger — //"no precise figure appears anywhere in either response"// — which is a negative over a whole document that no regex here supports | **recorded as a stated limit**, in a comment at the check and here, rather than dressed up. It was read by hand on 2026-09-02 and again on 2026-09-22 |
-The self-audit below is not a substitute for it. It found three errors of one
+Unchanged by the conversion: every figure on both pages. Nothing here moved a
-specific kind, which is what a self-audit is good for; it is not an independent
+number. The 26-check run after conversion is at ''out/ext/external.txt'' and
-reader.
+exits 0.
 ==== What no reviewer caught ====
@@ Line 795: / Line 951: @@
 <WRAP todo>
   * The ''use_dynamic_url'' sample is 200 items. A store-wide figure is one crawl away and would settle whether extension enumeration is a live threat or a historical one.
-  * The 40-item store-composition sample supports "roughly three-quarters extensions" and nothing narrower.
+  * The store-composition figure rests on a 200-item CRX sample (146 of 198 served are extensions). A larger draw would tighten the //on the order of 260,000// estimate, which is currently an order-of-magnitude statement and is written as one.
   * Nobody has re-measured what an installed extension does to a crawl since IMC 2016. That is the gap this page most wants filled.
   * The ''object'' / ''object-partial'' line is single-label and eleven papers sit close to it; a second reader would move some.
@@ Line 809: / Line 965: @@
 <bibtex bibliography></bibtex>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Quote-check refresh, 2026-09-21 =====
+The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''ext_quotecheck.mjs'' already carried the fallback; the verdict table in //The extraction's own quotes// was copied from a pre-fallback run. Re-run, unedited:
+<code>
+$ node scripts/ext_quotecheck.mjs
+extension-corpus population tuples with a quote: 74
+  exact:            54
+  partial (>=60% of 5-word windows): 15
+  rescued from the PDF: 4
+  below threshold in BOTH renderings:  1   <- read these, do not count them as fabrications
+    IEEE-SP/2011/verified-security-for-browser-extensions  n=1139  cov=0.56
+      We conducted a simple analysis of manifests for over 1,139 popular Chrome extensions
+</code>
+^ Figure ^ Was ^ Is ^ Why ^
+| tuples with a quote | 74 | 74 | population unchanged |
+| exact | 54 | 54 | unchanged |
+| partial (≥60% of 5-word windows) | 15 | 15 | unchanged |
+| rescued from the PDF | — | **4** | these were inside the old 5 |
+| below threshold | **5** | **1** (in both renderings) | 5 = 4 + 1 exactly |
+**What the refresh does not change.** All five were already read by hand and all five are present, so no claim on [[:privacy:browser_extensions]] rested on the 5. What it changes is the figure a reader would quote as this page's quote-failure rate: 1 of 74, not 5 of 74. And it exposes something worth stating — two of the four //rescued// are the two tuples the hand read calls paraphrases, so //rescued// is a statement about the rendering, not a warrant that the extraction is verbatim.
+**Scope of this edit.** The verdict table and the paragraph under it. The per-paper needle check (64 needles on that date, not the //70// this sentence originally said), the store-size table, the population figures and every citation stand as published; ''report_browser_extensions.mjs'' was **not** re-run. [[:privacy:browser_extensions]] carries no quote-check figure, so it was not edited.
+^ Item ^ Value ^
+| Date | 2026-09-21, unsupervised |
+| Command | ''%%node scripts/ext_quotecheck.mjs%%'' |
+| Artifact | ''out/qc0921b/ext.txt'' |
+| Script changes | none |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Pages saved | this page only |