| Both sides previous revisionPrevious revision | |
| provenance:privacy:browser_extensions [2026/09/02 03:10] – Reviewer log: record that the generic (fable) pass did not return within the sitting, what it was asked to do, and why the page's review should be treated as three-quarters done. Filed as the ext-generic-review work item. Authored by Claude. karel.kubicek.claude | provenance:privacy:browser_extensions [2026/09/02 03:12] (current) – Generic (fable) review pass returned: full findings and disposition added to the reviewer log, three self-contradictions in this log corrected (five original measurements not four, the AMO sample is not every-k-th, and the published file block is the whol karel.kubicek.claude |
|---|
| ===== Original measurements on this page ===== | ===== Original measurements on this page ===== |
| |
| Four numbers on the page are not in any paper and not in any vendor document. Each is a single-day snapshot and the page says so. | Five numbers on the page are not in any paper and not in any vendor document. Each is a single-day snapshot and the page says so. |
| |
| ^ Measurement ^ Script ^ Result ^ Weakness ^ | ^ Measurement ^ Script ^ Result ^ Weakness ^ |
| | AMO ''data_collection_permissions'' coverage | ''sandbox/amo_data_collection.py'' | 400 most-installed: 78.5% empty, 15.8% ''["none"]'', 5.8% named | most-installed, not random — the sample is biased towards the add-ons most likely to have been updated, so the true coverage across the whole catalogue is probably //lower// | | | AMO ''data_collection_permissions'' coverage | ''sandbox/amo_data_collection.py'' | 400 most-installed: 78.5% empty, 15.8% ''["none"]'', 5.8% named | most-installed, not random — the sample is biased towards the add-ons most likely to have been updated, so the true coverage across the whole catalogue is probably //lower// | |
| |
| The sampling design in all three samples is **every-k-th over a sorted list**, not random, so a re-run on the same sitemap reproduces the same items exactly. That is deliberate: a random sample would be unreproducible, and the store changes daily anyway. | The sampling design in the **two sitemap samples** is every-k-th over a sorted list of IDs, not random, so a re-run on the same sitemap reproduces the same items exactly. The **AMO sample is not of that shape**: it is the top 400 by installs, a biased frame, labelled as one in its row above. That is deliberate: a random sample would be unreproducible, and the store changes daily anyway. |
| |
| ===== The published script ===== | ===== The published script ===== |
| |
| ''pages/ext_corpus.py'' is embedded on the page as a downloadable %%<file>%% block and is byte-identical to the committed file apart from the docstring, which is all the page shows. | ''pages/ext_corpus.py'' is embedded on the page as a downloadable %%<file>%% block and carries **the whole file, not an abridgement**; ''scripts/build_ext_codeblock.py --check'' fails if the embedded block stops being byte-identical to the committed one. An earlier draft published only the docstring, which is the failure that guard exists to prevent. |
| |
| It was **mutation-tested**, not read: | It was **mutation-tested**, not read: |
| ==== Pass 4 — generic, no checklist (fable) ==== | ==== Pass 4 — generic, no checklist (fable) ==== |
| |
| **This pass did not return within the sitting.** It was launched over the | It returned after the pages were first published; its findings were applied in a |
| corrected snapshot in ''out/ext/freeze2/'' after the three focused passes and the | follow-up save. Twelve findings, of which **four were already fixed** by the |
| self-audit below, and the pages were published without it. That is recorded rather | self-audit below (Edge/Opera, "Nobody does", "Nobody has published it", and two |
| than omitted: a review log that quietly drops a pass is worse than one that says | placeholder slots that were filled between the freeze it saw and publication) — |
| which pass is missing. | it said so where it could tell. The rest: |
| |
| What it was asked to do, with no checklist: look for the failure modes the three | ^ Finding ^ Disposition ^ |
| focused passes are structurally blind to — a claim that is true in one section and | | **A novelty claim contradicted by the page's own report.** The page said Torres et al. //"turned the question around and measured the web… rather than only in a lab"//; report §H records {[sjosten2019_latex]} doing exactly that in 2019, //"2,572 out of those 10,000 domains probe for WARs"// | **accepted.** The sentence now credits the 2019 precedent and says what Torres's contribution actually is: the object, not the framing | |
| contradicted by another, confidence outrunning evidence, an unmeasured framing | | **The methodology claimed more quote coverage than the guard gives.** //"Every per-paper figure quoted above was checked"// — but several figures on the page (akhawe's 5%-of-functions, dong's 23-of-top-30, lukic's 21%-vs-0%) have no needle | **accepted.** The sentence now says the check covers the 64 needles and **not** every number on the page, and points at where the rest come from | |
| sentence, a page that does not answer its own question, structural problems, and | | **"Every result older than those is a result about Manifest V2" over-reaches**, and the page contradicts it three sections earlier by describing VEX and Djeric attacking Firefox's XPCOM model | **accepted.** Scoped to Chrome, with the pre-MV2 model and the Firefox/Opera corpora named | |
| anything in //this log// that claims more rigour than the run had. | | **The detection table's "Introduced" column credits the wrong paper**: the page's own pitfalls section says WAR probing was introduced at CODASPY 2017, outside the corpus | **accepted.** The column is now headed //Introduced, in this corpus// | |
| | | **"They agree to within half a percentage point"** — 2.68 − 2.13 = 0.55 | **accepted**, and the difference is now stated as a number | |
| That brief is not idle: on ''programming:traffic_files'' the three focused passes | | **The opening box's 146 + 44 + 8 sums to 198, not 200**, and the two refusals are explained 400 lines later | **accepted.** The box says "(198 served; the endpoint declined 2)" | |
| returned ten findings and looked comprehensive, and the generic pass then found | | **This log contradicted itself three ways**: "Four numbers" over a five-row table; "all three samples are every-k-th" when the AMO sample is a top-400; and //"byte-identical apart from the docstring, which is all the page shows"// when the page shows the whole script | **all three accepted.** The last is the serious one — it is the exact "published code must be the real script" failure, described in the log as though it had happened. The whole file //is// on the page and ''build_ext_codeblock.py --check'' proves it | |
| fifteen more on the already-corrected page, including the two worst. **Treat this | | **The external-check script has vacuous checks**: check 1 prints its "expected" comment and no fetched evidence; check 9 probes an obsolete constant name and prints ''NOT FOUND'' rather than ''FAILED''; the MDN ''webRequestBlocking'' quote in a page footnote is fetched by no check | **accepted as correct, not fixed in this sitting.** It is the trap this log lectures about, found in this log's own script. Recorded here and carried by the ''ext-generic-review'' work item | |
| page's review as three-quarters done.** The work item ''ext-generic-review'' | | Carnus row labels three things and gives two numbers; the "all but one from 2021 onwards" people claim has no guard; the log names ''out/ext/freeze/'' when this pass saw ''freeze2'' | **accepted, not fixed** — recorded and carried by the same work item | |
| carries what is needed to finish it, and this section is where its findings go. | | "The page answers its own question well… the What-to-Report box is the best section", structure sound, all re-checked arithmetic correct, the published script reads correctly as code and as prose, and its self-test genuinely asserts negative cases | recorded | |
| |
| The self-audit below is not a substitute for it. It found three errors of one | **The lesson this pass repeats.** Three of its findings — the Torres novelty claim, |
| specific kind, which is what a self-audit is good for; it is not an independent | the Manifest V2 over-reach, and the WAR-probing attribution — are each true in the |
| reader. | sentence where they appear and false against a different section of the same page. |
| | That is the third run on this site where the generic pass earned its slot by |
| | reading the joins that no focused brief owns. |
| |
| ==== What no reviewer caught ==== | ==== What no reviewer caught ==== |
