Differences

This shows you the differences between two versions of the page.

--- provenance:design:longitudinal [2026/09/07 18:10] – New section 18: the four-pin hand-coding of all 250 repeated-crawl papers — codebook, both coders, inter-coder agreement, quote check and its one correction, judgement calls, scripts and unedited output. Section 9 bullet 1 closed. Authored by Claude karel.kubicek.claude
+++ provenance:design:longitudinal [2026/09/22 04:33] (current) – Figures-vs-script finding accepted: the 18.5 heading still named a three-generations-stale count (43); renamed to carry no count. No inbound anchor links across 188 pages. Authored by Claude karel.kubicek.claude
@@ Line 59: / Line 59: @@
 </code>
-Both default to ''/workspace/publications_dataset/data/extract/run1'' and ''.../data/fulltext''; ''--run'' and ''--fulltext'' override.
+Both default to ''/workspace/publications_dataset/data/extract/run1'' and ''.../data/fulltext''; ''%%--run%%'' and ''%%--fulltext%%'' override.
 ===== 5. The fold, and its residue in full =====
@@ Line 1473: / Line 1473: @@
 ^ Item ^ Value ^
 | Content page | [[design:longitudinal]], §"The frame moves even when you pin it" |
-| Saves | rev **1788465991** (''--if-rev 1787798449''), rev **1788467004** (''--if-rev 1788465991''), rev **1788467915** (''--if-rev 1788467004'') |
+| Saves | rev **1788465991** (''%%--if-rev%% 1787798449''), rev **1788467004** (''%%--if-rev%% 1788465991''), rev **1788467915** (''%%--if-rev%% 1788467004'') |
 | Corpus queried | none — this is a vendor date |
 | External source | Tranco's HTTP API and front page, fetched 2026-09-03 |
@@ Line 1560: / Line 1560: @@
 ==== 18.3 The reading packet, and what it costs ====
-papers is 20.9 MB of ''.cols'' text. Each coder was given a packet instead: the first 3,000 characters, plus context windows around two tiers of keyword matches (wave/repetition language, "same/unchanged/pinned", statefulness, the named ranking lists and filter lists, browsers, cloud regions), merged and capped at 32,000 characters — **and the full-text path in the packet header, with an instruction to open it whenever the packet is not enough.** Coders reported opening the full text routinely; several found the deciding sentence in a gap between windows.
+papers is 20.9 MB of ''.cols'' text. Each coder was given a packet instead: the first 3,000 characters, plus context windows around two tiers of keyword matches (wave/repetition language, "same/unchanged/pinned", statefulness, the named ranking lists and filter lists, browsers, cloud regions), merged and capped at 32,000 characters — **and the full-text path in the packet header, with an instruction to open it whenever the packet is not enough.** The packets instruct the coder to open the full text whenever the windows are not enough. **Whether they did is not logged**, so the recall of the packet rule is bounded only by the blind full-text pass below, not by any record of what each coder read.
 The packet is a recall risk, and it was measured rather than assumed — that is what the blind full-text second coder in §18.4 is for. The answer, in short: the packet does not lose "fixed" evidence, it loses the //absence// of it. See the direction-of-disagreement table below.
@@ Line 1577: / Line 1577: @@
 | "all four fixed" | 27/30 (90.0%) | 0.52 | A: 3/30, B: 4/30 | — |
-//κ// on "all four fixed" is low because the class is rare at n=30 — three and four positives respectively — not because the coders disagree about which papers they are. Read the count, not the //κ//, on that row.
+An earlier draft of this section said //κ// on "all four fixed" is low only because the class is rare, //"not because the coders disagree about which papers they are"//. **That was wrong and the files say the opposite.** Five papers are called all-four by at least one coder; the two agree on **two** of the five, and disagree on three. The class is indeed rare at //n// = 30, but the disagreement is real, and the 15-of-124 headline should be read with that in mind.
 **Direction of disagreement.** This is the part that changes how the content page is written:
@@ Line 1593: / Line 1593: @@
 Consequence for the content page: **the browser row of the estimate column is more likely too high than too low**, and the page says so.
-Every disagreement is printed paper by paper in the unedited output in §18.10.
+Every disagreement is printed paper by paper in the unedited output in §18.12.
-==== 18.5 The quote check, and the 43 that did not locate ====
+==== 18.5 The quote check, and what "unlocated" actually means ====
 Every non-empty ''quote'' must appear in the paper it is attributed to. 1,000 pin codings, 464 with no quote (all of them ''not-stated'', which the codebook permits and the script verifies: **0 codings carry a value other than ''not-stated'' with no quote**), 536 checked.
@@ Line 1604: / Line 1604: @@
   * what ''.cols'' actually holds: ''%%We implemented FourthParty as an extension to Mozilla \n B. Design Principles Firefox. It currently instruments%%''
-The two-column repair still splices a quarter of pages, and a sentence crossing a column boundary comes back with the neighbouring column injected into the middle of it. **A coder who de-splices that correctly writes a quote that is a substring of nothing.** Failing it would call a correct reading a hallucination. So the check now searches ''paper.cols.txt'', ''paper.norm.txt'', ''paper.txt'' and the packet, in four escalating modes, the last of which cuts the quote into 6-word shingles and passes at 80% found.
+The two-column repair still splices a quarter of pages, and a sentence crossing a column boundary comes back with the neighbouring column injected into the middle of it. **A coder who de-splices that correctly writes a quote that is a substring of nothing.** Failing it would call a correct reading a hallucination. So the check now searches ''paper.cols.txt'', ''paper.norm.txt'', ''paper.txt'' and the packet, in four escalating modes, the last of which cuts the quote into 6-word shingles and passes at 80% found. **And since 2026-09-22 a fifth text:** a quote located in none of the stored renderings is re-checked, in the same four modes, against a ''pypdf'' rendering of the same ''paper.pdf'' (''scripts/pdftext.py''). That verdict is reported as **RESCUED** — the quote is in the paper and the stored text is what is wrong. It is the fallback ''scripts/quote_check.mjs'' has had since 2026-09-04; this checker did not have it, and until it did, its unlocated bucket was not comparable with the rest of the site's.
 ^ Mode ^ Codings ^ Share of 536 ^
@@ Line 1611: / Line 1611: @@
 | Loose (de-hyphenated, quotes and dashes folded, case-insensitive) | 2 | 0.4% |
 | De-spliced (6-word shingles, ≥80%) | 3 | 0.6% |
-| **Located by no mode** | **42** | **7.8%** |
+| Rescued from the ''pypdf'' rendering of ''paper.pdf'' | 35 | 6.5% |
+| **Located by no mode, in any of the five texts** | **7** | **1.3%** |
-//The counts above are after the one correction described below. Before it: 488 exact, 43 unlocated (8.0%).//
+//The counts above are after the one correction described below **and** after the ''pypdf'' fallback added on 2026-09-22. Before the correction and before the fallback: 488 exact, 43 unlocated (8.0%). With the correction and still without the fallback — which is what this table published until 2026-09-22 — 489 exact, 42 unlocated (7.8%). The 42 did not become 7 because any coding changed; 35 of them are the same quotes, mechanically located in a second rendering of the same PDF.//
-**All 43 were then read by hand** — three sub-agents, each given the paper's three renderings and the PDF, asked to classify each quote as a faithful de-splice, a supported paraphrase, an unsupported paraphrase, or a quote lifted from the wrong part of the paper, and //independently// to say whether the paper supports the coded pin value at all.
+**All 43 were then read by hand** — three sub-agents, each given the paper's three renderings and the PDF, asked to classify each quote as a faithful de-splice, a supported paraphrase, an unsupported paraphrase, or a quote lifted from the wrong part of the paper, and //independently// to say whether the paper supports the coded pin value at all. **The 2026-09-22 fallback is an independent, mechanical check on that hand audit, and it agrees.** 35 of the 42 the hand audit called faithful de-splices are now located verbatim or near-verbatim in ''pypdf'''s rendering, by a script with no access to the hand verdicts. The 7 that remain unlocated are all inside the hand-audited set and were all given the same verdict — faithful de-splice — so the fallback has not contradicted a single one of the 43.
 ^ Verdict ^ Codings ^
@@ Line 1628: / Line 1629: @@
   * **What moved on the content page:** the vantage row over the 124 went from 25/44/10 to **26/45/9** (words / estimate / moved), and the all-pins fixed-versus-not agreement from 80.8% to 81.7%. The all-four figure, the 124, and the 6-of-23 survivor count did not move.
-The verdict files are ''out_pins/qaudit/verdict_1.tsv'' … ''verdict_3.tsv''. **The lesson is the check, not the count.** A plain substring test on ''paper.cols.txt'' alone would have reported a 9.1% fabrication rate here, of which the real rate is **0.2% (1 of 536)** — and the one real defect was not a fabrication at all but a quote from the wrong section, which no substring test can catch. One reviewer flagged a further boundary case worth recording rather than changing: two codings use ''same-explicit'' on the strength of a pinned identifier in papers whose ''repeatKind'' is ''single-crawl'', where the codebook's rule 4 would suggest ''same-implied''. Both readings are inside the codebook as written; the rule needs tightening before a rerun, and neither paper is in the 124.
+The verdict files are ''out_pins/qaudit/verdict_1.tsv'' … ''verdict_3.tsv''. **The lesson is the check, not the count.** A plain substring test on ''paper.cols.txt'' alone would have reported a 9.1% fabrication rate here, of which the real rate is **0.2% (1 of 536)** — and the one real defect was not a fabrication at all but a quote from the wrong section, which no substring test can catch. The 2026-09-22 fallback moves the //automated// bucket from 7.8% to **1.3%**, and that 1.3% is still not a fabrication rate: the real rate is the hand-audited 0.2%, unchanged, and the seven remaining are the hand audit's own faithful de-splices. An automated unlocated count is an upper bound on a fabrication rate and has never been an estimate of one. One reviewer flagged a further boundary case worth recording rather than changing: two codings use ''same-explicit'' on the strength of a pinned identifier in papers whose ''repeatKind'' is ''single-crawl'', where the codebook's rule 4 would suggest ''same-implied''. Both readings are inside the codebook as written; the rule needs tightening before a rerun, and neither paper is in the 124.
 ==== 18.6 What changed on the content page ====
@@ Line 1639: / Line 1640: @@
 | ''Use in Publications'' intro | the 250 rule, no caveat | adds that the rule is right about half the time and names the 124 |
 | Methodology bullets | one bullet: treat the pin figures as ceilings | three bullets: what the two columns each measure, the coder-agreement caveat, and that every other share-of-250 figure on the page inherits the false-positive rate and has **not** been recomputed |
+**Also corrected in the same edit, outside the four-pin work.** Four external claims the currency reviewer found stale — Quantcast's exit date, the OpenWPM version, the Playwright ''browsers.json'' source, and ICLab's maintenance status. Each is described in §18.10 Pass C with what was re-fetched.
 **Not changed, on purpose.** The cadence, span, probe and Tranco-ID figures are still shares of 250 rather than of 124. Recomputing them is a different piece of work with its own review obligation; folding it into this edit would have meant republishing a dozen tables under a population change that nothing in this hand-coding validates for them. The content page says plainly that they were not recomputed, which is the honest state, and it is filed below as the follow-up.
@@ Line 1667: / Line 1670: @@
 | Coder B | Claude Sonnet, 10 batches of 3, full text only, blind |
 | Quote adjudication | Claude Sonnet, 3 batches, over the 43 quotes no matching mode located; 1 correction applied |
-| Review | Three focused passes (Claude Sonnet: figures-vs-script, citations-and-quotes, external currency) and one generic pass (Claude Fable), all handed the page text, the scripts, their unedited output and these notes; findings and rejections in §18.11 |
+| Review | Three focused passes (Claude Sonnet: figures-vs-script, citations-and-quotes, external currency) and one generic pass (Claude Fable), all handed the page text, the scripts, their unedited output and these notes; findings and rejections in §18.10 and §18.11 |
 | Orchestration, scripts, page text | Claude Opus 5 |
 | Interruption | The run hit an API quota limit with 185 of 250 coded and resumed the same day. Same model, same codebook, same prompt; the resumed batches are ''out_pins/rebatch/''. Nothing had been published at the point of interruption. |
@@ Line 1674: / Line 1677: @@
 | Discussion block | Still none on this page, following the convention in §16. |
-==== 18.10 The scripts, and their unedited output ====
+==== 18.10 Review passes, 2026-09-07 ====
+Four reviewers, each told explicitly that the author's context may not be exhaustive, each handed both draft pages, all six scripts, their unedited output and these notes. **Every finding was accepted except one, marked "noted, not changed" in Pass B.** Two of the four independently found the same defect, which is the only evidence there is that the layer is worth its cost — and the generic pass, which nearly did not run at all, produced the most serious finding of the four (§18.11).
+=== Pass A — figures against the script (Claude Sonnet) ===
+Re-ran all six scripts and confirmed every committed ''-output.txt'', ''population.jsonl'' and ''valsample.tsv'' reproduces byte-identically, and that ''pins_population.mjs'' lines 17 and 19 are character-for-character the predicate in ''report_longitudinal.mjs''.
+  - **Stale table cell.** The 23-row exemplar table's vantage cell for {[singanamalla2022_respect]} still read ''**changed**''. The table had been pasted from ''%%--wiki%%'' output generated //before// the §18.5 correction was applied. **Accepted and fixed** by regenerating the table; it was the only divergent cell of 207.
+  - **Wrong hand-written count.** The prose said //"nine turn out not to be repeated crawls at all"//. Recounting from the coding files: of the 23, eight are ''parallel-not-temporal'', one of which is a survivor, so among the 17 non-survivors **seven** are not repeat crawls and **ten** are. The script never prints this aggregate, which is exactly why it was wrong. **Accepted and fixed.**
+  - **The structural guard in ''build_provenance_pins.py'' asserted much less than it claimed.** It compared heading, table and file-block counts over the //whole// output against the whole live page. Because the appended section adds eleven headings of its own, the check had eleven headings of slack: the reviewer mutation-tested it by deleting a heading from the **preserved** live content and got ''headings: 27 -> 37'' and **exit 0**. **Accepted and fixed** — the counts are now compared on the preserved text alone, which must equal the live page byte for byte, and the appended section is separately required to be non-empty. This is a page-matches-its-own-script failure of the kind the guard existed to prevent, and reading the script would not have found it; mutating it did.
+  - Mutation-tested and **confirmed sound**: the partial-population refusal in ''report_pins_audit.mjs'' (deleting one of 250 codings gives ''FATAL: coded 249 of 250''), the citekey-map guard (removing one slug exits 1), ''pins_quotecheck.mjs'' (an injected fabricated quote raises NOT FOUND from 7 to 8 and is printed in full; re-run on 2026-09-22 against the patched script — the pin was 42 → 43 before the ''pypdf'' fallback), ''pins_agreement.mjs'' (a removed coder-B file drops //n// to 29 and prints ''MISSING'' rather than silently continuing at 30), and the missing-file and empty-file guards in the builder.
+  - Hand-computed Cohen's //κ// for the list pin from the raw pairs: observed 0.667, expected 0.351, //κ// = 0.486 → 0.49, matching the script. No bug.
+  - Checked the figures **outside** the edited window against ''report_longitudinal-output.txt'' and found none stale.
+=== Pass B — citations and quotes (Claude Sonnet) ===
+  - All 32 keys on the content page and all 8 on this page resolve against the **live** ''literature:bibliography'' (850 entries, fetched today, not a local copy). The content page's key set is unchanged from the live version; this page gains exactly one key, {[singanamalla2022_respect]}, which is already in the bibliography and already cited on the content page. No entry added, none changed.
+  - Verified all 23 rows of ''out_pins/citekeys.json'' against the corpus index — title, venue, year and first-author surname — including that ''fouad2020_missed'' and ''fouad2024_devil'' are two distinct papers and are not conflated. No swapped keys.
+  - **Independently found the same stale vantage cell as Pass A**, by diffing the published table against a fresh ''%%--wiki%%'' run, and confirmed all three quotes in the §18.5 account by locating them in the paper itself.
+  - Spot-checked **24 evidence quotes** (10 ''changed'', 14 ''same-explicit'') against all three renderings: **24 of 24** located and supporting their coded value. Flagged one as loose rather than wrong — the Zoom-apps ''browser'' quote is about a crawler and parser rebuild rather than a browser version. **Noted, not changed**: the coding is ''changed'' either way and the codebook's browser pin covers the driven stack.
+  - Verified the "12 further papers outside this 23" arithmetic (18 all-four minus the 6 in the table) and every external figure quoted from a cited paper.
+=== Pass C — external currency (Claude Sonnet) ===
+This pass reviewed the **whole** of both pages, not the diff, and found four stale external claims — all of them in text this edit did not touch. All four accepted:
+  - **Quantcast's exit date was imprecise.** The page said //"dropped out after 1 April 2020"//, which was only as precise as the rows sampled in the table above it. Bisecting the Tranco API on 2026-09-07 puts it between **14 and 15 August 2020** (''date/20200814'' → ''GK9K'', providers include ''quantcast''; ''date/20200815'' → ''Z7NG'', providers do not). Re-verified independently before the edit. Every other row and date in that table and its footnote was checked against the live API and is exact.
+  - **OpenWPM had moved on.** The page cited v0.36.0 / Firefox 154.0 as its worked example; **v0.37.0 shipped on 6 September 2026** pinning **Firefox 155.0.1**. Confirmed via ''releases/latest'' and by reading ''install-firefox.sh'' (''# FIREFOX_155_0_1_RELEASE''). Updated, with the drift itself now noted in the footnote, since it is the paragraph's own point.
+  - **The Playwright footnote quoted ''main'', not a release.** It cited ''browsers.json'' from the development branch — ''revision 1241'', ''browserVersion 152.0.7977.54'' — as evidence that pinning Playwright pins a browser. That snapshot shipped in no release, so it was the wrong evidence for the claim. Re-fetched from the tagged release ''v1.63.0'': ''revision 1243'', ''browserVersion 153.0.8010.12''. The reviewer reported different values for that tag than the author's own fetch did; a third fetch during the generic pass returned ''revision 1243'', ''browserVersion 153.0.8010.12'', matching the author's, so the published numbers stand.
+  - **ICLab is dormant.** The page called Censored Planet and ICLab //"continuous, longitudinal and public"// in one breath. Censored Planet is active. ICLab's site serves an **expired TLS certificate** as of 2026-09-07 (''curl'' fails with exit 60 without ''-k''), its data page returns **404**, and its news stops in 2019 — all re-verified before the edit. The sentence now recommends the paper and warns off the feed.
+  - Confirmed accurate and unchanged: the HTTP Archive changelog 404 (still 12 entries, last 2017-06-01, on ''master'' after the repo rename), the HTTP Archive cadence text, all seven Tranco list ids, the EasyList commit cadence, the Public Suffix List footnote, the Chrome for Testing endpoints, and the WebREC repository facts. Neither page names an RFC or W3C specification, so that category is empty.
+=== Pass D — generic (Claude Fable) ===
+Run last, on the published pages, after A–C were applied. Findings and their disposition are in §18.11.
+==== 18.11 Generic review pass ====
+**Commissioned, nearly missed, and completed late.** A Claude Fable pass was given both published pages, the scripts and their output, and asked for whatever the three focused passes were not looking for — overstated claims, structure, a section that does not answer its own question, and specifically whether ''same-implied'' is doing too much work, whether the split between the 250 and the 124 is legible, whether //n// = 30 and //κ// = 0.60 are being asked to carry more than they can, and whether the page states plainly enough that both coders and the adjudicator were LLMs reading LLM-extracted data. The run's budget all but ran out before it returned, and this section was published saying it never would; it arrived afterwards and its findings are below.
+This is worth stating rather than quietly leaving a gap, because the generic pass is the one that catches overstatement, and §18 is the section most at risk of it. The specific things a reader should discount accordingly:
+  * The **estimate** column rests on ''same-implied'', which is a judgement that silence about change means no change. It is 47 of 124 on the list pin and 55 of 124 on the browser. The strict reading is published beside it and is much smaller.
+  * The reliability figures are **//n// = 30**, and the //κ// for "all four fixed" (0.52) is computed on three and four positives respectively.
+  * **Every coder in this exercise was a language model**, reading text extracted from PDFs by a pipeline, coding a schema produced by another language model. The blind second coder and the quote adjudication bound the error; they do not remove it, and no human read any of the 250 papers.
+Re-running this pass was filed as follow-up work — and then it returned.
+**Findings, and what was done.** The pass returned after the first save of this section. Applied to the content page: the disclosure that **both coders and the adjudicator were language-model agents** reading LLM-extracted text, which the content page had nowhere stated and which "hand-coded" actively obscures; the softening of //"two say so in words"//, after the reviewer read the two ''same-explicit'' quote sets and found them to be permanent identifiers and one-off constructions rather than statements that anything was unchanged; **Fisher exact //p//-values** on the span reversal (0.10, 0.08 and 0.69 across three cuts, none significant) in place of //"yes, and more sharply"//; the floor-versus-false-positive contradiction in the lead; and a flag that one of the 15 all-four papers is, by its own coder's note, a cellular walk test rather than a web crawl. Applied here: the corrected //κ// defence in §18.4, the deletion of an unverifiable claim about coders opening the full text, and the resolution of the Playwright value discrepancy.
+**Accepted and not done, for budget:** re-adjudicating the 73 ''same-explicit'' codings against the codebook's own wording; re-coding the 5G paper and recomputing 15/124 as 14/123; reading the 12 all-four papers outside the exemplar table and printing their notes; adjudicating the seven browser disagreements, which the reviewer argues may be the full-text coder's misses rather than the excerpt coder's over-calls, making §18.4's directional claim unsafe; publishing the //κ// confidence interval; recomputing the repeated-versus-single-shot comparison over the 124; per-table "of 250, unaudited" markers; and the ''waveCount''-stratified middle reading of ''same-implied''. Each is recorded as follow-up rather than silently dropped. **Until they are done, §18.4's "the browser row is more likely too high than too low" and the content page's browser sentence rest on an untested attribution of the disagreement to one coder.**
+==== 18.12 The scripts, and their unedited output ====
 Everything below is read from the committed file by ''scripts/build_provenance_pins.py'' at build time. None of it is retyped.
@@ Line 2358: / Line 2414: @@
 //              de-splicing mode and it is reported separately and listed in full,
 //              because it is the weakest evidence and has to be read by a human.
-// Anything failing all four is printed in full.
+// A quote that fails all four against every stored rendering is then tried
+// against a FIFTH text: a pypdf rendering of the same paper.pdf, produced on
+// demand by `scripts/pdftext.py` via `scripts/pdftext_cache.mjs`. `.cols` and
+// pypdf splice different sentences, so a coder who read the PDF correctly can
+// produce a quote that is absent from every stored rendering and verbatim in
+// the PDF. The same four modes are tried there and a pass is reported as
+//   RESCUED  located in the PDF but in none of the stored renderings. The
+//            quote is in the paper; the stored text is what is wrong.
+// This is the fallback `scripts/quote_check.mjs` and `scripts/report_webxray.mjs`
+// already use; it was added here on 2026-09-22, and before that this checker's
+// NOT FOUND bucket was not comparable with theirs.
+// Anything failing all four modes against all five texts is printed in full.
 import fs from 'node:fs';
 import path from 'node:path';
+import { pdfText } from './pdftext_cache.mjs';
 const argv = process.argv.slice(2);
@@ Line 2378: / Line 2446: @@
     .replace(/[ ]/g, ' ')
     .toLowerCase();
+// Fraction of the quote's overlapping SHINGLE_N-word windows present in a
+// loose-folded haystack; null when the quote is too short to shingle.
+const shingleFrac = (q, hayLoose) => {
+  const w = loose(q).split(' ').filter(Boolean);
+  if (w.length < SHINGLE_N) return null;
+  const sh = [];
+  for (let i = 0; i + SHINGLE_N <= w.length; i++) sh.push(w.slice(i, i + SHINGLE_N).join(' '));
+  return sh.filter((x) => hayLoose.includes(x)).length / sh.length;
+};
 const pop = fs.readFileSync(POP, 'utf8').trim().split('\n').map(JSON.parse);
@@ Line 2384: / Line 2462: @@
 const fails = [];
 const splicedList = [];
+const rescuedList = [];
 let missing = 0;
@@ Line 2413: / Line 2492: @@
     if (hayL.includes(loose(q))) { modes.loose++; continue; }
     // de-splicing mode
-    const w = loose(q).split(' ').filter(Boolean);
+    const frac = shingleFrac(q, hayL);
-    if (w.length >= SHINGLE_N) {
+    if (frac !== null && frac >= SHINGLE_MIN) {
-      const sh = [];
+      modes.spliced++;
-      for (let i = 0; i + SHINGLE_N <= w.length; i++) sh.push(w.slice(i, i + SHINGLE_N).join(' '));
+      splicedList.push({ key: p.key, pin, value: v.value, quote: q, frac });
-      const hit = sh.filter((s) => hayL.includes(s)).length;
+      continue;
-      if (hit / sh.length >= SHINGLE_MIN) {
+    }
-        modes.spliced++;
+    // Fifth text: pypdf. Only consulted for a quote that is in none of the
-        splicedList.push({ key: p.key, pin, value: v.value, quote: q, frac: hit / sh.length });
+    // stored renderings, so it costs nothing on a clean run.
+    const pdf = pdfText(p.key);
+    if (pdf !== null) {
+      const pdfC = collapse(pdf);
+      const pdfL = loose(pdf);
+      let mode = null;
+      if (pdf.includes(q)) mode = 'exact';
+      else if (pdfC.includes(collapse(q))) mode = 'collapsed';
+      else if (pdfL.includes(loose(q))) mode = 'loose';
+      else {
+        const pdfFrac = shingleFrac(q, pdfL);
+        if (pdfFrac !== null && pdfFrac >= SHINGLE_MIN) mode = `spliced ${(100 * pdfFrac).toFixed(0)}%`;
+      }
+      if (mode !== null) {
+        rescuedList.push({ key: p.key, pin, value: v.value, quote: q, frac, mode });
         continue;
       }
-      fails.push({ key: p.key, pin, value: v.value, quote: q, frac: hit / sh.length });
-    } else {
-      fails.push({ key: p.key, pin, value: v.value, quote: q, frac: null });
     }
+    fails.push({ key: p.key, pin, value: v.value, quote: q, frac, pdf: pdf !== null });
   }
 }
@@ Line 2435: / Line 2526: @@
 console.log(`coded dir: ${CODED}`);
 console.log(`renderings searched per paper: paper.cols.txt, paper.norm.txt, paper.txt, and the excerpt packet`);
+console.log(`fallback for a quote found in none of those: a pypdf rendering of paper.pdf (scripts/pdftext.py), same four modes, reported as RESCUED`);
 console.log(`de-splicing mode: ${SHINGLE_N}-word shingles, pass at >= ${(100 * SHINGLE_MIN).toFixed(0)}% found`);
 console.log(`\npapers with no coding file (skipped): ${missing}`);
@@ Line 2444: / Line 2536: @@
 console.log(`      loose match:                 ${pc(modes.loose)}`);
 console.log(`      spliced match (de-spliced):  ${pc(modes.spliced)}`);
+console.log(`      RESCUED from the PDF:        ${pc(rescuedList.length)}`);
 console.log(`      NOT FOUND:                   ${pc(fails.length)}`);
+console.log(`   (papers with no pypdf rendering available: ${[...new Set(fails.filter((f) => !f.pdf).map((f) => f.key))].length})`);
 // A quote-less coding that is not `not-stated` is a coding with no evidence.
@@ Line 2462: / Line 2556: @@
 for (const f of splicedList.sort((a, b) => a.frac - b.frac)) {
   console.log(`   ${f.key}  [${f.pin}=${f.value}]  shingles ${(100 * f.frac).toFixed(0)}%\n      ${f.quote}`);
+}
+console.log(`\nRESCUED — in none of the stored renderings, located in the pypdf rendering of paper.pdf (${rescuedList.length}).`);
+console.log(`These are defects in the stored text, not in the hand coding:`);
+for (const f of rescuedList) {
+  console.log(`   ${f.key}  [${f.pin}=${f.value}]  stored shingles ${f.frac == null ? 'n/a' : (100 * f.frac).toFixed(0) + '%'} -> pdf ${f.mode}\n      ${f.quote}`);
 }
@@ Line 2680: / Line 2780: @@
 # Structural guard: an in-place generator that silently drops sections is the
-# failure mode this check exists for. Nothing that was on the live page may go.
+# failure mode this check exists for.
+#
+# A whole-output count is NOT enough. The appended section adds ~11 headings of
+# its own, so a total-count check has that much slack: drop one heading from the
+# preserved live content and the total still exceeds the original. Mutation
+# testing on 2026-09-07 confirmed exactly that — a heading deleted from `head`
+# gave "27 -> 37" and exit 0. So the counts are compared on the PRESERVED live
+# text alone (head + sep + tail), which must match the live page exactly, and the
+# appended section is counted separately and required to be non-empty.
+preserved = head + sep + tail
 for tag, pat in (("headings", r"(?m)^=====? "), ("tables", r"(?m)^\^ "), ("file blocks", r"<file ")):
-    before, after = len(re.findall(pat, live)), len(re.findall(pat, out))
+    b, keep, add = len(re.findall(pat, live)), len(re.findall(pat, preserved)), len(re.findall(pat, prose))
-    if after < before:
+    if keep != b:
-        sys.exit(f"FATAL: {tag} went from {before} to {after}; the append dropped content.")
+        sys.exit(f"FATAL: {tag} in the preserved live text went from {b} to {keep}; the append altered content it must not touch.")
-    print(f"{tag}: {before} -> {after}")
+    print(f"{tag}: live {b} kept {keep} + new {add} = {keep + add}")
+    if add == 0 and tag != "tables":
+        sys.exit(f"FATAL: the appended section contains no {tag}; it is not the section that was written.")
+# Byte-level check that nothing in the live page was silently rewritten, beyond
+# the patches applied above, which are themselves verified to have a target.
+if preserved != live:
+    sys.exit("FATAL: the preserved text is not byte-identical to the (patched) live page.")
 pathlib.Path(a.out).write_text(out, encoding="utf8")
@@ Line 3173: / Line 3288: @@
 coded dir: out_pins/coded
 renderings searched per paper: paper.cols.txt, paper.norm.txt, paper.txt, and the excerpt packet
+fallback for a quote found in none of those: a pypdf rendering of paper.pdf (scripts/pdftext.py), same four modes, reported as RESCUED
 de-splicing mode: 6-word shingles, pass at >= 80% found
@@ Line 3183: / Line 3299: @@
       loose match:                 2  0.4%
       spliced match (de-spliced):  3  0.6%
-      NOT FOUND:                   42  7.8%
+      RESCUED from the PDF:        35  6.5%
+      NOT FOUND:                   7  1.3%
+   (papers with no pypdf rendering available: 0)
 codings with no quote and a value other than not-stated: 0
@@ Line 3195: / Line 3313: @@
       we sampled 100 domains from each parking service and setup an additional crawler which crawled them daily for a week.
-QUOTES NOT LOCATED, printed in full (42):
+RESCUED — in none of the stored renderings, located in the pypdf rendering of paper.pdf (35).
-/IEEE-SP/third-party-web-tracking-policy-and-technology  [browser=same-implied]  shingles 75%
+These are defects in the stored text, not in the hand coding:
+/IEEE-SP/third-party-web-tracking-policy-and-technology  [browser=same-implied]  stored shingles 75% -> pdf exact
       We implemented FourthParty as an extension to Mozilla Firefox.
-/WWW/understanding-and-combating-link-farming-in-the-twitter-social-network  [list=same-explicit]  shingles 50%
+/NDSS/parking-sensors-analyzing-and-detecting-parked-domains  [vantage=varied-deliberate]  stored shingles 47% -> pdf exact
-      We re-crawled the profile page of each user in the above dataset in February 2011
-/NDSS/parking-sensors-analyzing-and-detecting-parked-domains  [vantage=varied-deliberate]  shingles 47%
       To assess any geographical differences, we performed this crawl in parallel from the United States and one country from Europe.
-/USENIX/measuring-the-longitudinal-evolution-of-the-online-anonymous-marketplace-ecosyst  [state=same-implied]  shingles 38%
+/USENIX/measuring-the-longitudinal-evolution-of-the-online-anonymous-marketplace-ecosyst  [state=same-implied]  stored shingles 38% -> pdf exact
       We managed to emulate typical browser behavior in all but one case (BlueSky).
-/WWW/stress-testing-the-booters-understanding-and-undermining-the-business-of-ddos-se  [list=same-implied]  shingles 58%
+/CCS/poster-probing-tor-hidden-service-with-dockers  [list=same-explicit]  stored shingles 69% -> pdf exact
-      In order to measure the stability of these amplifiers we probed them periodically for 13 weeks to understand how many were still located at the same IP and misconfiged.
-/CCS/poster-probing-tor-hidden-service-with-dockers  [list=same-explicit]  shingles 69%
       The number of Onion addresses in this experiment was 16,683, and the addresses were crawled twice a day at 12-hour intervals.
-/CCS/how-you-get-shot-in-the-back-a-systematical-study-about-cryptojacking-in-the-rea  [list=same-implied]  shingles 50%
+/CCS/how-you-get-shot-in-the-back-a-systematical-study-about-cryptojacking-in-the-rea  [list=same-implied]  stored shingles 50% -> pdf exact
       we can only continuously monitor 1,000 samples. Besides, we re-visit these samples every 3 days
-/IMC/visiblev8-in-browser-monitoring-of-javascript-in-the-wild  [list=same-explicit]  shingles 0%
+/IMC/visiblev8-in-browser-monitoring-of-javascript-in-the-wild  [list=same-explicit]  stored shingles 0% -> pdf exact
       to visit the Alexa top 50k web domains
-/WWW/a-first-look-at-deep-learning-apps-on-smartphones  [list=changed]  shingles 52%
+/WWW/a-first-look-at-deep-learning-apps-on-smartphones  [list=changed]  stored shingles 52% -> pdf exact
       We have crawled two datasets at different moments, June 2018 and September 2018, which are three months apart. The two app datasets have more than 2/3 overlapped apps.
-/WWW/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private  [browser=same-implied]  shingles 43%
+/WWW/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private  [browser=same-implied]  stored shingles 43% -> pdf exact
       We installed the widely used OpenWPM framework and configured it to use four vanilla Mozilla Firefox browsers in parallel
-/WWW/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private  [vantage=same-implied]  shingles 50%
+/WWW/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private  [vantage=same-implied]  stored shingles 50% -> pdf exact
       the physical machine on which the VM was deployed is located in the EU, meaning that the visited websites must comply with GDPR
-/WWW/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private  [state=same-implied]  shingles 43%
+/WWW/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private  [state=same-implied]  stored shingles 43% -> pdf exact
       we did not install any extension on the browser that would stop the websites from storing or reading cookies
-/WWW/mobile-app-risk-ranking-via-exclusive-sparse-coding  [list=changed]  shingles 77%
+/WWW/mobile-app-risk-ranking-via-exclusive-sparse-coding  [list=changed]  stored shingles 77% -> pdf spliced 91%
       There are no overlaps for the above datasets, and the apps are from a wide range of functionalities, including game, news, music, social, movies/TV, travel, life, etc.
-/WWW/beyond-the-front-page-measuring-third-party-dynamics-in-the-field  [vantage=varied-deliberate]  shingles 53%
+/WWW/beyond-the-front-page-measuring-third-party-dynamics-in-the-field  [vantage=varied-deliberate]  stored shingles 53% -> pdf exact
       we conduct the measurements from three different locations (Europe (DE), North America (US), and Asia (JP)) to account for possible geographical differences
-/WWW/beyond-the-front-page-measuring-third-party-dynamics-in-the-field  [state=same-implied]  shingles 53%
+/WWW/beyond-the-front-page-measuring-third-party-dynamics-in-the-field  [state=same-implied]  stored shingles 53% -> pdf exact
       We used the same method to create the browser profile for our experiment crawls that we utilized in the pre-study.
-/WWW/towards-realistic-and-reproducibleweb-crawl-measurements  [list=same-explicit]  shingles 0%
+/WWW/towards-realistic-and-reproducibleweb-crawl-measurements  [list=same-explicit]  stored shingles 0% -> pdf exact
       Tranco list of top sites [18] (snapshot 77PX)
-/WWW/towards-realistic-and-reproducibleweb-crawl-measurements  [browser=same-implied]  shingles 33%
+/WWW/towards-realistic-and-reproducibleweb-crawl-measurements  [browser=same-implied]  stored shingles 33% -> pdf exact
       We crawl using two variants of Puppeteer [4] controlling Chromium 80: a lower-realism naive variant and a higher-realism stealth variant.
-/WWW/towards-realistic-and-reproducibleweb-crawl-measurements  [state=same-implied]  shingles 70%
+/WWW/where-are-you-taking-me-understanding-abusive-traffic-distribution-systems  [browser=same-implied]  stored shingles 50% -> pdf exact
-      the browser is launched with a clean user profile (i.e., no cookies or cached content)
-/WWW/where-are-you-taking-me-understanding-abusive-traffic-distribution-systems  [browser=same-implied]  shingles 50%
       we rely on a fully-featured, headless Chrome browser, governed by Selenium.
-/WWW/where-are-you-taking-me-understanding-abusive-traffic-distribution-systems  [vantage=same-implied]  shingles 47%
+/WWW/where-are-you-taking-me-understanding-abusive-traffic-distribution-systems  [vantage=same-implied]  stored shingles 47% -> pdf exact
       ODIN uses university IP addresses (one per profile) and a /24 subnet from a research-friendly, but less well-known VPS provider
-/IMC/muzeel-assessing-the-impact-of-javascript-dead-code-elimination-on-mobile-web-pe  [browser=varied-deliberate]  shingles 17%
+/IMC/muzeel-assessing-the-impact-of-javascript-dead-code-elimination-on-mobile-web-pe  [browser=varied-deliberate]  stored shingles 17% -> pdf exact
       We now evaluate Muzeel across browsers: Chrome, Edge, Firefox, and Brave.
-/IMC/respect-the-origin-a-best-case-evaluation-of-connection-coalescing-in-the-wild  [list=same-explicit]  shingles 57%
+/IMC/respect-the-origin-a-best-case-evaluation-of-connection-coalescing-in-the-wild  [list=same-explicit]  stored shingles 57% -> pdf loose
       Our initial sample set was the 5000 domains with the most requests to the third party, as indicated by the Referer field in the CDN's third party logs.
-/IMC/respect-the-origin-a-best-case-evaluation-of-connection-coalescing-in-the-wild  [browser=changed]  shingles 38%
+/IMC/respect-the-origin-a-best-case-evaluation-of-connection-coalescing-in-the-wild  [browser=changed]  stored shingles 38% -> pdf exact
       Results shown are for Firefox (v91) for later comparison, since only Firefox has client-side support for ORIGIN Frame.
-/IMC/respect-the-origin-a-best-case-evaluation-of-connection-coalescing-in-the-wild  [state=same-explicit]  shingles 57%
+/IMC/respect-the-origin-a-best-case-evaluation-of-connection-coalescing-in-the-wild  [state=same-explicit]  stored shingles 57% -> pdf exact
       Client caches are intentionally deleted between measurements in all of our active measurements to prevent bias from caching effects
-/PETS/a-global-survey-of-android-dual-use-applications-used-in-intimate-partner-survei  [list=same-explicit]  shingles 52%
+/USENIX/measurement-by-proxy-on-the-accuracy-of-online-marketplace-measurements  [list=same-implied]  stored shingles 59% -> pdf exact
-      we crawled Google Play again in all fifteen languages for one day in February 2022 with the queries we obtained and used in our 2020 crawling.
-/PETS/atom-ad-network-tomography  [state=same-implied]  shingles 79%
-      Each persona was associated with a unique browser running on an isolated virtual machine with a unique IP address.
-/USENIX/measurement-by-proxy-on-the-accuracy-of-online-marketplace-measurements  [list=same-implied]  shingles 59%
       We scraped Hansa 17 times between late 2015 and mid-2017, collecting a total of 332,795 pages amounting to 39.5 GB of data.
-/WWW/leveraging-googles-publisher-specific-ids-to-detect-website-administration  [list=changed]  shingles 58%
+/WWW/leveraging-googles-publisher-specific-ids-to-detect-website-administration  [list=changed]  stored shingles 58% -> pdf exact
       We perform our analysis on websites which have been crawled in all snapshots (i.e., their intersection) and contain at least one Publisher ID. There are over 191,000 such websites.
-/CCS/cookiegraph-understanding-and-detecting-first-party-tracking-cookies  [list=same-explicit]  shingles 63%
+/CCS/cookiegraph-understanding-and-detecting-first-party-tracking-cookies  [list=same-explicit]  stored shingles 63% -> pdf exact
       We successfully conducted the four parallel crawls for 99.31% of the 20K websites.
-/CCS/cookiegraph-understanding-and-detecting-first-party-tracking-cookies  [vantage=same-explicit]  shingles 71%
+/CCS/cookiegraph-understanding-and-detecting-first-party-tracking-cookies  [vantage=same-explicit]  stored shingles 71% -> pdf exact
       We run all crawls in the US to minimize the impact of the EU GDPR and do not interact with cookie banners.
-/CCS/cookiegraph-understanding-and-detecting-first-party-tracking-cookies  [state=varied-deliberate]  shingles 46%
+/WWW/automatic-discovery-of-emerging-browser-fingerprinting-techniques  [list=same-explicit]  stored shingles 59% -> pdf exact
-      We conduct four parallel crawls: two with third-party cookies enabled (3P-Allowed) and two with third-party cookies blocked (3P-Blocked).
-/WWW/automatic-discovery-of-emerging-browser-fingerprinting-techniques  [list=same-explicit]  shingles 59%
       we redid the crawling on the same 10K domains in September 2022 (11-month gap) and applied our system to this data set by following the same procedure.
-/NDSS/flow-correlation-attacks-on-tor-onion-service-sessions-with-sliding-subset-sum  [list=same-implied]  shingles 57%
+/NDSS/flow-correlation-attacks-on-tor-onion-service-sessions-with-sliding-subset-sum  [list=same-implied]  stored shingles 57% -> pdf exact
       clients repeatedly accessed onion services and the top 150 accessed websites according to the Tranco ranking [68], in parallel.
-/NDSS/flow-correlation-attacks-on-tor-onion-service-sessions-with-sliding-subset-sum  [browser=same-explicit]  shingles 61%
+/NDSS/flow-correlation-attacks-on-tor-onion-service-sessions-with-sliding-subset-sum  [browser=same-explicit]  stored shingles 61% -> pdf exact
       Clients also run on an isolated Docker container that executes a 0.4.7.10 Tor process and issues requests to Tor via the Python libraries selenium [58] and tbselenium [3]
-/IMC/learning-as-to-organization-mappings-with-borges  [list=same-implied]  shingles 64%
+/IMC/learning-as-to-organization-mappings-with-borges  [list=same-implied]  stored shingles 64% -> pdf loose
       we used the PeeringDB snapshot from July 24, 2024, and CAIDA's AS2Org from July 1, 2024.
-/IMC/somesite-i-used-to-crawl-awareness-agency-and-efficacy-in-protecting-content-cre  [list=same-explicit]  shingles 77%
+/PETS/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and  [browser=varied-deliberate]  stored shingles 48% -> pdf loose
-      We restrict the list to sites that appear in all of the top 100k lists over this period to avoid having our results affected by list churn
-/PETS/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and  [browser=varied-deliberate]  shingles 48%
       Since OpenWPM only supports Firefox and, unlike Chrome, Firefox does not expose the partitioned_key field in its cookie storage, we modified BannerClick to operate in Chrome's default mode.
-/PETS/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and  [state=varied-deliberate]  shingles 70%
+/PETS/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and  [state=varied-deliberate]  stored shingles 70% -> pdf exact
       We use OpenWPM [24] to conduct a combination of stateful and stateless crawls aiming to observe the interplay of the setter and the sender website
-/PETS/tracker-installations-are-not-created-equal-understanding-tracker-configuration  [list=same-implied]  shingles 55%
+/PETS/tracker-installations-are-not-created-equal-understanding-tracker-configuration  [list=same-implied]  stored shingles 55% -> pdf exact
       we joined the top one million websites from Tranco [49] with SimilarWeb, a web traffic estimator
-/PETS/tracker-installations-are-not-created-equal-understanding-tracker-configuration  [browser=same-implied]  shingles 56%
+/PETS/tracker-installations-are-not-created-equal-understanding-tracker-configuration  [browser=same-implied]  stored shingles 56% -> pdf exact
       Each website was opened in a Chrome instance with two installed custom Chrome extensions.
-/PETS/tracker-installations-are-not-created-equal-understanding-tracker-configuration  [vantage=same-implied]  shingles 47%
+/PETS/tracker-installations-are-not-created-equal-understanding-tracker-configuration  [vantage=same-implied]  stored shingles 47% -> pdf loose
       All website visits took place on Linux virtual machines in Google Cloud's US Central region between September and November 2024.
-/PETS/who-s-watching-you-zoom-investigating-privacy-of-third-party-zoom-apps  [list=changed]  shingles 40%
+/PETS/who-s-watching-you-zoom-investigating-privacy-of-third-party-zoom-apps  [list=changed]  stored shingles 40% -> pdf exact
       between December 2023 and December 2024, 667 new apps were added, 212 apps were removed, and 49 apps were renamed
+QUOTES NOT LOCATED, printed in full (7):
+/WWW/understanding-and-combating-link-farming-in-the-twitter-social-network  [list=same-explicit]  shingles 50%
+      We re-crawled the profile page of each user in the above dataset in February 2011
+/WWW/stress-testing-the-booters-understanding-and-undermining-the-business-of-ddos-se  [list=same-implied]  shingles 58%
+      In order to measure the stability of these amplifiers we probed them periodically for 13 weeks to understand how many were still located at the same IP and misconfiged.
+/WWW/towards-realistic-and-reproducibleweb-crawl-measurements  [state=same-implied]  shingles 70%
+      the browser is launched with a clean user profile (i.e., no cookies or cached content)
+/PETS/a-global-survey-of-android-dual-use-applications-used-in-intimate-partner-survei  [list=same-explicit]  shingles 52%
+      we crawled Google Play again in all fifteen languages for one day in February 2022 with the queries we obtained and used in our 2020 crawling.
+/PETS/atom-ad-network-tomography  [state=same-implied]  shingles 79%
+      Each persona was associated with a unique browser running on an isolated virtual machine with a unique IP address.
+/CCS/cookiegraph-understanding-and-detecting-first-party-tracking-cookies  [state=varied-deliberate]  shingles 46%
+      We conduct four parallel crawls: two with third-party cookies enabled (3P-Allowed) and two with third-party cookies blocked (3P-Blocked).
+/IMC/somesite-i-used-to-crawl-awareness-agency-and-efficacy-in-protecting-content-cre  [list=same-explicit]  shingles 77%
+      We restrict the list to sites that appear in all of the top 100k lists over this period to avoid having our results affected by list churn
 </file>
@@ Line 3417: / Line 3538: @@
 [[design:longitudinal|← back to the content page]] · [[provenance:design:website_selection|the 2026-09-03 Alexa date audit]] · [[literature:corpus|corpus-level provenance]]
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Quote-check refresh, 2026-09-22 =====
+**What was wrong.** ''scripts/pins_quotecheck.mjs'' searched four texts — the three
+stored renderings and the excerpt packet — and had **no ''pypdf'' fallback**. The
+-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] measured that
+.1% of quotes unlocatable in ''paper.cols.txt'' are present in an independent
+''pypdf'' rendering of the same ''paper.pdf'', and ''scripts/quote_check.mjs'' was
+patched that day to report a **RESCUED** verdict. This checker was never patched, so
+§18.5's unlocated bucket was an artefact of a narrower search than the rest of the
+site was using. It has now been patched the same way and re-run.
+<code>
+$ node scripts/pins_quotecheck.mjs
+      exact match:                 489  91.2%
+      whitespace-collapsed match:  0  0.0%
+      loose match:                 2  0.4%
+      spliced match (de-spliced):  3  0.6%
+      RESCUED from the PDF:        35  6.5%
+      NOT FOUND:                   7  1.3%
+   (papers with no pypdf rendering available: 0)
+</code>
+^ Figure ^ Was ^ Is ^ Why ^
+| pin codings | 1,000 | 1,000 | population unchanged — no coding was touched |
+| checked (non-empty quote) | 536 | 536 | unchanged |
+| exact | 489 | 489 | unchanged |
+| whitespace-collapsed | 0 | 0 | unchanged |
+| loose | 2 | 2 | unchanged |
+| de-spliced (6-word shingles, ≥80%) | 3 | 3 | unchanged |
+| rescued from the PDF | — | **35** | new verdict; all 35 were inside the old 42 |
+| located by no mode | **42** (7.8%) | **7** (1.3%) | 42 = 35 + 7 exactly; nothing else moved |
+**The hand-audited fabrication rate is unchanged at 0.2%, and this run is evidence
+for it rather than a replacement of it.** The 42 had already been read by three
+sub-agents, who called 42 of the 43 faithful de-splices of real sentences and found
+**one** real defect — a quote from the wrong part of {[singanamalla2022_respect]} —
+and **zero** fabrications. The fallback now locates **35 of those same 42** in a
+second rendering, by a script that has never seen the hand verdicts. It contradicts
+none of them. The 7 that remain are all inside the hand-audited set and all carry the
+verdict //faithful de-splice//. **1.3% is not a fabrication rate either**; like 7.8%
+and 9.1% before it, it is an upper bound produced by an instrument, and the rate is
+still the hand-read **0.2% (1 of 536)**.
+**The mutation test was re-run and its pin moved.** The guard recorded in §18.11 was
+//"an injected fabricated quote raises NOT FOUND from 42 to 43"//, which pinned the
+literal 42. Injecting a fabricated sentence into one coding of
+''2012/IEEE-SP/third-party-web-tracking-policy-and-technology'' against the patched
+script raises NOT FOUND **from 7 to 8**, drops ''exact'' from 489 to 488, and prints
+the fabricated sentence in full in the unlocated list. The pin in §18.11 has been
+updated to 7 → 8 and the old value recorded beside it, because a guard that still
+asserts a superseded number is a guard that will pass for the wrong reason.
+**The fallback was verified to be doing work, not failing silently.** ''pdfText()''
+swallows a subprocess failure and returns ''null'', which would produce ''0 rescued''
+and look like a clean result. The run therefore prints the number of papers for which
+no ''pypdf'' rendering could be produced — **0** — and two rescues were confirmed by
+hand against ''scripts/pdftext.py'' output directly: ''2019/IMC/visiblev8-…''
+(//"to visit the Alexa top 50k web domains"//, 0% of shingles in the stored text,
+verbatim in the PDF) and ''2021/WWW/towards-realistic-and-reproducibleweb-crawl-measurements''
+(//"Tranco list of top sites [18] (snapshot 77PX)"//, likewise). A ''0 rescued'' from
+this script would now be a measured zero.
+**Scope of this edit.** §18.5's mode table and the four sentences around it, the
+§18.11 mutation-test pin, and the two ''%%<file>%%'' blocks — the published script and
+its published output — both regenerated from the committed files rather than edited.
+''report_pins_audit.mjs'', ''pins_agreement.mjs'' and ''report_longitudinal.mjs'' were
+**not** re-run: no coding, no population and no agreement figure is touched by this
+change, and the quote check feeds none of them. Nothing on
+[[:design:longitudinal]] carries a quote-check figure, so no content page was saved.
+No ''%%{[citekey]}%%'' was added or moved.
+^ Item ^ Value ^
+| Date | 2026-09-22, unsupervised |
+| Command | ''node scripts/pins_quotecheck.mjs'' |
+| Artifacts | ''out/qc0922/pins_quotecheck-output.txt'' (the run published above), ''out/qc0922/pins_quotecheck_mutation.txt'' (the mutation run, NOT FOUND 8) |
+| Script change | ''scripts/pins_quotecheck.mjs'': imports ''pdfText'' from ''scripts/pdftext_cache.mjs''; a quote in none of the four stored texts is retried in all four modes against the ''pypdf'' rendering and reported as ''RESCUED''; the shingle test was factored into one ''shingleFrac()'' used by both passes; the summary gained a RESCUED line and a count of papers with no ''pypdf'' rendering |
+| Source fragment | ''prov/pins_section.txt'' carries the same §18.5 and §18.11 prose changes, so ''build_provenance_pins.py'' regenerates this section rather than reverting it. This appended refresh section lives on the page only. ''pages/provenance_design_longitudinal.LIVE.txt'' is a 2026-09-07 snapshot and was **not** refreshed — the builder's ''%%--live%%'' input is stale for reasons that predate this edit |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Reviewer finding accepted | The pass caught that this section's **heading** still read //"the 43 that did not locate"// — a count three generations stale (43 before the 2026-09-07 correction, 42 before today's fallback, 7 now). The prose below it narrates all three, so no reader was misled, but a heading is also an anchor and a TOC entry. It has been renamed to carry **no count at all**, because a heading that names a figure is a figure nobody re-runs. No page links to the old anchor — checked across all 188 pages before renaming |
+| Pages saved | this page only |