Differences

This shows you the differences between two versions of the page.

--- provenance:design:archives [2026/09/17 07:23] – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude
+++ provenance:design:archives [2026/09/21 13:37] (current) – Quote-check refresh 2026-09-21: re-ran quote_check.mjs --temporal with the pypdf fallback; 21 below threshold -> 13 rescued + 8 below in both; all three hand-read cases are still in the 8. Authored by Claude karel.kubicek.claude
@@ Line 133: / Line 133: @@
 ===== 8. Quotes and figures spot-checked =====
-''node scripts/quote_check.mjs --temporal '...' '' over the archive-related temporal tuples: **118 quotes checked, 62 exact, 35 partial (≥60% of five-word windows), 21 below threshold, 0 with no full text on disk.** Three of the below-threshold cases were then read by hand against ''paper.cols.txt'':
+''node scripts/quote_check.mjs %%--temporal%% '...' '' over the archive-related temporal tuples. **Re-run 2026-09-21** with the PDF fallback — see //Quote-check refresh, 2026-09-21// at the foot of this page: **118 quotes checked, 62 exact, 35 partial (≥60% of five-word windows), 13 rescued from the PDF, 8 below threshold in both renderings, 0 with no full text on disk.** The old figure was **21** below threshold; 13 of those 21 are located in an independent ''pypdf'' rendering of the same ''paper.pdf'' and are a defect in the stored text rather than in the extraction. ''exact'' (62) and ''partial'' (35) did not move.
+Three of the then-21 were read by hand against ''paper.cols.txt''. **All three are still below threshold in both renderings**, so they are in the 8 and the hand read below remains the only evidence for them — the fallback rescues 13 other quotes and none of these:
 ^ Paper ^ Quote ^ Verdict ^
@@ Line 139: / Line 141: @@
 | NDSS 2025, //Duumviri// | "using Wayback Machine [28]." | **Real**, line 696. Fails only because a five-word-window test on a five-word quote is degenerate. |
 | CCS 2016, //Lurking Malice in the Cloud// | "Searching the Common Crawl dataset, collected in February 2015" | **Real.** The source reads "Common Crawl [?] dataset, collected in February 2015" — the extractor dropped an unresolved citation marker. |
+**Why none of the three moved is the point, not a disappointment.** Two of them (//Duumviri// at 0%, //Through the Lens// at 0%) are short quotes, where a five-word-window test is degenerate regardless of how the PDF renders; the third is a dropped citation marker, which lives in the extraction's own string and no rendering can repair. The fallback fixes de-columning, and de-columning was not what broke these three. The remaining five of the 8 have **not** been read: ''WWW/2020/an-empirical-study-of-the-use-of-integrity-verification-mechanisms…'' (58%), ''PETS/2021/privacy-preference-signals…'' (56%), ''PETS/2021/the-cname-of-the-game…'' (43% and 29%, two tuples in the same paper) and ''CCS/2023/black-ostrich…'' (33%).
 Figures quoted on the content page from individual papers were read in their own full text, not taken from the extraction:
@@ Line 587: / Line 591: @@
 Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Quote-check refresh, 2026-09-21 =====
+The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs'' was patched the same day to re-check everything below threshold against that second rendering and report a fourth verdict, **RESCUED**. §8's figure predates the patch. Re-run with the same ''%%--temporal%%'' pattern §3 records, unedited first line:
+<code>
+$ node scripts/quote_check.mjs --temporal 'wayback|internet archive|archive\.org|web-archive|common ?crawl|http ?archive' --show 30
+quotes checked: 62 exact, 35 partial (>=60% of 5-word windows), 13 rescued from the PDF, 8 below threshold in both renderings, 0 with no full text on disk.
+</code>
+The command is the code block above — the pattern is a regex alternation, and a DokuWiki table cell cannot hold a ''%%|%%''.
+^ Figure ^ Was ^ Is ^ Why ^
+| quotes checked | 118 | 118 | population unchanged — the corpus has not moved |
+| exact | 62 | 62 | unchanged |
+| partial (≥60% of 5-word windows) | 35 | 35 | unchanged |
+| rescued from the PDF | — | **13** | new verdict; these were inside the old 21 |
+| below threshold | **21** | **8** (in both renderings) | 21 = 13 + 8 exactly; nothing else moved |
+| below-threshold rate | 17.8% | **6.8%** | 8 of 118 |
+**This is the page where the fallback and the hand pass do not overlap at all.** All three quotes §8 read by hand are still below threshold in both renderings, because two are too short for the window test and the third is a dropped citation marker. §8 now says so and names the five of the 8 that nobody has read.
+**Scope of this edit.** §8's bulk figure and the paragraph under its verdict table. ''arch_fold.mjs'' and ''report_archives.mjs'' were **not** re-run in this pass; the 122/85-paper archive counts, the fold residue, the per-paper figure table in §8 (read from the papers, not from the extraction) and every citation stand as published. [[:design:archives]] carries no quote-check figure, so it was not edited.
+^ Item ^ Value ^
+| Date | 2026-09-21, unsupervised |
+| Artifact | ''out/qc0921/temporal_archives.txt'' (full run, 13 RESCUED rows and 8 below-threshold rows listed) |
+| Script changes | none — ''quote_check.mjs'' was already patched on 2026-09-04 |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Pages saved | this page only |