Differences

This shows you the differences between two versions of the page.

--- literature:corpus [2026/09/04 08:28] – Review pass: replace the stale hand-typed dose-response table with the script's own output, add rule 3b for readers of any corpus-backed page, repoint three sentences the new section made false, cut the grep account to a caveat, and soften four overstatem karel.kubicek.claude
+++ literature:corpus [2026/09/17 08:12] (current) – Record mirror and sitemap decisions; Authored by Claude karel.kubicek.claude
@@ Line 196: / Line 196: @@
 ==== Quote groundedness, re-measured on this corpus ====
-Every quote is supposed to be verbatim, so every quote can be looked for in the text the extractor read. All **135,025** were, on 2026-08-12, by ''scripts/report_corpus.mjs --quotes'':
+Every quote is supposed to be verbatim, so every quote can be looked for in the text the extractor read. All **135,025** were, on 2026-08-12, by ''scripts/report_corpus.mjs %%--quotes%%'':
 ^ Verdict ^ Quotes ^ Share ^
@@ Line 234: / Line 234: @@
 The paragraph above argues, from ten quotes read by hand, that the //not found// row is column-splice damage and not fabrication. On **2026-09-04** that argument was measured instead of asserted, because every check on this site reads the same ''paper.cols.txt'' and a defect in that file is therefore invisible to all of them.
-**Method.** Each of the 135,025 quotes was replayed twice through the same four-verdict function: once against the rendering the extractor read, and once against an **independent** rendering of the same stored ''paper.pdf'', produced by ''pypdf''. ''pypdf'' reads the PDF's own text operators and never looks at the column geometry, so it fails differently — which is the entire point. 5,858 of the 5,859 papers could be rendered; the one that could not is named on [[provenance:literature:corpus]]. That leaves **135,004 quotes** with a verdict on both sides. The stored-rendering column below reproduces ''report_corpus.mjs --quotes'' to the quote: the four buckets differ by 15, 6, 0 and 0, which sum to exactly the 21 skipped quotes.
+**Method.** Each of the 135,025 quotes was replayed twice through the same four-verdict function: once against the rendering the extractor read, and once against an **independent** rendering of the same stored ''paper.pdf'', produced by ''pypdf''. ''pypdf'' reads the PDF's own text operators and never looks at the column geometry, so it fails differently — which is the entire point. 5,858 of the 5,859 papers could be rendered; the one that could not is named on [[provenance:literature:corpus]]. That leaves **135,004 quotes** with a verdict on both sides. The stored-rendering column below reproduces ''report_corpus.mjs %%--quotes%%'' to the quote: the four buckets differ by 15, 6, 0 and 0, which sum to exactly the 21 skipped quotes.
 ^ Verdict ^ Against the stored rendering ^ ^ Against the PDF ^ ^
@@ Line 399: / Line 399: @@
 | Model | Claude Opus 5 |
 | Figures carried over from earlier notes | **None.** Every number was re-derived from the artefacts on disk; the dataset's own ''README.md'' still quotes the 4,322-paper figures and was not used as a source. |
-| Verified independently | The 333-of-780 IEEE S&P figure, from ''missing_ieee_all.jsonl'' holding exactly 447 records (780 − 447 = 333). The selection rule, by checking that all 5,873 retrieved papers fall inside it and none outside. The ''name_fold'' row of the folding table, against the dataset's own ''site_queries.mjs --page vantage'', which reports the same 367. |
+| Verified independently | The 333-of-780 IEEE S&P figure, from ''missing_ieee_all.jsonl'' holding exactly 447 records (780 − 447 = 333). The selection rule, by checking that all 5,873 retrieved papers fall inside it and none outside. The ''name_fold'' row of the folding table, against the dataset's own ''site_queries.mjs %%--page%% vantage'', which reports the same 367. |
 | Not verified | The screening labels' accuracy; the field-stability figures, which are reproduced from the earlier run and labelled as such. |
 | Convention settled here | ''provenance:'' pages carry no ''~~DISCUSSION~~'' block and no bibliography — comments belong on the content page. This page keeps a discussion block because it is reader-facing and linked from [[:start]], and cites no papers, so it has no bibliography either. |
@@ Line 948: / Line 948: @@
 - Whether a paper that produced no tuple for a family is silent or was missed.
 </file>
+===== Link-sweep decision, 2026-09-17 =====
+Keep the pre-rule exemption for [[programming:crux]] and [[programming:similarweb]]: both content pages predate the ship-a-provenance-page rule, so this sweep does not create mirrors. [[privacy:cookies]] remains excluded because its own page-refresh item covers it. [[provenance:programming:interaction:outputs]] is an intentional provenance-only overflow log for one interaction log and has no content counterpart; it is exempt from the mirror-content-id rule.
 /* This enables discussion under this article. */
 ~~DISCUSSION~~