| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:design:archives [2026/09/04 17:29] – Citekey consolidation 2026-09-04: repoint duplicate bibliography keys to the kept key and add a dated amendment; no prose or figure changed. Authored by Claude karel.kubicek.claude | provenance:design:archives [2026/09/21 13:37] (current) – Quote-check refresh 2026-09-21: re-ran quote_check.mjs --temporal with the pypdf fallback; 21 below threshold -> 13 rescued + 8 below in both; all three hand-read cases are still in the 8. Authored by Claude karel.kubicek.claude |
|---|
| | Report script | ''scripts/report_archives.mjs'' | | | Report script | ''scripts/report_archives.mjs'' | |
| | Fold it depends on | ''scripts/arch_fold.mjs'' (archive-name families) | | | Fold it depends on | ''scripts/arch_fold.mjs'' (archive-name families) | |
| | Quote checker extension | ''scripts/quote_check.mjs'' gained a ''--temporal'' mode; no other flag reaches ''temporal'' tuples | | | Quote checker extension | ''scripts/quote_check.mjs'' gained a ''%%--temporal%%'' mode; no other flag reaches ''temporal'' tuples | |
| | Data | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues, 2010–2026 | | | Data | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues, 2010–2026 | |
| | Bibliography | 3 new entries (''lerner2017_rewriting'', ''singh2026_empire'', ''roth2022_security''), 1 entry corrected in place (''hantke2025web'': arXiv preprint → published USENIX Security 2025) | | | Bibliography | 3 new entries (''lerner2017_rewriting'', ''singh2026_empire'', ''roth2022_security''), 1 entry corrected in place (''hantke2025web'': arXiv preprint → published USENIX Security 2025) | |
| ===== 8. Quotes and figures spot-checked ===== | ===== 8. Quotes and figures spot-checked ===== |
| |
| ''node scripts/quote_check.mjs --temporal '...' '' over the archive-related temporal tuples: **118 quotes checked, 62 exact, 35 partial (≥60% of five-word windows), 21 below threshold, 0 with no full text on disk.** Three of the below-threshold cases were then read by hand against ''paper.cols.txt'': | ''node scripts/quote_check.mjs %%--temporal%% '...' '' over the archive-related temporal tuples. **Re-run 2026-09-21** with the PDF fallback — see //Quote-check refresh, 2026-09-21// at the foot of this page: **118 quotes checked, 62 exact, 35 partial (≥60% of five-word windows), 13 rescued from the PDF, 8 below threshold in both renderings, 0 with no full text on disk.** The old figure was **21** below threshold; 13 of those 21 are located in an independent ''pypdf'' rendering of the same ''paper.pdf'' and are a defect in the stored text rather than in the extraction. ''exact'' (62) and ''partial'' (35) did not move. |
| | |
| | Three of the then-21 were read by hand against ''paper.cols.txt''. **All three are still below threshold in both renderings**, so they are in the 8 and the hand read below remains the only evidence for them — the fallback rescues 13 other quotes and none of these: |
| |
| ^ Paper ^ Quote ^ Verdict ^ | ^ Paper ^ Quote ^ Verdict ^ |
| | NDSS 2025, //Duumviri// | "using Wayback Machine [28]." | **Real**, line 696. Fails only because a five-word-window test on a five-word quote is degenerate. | | | NDSS 2025, //Duumviri// | "using Wayback Machine [28]." | **Real**, line 696. Fails only because a five-word-window test on a five-word quote is degenerate. | |
| | CCS 2016, //Lurking Malice in the Cloud// | "Searching the Common Crawl dataset, collected in February 2015" | **Real.** The source reads "Common Crawl [?] dataset, collected in February 2015" — the extractor dropped an unresolved citation marker. | | | CCS 2016, //Lurking Malice in the Cloud// | "Searching the Common Crawl dataset, collected in February 2015" | **Real.** The source reads "Common Crawl [?] dataset, collected in February 2015" — the extractor dropped an unresolved citation marker. | |
| | |
| | **Why none of the three moved is the point, not a disappointment.** Two of them (//Duumviri// at 0%, //Through the Lens// at 0%) are short quotes, where a five-word-window test is degenerate regardless of how the PDF renders; the third is a dropped citation marker, which lives in the extraction's own string and no rendering can repair. The fallback fixes de-columning, and de-columning was not what broke these three. The remaining five of the 8 have **not** been read: ''WWW/2020/an-empirical-study-of-the-use-of-integrity-verification-mechanisms…'' (58%), ''PETS/2021/privacy-preference-signals…'' (56%), ''PETS/2021/the-cname-of-the-game…'' (43% and 29%, two tuples in the same paper) and ''CCS/2023/black-ostrich…'' (33%). |
| |
| Figures quoted on the content page from individual papers were read in their own full text, not taken from the extraction: | Figures quoted on the content page from individual papers were read in their own full text, not taken from the extraction: |
| | Accepted | 29 | | | Accepted | 29 | |
| | Rejected | 1 — repairing four pre-existing duplicate key pairs in the shared bibliography. Out of scope for this item and riskier than leaving them: both keys in each pair resolve, and deleting one could break a page we are not editing. Recorded in §10 instead. | | | Rejected | 1 — repairing four pre-existing duplicate key pairs in the shared bibliography. Out of scope for this item and riskier than leaving them: both keys in each pair resolve, and deleting one could break a page we are not editing. Recorded in §10 instead. | |
| | Code written | ''scripts/arch_fold.mjs'' (new), ''scripts/report_archives.mjs'' (new), ''scripts/quote_check.mjs'' (''--temporal'' mode added) | | | Code written | ''scripts/arch_fold.mjs'' (new), ''scripts/report_archives.mjs'' (new), ''scripts/quote_check.mjs'' (''%%--temporal%%'' mode added) | |
| | Pages saved | [[design:archives]] (extended), [[provenance:design:archives]] (new), [[literature:bibliography]] (3 entries added, 1 corrected in place) | | | Pages saved | [[design:archives]] (extended), [[provenance:design:archives]] (new), [[literature:bibliography]] (3 entries added, 1 corrected in place) | |
| | Discussion block | None, following the convention already set by ''provenance:design:sampling'' and ''provenance:design:crawling_location'' — comments belong on the content page. The citekeys here therefore render as markers with no reference list, which is the convention rather than an omission. | | | Discussion block | None, following the convention already set by ''provenance:design:sampling'' and ''provenance:design:crawling_location'' — comments belong on the content page. The citekeys here therefore render as markers with no reference list, which is the convention rather than an omission. | |
| |
| [[design:archives|← back to the content page]] · [[literature:corpus|corpus-level provenance]] | [[design:archives|← back to the content page]] · [[literature:corpus|corpus-level provenance]] |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| | |
| | ===== Quote-check refresh, 2026-09-21 ===== |
| | |
| | The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs'' was patched the same day to re-check everything below threshold against that second rendering and report a fourth verdict, **RESCUED**. §8's figure predates the patch. Re-run with the same ''%%--temporal%%'' pattern §3 records, unedited first line: |
| | |
| | <code> |
| | $ node scripts/quote_check.mjs --temporal 'wayback|internet archive|archive\.org|web-archive|common ?crawl|http ?archive' --show 30 |
| | 118 quotes checked: 62 exact, 35 partial (>=60% of 5-word windows), 13 rescued from the PDF, 8 below threshold in both renderings, 0 with no full text on disk. |
| | </code> |
| | |
| | The command is the code block above — the pattern is a regex alternation, and a DokuWiki table cell cannot hold a ''%%|%%''. |
| | |
| | ^ Figure ^ Was ^ Is ^ Why ^ |
| | | quotes checked | 118 | 118 | population unchanged — the corpus has not moved | |
| | | exact | 62 | 62 | unchanged | |
| | | partial (≥60% of 5-word windows) | 35 | 35 | unchanged | |
| | | rescued from the PDF | — | **13** | new verdict; these were inside the old 21 | |
| | | below threshold | **21** | **8** (in both renderings) | 21 = 13 + 8 exactly; nothing else moved | |
| | | below-threshold rate | 17.8% | **6.8%** | 8 of 118 | |
| | |
| | **This is the page where the fallback and the hand pass do not overlap at all.** All three quotes §8 read by hand are still below threshold in both renderings, because two are too short for the window test and the third is a dropped citation marker. §8 now says so and names the five of the 8 that nobody has read. |
| | |
| | **Scope of this edit.** §8's bulk figure and the paragraph under its verdict table. ''arch_fold.mjs'' and ''report_archives.mjs'' were **not** re-run in this pass; the 122/85-paper archive counts, the fold residue, the per-paper figure table in §8 (read from the papers, not from the extraction) and every citation stand as published. [[:design:archives]] carries no quote-check figure, so it was not edited. |
| | |
| | ^ Item ^ Value ^ |
| | | Date | 2026-09-21, unsupervised | |
| | | Artifact | ''out/qc0921/temporal_archives.txt'' (full run, 13 RESCUED rows and 8 below-threshold rows listed) | |
| | | Script changes | none — ''quote_check.mjs'' was already patched on 2026-09-04 | |
| | | Reviewers | one ''sonnet'' figures-vs-script pass | |
| | | Pages saved | this page only | |
| |
