Differences

This shows you the differences between two versions of the page.

--- provenance:programming:traffic_files [2026/09/17 07:24] – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude
+++ provenance:programming:traffic_files [2026/09/21 13:36] (current) – Quote-check refresh 2026-09-21: re-ran quote_check.mjs --tools mitmproxy/tcpdump/wireshark/tshark with the pypdf fallback; 41 below threshold -> 27 rescued + 14 below in both; new provenance section. Authored by Claude karel.kubicek.claude
@@ Line 792: / Line 792: @@
 ==== 8.2 Evidence quotes behind the capture-tool figures ====
-''node scripts/quote_check.mjs --tools 'mitmproxy|tcpdump|wireshark|tshark''' over the whole corpus:
+''node scripts/quote_check.mjs %%--tools%% 'mitmproxy|tcpdump|wireshark|tshark''' over the whole corpus. **Re-run 2026-09-21** with the PDF fallback — see //Quote-check refresh, 2026-09-21// at the foot of this page:
 <code>
-quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 41 below threshold, 0 with no full text on disk.
+quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 27 rescued from the PDF, 14 below threshold in both renderings, 0 with no full text on disk.
 </code>
-Below-threshold is not "unsupported". The lowest-scoring one is the single most-quoted sentence on the content page, Englehardt & Narayanan's, at **20%** — and it is verbatim in the paper. The ''.cols'' rendering reads //"After examining several Python HTTP  proxies, we chose to use Mitmproxy6 to record all HTTP Re-quest and Response headers"//: a footnote marker and a hyphenation split by the two-column repair. Checked by hand; the extraction is faithful. Two others were spot-checked the same way (''respect-the-origin'', ''comparing-the-effects-of-dns-dot-and-doh'') and both are present verbatim.
+**The old figure was 41 below threshold.** 27 of the 41 are located in an independent ''pypdf'' rendering of the same ''paper.pdf'' and are a defect in the stored text, not in the extraction; **14** are below threshold in both, i.e. 3.9% of 357. ''exact'' (229) and ''partial'' (87) did not move.
+Below-threshold is not "unsupported", and the example this section has always given is now the clearest case of why. The lowest-scoring quote under the old single-rendering test was the single most-quoted sentence on the content page, Englehardt & Narayanan's, at **20%** — and it is verbatim in the paper. The ''.cols'' rendering reads //"After examining several Python HTTP  proxies, we chose to use Mitmproxy6 to record all HTTP Re-quest and Response headers"//: a footnote marker and a hyphenation split by the two-column repair. It was checked by hand and the extraction is faithful; **the fallback now reaches it on its own, scoring it 73% against the PDF**, so it is one of the 27 rather than a failure. Two others were spot-checked the same way (''respect-the-origin'', ''comparing-the-effects-of-dns-dot-and-doh'') and both are present verbatim; neither is below threshold under the current script. The lowest-scoring of the 14 that remain is ''PETS/2026/clicking-into-exposure…'' at 25%, and none of the 14 has been read by hand.
 ==== 8.3 External facts ====
@@ Line 939: / Line 941: @@
 Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Quote-check refresh, 2026-09-21 =====
+The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs'' was patched the same day to re-check everything below threshold against that second rendering and report a fourth verdict, **RESCUED**. §8.2's figure predates the patch. Re-run, unedited first line:
+<code>
+$ node scripts/quote_check.mjs --tools 'mitmproxy|tcpdump|wireshark|tshark' --show 400
+quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 27 rescued from the PDF, 14 below threshold in both renderings, 0 with no full text on disk.
+</code>
+The command is the code block above — it contains a regex alternation, and a DokuWiki table cell cannot hold a ''%%|%%''.
+^ Figure ^ Was ^ Is ^ Why ^
+| quotes checked | 357 | 357 | population unchanged — the corpus has not moved |
+| exact | 229 | 229 | unchanged |
+| partial (≥60% of 5-word windows) | 87 | 87 | unchanged |
+| rescued from the PDF | — | **27** | new verdict; these were inside the old 41 |
+| below threshold | **41** | **14** (in both renderings) | 41 = 27 + 14 exactly; nothing else moved |
+| below-threshold rate | 11.5% | **3.9%** | 14 of 357 |
+**The page's own worked example is now a positive control for the fallback.** Englehardt & Narayanan's mitmproxy sentence was the lowest-scoring quote in the whole pass (20%), was read by hand, and was found verbatim. The PDF rendering scores it 73% with nobody reading anything. That is the mechanism the [[:provenance:literature:corpus]] audit measured, hitting the one quote on this page whose correctness was already independently known.
+**Scope of this edit.** §8.2 only. ''report_traffic_files.mjs'' and the HAR full-text sweep were **not** re-run in this pass; the 10-paper HAR tool count, the 95 hand-classified sweep hits, the tool fold, §8.3's external checks and every citation stand as published. [[:programming:traffic_files]] carries no quote-check figure — it points at this page for the spot-checked quotes — so it was not edited.
+^ Item ^ Value ^
+| Date | 2026-09-21, unsupervised |
+| Artifact | ''out/qc0921/tools_traffic.txt'' (full run, 27 RESCUED rows and 14 below-threshold rows listed) |
+| Script changes | none — ''quote_check.mjs'' was already patched on 2026-09-04 |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Pages saved | this page only |