Differences

This shows you the differences between two versions of the page.

--- provenance:programming:traffic_files [2026/08/14 03:38] – Add the review log (10 accepted findings across three passes), correct the fold's proxy families and every figure downstream of them, refresh the embedded script output. Authored by Claude. karel.kubicek.claude
+++ provenance:programming:traffic_files [2026/09/21 13:36] (current) – Quote-check refresh 2026-09-21: re-ran quote_check.mjs --tools mitmproxy/tcpdump/wireshark/tshark with the pypdf fallback; 41 below threshold -> 27 rescued + 14 below in both; new provenance section. Authored by Claude karel.kubicek.claude
@@ Line 217: / Line 217: @@
 ===== 6. Unedited report output =====
-The full output of ''node scripts/report_traffic_files.mjs'' is below, unedited. ''--list'' output (the 514-paper population with its families) is not reproduced here for length; regenerate it with the command in §3.
+The full output of ''node scripts/report_traffic_files.mjs'' is below, unedited. ''%%--list%%'' output (the 514-paper population with its families) is not reproduced here for length; regenerate it with the command in §3.
 <file text report_traffic_files.txt>
@@ Line 487: / Line 487: @@
       "recorded a trace of all HTTP requests initiated by the web page (HAR)"; five of its features are HAR features
   PETS/2019/4-years-of-eu-cookie-law-results-and-lessons-learned
-      "they dump to file the HTTP Archive (HAR) [55], a JSON-formatted … We look at all HTTP responses with Set-Cookie header in the HAR file." Also analyses the httparchive.org corpus — counted in both.
+      "they dump to file the HTTP Archive (HAR) [55], a JSON-formatted … We look at all HTTP responses with Set-Cookie header in the HAR file." It ALSO analyses the httparchive.org corpus, so it is the one paper that is both instrument and dataset. This map is single-label and it is filed under instrument, so the dataset row of 14 excludes it — the content page says so rather than implying 15. (The earlier comment here claimed it was "counted in both", which the arithmetic disproves: the five verdicts sum to exactly 95.)
   PETS/2019/oblivious-dns-practical-privacy-for-dns-queries
       "Chrome webdriver and record HAR files for each browsing session"
@@ Line 792: / Line 792: @@
 ==== 8.2 Evidence quotes behind the capture-tool figures ====
-''node scripts/quote_check.mjs --tools 'mitmproxy|tcpdump|wireshark|tshark''' over the whole corpus:
+''node scripts/quote_check.mjs %%--tools%% 'mitmproxy|tcpdump|wireshark|tshark''' over the whole corpus. **Re-run 2026-09-21** with the PDF fallback — see //Quote-check refresh, 2026-09-21// at the foot of this page:
 <code>
-quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 41 below threshold, 0 with no full text on disk.
+quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 27 rescued from the PDF, 14 below threshold in both renderings, 0 with no full text on disk.
 </code>
-Below-threshold is not "unsupported". The lowest-scoring one is the single most-quoted sentence on the content page, Englehardt & Narayanan's, at **20%** — and it is verbatim in the paper. The ''.cols'' rendering reads //"After examining several Python HTTP  proxies, we chose to use Mitmproxy6 to record all HTTP Re-quest and Response headers"//: a footnote marker and a hyphenation split by the two-column repair. Checked by hand; the extraction is faithful. Two others were spot-checked the same way (''respect-the-origin'', ''comparing-the-effects-of-dns-dot-and-doh'') and both are present verbatim.
+**The old figure was 41 below threshold.** 27 of the 41 are located in an independent ''pypdf'' rendering of the same ''paper.pdf'' and are a defect in the stored text, not in the extraction; **14** are below threshold in both, i.e. 3.9% of 357. ''exact'' (229) and ''partial'' (87) did not move.
+Below-threshold is not "unsupported", and the example this section has always given is now the clearest case of why. The lowest-scoring quote under the old single-rendering test was the single most-quoted sentence on the content page, Englehardt & Narayanan's, at **20%** — and it is verbatim in the paper. The ''.cols'' rendering reads //"After examining several Python HTTP  proxies, we chose to use Mitmproxy6 to record all HTTP Re-quest and Response headers"//: a footnote marker and a hyphenation split by the two-column repair. It was checked by hand and the extraction is faithful; **the fallback now reaches it on its own, scoring it 73% against the PDF**, so it is one of the 27 rather than a failure. Two others were spot-checked the same way (''respect-the-origin'', ''comparing-the-effects-of-dns-dot-and-doh'') and both are present verbatim; neither is below threshold under the current script. The lowest-scoring of the 14 that remain is ''PETS/2026/clicking-into-exposure…'' at 25%, and none of the 14 has been read by hand.
 ==== 8.3 External facts ====
@@ Line 811: / Line 813: @@
 | WARC | ''iipc.github.io/warc-specifications/'' | WARC 1.0 and 1.1 listed as standard; ISO 28500:2017 is the published edition (WebSearch only — ''iso.org'' answered 403). |
 | mitmproxy current | GitHub releases API | v12.2.3, 2026-05-12. **The runs on this page used 11.0.2** and the page says so. |
-| ''--set hardump'' still exists | ''docs.mitmproxy.org/stable/concepts/options/'' | Yes: "Save a HAR file with all flows on exit." |
+| ''%%--set%% hardump'' still exists | ''docs.mitmproxy.org/stable/concepts/options/'' | Yes: "Save a HAR file with all flows on exit." |
 | Wireshark / tcpdump / libpcap | wireshark.org, GitHub tags | 4.6.8 / tcpdump-4.99.6 / libpcap-1.10.6 |
 | chrome-har-capturer abandoned | npm registry + GitHub | **No** — 0.14.4 published 2026-01-27, repo pushed 2026-08-12. Actively maintained. |
@@ Line 824: / Line 826: @@
 | Okta sanitiser guidance | ''support.okta.com/help/s/article/sanitizing-http-traces'' | "There is no attempt to remove user information as part of the sanitization process" |
 | A maintained HAR sanitiser exists | GitHub API | **No.** ''google/har-sanitizer'' and ''cloudflare/har-sanitizer'' are both ''archived: true'' as of 2026-08-14 |
-| Chromium writes a TLS key log | run locally | Yes — ''--ssl-key-log-file'' produced a 5-line NSS key log for one ''https://example.com/'' load |
+| Chromium writes a TLS key log | run locally | Yes — ''%%--ssl-key-log-file%%'' produced a 5-line NSS key log for one ''https://example.com/'' load |
 ==== 8.4 Sources rejected ====
@@ Line 849: / Line 851: @@
 | 2 | The sweep found **95** papers; a ''grep -rlP'' with the same regex had found 94. ''WWW/2022/reproducibility-and-replicability-of-web-measurement-studies'' was missing from the shell result. | Cross-checking the two sweeps. The Node sweep is authoritative and is the one in the report. |
 | 3 | "By year bucket" and "By venue" printed each other's values, because the key-order fix was applied to the map but not to the two ''split('/')'' index reads below it. | Reading the output: it said "By venue: 2025 5" followed by "2021 4". |
-| 4 | Adding ''adafruit'' to the radio family moved one paper out of packet capture. Seven figures on the page went stale in one edit: 288→287, 512→511, 164→165, 42.4→42.3, 62.3→62.2, 36.9→37.0, 73→72 — and 659→662, 129→130, 12.4→12.3 from an unrelated fold edit. The page had already been written with all of them. | ''check_page_numbers.mjs'', run whole-page with ''--code''. It also caught an eighth, "164 excluded papers", sitting in the limitations section outside any table. |
+| 4 | Adding ''adafruit'' to the radio family moved one paper out of packet capture. Seven figures on the page went stale in one edit: 288→287, 512→511, 164→165, 42.4→42.3, 62.3→62.2, 36.9→37.0, 73→72 — and 659→662, 129→130, 12.4→12.3 from an unrelated fold edit. The page had already been written with all of them. | ''check_page_numbers.mjs'', run whole-page with ''%%--code%%''. It also caught an eighth, "164 excluded papers", sitting in the limitations section outside any table. |
 | 5 | The ''%%<code>%%'' block quoting the mitmproxy comparison held byte counts from an earlier run (20472/261704/232414 against a later run's 20453/261692/232420). | A line-by-line equality check of every ''%%<code>%%'' block against the run output it claims to be, both directions. A substring check would have passed. |
 | 6 | The three-line output beside the published ''%%<file javascript>%%'' snippet had never been produced by that snippet. | Extracting the block from the page source, writing it to ''sandbox/har_vs_cdp_core.mjs'' and running it. It does produce exactly those three lines. |
@@ Line 862: / Line 864: @@
 Three focused passes and the author's own read, all against the frozen snapshot **rev 1786677596** (''pages/programming_traffic_files.txt'' md5 ''d165a98f56316b0d2c618128bc9d801d''). Each reviewer was told its context might not be exhaustive and was handed the page, the provenance, the scripts and their committed output. Nothing was edited while they ran.
-==== 10.1 Figures vs script (''sonnet'') ====
+==== 10.1 Figures vs script (sonnet) ====
 ^ # ^ Finding ^ Verdict ^
@@ Line 870: / Line 872: @@
 | F4 | ''report_traffic_files.mjs''’s closing block hardcoded "mitmproxy 12.1.2 / Wireshark 4.6.2 / Chrome 127" as current while the page correctly said 12.2.3 / 4.6.8 / 130 — and the stale copy was reproduced verbatim in §6 of this page. | **Accepted.** That block is now a pointer to ''verify_traffic_figures.mjs'' and ''external_checks_traffic_files.sh'' rather than a second, drifting copy. **A provenance page that embeds a script's output inherits that script's stale numbers.** |
-The same pass re-ran the report (plain and ''--wiki'', byte-identical to the committed output), re-ran the figure verifier, re-ran both experiments three times, and confirmed papers-not-tuples, sentinel handling, ''usedOrMentioned'' filtering, both-ended year buckets and every denominator in "Use in Publications". No other issue found.
+The same pass re-ran the report (plain and ''%%--wiki%%'', byte-identical to the committed output), re-ran the figure verifier, re-ran both experiments three times, and confirmed papers-not-tuples, sentinel handling, ''usedOrMentioned'' filtering, both-ended year buckets and every denominator in "Use in Publications". No other issue found.
-==== 10.2 Citations and quotes (''sonnet'') ====
+==== 10.2 Citations and quotes (sonnet) ====
 ^ # ^ Finding ^ Verdict ^
@@ Line 880: / Line 882: @@
 All 16 ''%%{[key]}%%'' citations resolve; 279 keys in the bibliography, 279 unique; all 14 new entries match Crossref on authors, title, year, DOI, and the three hand-written PoPETs entries match on volume, issue and pages. Every quotation verified verbatim after whitespace normalisation. Every claim-to-paper attribution checked, including all eleven Hantke et al. figures for numerator/denominator/percentage coming from the same sentence. The "six papers cite a W3C HAR URL" figure was independently reproduced (two ''dvcs.w3.org'', four ''w3c.github.io''). No missing citations, no footnote/citation mismatches.
-==== 10.3 External currency (''sonnet'') ====
+==== 10.3 External currency (sonnet) ====
 ^ # ^ Finding ^ Verdict ^
@@ Line 889: / Line 891: @@
 | E5 | Two small imprecisions: BrowserMob's tag year (same as C1), and this page's "''har-spec'' has not moved since 2019" against a last commit the reviewer dates to April 2018. | **Accepted with a caveat.** The two reviewers give different years and GitHub rate-limited the re-check, so the page now says "dormant for years" — which both agree on — rather than a year neither could confirm together. |
-The same pass independently re-verified the HAR abandonment banner, the two ''dvcs.w3.org''/''w3c.github.io'' copies, ''har.fyi''’s subject, both IETF drafts, WARC/ISO 28500, seven tool versions, every maintenance-status call, the Chrome 130 wording, all ten Firefox prefs read from ''firefox.js'', ''--set hardump'', the QUIC/custom-CA limitation, ''SSLKEYLOGFILE'' for both browsers, and ~20 URLs for liveness. One 403 (''iso.org''), already disclosed in §8.3. Nothing else superseded.
+The same pass independently re-verified the HAR abandonment banner, the two ''dvcs.w3.org''/''w3c.github.io'' copies, ''har.fyi''’s subject, both IETF drafts, WARC/ISO 28500, seven tool versions, every maintenance-status call, the Chrome 130 wording, all ten Firefox prefs read from ''firefox.js'', ''%%--set%% hardump'', the QUIC/custom-CA limitation, ''SSLKEYLOGFILE'' for both browsers, and ~20 URLs for liveness. One 403 (''iso.org''), already disclosed in §8.3. Nothing else superseded.
 ==== 10.4 The author's own read ====
@@ Line 897: / Line 899: @@
 ==== 10.5 What the review layer cost and returned ====
-Ten accepted findings, zero rejected. The three passes disagreed usefully: only the figures pass could have found F3 and F4 (they need the scripts re-run), only the citation pass could have found C2 (it needs the cited paper read), and only the currency pass could have found E1 (it needs today's release notes). **A fourth, generic pass (''fable'', no checklist) was launched after the other three had been applied, and had not returned when this log was first published.** It is reading the page and this provenance page against the "no MDN, no textbook" test, for overstated claims, unmeasured framing sentences, internal contradictions and scope overlap with [[design:archives]] and [[programming:crawler]]. Its findings will be appended here as §10.6 rather than folded silently into the sections above, so that what the first three passes missed stays visible.
+Ten accepted findings, zero rejected. The three passes disagreed usefully: only the figures pass could have found F3 and F4 (they need the scripts re-run), only the citation pass could have found C2 (it needs the cited paper read), and only the currency pass could have found E1 (it needs today's release notes). A fourth, **generic** pass (''fable'', no checklist) was run after the other three had been applied — see §10.6. An earlier version of this section justified //not// running it on the grounds that the three focused passes had covered every category one would. That justification was wrong, and §10.6 is the evidence: the generic pass returned fifteen findings, including two the focused passes were structurally unable to catch.
+==== 10.6 Generic (fable, no checklist) ====
+Fifteen findings, **all accepted**. Two of them are the reason this pass exists, because no focused pass could have found them:
+^ # ^ Finding ^ Verdict ^
+| G1 | The opening ''%%<WRAP important>%%'' box said "Measured on a single page load of a local fixture" and then made the ''serverIPAddress'' claim — which §7.2's own output says the fixture **cannot** show, because both sides are loopback. The claim came from the separate remote re-run, which "What We Ran" did not even mention. | **Accepted.** This is the documented failure mode exactly: the body text carried the qualifier and the prominent box dropped it. Box reworded, the bullet now names the remote run, and "What We Ran" now says the mitmproxy script was run twice. |
+| G2 | "14 papers analyse the dataset and 32 use the format, and only one paper does both" contradicts the data structure: ''HAR_VERDICT'' is **single-label**, the five verdicts sum to exactly 95, and the fold's own comment claimed the paper was "counted in both". So the paper the box names as a dataset user is excluded from the 14. | **Accepted**, and the sharpest finding of any pass. Neither a figures check (the counts are all correct) nor a citation check (the paper is right) could see it — it needs the tip box read against the fold's data model. Prose corrected to state the single-label rule explicitly; the false comment in ''traffic_fold.mjs'' corrected. |
+| G3 | "in our corpus most papers give none of them" heads a six-item checklist of which exactly **one** was measured. | **Accepted.** An unmeasured claim hiding in a section lead. Replaced with the one figure we have. |
+| G4 | The limitations bullet "Nothing on this page claims a HAR loses //x%// of anything on real sites" contradicts §Replay, which quotes 13.9% and <0.3% over 8,544 real origins. | **Accepted.** The bullet meant "nothing //from our fixture//"; it now says that, and credits the percentages to Hantke et al. rather than to us. |
+| G5 | The "Volume per 1,000 sites" row traces to nothing — every other number on the page traces to a script or a source. | **Accepted.** Kept, but footnoted as an order-of-magnitude planning estimate with the reason it is not a measurement. |
+| G6 | The mitmproxy comparison ran on 11.0.2 while the surrounding section recommends 12.2.3, and the page argues elsewhere that mitmproxy's major versions matter — but the disclosure sat only in "What We Ran". | **Accepted.** Version now stated beside the table, with the page's own argument turned on itself. |
+| G7 | The published snippet contains ''.catch(() => {})'' — a silent error swallow, in code a newcomer will copy into a crawler, on a page whose whole argument is that inconsistently-recorded failures are the trap. | **Accepted**, and it is a fair hit: the page published the anti-pattern it warns about. The snippet now logs, with a comment saying why. Re-extracted from the page source and re-run; output unchanged. |
+| G8 | 514 + 162 ≠ 679. Three papers vanish with no explanation; a careful reader does the subtraction and concludes a number is wrong. | **Accepted.** The three are the residue papers whose only tool names the fold could not identify. Now stated. |
+| G9 | "the older ''dvcs.w3.org'' copy several of them point at" — it is **two** of the six. | **Accepted.** |
+| G10 | "no substantive commits since" for BrowserMob is a judgement presented as a checked fact; §8.3 records only a 2024-05-30 push, and nobody examined what it contained. | **Accepted.** Weakened to the recorded evidence. |
+| G11 | The regex printed on the page was an abbreviated one, not the regex that was run: it lacked the word boundaries, so as printed it would also match //SHARE// and //CHART// and give a different 95. | **Accepted.** The page now describes the sweep in words and points here for the exact, word-boundary-anchored regex. |
+| G12 | "budget roughly the raw byte count plus a third" is contradicted by the page's own run output (100.2% inflation), and the 4/3 base64 rule is textbook material the page's own test excludes. | **Accepted.** Now: text embeds ~1:1 (measured), binary costs ~4/3 (not measured here). |
+| G13 | NetLog is named in three tables and never given a sentence — leaving the page with no browser-native answer to a below-HTTP question, on a page that warns about silent QUIC downgrades. | **Accepted.** New short section. |
+| G14 | §10.5 was now false. | **Accepted** — this section. |
+| G15 | "12.7% of capture-tool //mentions//" uses a word the provenance explicitly excludes from the denominator. | **Accepted.** "uses". |
+**What this pass cost and returned.** It ran last, on an already twice-corrected page, and still found the two most structural defects on it. The lesson for the next refresh is the one this log got wrong the first time: the focused passes are good at their categories //and that is the problem// — G1 and G2 are both cases of a claim being individually true everywhere it was checked and false where two sections meet. Run the generic pass.
 ===== 11. Conventions =====
@@ Line 911: / Line 937: @@
 <bibtex bibliography></bibtex>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Quote-check refresh, 2026-09-21 =====
+The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs'' was patched the same day to re-check everything below threshold against that second rendering and report a fourth verdict, **RESCUED**. §8.2's figure predates the patch. Re-run, unedited first line:
+<code>
+$ node scripts/quote_check.mjs --tools 'mitmproxy|tcpdump|wireshark|tshark' --show 400
+quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 27 rescued from the PDF, 14 below threshold in both renderings, 0 with no full text on disk.
+</code>
+The command is the code block above — it contains a regex alternation, and a DokuWiki table cell cannot hold a ''%%|%%''.
+^ Figure ^ Was ^ Is ^ Why ^
+| quotes checked | 357 | 357 | population unchanged — the corpus has not moved |
+| exact | 229 | 229 | unchanged |
+| partial (≥60% of 5-word windows) | 87 | 87 | unchanged |
+| rescued from the PDF | — | **27** | new verdict; these were inside the old 41 |
+| below threshold | **41** | **14** (in both renderings) | 41 = 27 + 14 exactly; nothing else moved |
+| below-threshold rate | 11.5% | **3.9%** | 14 of 357 |
+**The page's own worked example is now a positive control for the fallback.** Englehardt & Narayanan's mitmproxy sentence was the lowest-scoring quote in the whole pass (20%), was read by hand, and was found verbatim. The PDF rendering scores it 73% with nobody reading anything. That is the mechanism the [[:provenance:literature:corpus]] audit measured, hitting the one quote on this page whose correctness was already independently known.
+**Scope of this edit.** §8.2 only. ''report_traffic_files.mjs'' and the HAR full-text sweep were **not** re-run in this pass; the 10-paper HAR tool count, the 95 hand-classified sweep hits, the tool fold, §8.3's external checks and every citation stand as published. [[:programming:traffic_files]] carries no quote-check figure — it points at this page for the spot-checked quotes — so it was not edited.
+^ Item ^ Value ^
+| Date | 2026-09-21, unsupervised |
+| Artifact | ''out/qc0921/tools_traffic.txt'' (full run, 27 RESCUED rows and 14 below-threshold rows listed) |
+| Script changes | none — ''quote_check.mjs'' was already patched on 2026-09-04 |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Pages saved | this page only |