| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:programming:traffic_files [2026/08/14 03:42] – Add §10.6, the generic review pass: 15 accepted findings including two internal contradictions the focused passes could not catch; correct §10.5, which had wrongly justified skipping it. Refresh embedded script output. Authored by Claude. karel.kubicek.claude | provenance:programming:traffic_files [2026/09/21 13:36] (current) – Quote-check refresh 2026-09-21: re-ran quote_check.mjs --tools mitmproxy/tcpdump/wireshark/tshark with the pypdf fallback; 41 below threshold -> 27 rescued + 14 below in both; new provenance section. Authored by Claude karel.kubicek.claude |
|---|
| ===== 6. Unedited report output ===== | ===== 6. Unedited report output ===== |
| |
| The full output of ''node scripts/report_traffic_files.mjs'' is below, unedited. ''--list'' output (the 514-paper population with its families) is not reproduced here for length; regenerate it with the command in §3. | The full output of ''node scripts/report_traffic_files.mjs'' is below, unedited. ''%%--list%%'' output (the 514-paper population with its families) is not reproduced here for length; regenerate it with the command in §3. |
| |
| <file text report_traffic_files.txt> | <file text report_traffic_files.txt> |
| ==== 8.2 Evidence quotes behind the capture-tool figures ==== | ==== 8.2 Evidence quotes behind the capture-tool figures ==== |
| |
| ''node scripts/quote_check.mjs --tools 'mitmproxy|tcpdump|wireshark|tshark''' over the whole corpus: | ''node scripts/quote_check.mjs %%--tools%% 'mitmproxy|tcpdump|wireshark|tshark''' over the whole corpus. **Re-run 2026-09-21** with the PDF fallback — see //Quote-check refresh, 2026-09-21// at the foot of this page: |
| |
| <code> | <code> |
| 357 quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 41 below threshold, 0 with no full text on disk. | 357 quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 27 rescued from the PDF, 14 below threshold in both renderings, 0 with no full text on disk. |
| </code> | </code> |
| |
| Below-threshold is not "unsupported". The lowest-scoring one is the single most-quoted sentence on the content page, Englehardt & Narayanan's, at **20%** — and it is verbatim in the paper. The ''.cols'' rendering reads //"After examining several Python HTTP proxies, we chose to use Mitmproxy6 to record all HTTP Re-quest and Response headers"//: a footnote marker and a hyphenation split by the two-column repair. Checked by hand; the extraction is faithful. Two others were spot-checked the same way (''respect-the-origin'', ''comparing-the-effects-of-dns-dot-and-doh'') and both are present verbatim. | **The old figure was 41 below threshold.** 27 of the 41 are located in an independent ''pypdf'' rendering of the same ''paper.pdf'' and are a defect in the stored text, not in the extraction; **14** are below threshold in both, i.e. 3.9% of 357. ''exact'' (229) and ''partial'' (87) did not move. |
| | |
| | Below-threshold is not "unsupported", and the example this section has always given is now the clearest case of why. The lowest-scoring quote under the old single-rendering test was the single most-quoted sentence on the content page, Englehardt & Narayanan's, at **20%** — and it is verbatim in the paper. The ''.cols'' rendering reads //"After examining several Python HTTP proxies, we chose to use Mitmproxy6 to record all HTTP Re-quest and Response headers"//: a footnote marker and a hyphenation split by the two-column repair. It was checked by hand and the extraction is faithful; **the fallback now reaches it on its own, scoring it 73% against the PDF**, so it is one of the 27 rather than a failure. Two others were spot-checked the same way (''respect-the-origin'', ''comparing-the-effects-of-dns-dot-and-doh'') and both are present verbatim; neither is below threshold under the current script. The lowest-scoring of the 14 that remain is ''PETS/2026/clicking-into-exposure…'' at 25%, and none of the 14 has been read by hand. |
| |
| ==== 8.3 External facts ==== | ==== 8.3 External facts ==== |
| | WARC | ''iipc.github.io/warc-specifications/'' | WARC 1.0 and 1.1 listed as standard; ISO 28500:2017 is the published edition (WebSearch only — ''iso.org'' answered 403). | | | WARC | ''iipc.github.io/warc-specifications/'' | WARC 1.0 and 1.1 listed as standard; ISO 28500:2017 is the published edition (WebSearch only — ''iso.org'' answered 403). | |
| | mitmproxy current | GitHub releases API | v12.2.3, 2026-05-12. **The runs on this page used 11.0.2** and the page says so. | | | mitmproxy current | GitHub releases API | v12.2.3, 2026-05-12. **The runs on this page used 11.0.2** and the page says so. | |
| | ''--set hardump'' still exists | ''docs.mitmproxy.org/stable/concepts/options/'' | Yes: "Save a HAR file with all flows on exit." | | | ''%%--set%% hardump'' still exists | ''docs.mitmproxy.org/stable/concepts/options/'' | Yes: "Save a HAR file with all flows on exit." | |
| | Wireshark / tcpdump / libpcap | wireshark.org, GitHub tags | 4.6.8 / tcpdump-4.99.6 / libpcap-1.10.6 | | | Wireshark / tcpdump / libpcap | wireshark.org, GitHub tags | 4.6.8 / tcpdump-4.99.6 / libpcap-1.10.6 | |
| | chrome-har-capturer abandoned | npm registry + GitHub | **No** — 0.14.4 published 2026-01-27, repo pushed 2026-08-12. Actively maintained. | | | chrome-har-capturer abandoned | npm registry + GitHub | **No** — 0.14.4 published 2026-01-27, repo pushed 2026-08-12. Actively maintained. | |
| | Okta sanitiser guidance | ''support.okta.com/help/s/article/sanitizing-http-traces'' | "There is no attempt to remove user information as part of the sanitization process" | | | Okta sanitiser guidance | ''support.okta.com/help/s/article/sanitizing-http-traces'' | "There is no attempt to remove user information as part of the sanitization process" | |
| | A maintained HAR sanitiser exists | GitHub API | **No.** ''google/har-sanitizer'' and ''cloudflare/har-sanitizer'' are both ''archived: true'' as of 2026-08-14 | | | A maintained HAR sanitiser exists | GitHub API | **No.** ''google/har-sanitizer'' and ''cloudflare/har-sanitizer'' are both ''archived: true'' as of 2026-08-14 | |
| | Chromium writes a TLS key log | run locally | Yes — ''--ssl-key-log-file'' produced a 5-line NSS key log for one ''https://example.com/'' load | | | Chromium writes a TLS key log | run locally | Yes — ''%%--ssl-key-log-file%%'' produced a 5-line NSS key log for one ''https://example.com/'' load | |
| |
| ==== 8.4 Sources rejected ==== | ==== 8.4 Sources rejected ==== |
| | 2 | The sweep found **95** papers; a ''grep -rlP'' with the same regex had found 94. ''WWW/2022/reproducibility-and-replicability-of-web-measurement-studies'' was missing from the shell result. | Cross-checking the two sweeps. The Node sweep is authoritative and is the one in the report. | | | 2 | The sweep found **95** papers; a ''grep -rlP'' with the same regex had found 94. ''WWW/2022/reproducibility-and-replicability-of-web-measurement-studies'' was missing from the shell result. | Cross-checking the two sweeps. The Node sweep is authoritative and is the one in the report. | |
| | 3 | "By year bucket" and "By venue" printed each other's values, because the key-order fix was applied to the map but not to the two ''split('/')'' index reads below it. | Reading the output: it said "By venue: 2025 5" followed by "2021 4". | | | 3 | "By year bucket" and "By venue" printed each other's values, because the key-order fix was applied to the map but not to the two ''split('/')'' index reads below it. | Reading the output: it said "By venue: 2025 5" followed by "2021 4". | |
| | 4 | Adding ''adafruit'' to the radio family moved one paper out of packet capture. Seven figures on the page went stale in one edit: 288→287, 512→511, 164→165, 42.4→42.3, 62.3→62.2, 36.9→37.0, 73→72 — and 659→662, 129→130, 12.4→12.3 from an unrelated fold edit. The page had already been written with all of them. | ''check_page_numbers.mjs'', run whole-page with ''--code''. It also caught an eighth, "164 excluded papers", sitting in the limitations section outside any table. | | | 4 | Adding ''adafruit'' to the radio family moved one paper out of packet capture. Seven figures on the page went stale in one edit: 288→287, 512→511, 164→165, 42.4→42.3, 62.3→62.2, 36.9→37.0, 73→72 — and 659→662, 129→130, 12.4→12.3 from an unrelated fold edit. The page had already been written with all of them. | ''check_page_numbers.mjs'', run whole-page with ''%%--code%%''. It also caught an eighth, "164 excluded papers", sitting in the limitations section outside any table. | |
| | 5 | The ''%%<code>%%'' block quoting the mitmproxy comparison held byte counts from an earlier run (20472/261704/232414 against a later run's 20453/261692/232420). | A line-by-line equality check of every ''%%<code>%%'' block against the run output it claims to be, both directions. A substring check would have passed. | | | 5 | The ''%%<code>%%'' block quoting the mitmproxy comparison held byte counts from an earlier run (20472/261704/232414 against a later run's 20453/261692/232420). | A line-by-line equality check of every ''%%<code>%%'' block against the run output it claims to be, both directions. A substring check would have passed. | |
| | 6 | The three-line output beside the published ''%%<file javascript>%%'' snippet had never been produced by that snippet. | Extracting the block from the page source, writing it to ''sandbox/har_vs_cdp_core.mjs'' and running it. It does produce exactly those three lines. | | | 6 | The three-line output beside the published ''%%<file javascript>%%'' snippet had never been produced by that snippet. | Extracting the block from the page source, writing it to ''sandbox/har_vs_cdp_core.mjs'' and running it. It does produce exactly those three lines. | |
| Three focused passes and the author's own read, all against the frozen snapshot **rev 1786677596** (''pages/programming_traffic_files.txt'' md5 ''d165a98f56316b0d2c618128bc9d801d''). Each reviewer was told its context might not be exhaustive and was handed the page, the provenance, the scripts and their committed output. Nothing was edited while they ran. | Three focused passes and the author's own read, all against the frozen snapshot **rev 1786677596** (''pages/programming_traffic_files.txt'' md5 ''d165a98f56316b0d2c618128bc9d801d''). Each reviewer was told its context might not be exhaustive and was handed the page, the provenance, the scripts and their committed output. Nothing was edited while they ran. |
| |
| ==== 10.1 Figures vs script (''sonnet'') ==== | ==== 10.1 Figures vs script (sonnet) ==== |
| |
| ^ # ^ Finding ^ Verdict ^ | ^ # ^ Finding ^ Verdict ^ |
| | F4 | ''report_traffic_files.mjs''’s closing block hardcoded "mitmproxy 12.1.2 / Wireshark 4.6.2 / Chrome 127" as current while the page correctly said 12.2.3 / 4.6.8 / 130 — and the stale copy was reproduced verbatim in §6 of this page. | **Accepted.** That block is now a pointer to ''verify_traffic_figures.mjs'' and ''external_checks_traffic_files.sh'' rather than a second, drifting copy. **A provenance page that embeds a script's output inherits that script's stale numbers.** | | | F4 | ''report_traffic_files.mjs''’s closing block hardcoded "mitmproxy 12.1.2 / Wireshark 4.6.2 / Chrome 127" as current while the page correctly said 12.2.3 / 4.6.8 / 130 — and the stale copy was reproduced verbatim in §6 of this page. | **Accepted.** That block is now a pointer to ''verify_traffic_figures.mjs'' and ''external_checks_traffic_files.sh'' rather than a second, drifting copy. **A provenance page that embeds a script's output inherits that script's stale numbers.** | |
| |
| The same pass re-ran the report (plain and ''--wiki'', byte-identical to the committed output), re-ran the figure verifier, re-ran both experiments three times, and confirmed papers-not-tuples, sentinel handling, ''usedOrMentioned'' filtering, both-ended year buckets and every denominator in "Use in Publications". No other issue found. | The same pass re-ran the report (plain and ''%%--wiki%%'', byte-identical to the committed output), re-ran the figure verifier, re-ran both experiments three times, and confirmed papers-not-tuples, sentinel handling, ''usedOrMentioned'' filtering, both-ended year buckets and every denominator in "Use in Publications". No other issue found. |
| |
| ==== 10.2 Citations and quotes (''sonnet'') ==== | ==== 10.2 Citations and quotes (sonnet) ==== |
| |
| ^ # ^ Finding ^ Verdict ^ | ^ # ^ Finding ^ Verdict ^ |
| All 16 ''%%{[key]}%%'' citations resolve; 279 keys in the bibliography, 279 unique; all 14 new entries match Crossref on authors, title, year, DOI, and the three hand-written PoPETs entries match on volume, issue and pages. Every quotation verified verbatim after whitespace normalisation. Every claim-to-paper attribution checked, including all eleven Hantke et al. figures for numerator/denominator/percentage coming from the same sentence. The "six papers cite a W3C HAR URL" figure was independently reproduced (two ''dvcs.w3.org'', four ''w3c.github.io''). No missing citations, no footnote/citation mismatches. | All 16 ''%%{[key]}%%'' citations resolve; 279 keys in the bibliography, 279 unique; all 14 new entries match Crossref on authors, title, year, DOI, and the three hand-written PoPETs entries match on volume, issue and pages. Every quotation verified verbatim after whitespace normalisation. Every claim-to-paper attribution checked, including all eleven Hantke et al. figures for numerator/denominator/percentage coming from the same sentence. The "six papers cite a W3C HAR URL" figure was independently reproduced (two ''dvcs.w3.org'', four ''w3c.github.io''). No missing citations, no footnote/citation mismatches. |
| |
| ==== 10.3 External currency (''sonnet'') ==== | ==== 10.3 External currency (sonnet) ==== |
| |
| ^ # ^ Finding ^ Verdict ^ | ^ # ^ Finding ^ Verdict ^ |
| | E5 | Two small imprecisions: BrowserMob's tag year (same as C1), and this page's "''har-spec'' has not moved since 2019" against a last commit the reviewer dates to April 2018. | **Accepted with a caveat.** The two reviewers give different years and GitHub rate-limited the re-check, so the page now says "dormant for years" — which both agree on — rather than a year neither could confirm together. | | | E5 | Two small imprecisions: BrowserMob's tag year (same as C1), and this page's "''har-spec'' has not moved since 2019" against a last commit the reviewer dates to April 2018. | **Accepted with a caveat.** The two reviewers give different years and GitHub rate-limited the re-check, so the page now says "dormant for years" — which both agree on — rather than a year neither could confirm together. | |
| |
| The same pass independently re-verified the HAR abandonment banner, the two ''dvcs.w3.org''/''w3c.github.io'' copies, ''har.fyi''’s subject, both IETF drafts, WARC/ISO 28500, seven tool versions, every maintenance-status call, the Chrome 130 wording, all ten Firefox prefs read from ''firefox.js'', ''--set hardump'', the QUIC/custom-CA limitation, ''SSLKEYLOGFILE'' for both browsers, and ~20 URLs for liveness. One 403 (''iso.org''), already disclosed in §8.3. Nothing else superseded. | The same pass independently re-verified the HAR abandonment banner, the two ''dvcs.w3.org''/''w3c.github.io'' copies, ''har.fyi''’s subject, both IETF drafts, WARC/ISO 28500, seven tool versions, every maintenance-status call, the Chrome 130 wording, all ten Firefox prefs read from ''firefox.js'', ''%%--set%% hardump'', the QUIC/custom-CA limitation, ''SSLKEYLOGFILE'' for both browsers, and ~20 URLs for liveness. One 403 (''iso.org''), already disclosed in §8.3. Nothing else superseded. |
| |
| ==== 10.4 The author's own read ==== | ==== 10.4 The author's own read ==== |
| |
| |
| ==== 10.6 Generic (''fable'', no checklist) ==== | ==== 10.6 Generic (fable, no checklist) ==== |
| |
| Fifteen findings, **all accepted**. Two of them are the reason this pass exists, because no focused pass could have found them: | Fifteen findings, **all accepted**. Two of them are the reason this pass exists, because no focused pass could have found them: |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| | |
| | ===== Quote-check refresh, 2026-09-21 ===== |
| | |
| | The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs'' was patched the same day to re-check everything below threshold against that second rendering and report a fourth verdict, **RESCUED**. §8.2's figure predates the patch. Re-run, unedited first line: |
| | |
| | <code> |
| | $ node scripts/quote_check.mjs --tools 'mitmproxy|tcpdump|wireshark|tshark' --show 400 |
| | 357 quotes checked: 229 exact, 87 partial (>=60% of 5-word windows), 27 rescued from the PDF, 14 below threshold in both renderings, 0 with no full text on disk. |
| | </code> |
| | |
| | The command is the code block above — it contains a regex alternation, and a DokuWiki table cell cannot hold a ''%%|%%''. |
| | |
| | ^ Figure ^ Was ^ Is ^ Why ^ |
| | | quotes checked | 357 | 357 | population unchanged — the corpus has not moved | |
| | | exact | 229 | 229 | unchanged | |
| | | partial (≥60% of 5-word windows) | 87 | 87 | unchanged | |
| | | rescued from the PDF | — | **27** | new verdict; these were inside the old 41 | |
| | | below threshold | **41** | **14** (in both renderings) | 41 = 27 + 14 exactly; nothing else moved | |
| | | below-threshold rate | 11.5% | **3.9%** | 14 of 357 | |
| | |
| | **The page's own worked example is now a positive control for the fallback.** Englehardt & Narayanan's mitmproxy sentence was the lowest-scoring quote in the whole pass (20%), was read by hand, and was found verbatim. The PDF rendering scores it 73% with nobody reading anything. That is the mechanism the [[:provenance:literature:corpus]] audit measured, hitting the one quote on this page whose correctness was already independently known. |
| | |
| | **Scope of this edit.** §8.2 only. ''report_traffic_files.mjs'' and the HAR full-text sweep were **not** re-run in this pass; the 10-paper HAR tool count, the 95 hand-classified sweep hits, the tool fold, §8.3's external checks and every citation stand as published. [[:programming:traffic_files]] carries no quote-check figure — it points at this page for the spot-checked quotes — so it was not edited. |
| | |
| | ^ Item ^ Value ^ |
| | | Date | 2026-09-21, unsupervised | |
| | | Artifact | ''out/qc0921/tools_traffic.txt'' (full run, 27 RESCUED rows and 14 below-threshold rows listed) | |
| | | Script changes | none — ''quote_check.mjs'' was already patched on 2026-09-04 | |
| | | Reviewers | one ''sonnet'' figures-vs-script pass | |
| | | Pages saved | this page only | |
| |
