| Next revision | Previous revision |
| provenance:programming:crawler:foxhound [2026/08/17 17:39] – New page: provenance for programming:crawler:foxhound — every query with its denominator, the 13-to-9 population narrowing with role verdicts and deciding quotes, the taint-instrument fold with its full residue (which is where the pre-2024 lineage came fr karel.kubicek.claude | provenance:programming:crawler:foxhound [2026/08/17 17:56] (current) – Record the figures-and-code re-review: the UTF-16 offset false negative and the iframe.srcdoc gap (both fixed), the confirmed weak needle, the byte-vs-code-point unit error, and a process defect on my side — reviewers were handed files that changed under karel.kubicek.claude |
|---|
| * **Schema evidence quotes** (section G): the 9 ''tools[].evidence.quote'' values, normalised for whitespace and line-break hyphens, then by five-word windows — **5 exact, 4 partial (≥60% of windows), 0 below threshold, 0 without full text.** The four partials are column splices in the repaired rendering, not unsupported claims. Cross-checked with the shared bulk verifier, which is a separate implementation: ''node scripts/quote_check.mjs --tools 'fox.?hound' --show'' returns the identical 5/4/0/0. | * **Schema evidence quotes** (section G): the 9 ''tools[].evidence.quote'' values, normalised for whitespace and line-break hyphens, then by five-word windows — **5 exact, 4 partial (≥60% of windows), 0 below threshold, 0 without full text.** The four partials are column splices in the repaired rendering, not unsupported claims. Cross-checked with the shared bulk verifier, which is a separate implementation: ''node scripts/quote_check.mjs --tools 'fox.?hound' --show'' returns the identical 5/4/0/0. |
| * **Hand deciding quotes** (also section G): the 13 sentences in the ''ROLE'' map — **13 exact, 0 not exact**, as the report says. Two of them are exact only because they were //shortened// during this run to a contiguous fragment: the CCS 2025 extension sentence and the NDSS 2026 baseline sentence are each spliced across two columns in ''paper.cols.txt'', and the longer versions I first wrote matched at 50% and 60%. Both are marked with an inline comment in ''fh_fold.mjs'' saying why they are short, so a later reader does not "restore" them. (An earlier draft of this page said "11 exact, 2 not", describing the state before the shortening; the citations reviewer caught the mismatch against the artefact.) | * **Hand deciding quotes** (also section G): the 13 sentences in the ''ROLE'' map — **13 exact, 0 not exact**, as the report says. Two of them are exact only because they were //shortened// during this run to a contiguous fragment: the CCS 2025 extension sentence and the NDSS 2026 baseline sentence are each spliced across two columns in ''paper.cols.txt'', and the longer versions I first wrote matched at 50% and 60%. Both are marked with an inline comment in ''fh_fold.mjs'' saying why they are short, so a later reader does not "restore" them. (An earlier draft of this page said "11 exact, 2 not", describing the state before the shortening; the citations reviewer caught the mismatch against the artefact.) |
| * **Per-paper figures and quoted sentences used on the page** (section H): **48 needles, 0 not located** verbatim in ''paper.cols.txt''. This check exists because ''detection[].prevalence'' is a model summary rather than a quotation — the same reason ''verify_requests_figures.mjs'' exists. It started at 31 needles and grew to 48 on the figures reviewer's finding that the comparison table's per-tool numbers were disclosed as hand-checked but not machine-checked; adding them also caught three of my own needles that were column-spliced and would have read as failures. | * **Per-paper figures and quoted sentences used on the page** (section H): **50 needles, 0 not located** verbatim in ''paper.cols.txt''. This check exists because ''detection[].prevalence'' is a model summary rather than a quotation — the same reason ''verify_requests_figures.mjs'' exists. It started at 31 needles and grew to 50 on the figures reviewer's finding that the comparison table's per-tool numbers were disclosed as hand-checked but not machine-checked; adding them also caught three of my own needles that were column-spliced and would have read as failures. |
| | |
| | A self-caught defect in the same check, worth recording because nothing would have flagged it: **three needles were too short to mean anything.** ''"38.5"'', ''"13"'' and ''"55 event handlers"'' will match almost any paper's digits, so they were passing for the wrong reason — a false pass is worse than a failure, because it looks like verification. All three were replaced with the full clause from the source (''"false negative rate of 38.5%"'', ''"From each of the remaining 13 libraries we create a benchmark sample"'', ''"a total of 55 event handlers"''), and ''"9.6% of the top 10K"'' was split into two needles because the paper states the share and the flow count in different sentences. The rule this suggests for the next page: audit the needle list for anything under about twenty characters before trusting the "0 not located" line. |
| |
| One figure is quoted from the extraction with a formatting change: FP-tracer's ''269 784 fingerprinting flows'' (thin space in the paper and in the extraction) is written ''269,784'' on the page. That is a rendering choice, logged here so a future check does not read it as a mismatch. | One figure is quoted from the extraction with a formatting change: FP-tracer's ''269 784 fingerprinting flows'' (thin space in the paper and in the extraction) is written ''269,784'' on the page. That is a rendering choice, logged here so a future check does not read it as a mismatch. |
| ===== The published code ===== | ===== The published code ===== |
| |
| ''foxhound_flows.py'' on the content page is not a wrapper around a browser: it reduces already-collected taint reports. **The first version of it had two real bugs, both found by the generic reviewer and both visible in its own self-test** — see findings 1 and 2 in the [[#Review log]]. That is the strongest argument on this page for publishing code with a self-test rather than a snippet: the fixture disagreed with the prose, and it was checkable. That choice was deliberate — a script that needs a Foxhound binary could not have been tested in this run, and an untested script on the page would violate the task's own rule. As written it is testable, and its self-test is the fixture from the project's documented worked example plus five mutations of it: | ''foxhound_flows.py'' on the content page is not a wrapper around a browser: it reduces already-collected taint reports. **The first version of it had two real bugs, both found by the generic reviewer and both visible in its own self-test** — see findings 1 and 2 in the [[#Review log]] — and a third, found afterwards by writing an adversarial stress harness against the rewritten version: ''script_of'' required a ''scripthash'' and therefore threw away the filename, so the documented ''%%--unit script%%'' fallback to a filename never fired and every unhashed flow collapsed into one empty key. Fixed, with a regression test, plus tests for an empty flow and for the encode/decode ordering. The stress harness also confirmed one limitation that cannot be fixed by a name heuristic and is now documented in the code: a //page// function called something like ''reportTaintSinkStats'' would be misclassified as harness code, which is why ''%%--harness%%'' exists. That is the strongest argument on this page for publishing code with a self-test rather than a snippet: the fixture disagreed with the prose, and it was checkable. That choice was deliberate — a script that needs a Foxhound binary could not have been tested in this run, and an untested script on the page would violate the task's own rule. As written it is testable, and its self-test is the fixture from the project's documented worked example plus five mutations of it: |
| |
| <code> | <code> |
| $ python3 pages/foxhound_flows.py --selftest | $ python3 pages/foxhound_flows.py --selftest |
| selftest: 19 checks passed | selftest: 31 checks passed |
| |
| flows: 1 sites: 1 pages: 1 scripts: 1 | flows: 1 sites: 1 pages: 1 scripts: 1 |
| </code> | </code> |
| |
| The copy embedded in the page's ''<file python>'' block was byte-compared against ''pages/foxhound_flows.py'' before publication (20,338 bytes each, identical), because a published script that has drifted from the one that was tested is worse than no script: | The copy embedded in the page's ''<file python>'' block was byte-compared against ''pages/foxhound_flows.py'' before publication (24,456 bytes each, identical — a real ''wc -c'' byte count, not Python's ''len()'', which counts code points and was what an earlier draft printed), because a published script that has drifted from the one that was tested is worse than no script: |
| |
| <code bash> | <code bash> |
| NDSS/2026 | 55 event handlers | true | NDSS/2026 | 55 event handlers | true |
| CCS/2025 | one site per library in the benchmark | true | CCS/2025 | one site per library in the benchmark | true |
| | CCS/2025 | 8 of 13 detected = 38.5% FN rate | true |
| PETS/2026 | 40,605 requests on 7,614 sites | true | PETS/2026 | 40,605 requests on 7,614 sites | true |
| PETS/2026 | 23,109 by taint tracking | true | PETS/2026 | 23,109 by taint tracking | true |
| IEEE-SP/2024 | Playwright + Foxhound v98.0.2 + DevTools | true | IEEE-SP/2024 | Playwright + Foxhound v98.0.2 + DevTools | true |
| IEEE-SP/2024 | 202,834 flows, 9.6% of top 10K | true | IEEE-SP/2024 | 202,834 flows, 9.6% of top 10K | true |
| | IEEE-SP/2024 | 202,834 verified flows | true |
| IEEE-SP/2024 | no iframe.srcdoc support | true | IEEE-SP/2024 | no iframe.srcdoc support | true |
| IEEE-SP/2024 | Playwright 1.33, commit 2916e01 | true | IEEE-SP/2024 | Playwright 1.33, commit 2916e01 | true |
| USENIX/2025 | rejected: interpreter-level string taint | true | USENIX/2025 | rejected: interpreter-level string taint | true |
| |
| 48 figures checked, 0 not located. | 50 figures checked, 0 not located. |
| |
| === Z. Every number on the page that does NOT come from this corpus === | === Z. Every number on the page that does NOT come from this corpus === |
| taintfox.escapeURL, default false, line 116; 0 occurrences of "taintfox" in all.js | netwerk/base/nsStandardURL.cpp and all.js | taintfox.escapeURL, default false, line 116; 0 occurrences of "taintfox" in all.js | netwerk/base/nsStandardURL.cpp and all.js |
| 68 -> 347 confirmed DOM-XSS flows, i.e. 5x | Sabino et al., NDSS 2026, already in section H | 68 -> 347 confirmed DOM-XSS flows, i.e. 5x | Sabino et al., NDSS 2026, already in section H |
| 19 self-test checks, 7 mutations of the documented fixture | python3 pages/foxhound_flows.py --selftest | 31 self-test checks, 13 mutations of the documented fixture | python3 pages/foxhound_flows.py --selftest |
| | UTF-16 code units: 13 for a two-emoji fixture, offsets 4..6 | the utf16_slice self-test in pages/foxhound_flows.py |
| PR #198 touched 36 files | the PR object's changed_files field. A naive GET of /pulls/198/files returns 30 (its default page size) and under-counts; the external-currency reviewer reported 29 for this reason. The probe asks for per_page=50 and prints changed_files beside it. | PR #198 touched 36 files | the PR object's changed_files field. A naive GET of /pulls/198/files returns 30 (its default page size) and under-counts; the external-currency reviewer reported 29 for this reason. The probe asks for per_page=50 and prints changed_files beside it. |
| 12345, TRK-a55bd7c6, 11-16, 22 characters, 5 characters | strings from the documented worked example and from a cited paper — not figures | 12345, TRK-a55bd7c6, 11-16, 22 characters, 5 characters | strings from the documented worked example and from a cited paper — not figures |
| |
| Two things the pass explicitly endorsed and which were therefore left alone: the fourteen-zero-years table with its lineage explanation, and the calibrated wording of the judgement-calls table ("did not outlive their paper" rather than "superseded"). | Two things the pass explicitly endorsed and which were therefore left alone: the fourteen-zero-years table with its lineage explanation, and the calibrated wording of the judgement-calls table ("did not outlive their paper" rather than "superseded"). |
| | |
| | ==== Re-review round ==== |
| | |
| | The task's rule is to re-run any reviewer whose findings were acted on. All four rounds' findings were acted on, so the figures/code pass and the citations pass were re-run against the **published** text (fetched back with ''?do=export_raw'' and byte-identical to the local drafts). |
| | |
| | One re-run **failed to do its job**: the first citations re-review returned a single line saying it would wait for a background agent, with no findings at all. That is recorded rather than quietly dropped — a reviewer that returns nothing is not a pass — and it was replaced with a re-run given an explicit A–K checklist of the specific claims that were new or changed since the first round, with an instruction not to delegate. |
| | |
| | The re-review round also produced one finding I caught myself while it ran, recorded above under [[#Quotes and figures checked]]: three of the section-H needles were short enough to match almost anything, so they were passing for the wrong reason. |
| | |
| | The re-run of the claims check verified 15 items against primary sources and confirmed all of them except two, both accepted: |
| | |
| | ^ # ^ Finding ^ Disposition ^ |
| | | 1 | The page said Sabino et al. read the decade-long decline in DOM-XSS as this setting "rather than the web having got safer". The paper actually names **several** contributing factors and credits genuine improvements among them — better ad blocking, HTTPS/mixed-content enforcement — so the "rather than" was a false dichotomy the authors do not assert. | **Accepted.** Rewritten to quote "several factors, such as dataset and methodology, may contribute", to say the encoding is the first of them, and to name the genuine improvements alongside. The page now calls it "one measurement artefact among real effects rather than a debunking". | |
| | | 2 | The page stated as fact that PanoptiChrome's own authors avoided the unresponsiveness because they drove it by hand; the paper hedges with "likely because". | **Accepted.** The hedge is restored. | |
| | |
| | Everything else it checked came back verbatim, including the ''taintfox.escapeURL'' call site and its ''false'' default, the absence of ''taintfox'' from ''all.js'', the 34/61/1 pref counts, both issue states and the maintainer's legacy-IE sentence, all three mozconfigs and the missing unsuffixed one, the PR #350 diff and the trace-versus-tracking reading of it, the 68→347 quintupling and "would not be exploitable in any modern browser", every cell of the bake-off table including the five compatibility and five transparency figures, the "presense" misspelling (confirmed in both renderings, so the ''[sic]'' is right), and the neighbours' published count of 8. It also noted that for the NDSS 2026 paper ''paper.cols.txt'' is the well-ordered rendering and ''paper.txt'' the interleaved one — the opposite of the case flagged in round one, which is a useful reminder that neither rendering is reliably the better one. |
| | |
| | The figures-and-code re-review returned last and found **two more real defects in the published script**, both accepted, plus one confirmation and one cosmetic fix: |
| | |
| | ^ # ^ Finding ^ Disposition ^ |
| | | 1 | **''begin''/''end'' are UTF-16 code-unit offsets, and the script sliced with them in Python, which indexes by code point.** One astral character earlier in the string — an emoji, some CJK extensions — shifts every later index, so the wrong substring is read. The reviewer built the case: on ''"\\U0001F600\\U0001F600<>PADDING"'' the engine's offsets 4..6 bound ''"<>"'', while ''value[4:6]'' returns ''"PA"'', and the flow is therefore reported as having held no syntax character. A **false negative on exactly the screen the script exists to provide**. | **Accepted; reproduced first, then fixed.** ''utf16_slice()'' and ''utf16_len()'' do the slicing in the engine's own unit, four regression tests cover it (including one asserting what the naive Python slice //would// have returned, so the bug cannot come back unnoticed), and the content page now states the offsets are UTF-16 and names the languages this bites in. | |
| | | 2 | ''iframe.srcdoc'' was missing from the HTML/JS sink set although its content is parsed as a whole document, so ''no_syntax_chars'' could never fire for it — on the same page that calls out ''iframe.srcdoc'' as a sink a 2024 paper found missing. | **Accepted.** Added, along with ''script.textContent''. The set now also carries an explicit note that navigation sinks (''location.href'', ''a.href'', ''window.open'') are **deliberately excluded**, because they are dangerous through the ''javascript:'' scheme rather than through markup syntax, and a screen that does not model that should stay silent about them rather than look reassuring. A test asserts both behaviours. | |
| | | 3 | The weak ''"13"'' needle in section H, which occurs 26 times in that paper for unrelated reasons. | **Accepted as confirmation** — independently caught and fixed during the same window; see above. | |
| | | 4 | The provenance page said "20,338 bytes each" for the byte-comparison, but that number is Python's ''len()'' — code points. The file has em dashes, so the real ''wc -c'' differs. | **Accepted.** The claim of identity was right; the unit was wrong, and the figure is now a real byte count. | |
| | |
| | **A process defect on my side, recorded because it degraded the review:** this reviewer reported that the files under review changed on disk while it worked, because I was applying the other reviewers' fixes in parallel instead of freezing the tree. It anchored its findings to the snapshot it started from and flagged the drift, which is the right response — but it should not have had to. The next run should copy the artefacts to a frozen directory and point reviewers at that. |
| |
| ===== Rendering check ===== | ===== Rendering check ===== |
| |
| Rendering check pending: run after publication. | Both pages were published, then ''literature:bibliography'' and both new pages were purged with ''?purge=true'' and the rendered HTML re-read. Without the purge the ''bibtex'' plugin serves a cached parse and every new citation renders broken while both raw sources look perfect — the trap recorded on ''programming:crawler:webxray'', where 14 of 20 citations were broken until the purge. |
| | |
| | ^ Check ^ Result ^ |
| | | ''literature:bibliography'' entries after the append | 314 → **326** (12 added, 0 collisions, 0 duplicates) | |
| | | ''{[citekey]}'' markers rendered on the content page | **90** ''bibtex_citekey'' spans, 0 broken | |
| | | Distinct citekeys on the content page | **16**, all resolving | |
| | | Reference list at the foot of the content page | **16 entries**, each with author list, venue and a DOI or venue URL | |
| | | ''<WRAP important>'' boxes | 5, all rendered as boxes | |
| | | Footnotes | 20 | |
| | | Tables | 10 on the content page, 8 here | |
| | | Downloadable ''foxhound_flows.py'' snippet | present | |
| | | Literal ''%%{[%%'' sequences left unrendered in prose | 0 | |
| | | Real citations on this provenance page | **0**, as intended; the one textual match is the words "bibtex_citekey" in the prose above | |
| | | ''%%{[key]}%%'' written as a literal example here | escaped with ''%%%%'', renders as text | |
| | | Unfilled double-brace marker slots in this page | 0 — and ''scripts/build_provenance_foxhound.py'' refuses to build if a slot has no content. It fired on the first draft of this very row, which wrote the marker syntax out literally; that is the guard working, and the row is now phrased so it cannot trip it | |
| | | Both ''<file text>'' blocks | render as downloadable snippets with the full output inline | |
| | | Reachability | the content page was already linked from ''programming:crawler'' and ''privacy:javascript'' before it existed, so no new inbound link was needed; the two pages link each other, and this one links [[literature:corpus]] | |
| | |
| | Guards run against the final text, all passing: ''check_tables.mjs'' on both pages, ''check_page_numbers.mjs'' on the content page both windowed and whole-page and with ''%%--code%%'', ''check_attributions.mjs'' (9 attributions), a citekey-resolution check, a byte-comparison of the embedded script against ''pages/foxhound_flows.py'', and the script's own 19-check self-test. |
| |
| ====== References ====== | ====== References ====== |
