Differences

This shows you the differences between two versions of the page.

--- provenance:programming:crawler:foxhound [2026/08/17 17:50] – Record the third code bug, found after the review rounds by writing an adversarial stress harness against the rewritten reducer, and the one limitation a name heuristic cannot fix. Refresh the self-test output (25 checks) and the byte comparison. Authored karel.kubicek.claude
+++ provenance:programming:crawler:foxhound [2026/09/17 07:41] (current) – Markup rendering sweep correction: escape residual CLI flags. Authored by Claude. karel.kubicek.claude
@@ Line 112: / Line 112: @@
 Three separate checks, all in the report:
-  * **Schema evidence quotes** (section G): the 9 ''tools[].evidence.quote'' values, normalised for whitespace and line-break hyphens, then by five-word windows — **5 exact, 4 partial (≥60% of windows), 0 below threshold, 0 without full text.** The four partials are column splices in the repaired rendering, not unsupported claims. Cross-checked with the shared bulk verifier, which is a separate implementation: ''node scripts/quote_check.mjs --tools 'fox.?hound' --show'' returns the identical 5/4/0/0.
+  * **Schema evidence quotes** (section G): the 9 ''tools[].evidence.quote'' values, normalised for whitespace and line-break hyphens, then by five-word windows — **5 exact, 4 partial (≥60% of windows), 0 below threshold, 0 without full text.** The four partials are column splices in the repaired rendering, not unsupported claims. Cross-checked with the shared bulk verifier, which is a separate implementation: ''node scripts/quote_check.mjs %%--tools%% 'fox.?hound' %%--show%%'' returns the identical 5/4/0/0.
   * **Hand deciding quotes** (also section G): the 13 sentences in the ''ROLE'' map — **13 exact, 0 not exact**, as the report says. Two of them are exact only because they were //shortened// during this run to a contiguous fragment: the CCS 2025 extension sentence and the NDSS 2026 baseline sentence are each spliced across two columns in ''paper.cols.txt'', and the longer versions I first wrote matched at 50% and 60%. Both are marked with an inline comment in ''fh_fold.mjs'' saying why they are short, so a later reader does not "restore" them. (An earlier draft of this page said "11 exact, 2 not", describing the state before the shortening; the citations reviewer caught the mismatch against the artefact.)
   * **Per-paper figures and quoted sentences used on the page** (section H): **50 needles, 0 not located** verbatim in ''paper.cols.txt''. This check exists because ''detection[].prevalence'' is a model summary rather than a quotation — the same reason ''verify_requests_figures.mjs'' exists. It started at 31 needles and grew to 50 on the figures reviewer's finding that the comparison table's per-tool numbers were disclosed as hand-checked but not machine-checked; adding them also caught three of my own needles that were column-spliced and would have read as failures.
@@ Line 141: / Line 141: @@
 | 6.1 GiB checkout, GPL-3.0, repo created 2021-06-04 | GitHub repos API | API fetch |
 | TU Braunschweig hosts 31 per-commit Linux builds under Debian Bookworm, newest ''1bb8dbd6'' (2026-05-19), with SHA-256 checksums and the matching Playwright version | [[https://foxhound.ias.tu-bs.de/|foxhound.ias.tu-bs.de]] | HTML fetch with a browser User-Agent, then regex extraction of commits, dates and Playwright versions |
-| ''report.detail'' field list; the worked taint-flow JSON; "one release behind Playwright"; the gkrust OOM; the Playwright patch workflow constraints; the git-history resync | the project wiki, cloned as a git repository (''project-foxhound.wiki.git'') and the README | ''git clone --depth 1'', then read; the probe asserts nine specific README substrings still exist |
+| ''report.detail'' field list; the worked taint-flow JSON; "one release behind Playwright"; the gkrust OOM; the Playwright patch workflow constraints; the git-history resync | the project wiki, cloned as a git repository (''project-foxhound.wiki.git'') and the README | ''git clone %%--depth%% 1'', then read; the probe asserts nine specific README substrings still exist |
 | The JIT / best-effort caveat | ''taint/docs/TaintFlow.md'' on ''main'' | raw fetch, and the probe greps the two sentences |
 | Klein et al., EuroS&P 2022 is the requested citation | README ''Cite us!'' section | probe asserts the BibTeX key ''KleBarBen'' is still present |
@@ Line 161: / Line 161: @@
 <code>
 $ python3 pages/foxhound_flows.py --selftest
-selftest: 25 checks passed
+selftest: 31 checks passed
 flows: 1   sites: 1   pages: 1   scripts: 1
@@ Line 173: / Line 173: @@
 </code>
-The copy embedded in the page's ''<file python>'' block was byte-compared against ''pages/foxhound_flows.py'' before publication (21,889 bytes each, identical), because a published script that has drifted from the one that was tested is worse than no script:
+The copy embedded in the page's ''%%<file python>%%'' block was byte-compared against ''pages/foxhound_flows.py'' before publication (24,456 bytes each, identical — a real ''wc -c'' byte count, not Python's ''len()'', which counts code points and was what an earlier draft printed), because a published script that has drifted from the one that was tested is worse than no script:
 <code bash>
@@ Line 226: / Line 226: @@
 </code>
-''check_page_numbers.mjs'' passes both windowed and whole-page, and with ''--code''. Getting it to pass required a ''Z2'' block in the report listing every literal on the page whose source is not the corpus — the build minutes, the version pins, the pref counts, the per-paper figures from cited papers, and the tokenizer artefacts (''22.04'' yielding ''2.0''; the line numbers inside the fixture JSON). That block is the audit trail for every number the corpus cannot account for, and it is why the guard is a guard rather than a formality.
+''check_page_numbers.mjs'' passes both windowed and whole-page, and with ''%%--code%%''. Getting it to pass required a ''Z2'' block in the report listing every literal on the page whose source is not the corpus — the build minutes, the version pins, the pref counts, the per-paper figures from cited papers, and the tokenizer artefacts (''22.04'' yielding ''2.0''; the line numbers inside the fixture JSON). That block is the audit trail for every number the corpus cannot account for, and it is why the guard is a guard rather than a formality.
 ''check_attributions.mjs'' reported "0 attributions checked" against this page on first run, because its regex required the crawler-page style ''%%| Name et al., VENUE YEAR, //Title// {[key]} |%%'' and this page's tables omit the title. The regex was generalised to accept either form; both this page (9 attributions) and ''programming:crawler:pagegraph'' (8) pass afterwards. **"0 checked" is not a pass** — the guard says so itself, and this is the second page on which that message was the actual finding.
@@ Line 652: / Line 652: @@
 taintfox.escapeURL, default false, line 116; 0 occurrences of "taintfox" in all.js | netwerk/base/nsStandardURL.cpp and all.js
 -> 347 confirmed DOM-XSS flows, i.e. 5x | Sabino et al., NDSS 2026, already in section H
- self-test checks, 10 mutations of the documented fixture | python3 pages/foxhound_flows.py --selftest
+ self-test checks, 13 mutations of the documented fixture | python3 pages/foxhound_flows.py --selftest
+UTF-16 code units: 13 for a two-emoji fixture, offsets 4..6 | the utf16_slice self-test in pages/foxhound_flows.py
 PR #198 touched 36 files | the PR object's changed_files field. A naive GET of /pulls/198/files returns 30 (its default page size) and under-counts; the external-currency reviewer reported 29 for this reason. The probe asks for per_page=50 and prints changed_files beside it.
 , TRK-a55bd7c6, 11-16, 22 characters, 5 characters | strings from the documented worked example and from a cited paper — not figures
@@ Line 1193: / Line 1194: @@
 Everything else it checked came back verbatim, including the ''taintfox.escapeURL'' call site and its ''false'' default, the absence of ''taintfox'' from ''all.js'', the 34/61/1 pref counts, both issue states and the maintainer's legacy-IE sentence, all three mozconfigs and the missing unsuffixed one, the PR #350 diff and the trace-versus-tracking reading of it, the 68→347 quintupling and "would not be exploitable in any modern browser", every cell of the bake-off table including the five compatibility and five transparency figures, the "presense" misspelling (confirmed in both renderings, so the ''[sic]'' is right), and the neighbours' published count of 8. It also noted that for the NDSS 2026 paper ''paper.cols.txt'' is the well-ordered rendering and ''paper.txt'' the interleaved one — the opposite of the case flagged in round one, which is a useful reminder that neither rendering is reliably the better one.
+The figures-and-code re-review returned last and found **two more real defects in the published script**, both accepted, plus one confirmation and one cosmetic fix:
+^ # ^ Finding ^ Disposition ^
+| 1 | **''begin''/''end'' are UTF-16 code-unit offsets, and the script sliced with them in Python, which indexes by code point.** One astral character earlier in the string — an emoji, some CJK extensions — shifts every later index, so the wrong substring is read. The reviewer built the case: on ''"\\U0001F600\\U0001F600<>PADDING"'' the engine's offsets 4..6 bound ''"<>"'', while ''value[4:6]'' returns ''"PA"'', and the flow is therefore reported as having held no syntax character. A **false negative on exactly the screen the script exists to provide**. | **Accepted; reproduced first, then fixed.** ''utf16_slice()'' and ''utf16_len()'' do the slicing in the engine's own unit, four regression tests cover it (including one asserting what the naive Python slice //would// have returned, so the bug cannot come back unnoticed), and the content page now states the offsets are UTF-16 and names the languages this bites in. |
+| 2 | ''iframe.srcdoc'' was missing from the HTML/JS sink set although its content is parsed as a whole document, so ''no_syntax_chars'' could never fire for it — on the same page that calls out ''iframe.srcdoc'' as a sink a 2024 paper found missing. | **Accepted.** Added, along with ''script.textContent''. The set now also carries an explicit note that navigation sinks (''location.href'', ''a.href'', ''window.open'') are **deliberately excluded**, because they are dangerous through the ''javascript:'' scheme rather than through markup syntax, and a screen that does not model that should stay silent about them rather than look reassuring. A test asserts both behaviours. |
+| 3 | The weak ''"13"'' needle in section H, which occurs 26 times in that paper for unrelated reasons. | **Accepted as confirmation** — independently caught and fixed during the same window; see above. |
+| 4 | The provenance page said "20,338 bytes each" for the byte-comparison, but that number is Python's ''len()'' — code points. The file has em dashes, so the real ''wc -c'' differs. | **Accepted.** The claim of identity was right; the unit was wrong, and the figure is now a real byte count. |
+**A process defect on my side, recorded because it degraded the review:** this reviewer reported that the files under review changed on disk while it worked, because I was applying the other reviewers' fixes in parallel instead of freezing the tree. It anchored its findings to the snapshot it started from and flagged the drift, which is the right response — but it should not have had to. The next run should copy the artefacts to a frozen directory and point reviewers at that.
 ===== Rendering check =====
@@ Line 1203: / Line 1214: @@
 | Distinct citekeys on the content page | **16**, all resolving |
 | Reference list at the foot of the content page | **16 entries**, each with author list, venue and a DOI or venue URL |
-| ''<WRAP important>'' boxes | 5, all rendered as boxes |
+| ''%%<WRAP important>%%'' boxes | 5, all rendered as boxes |
 | Footnotes | 20 |
 | Tables | 10 on the content page, 8 here |
@@ Line 1211: / Line 1222: @@
 | ''%%{[key]}%%'' written as a literal example here | escaped with ''%%%%'', renders as text |
 | Unfilled double-brace marker slots in this page | 0 — and ''scripts/build_provenance_foxhound.py'' refuses to build if a slot has no content. It fired on the first draft of this very row, which wrote the marker syntax out literally; that is the guard working, and the row is now phrased so it cannot trip it |
-| Both ''<file text>'' blocks | render as downloadable snippets with the full output inline |
+| Both ''%%<file text>%%'' blocks | render as downloadable snippets with the full output inline |
 | Reachability | the content page was already linked from ''programming:crawler'' and ''privacy:javascript'' before it existed, so no new inbound link was needed; the two pages link each other, and this one links [[literature:corpus]] |
@@ Line 1219: / Line 1230: @@
 <bibtex bibliography></bibtex>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.