| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:privacy:javascript [2026/09/17 07:23] – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude | provenance:privacy:javascript [2026/09/21 14:32] (current) – Foxhound count errata 2026-09-21: Foxhound 8 -> 9 (4.4%), no-tool 120 -> 119, the js_fold.mjs override that produced it. Authored by Claude karel.kubicek.claude |
|---|
| | Esprima (tool, in population) | 16 (10.0%) | 21 (10.2%) | | | Esprima (tool, in population) | 16 (10.0%) | 21 (10.2%) | |
| | OpenWPM (tool, in population) | 17 (10.6%) | 19 (9.2%) | | | OpenWPM (tool, in population) | 17 (10.6%) | 19 (9.2%) | |
| | **Project Foxhound** | **2 (1.3%)** | **8 (3.9%)** | | | **Project Foxhound** | **2 (1.3%)** | **9 (4.4%)**((Published as 8 (3.9%) until the Foxhound count errata of 2026-09-21, at the foot of this page.)) | |
| | **PageGraph** | **4 (2.5%)** | **8 (3.9%)** | | | **PageGraph** | **4 (2.5%)** | **8 (3.9%)** | |
| | Jalangi | 1 | 4 | | | Jalangi | 1 | 4 | |
| | Names no JS-analysis tool at all | 91 of 160 (56.9%) | 120 of 206 (58.3%) | | | Names no JS-analysis tool at all | 91 of 160 (56.9%) | 119 of 206 (57.8%)((Same errata: published as 120 of 206 (58.3%) until 2026-09-21.)) | |
| | ''heuristic-rules'' | 80 (52.3%) | 108 (54.5%) | | | ''heuristic-rules'' | 80 (52.3%) | 108 (54.5%) | |
| | ''dynamic-analysis'' | 21 (13.7%) | 33 (16.7%) | | | ''dynamic-analysis'' | 21 (13.7%) | 33 (16.7%) | |
| **Moved by more than a rounding step:** | **Moved by more than a rounding step:** |
| |
| * **Project Foxhound 2 → 8 and PageGraph 4 → 8.** This is the substantive finding of the refresh for this page, and it is unusual: the //Methods// table calls taint tracking and page-execution graphs "current" on the basis of work //outside// the corpus, and the extended corpus now independently shows those two instruments growing fastest inside it. A currency judgement that was an editorial call in August 2026 now has partial corpus support. The page says so. | * **Project Foxhound 2 → 9 and PageGraph 4 → 8.**((The Foxhound figure read 8 when this section was written; see the errata of 2026-09-21 below. The 2 is the pre-extension corpus and is unaffected — the paper the errata adds is from NDSS 2025, which that corpus did not reach.)) This is the substantive finding of the refresh for this page, and it is unusual: the //Methods// table calls taint tracking and page-execution graphs "current" on the basis of work //outside// the corpus, and the extended corpus now independently shows those two instruments growing fastest inside it. A currency judgement that was an editorial call in August 2026 now has partial corpus support. The page says so. |
| * **Esprima overtook OpenWPM** as the most-named tool in the population, 21 to 19. Small numbers; reported as a ranking. | * **Esprima overtook OpenWPM** as the most-named tool in the population, 21 to 19. Small numbers; reported as a ranking. |
| * **The ''llm'' classification method fired for the first time, twice.** Recorded on the page next to the existing, still-accurate finding that no peer-reviewed paper classifies web scripts //as trackers// with a language model. Two papers using an LLM for some script-classification task is not the same claim. | * **The ''llm'' classification method fired for the first time, twice.** Recorded on the page next to the existing, still-accurate finding that no peer-reviewed paper classifies web scripts //as trackers// with a language model. Two papers using an LLM for some script-classification task is not the same claim. |
| ===== 6. Quotes checked ===== | ===== 6. Quotes checked ===== |
| |
| //Recorded, 2026-08-12.// | //Recorded, 2026-08-12. Re-run 2026-09-21 with the PDF fallback — see //Quote-check refresh, 2026-09-21// at the foot of this page.// |
| |
| <code> | <code> |
| $ node scripts/quote_check.mjs --classification javascript | $ node scripts/quote_check.mjs --classification javascript --show 400 |
| 66 quotes checked: 34 exact, 19 partial (>=60% of 5-word windows), | 66 quotes checked: 34 exact, 19 partial (>=60% of 5-word windows), 12 rescued from the PDF, 1 below threshold in both renderings, 0 with no full text on disk. |
| 13 below threshold, 0 with no full text on disk. | |
| </code> | </code> |
| |
| Five of the 13 below-threshold quotes were read by hand against ''paper.cols.txt'' — ''2013/USENIX/revolver'', ''2017/IMC/the-ad-wars'', ''2017/PETS/towards-seamless-tracking-free-web'', ''2011/CCS/fortifying-web-based-applications'', ''2012/USENIX/privilege-separation-in-html5-applications'' — and **all five are present**. The last is a good example of why the automated test cannot be trusted alone; the source reads: | **The old figure was 13 below threshold. Twelve of the 13 are a defect in the stored text, not in the extraction** — they are located in an independent ''pypdf'' rendering of the same ''paper.pdf''. **One** quote is below threshold in both renderings: ''IEEE-SP/2012/rozzle-de-cloaking-internet-malware'' at 58%, //"The set contains 2,373 JavaScript files that were flagged by Zozzle, a static malicious JavaScript detector"//, which has **not** been read by hand. ''exact'' (34) and ''partial'' (19) did not move. |
| | |
| | Five of the then-13 were read by hand against ''paper.cols.txt'' — ''2013/USENIX/revolver'', ''2017/IMC/the-ad-wars'', ''2017/PETS/towards-seamless-tracking-free-web'', ''2011/CCS/fortifying-web-based-applications'', ''2012/USENIX/privilege-separation-in-html5-applications'' — and **all five are present**. **All five are also among the 12 the fallback rescues**, which is the strongest agreement between hand and script of any page in this sweep. The last is a good example of why the automated test could not be trusted alone; the source reads: |
| |
| <code> | <code> |
| </code> | </code> |
| |
| The extraction's "we (conservatively) identify all calls to privileged APIs (i.e., calls to the chrome object)" is correct; the two columns interleave inside the parenthesis. **The other 8 were not read.** | The extraction's "we (conservatively) identify all calls to privileged APIs (i.e., calls to the chrome object)" is correct; the two columns interleave inside the parenthesis — which is exactly the failure the PDF fallback undoes. **The other 8 were not read, and 7 of those 8 are now rescued; the unread remainder is the single Rozzle quote above.** |
| |
| //Reconstructed:// the original run recorded that "each number reproduced on this page was re-located in ''paper.cols.txt'' after whitespace normalisation", which is a stronger claim than the one above and applies to the //quoted prevalence figures// rather than to the schema tuples. Which numbers is not recoverable. One specific check from that run **is** recorded and is worth repeating here because it is the page's best cautionary tale: | //Reconstructed:// the original run recorded that "each number reproduced on this page was re-located in ''paper.cols.txt'' after whitespace normalisation", which is a stronger claim than the one above and applies to the //quoted prevalence figures// rather than to the schema tuples. Which numbers is not recoverable. One specific check from that run **is** recorded and is worth repeating here because it is the page's best cautionary tale: |
| |
| Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| | |
| | ===== Quote-check refresh, 2026-09-21 ===== |
| | |
| | The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs'' was patched the same day to re-check everything below threshold against that second rendering and report a fourth verdict, **RESCUED**. §6's figure predates the patch. Re-run, unedited first line: |
| | |
| | <code> |
| | $ node scripts/quote_check.mjs --classification javascript --show 400 |
| | 66 quotes checked: 34 exact, 19 partial (>=60% of 5-word windows), 12 rescued from the PDF, 1 below threshold in both renderings, 0 with no full text on disk. |
| | </code> |
| | |
| | ^ Figure ^ Was ^ Is ^ Why ^ |
| | | quotes checked | 66 | 66 | population unchanged — the corpus has not moved | |
| | | exact | 34 | 34 | unchanged | |
| | | partial (≥60% of 5-word windows) | 19 | 19 | unchanged | |
| | | rescued from the PDF | — | **12** | new verdict; these were inside the old 13 | |
| | | below threshold | **13** | **1** (in both renderings) | 13 = 12 + 1 exactly; nothing else moved | |
| | | below-threshold rate | 19.7% | **1.5%** | 1 of 66 | |
| | | unread below-threshold quotes | 8 | **1** | the ''rozzle-de-cloaking-internet-malware'' Zozzle quote | |
| | |
| | **Small population, so read the count and not the percentage.** 66 quotes is the smallest population in this sweep apart from the 9 Foxhound quotes, and a single quote is 1.5 points of it. What this run establishes is that **one** quote on this page is below threshold in both renderings and has not been read by hand — not that the page's quote-failure rate is 1.5%. |
| | |
| | **What this does and does not say.** It does not say 12 extractions were wrong and are now right — the quotes were always in the papers, and 5 of the 12 had already been read by hand and found present. It says the //stored text// could not locate them and a second rendering of the same PDF can. |
| | |
| | **Scope of this edit.** §6 only. ''report_javascript.mjs'' was **not** re-run in this pass; the tool table, the phenomenon fold, the 44-of-198 ground-truth figure, the PoPETs 2023 transposed-label reading and every citation stand as published. [[:privacy:javascript]] carries no quote-check figure, so it was not edited. |
| | |
| | ^ Item ^ Value ^ |
| | | Date | 2026-09-21, unsupervised | |
| | | Command | ''%%node scripts/quote_check.mjs --classification javascript --show 400%%'' | |
| | | Artifact | ''out/qc0921/class_javascript.txt'' (full run, 12 RESCUED rows and the 1 below-threshold row listed) | |
| | | Script changes | none — ''quote_check.mjs'' was already patched on 2026-09-04 | |
| | | Reviewers | one ''sonnet'' figures-vs-script pass | |
| | | Pages saved | this page only | |
| | |
| | ===== Foxhound count errata, 2026-09-21 ===== |
| | |
| | **What was wrong.** The toolchain table published ''Project Foxhound (taint tracking) | 8 | 3.9%'', and the sentence below it read "went from 2 papers to 8". The per-paper role audit on [[:programming:crawler:foxhound]] (''scripts/report_foxhound.mjs'' + ''scripts/fh_fold.mjs'', 2026-08-17) finds **9 papers in the corpus that actually ran the browser**, and its footnote named this page's table as one of the two to correct. |
| | |
| | **Why.** ''jsSignals()'' counts a tool only when the tuple says ''used'' or ''produced''. Khodayari et al.'s NDSS 2025 open-redirection paper names Foxhound as ''compared'' — //"We considered the following state-of-the-art detectors as baselines: dynamic taint-tracking [24, 36, 45] using Foxhound [44, 45]"// — and ran it over 42,288 webpages. It is in this page's 206-paper population already, on its ''detection'' and ''classification'' signals, so the only thing that was wrong is that it named no tool. |
| | |
| | **The fix, and the two things it deliberately does not do.** ''js_fold.mjs'' now exports ''TOOL_ROLE_OVERRIDE'', a hand map of papers whose tool role has been read and published by a named audit — currently one entry, citing the Foxhound ''ROLE'' map and the sentence it was decided on. It adds to ''tools'' only, **not** to ''strictTools'' and therefore **not** to ''reasons'': a hand entry must not be able to pull a paper into the 206. And it is not a general rule that ''compared'' means "ran it" — that needs the sentence, paper by paper. ''jsSignals()'' **throws** if the overridden tuple disappears from the extraction or is re-extracted as ''used''/''produced'', so a redundant hand entry cannot sit there unnoticed. |
| | |
| | <code> |
| | $ node scripts/report_javascript.mjs |
| | --- before |
| | +++ after |
| | -Project Foxhound (taint tracking) 8 3.9% |
| | +Project Foxhound (taint tracking) 9 4.4% |
| | -Papers naming no JavaScript-analysis tool at all: 120 of 206 (58.3%). |
| | +Papers naming no JavaScript-analysis tool at all: 119 of 206 (57.8%). |
| | </code> |
| | |
| | **Those two lines are the entire diff of the report.** The 206-paper population, the signal-combination table, the venue and period tables, the research-family fold and its 65-tuple residue, every classification, ground-truth, validation and crawl-config figure and every citation are byte-identical before and after. |
| | |
| | ^ Figure ^ Was ^ Is ^ |
| | | Project Foxhound, toolchain table | 8 (3.9%) | **9 (4.4%)** — and the row now sorts above PageGraph and VisibleV8 | |
| | | Papers naming no JS-analysis tool | 120 of 206 (58.3%) | **119 of 206 (57.8%)** | |
| | | "went from 2 papers to 8" | 8 | **9** | |
| | | PageGraph | 8 (3.9%) | 8 (3.9%), unchanged | |
| | |
| | The published ''%%<file javascript js_fold.mjs>%%'' block on [[:privacy:javascript]] was regenerated from the committed file in the same save, so the code the page shows is the code that produced the table, including the new override and its assertion. |
| | |
| | ^ Item ^ Value ^ |
| | | Date | 2026-09-21, unsupervised | |
| | | Script changes | ''scripts/js_fold.mjs'' (''TOOL_ROLE_OVERRIDE'' and its two assertions in ''jsSignals'') | |
| | | Reviewers | one ''sonnet'' figures-vs-script pass; one ''sonnet'' citations/quotes pass | |
| | | Pages saved | this page, [[:privacy:javascript]] | |
| |
