| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:privacy:javascript [2026/09/03 22:18] – Close the target=other caveat with the targetDetail probe a reviewer pointed out was available (16 hits, none a script or tracker classification), and record the second independent outside search. Authored by Claude karel.kubicek.claude | provenance:privacy:javascript [2026/09/21 14:32] (current) – Foxhound count errata 2026-09-21: Foxhound 8 -> 9 (4.4%), no-tool 120 -> 119, the js_fold.mjs override that produced it. Authored by Claude karel.kubicek.claude |
|---|
| ^ Item ^ Value ^ | ^ Item ^ Value ^ |
| | Content page | [[privacy:javascript]] | | | Content page | [[privacy:javascript]] | |
| | Report script | ''scripts/report_javascript.mjs'' (''--wiki'' for DokuWiki tables, ''--list'' for the full paper list) | | | Report script | ''scripts/report_javascript.mjs'' (''%%--wiki%%'' for DokuWiki tables, ''%%--list%%'' for the full paper list) | |
| | Population rule and folds | ''scripts/js_fold.mjs'' | | | Population rule and folds | ''scripts/js_fold.mjs'' | |
| | Quote verification | ''scripts/quote_check.mjs --classification javascript'' | | | Quote verification | ''scripts/quote_check.mjs %%--classification%% javascript'' | |
| | Data | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues, 2010–2026 | | | Data | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 7 venues, 2010–2026 | |
| | Refreshed | 2026-08-12 | | | Refreshed | 2026-08-12 | |
| | Esprima (tool, in population) | 16 (10.0%) | 21 (10.2%) | | | Esprima (tool, in population) | 16 (10.0%) | 21 (10.2%) | |
| | OpenWPM (tool, in population) | 17 (10.6%) | 19 (9.2%) | | | OpenWPM (tool, in population) | 17 (10.6%) | 19 (9.2%) | |
| | **Project Foxhound** | **2 (1.3%)** | **8 (3.9%)** | | | **Project Foxhound** | **2 (1.3%)** | **9 (4.4%)**((Published as 8 (3.9%) until the Foxhound count errata of 2026-09-21, at the foot of this page.)) | |
| | **PageGraph** | **4 (2.5%)** | **8 (3.9%)** | | | **PageGraph** | **4 (2.5%)** | **8 (3.9%)** | |
| | Jalangi | 1 | 4 | | | Jalangi | 1 | 4 | |
| | Names no JS-analysis tool at all | 91 of 160 (56.9%) | 120 of 206 (58.3%) | | | Names no JS-analysis tool at all | 91 of 160 (56.9%) | 119 of 206 (57.8%)((Same errata: published as 120 of 206 (58.3%) until 2026-09-21.)) | |
| | ''heuristic-rules'' | 80 (52.3%) | 108 (54.5%) | | | ''heuristic-rules'' | 80 (52.3%) | 108 (54.5%) | |
| | ''dynamic-analysis'' | 21 (13.7%) | 33 (16.7%) | | | ''dynamic-analysis'' | 21 (13.7%) | 33 (16.7%) | |
| **Moved by more than a rounding step:** | **Moved by more than a rounding step:** |
| |
| * **Project Foxhound 2 → 8 and PageGraph 4 → 8.** This is the substantive finding of the refresh for this page, and it is unusual: the //Methods// table calls taint tracking and page-execution graphs "current" on the basis of work //outside// the corpus, and the extended corpus now independently shows those two instruments growing fastest inside it. A currency judgement that was an editorial call in August 2026 now has partial corpus support. The page says so. | * **Project Foxhound 2 → 9 and PageGraph 4 → 8.**((The Foxhound figure read 8 when this section was written; see the errata of 2026-09-21 below. The 2 is the pre-extension corpus and is unaffected — the paper the errata adds is from NDSS 2025, which that corpus did not reach.)) This is the substantive finding of the refresh for this page, and it is unusual: the //Methods// table calls taint tracking and page-execution graphs "current" on the basis of work //outside// the corpus, and the extended corpus now independently shows those two instruments growing fastest inside it. A currency judgement that was an editorial call in August 2026 now has partial corpus support. The page says so. |
| * **Esprima overtook OpenWPM** as the most-named tool in the population, 21 to 19. Small numbers; reported as a ranking. | * **Esprima overtook OpenWPM** as the most-named tool in the population, 21 to 19. Small numbers; reported as a ranking. |
| * **The ''llm'' classification method fired for the first time, twice.** Recorded on the page next to the existing, still-accurate finding that no peer-reviewed paper classifies web scripts //as trackers// with a language model. Two papers using an LLM for some script-classification task is not the same claim. | * **The ''llm'' classification method fired for the first time, twice.** Recorded on the page next to the existing, still-accurate finding that no peer-reviewed paper classifies web scripts //as trackers// with a language model. Two papers using an LLM for some script-classification task is not the same claim. |
| ===== 6. Quotes checked ===== | ===== 6. Quotes checked ===== |
| |
| //Recorded, 2026-08-12.// | //Recorded, 2026-08-12. Re-run 2026-09-21 with the PDF fallback — see //Quote-check refresh, 2026-09-21// at the foot of this page.// |
| |
| <code> | <code> |
| $ node scripts/quote_check.mjs --classification javascript | $ node scripts/quote_check.mjs --classification javascript --show 400 |
| 66 quotes checked: 34 exact, 19 partial (>=60% of 5-word windows), | 66 quotes checked: 34 exact, 19 partial (>=60% of 5-word windows), 12 rescued from the PDF, 1 below threshold in both renderings, 0 with no full text on disk. |
| 13 below threshold, 0 with no full text on disk. | |
| </code> | </code> |
| |
| Five of the 13 below-threshold quotes were read by hand against ''paper.cols.txt'' — ''2013/USENIX/revolver'', ''2017/IMC/the-ad-wars'', ''2017/PETS/towards-seamless-tracking-free-web'', ''2011/CCS/fortifying-web-based-applications'', ''2012/USENIX/privilege-separation-in-html5-applications'' — and **all five are present**. The last is a good example of why the automated test cannot be trusted alone; the source reads: | **The old figure was 13 below threshold. Twelve of the 13 are a defect in the stored text, not in the extraction** — they are located in an independent ''pypdf'' rendering of the same ''paper.pdf''. **One** quote is below threshold in both renderings: ''IEEE-SP/2012/rozzle-de-cloaking-internet-malware'' at 58%, //"The set contains 2,373 JavaScript files that were flagged by Zozzle, a static malicious JavaScript detector"//, which has **not** been read by hand. ''exact'' (34) and ''partial'' (19) did not move. |
| | |
| | Five of the then-13 were read by hand against ''paper.cols.txt'' — ''2013/USENIX/revolver'', ''2017/IMC/the-ad-wars'', ''2017/PETS/towards-seamless-tracking-free-web'', ''2011/CCS/fortifying-web-based-applications'', ''2012/USENIX/privilege-separation-in-html5-applications'' — and **all five are present**. **All five are also among the 12 the fallback rescues**, which is the strongest agreement between hand and script of any page in this sweep. The last is a good example of why the automated test could not be trusted alone; the source reads: |
| |
| <code> | <code> |
| </code> | </code> |
| |
| The extraction's "we (conservatively) identify all calls to privileged APIs (i.e., calls to the chrome object)" is correct; the two columns interleave inside the parenthesis. **The other 8 were not read.** | The extraction's "we (conservatively) identify all calls to privileged APIs (i.e., calls to the chrome object)" is correct; the two columns interleave inside the parenthesis — which is exactly the failure the PDF fallback undoes. **The other 8 were not read, and 7 of those 8 are now rescued; the unread remainder is the single Rozzle quote above.** |
| |
| //Reconstructed:// the original run recorded that "each number reproduced on this page was re-located in ''paper.cols.txt'' after whitespace normalisation", which is a stronger claim than the one above and applies to the //quoted prevalence figures// rather than to the schema tuples. Which numbers is not recoverable. One specific check from that run **is** recorded and is worth repeating here because it is the page's best cautionary tale: | //Reconstructed:// the original run recorded that "each number reproduced on this page was re-located in ''paper.cols.txt'' after whitespace normalisation", which is a stronger claim than the one above and applies to the //quoted prevalence figures// rather than to the schema tuples. Which numbers is not recoverable. One specific check from that run **is** recorded and is worth repeating here because it is the page's best cautionary tale: |
| * **Whether the 8 unread below-threshold quotes check out.** See §6. | * **Whether the 8 unread below-threshold quotes check out.** See §6. |
| * **Whether the per-1,000 decline is real.** The series peaks at 44.2 in 2014–2017 and falls to 24.5 in 2025–2026, but the last point is provisional and the page's own explanation — the topic being absorbed into tracking and supply-chain papers that no longer call themselves JavaScript analysis — is an interpretation the extraction cannot test. | * **Whether the per-1,000 decline is real.** The series peaks at 44.2 in 2014–2017 and falls to 24.5 in 2025–2026, but the last point is provisional and the page's own explanation — the topic being absorbed into tracking and supply-chain papers that no longer call themselves JavaScript analysis — is an interpretation the extraction cannot test. |
| * **The false-positive tail of the 206.** An e-voting client audit, a router-attack paper and a PHP injection-sink study are in the population and analyse JavaScript incidentally. They were left in and the page says so; ''--list'' prints them all so a reader can judge. Nobody has audited the list paper by paper on the new corpus. | * **The false-positive tail of the 206.** An e-voting client audit, a router-attack paper and a PHP injection-sink study are in the population and analyse JavaScript incidentally. They were left in and the page says so; ''%%--list%%'' prints them all so a reader can judge. Nobody has audited the list paper by paper on the new corpus. |
| * **Whether ''classification.method'' shares are worth publishing at all.** 58% run-to-run agreement is the weakest field carrying a table on this page. It is labelled as a rough share, which is the most that can honestly be done with it. | * **Whether ''classification.method'' shares are worth publishing at all.** 58% run-to-run agreement is the weakest field carrying a table on this page. It is labelled as a rough share, which is the most that can honestly be done with it. |
| * **ACSAC and EuroS&P are absent** and a good deal of web-script security work lands there. Stated on the page. | * **ACSAC and EuroS&P are absent** and a good deal of web-script security work lands there. Stated on the page. |
| * The matcher was **substring**, not word-boundary, so ''report.includes('59')'' was satisfied by ''11.59 bits''. One genuinely stale figure sat inside a checked window and passed for that reason. | * The matcher was **substring**, not word-boundary, so ''report.includes('59')'' was satisfied by ''11.59 bits''. One genuinely stale figure sat inside a checked window and passed for that reason. |
| |
| Both are fixed in ''scripts/check_page_numbers.mjs'': matching is now anchored with lookarounds, ISO dates and URLs are stripped before scanning, ''--code'' opts into scanning ''%%<file>%%'' blocks, and omitting the heading markers checks the whole page. **Run it windowed //and// whole-page.** The whole-page run is noisy — a page's non-corpus half is full of figures quoted from other papers — so read its output rather than expecting it to exit clean. | Both are fixed in ''scripts/check_page_numbers.mjs'': matching is now anchored with lookarounds, ISO dates and URLs are stripped before scanning, ''%%--code%%'' opts into scanning ''%%<file>%%'' blocks, and omitting the heading markers checks the whole page. **Run it windowed //and// whole-page.** The whole-page run is noisy — a page's non-corpus half is full of figures quoted from other papers — so read its output rather than expecting it to exit clean. |
| Fixed on this page's content page as a result — and one of them is the most instructive defect in the whole refresh: | Fixed on this page's content page as a result — and one of them is the most instructive defect in the whole refresh: |
| |
| * **The embedded ''%%<file javascript js_fold.mjs>%%'' block still quoted the 4,322-corpus counts** — ''program-analysis'' on 959 papers, ''code-or-binary-analysis'' on 1,063, LLVM 66, Soot 61, IDA Pro 36, FlowDroid 34, angr 28, Esprima 15 — two screens below the page's own table saying 1,385 / 1,484 / 101 / 81 / 62 / 48 / 39 / 23. The published, downloadable source contradicted the prose. **''check_page_numbers.mjs'' strips ''%%<file>%%'' blocks and so could never have caught it**; it now takes ''--code''. Fixed in both ''scripts/js_fold.mjs'' and the embedded copy, and the two were diffed afterwards. | * **The embedded ''%%<file javascript js_fold.mjs>%%'' block still quoted the 4,322-corpus counts** — ''program-analysis'' on 959 papers, ''code-or-binary-analysis'' on 1,063, LLVM 66, Soot 61, IDA Pro 36, FlowDroid 34, angr 28, Esprima 15 — two screens below the page's own table saying 1,385 / 1,484 / 101 / 81 / 62 / 48 / 39 / 23. The published, downloadable source contradicted the prose. **''check_page_numbers.mjs'' strips ''%%<file>%%'' blocks and so could never have caught it**; it now takes ''%%--code%%''. Fixed in both ''scripts/js_fold.mjs'' and the embedded copy, and the two were diffed afterwards. |
| * Related-Pages shares **42.4% → 39.8%** and **28.6% → 27.6%**. | * Related-Pages shares **42.4% → 39.8%** and **28.6% → 27.6%**. |
| * "the privacy reader's own family is the largest but is barely a quarter of the field" was wrong against its own table: the largest single family is 29 of 206 = 14.1%. Rewritten to name the two families that together make 53 of 206 = 25.7%, which is what "barely a quarter" was always describing. | * "the privacy reader's own family is the largest but is barely a quarter of the field" was wrong against its own table: the largest single family is 29 of 206 = 14.1%. Rewritten to name the two families that together make 53 of 206 = 25.7%, which is what "barely a quarter" was always describing. |
| | the nearest work is three arXiv preprints and a de-minifier | **incomplete** | TGNN {[xiong2026_tgnn]} (TheWebConf 2026) is peer-reviewed, in-corpus, and was already cited nine times on [[privacy:requests]] | | | the nearest work is three arXiv preprints and a de-minifier | **incomplete** | TGNN {[xiong2026_tgnn]} (TheWebConf 2026) is peer-reviewed, in-corpus, and was already cited nine times on [[privacy:requests]] | |
| |
| ==== 12.2 The two ''llm'' papers, named ==== | ==== 12.2 The two llm papers, named ==== |
| |
| The page's method table counts a paper if it carries **any** ''llm'' classification tuple, whatever the target. In this page's population that is two papers, and the reason the old sentence read as a JavaScript finding is that the table sits under a JavaScript heading. Query: ''node scripts/report_llm_currency.mjs'' and, for the population restriction, ''node scripts/report_javascript.mjs''. | The page's method table counts a paper if it carries **any** ''llm'' classification tuple, whatever the target. In this page's population that is two papers, and the reason the old sentence read as a JavaScript finding is that the table sits under a JavaScript heading. Query: ''node scripts/report_llm_currency.mjs'' and, for the population restriction, ''node scripts/report_javascript.mjs''. |
| |
| [[privacy:javascript|← back to the content page]] · [[literature:corpus|corpus-level provenance]] | [[privacy:javascript|← back to the content page]] · [[literature:corpus|corpus-level provenance]] |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| | |
| | ===== Quote-check refresh, 2026-09-21 ===== |
| | |
| | The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs'' was patched the same day to re-check everything below threshold against that second rendering and report a fourth verdict, **RESCUED**. §6's figure predates the patch. Re-run, unedited first line: |
| | |
| | <code> |
| | $ node scripts/quote_check.mjs --classification javascript --show 400 |
| | 66 quotes checked: 34 exact, 19 partial (>=60% of 5-word windows), 12 rescued from the PDF, 1 below threshold in both renderings, 0 with no full text on disk. |
| | </code> |
| | |
| | ^ Figure ^ Was ^ Is ^ Why ^ |
| | | quotes checked | 66 | 66 | population unchanged — the corpus has not moved | |
| | | exact | 34 | 34 | unchanged | |
| | | partial (≥60% of 5-word windows) | 19 | 19 | unchanged | |
| | | rescued from the PDF | — | **12** | new verdict; these were inside the old 13 | |
| | | below threshold | **13** | **1** (in both renderings) | 13 = 12 + 1 exactly; nothing else moved | |
| | | below-threshold rate | 19.7% | **1.5%** | 1 of 66 | |
| | | unread below-threshold quotes | 8 | **1** | the ''rozzle-de-cloaking-internet-malware'' Zozzle quote | |
| | |
| | **Small population, so read the count and not the percentage.** 66 quotes is the smallest population in this sweep apart from the 9 Foxhound quotes, and a single quote is 1.5 points of it. What this run establishes is that **one** quote on this page is below threshold in both renderings and has not been read by hand — not that the page's quote-failure rate is 1.5%. |
| | |
| | **What this does and does not say.** It does not say 12 extractions were wrong and are now right — the quotes were always in the papers, and 5 of the 12 had already been read by hand and found present. It says the //stored text// could not locate them and a second rendering of the same PDF can. |
| | |
| | **Scope of this edit.** §6 only. ''report_javascript.mjs'' was **not** re-run in this pass; the tool table, the phenomenon fold, the 44-of-198 ground-truth figure, the PoPETs 2023 transposed-label reading and every citation stand as published. [[:privacy:javascript]] carries no quote-check figure, so it was not edited. |
| | |
| | ^ Item ^ Value ^ |
| | | Date | 2026-09-21, unsupervised | |
| | | Command | ''%%node scripts/quote_check.mjs --classification javascript --show 400%%'' | |
| | | Artifact | ''out/qc0921/class_javascript.txt'' (full run, 12 RESCUED rows and the 1 below-threshold row listed) | |
| | | Script changes | none — ''quote_check.mjs'' was already patched on 2026-09-04 | |
| | | Reviewers | one ''sonnet'' figures-vs-script pass | |
| | | Pages saved | this page only | |
| | |
| | ===== Foxhound count errata, 2026-09-21 ===== |
| | |
| | **What was wrong.** The toolchain table published ''Project Foxhound (taint tracking) | 8 | 3.9%'', and the sentence below it read "went from 2 papers to 8". The per-paper role audit on [[:programming:crawler:foxhound]] (''scripts/report_foxhound.mjs'' + ''scripts/fh_fold.mjs'', 2026-08-17) finds **9 papers in the corpus that actually ran the browser**, and its footnote named this page's table as one of the two to correct. |
| | |
| | **Why.** ''jsSignals()'' counts a tool only when the tuple says ''used'' or ''produced''. Khodayari et al.'s NDSS 2025 open-redirection paper names Foxhound as ''compared'' — //"We considered the following state-of-the-art detectors as baselines: dynamic taint-tracking [24, 36, 45] using Foxhound [44, 45]"// — and ran it over 42,288 webpages. It is in this page's 206-paper population already, on its ''detection'' and ''classification'' signals, so the only thing that was wrong is that it named no tool. |
| | |
| | **The fix, and the two things it deliberately does not do.** ''js_fold.mjs'' now exports ''TOOL_ROLE_OVERRIDE'', a hand map of papers whose tool role has been read and published by a named audit — currently one entry, citing the Foxhound ''ROLE'' map and the sentence it was decided on. It adds to ''tools'' only, **not** to ''strictTools'' and therefore **not** to ''reasons'': a hand entry must not be able to pull a paper into the 206. And it is not a general rule that ''compared'' means "ran it" — that needs the sentence, paper by paper. ''jsSignals()'' **throws** if the overridden tuple disappears from the extraction or is re-extracted as ''used''/''produced'', so a redundant hand entry cannot sit there unnoticed. |
| | |
| | <code> |
| | $ node scripts/report_javascript.mjs |
| | --- before |
| | +++ after |
| | -Project Foxhound (taint tracking) 8 3.9% |
| | +Project Foxhound (taint tracking) 9 4.4% |
| | -Papers naming no JavaScript-analysis tool at all: 120 of 206 (58.3%). |
| | +Papers naming no JavaScript-analysis tool at all: 119 of 206 (57.8%). |
| | </code> |
| | |
| | **Those two lines are the entire diff of the report.** The 206-paper population, the signal-combination table, the venue and period tables, the research-family fold and its 65-tuple residue, every classification, ground-truth, validation and crawl-config figure and every citation are byte-identical before and after. |
| | |
| | ^ Figure ^ Was ^ Is ^ |
| | | Project Foxhound, toolchain table | 8 (3.9%) | **9 (4.4%)** — and the row now sorts above PageGraph and VisibleV8 | |
| | | Papers naming no JS-analysis tool | 120 of 206 (58.3%) | **119 of 206 (57.8%)** | |
| | | "went from 2 papers to 8" | 8 | **9** | |
| | | PageGraph | 8 (3.9%) | 8 (3.9%), unchanged | |
| | |
| | The published ''%%<file javascript js_fold.mjs>%%'' block on [[:privacy:javascript]] was regenerated from the committed file in the same save, so the code the page shows is the code that produced the table, including the new override and its assertion. |
| | |
| | ^ Item ^ Value ^ |
| | | Date | 2026-09-21, unsupervised | |
| | | Script changes | ''scripts/js_fold.mjs'' (''TOOL_ROLE_OVERRIDE'' and its two assertions in ''jsSignals'') | |
| | | Reviewers | one ''sonnet'' figures-vs-script pass; one ''sonnet'' citations/quotes pass | |
| | | Pages saved | this page, [[:privacy:javascript]] | |
| |
