| |
| provenance:programming:crawler [2026/08/12 09:17] – Create provenance page for programming:crawler: queries, denominators, fold changes and residue, quote verification, and the 4,322 -> 5,859 refresh diff. Partly reconstructed after the fact; marked as such. Authored by Claude. karel.kubicek.claude | provenance:programming:crawler [2026/08/12 10:07] (current) – Add section 10: the Fable review pass, the windowed-guard defect it exposed (29 corpus figures stale outside the audited heading window across six pages), the hardened check_page_numbers.mjs, and the page-specific fixes. Authored by Claude. karel.kubicek.claude |
|---|
| </code> | </code> |
| |
| ''check_page_numbers.mjs'' left four figures unaccounted on 2026-08-12, all deliberate: ''100'' ("not against 100%"), ''19.1'' (the crawled share of the corpus, from ''data/extract/OVERVIEW.md''), ''53'' (the **old** OpenWPM count, quoted on the page as history), and ''93'' — which was a stale number from an intermediate run of the quote check and was corrected to 106 before saving. That last one is the guard doing exactly the job it exists for. | ''check_page_numbers.mjs'' left four figures unaccounted on 2026-08-12. **Three are permanent and legitimate** and are named here so the next run does not re-investigate them: ''100'' ("not against 100%"), ''19.1'' (the crawled share of the corpus, from ''data/extract/OVERVIEW.md'') and ''106'' (the quote-check total, from ''out/new/quote_check_crawler.txt'', not from this report). A fourth, ''53'', is the **old** OpenWPM count quoted on the page as history. The guard also caught a real error: the page briefly carried ''93'' from an intermediate run of the quote check, corrected to 106 before saving. Run it whole-page too (§10). |
| |
| ===== 4. What the refresh changed ===== | ===== 4. What the refresh changed ===== |
| * **The sandbox measurements were not re-run.** The Selenium/Puppeteer/Playwright/CDP comparison, the WebDriver BiDi ''responseCompleted'' finding and the "two network handlers cross-deliver" bug are from 2026-08-06 against ''selenium-webdriver'' 4.46. Library versions have moved since. Nothing in this refresh touched them, and nothing in this refresh verified them. | * **The sandbox measurements were not re-run.** The Selenium/Puppeteer/Playwright/CDP comparison, the WebDriver BiDi ''responseCompleted'' finding and the "two network handlers cross-deliver" bug are from 2026-08-06 against ''selenium-webdriver'' 4.46. Library versions have moved since. Nothing in this refresh touched them, and nothing in this refresh verified them. |
| |
| ===== 9. Run log ===== | ===== 10. Review pass, 2026-08-12 ===== |
| | |
| | //Recorded.// The refresh was reviewed by a second model (Claude Fable 5), told explicitly that the summary it was given might not be exhaustive, with instructions to hunt stale numbers. It found a **systematic defect, not a scatter of typos**, and it is worth stating because it will recur on the next refresh: |
| | |
| | * ''check_page_numbers.mjs'' was run with a **heading window** — normally ''Use in Publications'' to the next section — so it audited only the corpus section. Every corpus figure repeated in a page's **intro, tooling section, recommendations, footnotes, Related Pages or an embedded code block** was outside the window and stayed at its 4,322-corpus value. Across the six pages **29 such figures** survived the first pass. |
| | * The matcher was **substring**, not word-boundary, so ''report.includes('59')'' was satisfied by ''11.59 bits''. One genuinely stale figure sat inside a checked window and passed for that reason. |
| | |
| | Both are fixed in ''scripts/check_page_numbers.mjs'': matching is now anchored with lookarounds, ISO dates and URLs are stripped before scanning, ''--code'' opts into scanning ''<file>'' blocks, and omitting the heading markers checks the whole page. **Run it windowed //and// whole-page.** The whole-page run is noisy — a page's non-corpus half is full of figures quoted from other papers — so read its output rather than expecting it to exit clean. |
| | Fixed on this page's content page as a result: **11.6% → 12.0%** (twice), Selenium **21.8% → 21.6%**, Puppeteer **6.3% → 6.8%**, direct CDP **35 papers (4.1%) → 42 (3.8%)**, headless **12.9% → 12.5%**, and unnamed bespoke crawlers **34 → 75**. Two prose corrections the numbers forced: |
| | |
| | * The Playwright section still said "still rare in the literature — 10 papers", two screens from a corpus table saying 34 and a sentence saying Playwright overtakes Puppeteer. Rewritten. |
| | * "OpenWPM users are more than twice as good as average … Playwright's cohort is next best" **ranked the two backwards** against the table directly above it (Playwright 26.5%, OpenWPM 25.9%). On the old corpus Playwright led 50.0% to 25.5% on a base of ten papers; with 34 they are effectively tied, and the sentence now says so. |
| | * The framework table had drifted out of descending order after the refresh moved three rows. Restored. |
| | |
| | **''106'' is a permanent legitimate unaccounted figure** on this page: it is the quote-check total, from ''out/new/quote_check_crawler.txt'', not from ''report_crawler.mjs''. Recorded here so the next run does not re-investigate it. §3 above previously said "four … all deliberate" while listing one figure (''93'') that was a real error — corrected. |
| | |
| | ===== 11. Run log ===== |
| |
| ^ ^ ^ | ^ ^ ^ |
| | Mistake caught in review | The page briefly carried "231" for that combined count — arrived at by summing family counts, which double-counts papers naming two of them. The real union is 215. ''check_page_numbers.mjs'' flagged it as unaccounted, which is how it was caught; it was then computed in the report script rather than by hand. | | | Mistake caught in review | The page briefly carried "231" for that combined count — arrived at by summing family counts, which double-counts papers naming two of them. The real union is 215. ''check_page_numbers.mjs'' flagged it as unaccounted, which is how it was caught; it was then computed in the report script rather than by hand. | |
| | Deferred | LLM-agent browser automation in the 2025–2026 residue (§5.3), logged as a separate work item. | | | Deferred | LLM-agent browser automation in the 2025–2026 residue (§5.3), logged as a separate work item. | |
| | | Review | Reviewed by Claude Fable 5 on 2026-08-12 with the instruction that the summary might not be exhaustive. It found the windowed-guard defect in §10 and 7 stale figures plus a mis-ordered table and a backwards ranking sentence on this page. All fixes were applied and re-saved the same day. | |
| |
| [[programming:crawler|← back to the content page]] · [[literature:corpus|corpus-level provenance]] | [[programming:crawler|← back to the content page]] · [[literature:corpus|corpus-level provenance]] |
| |
