| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:programming:crawler:tracker_radar_collector [2026/09/03 08:50] – Add the two comparison papers and the Node release schedule to the external-sources table with their verbatim verifications, and record that the coordinator re-fetched Node's EOL dates from schedule.json because they were not in the HTML page pass (c) cit karel.kubicek.claude | provenance:programming:crawler:tracker_radar_collector [2026/09/17 07:24] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude |
|---|
| </code> | </code> |
| |
| Every comparable page in this namespace (openwpm, traffic_files, pagegraph, webxray, foxhound) needed three to five review passes, and every pass found blocking defects. So the absence of findings was evidence about the review, not about the page. The 2026-09-03 run therefore ran the full review layer over the published pages. **The first pass's corpus arithmetic held up completely** — the script reproduces byte for byte, an independent full-text probe finds the same 10 papers, and every per-paper claim survived a clause-by-clause check. What did not hold up was everything the model could only have got from reading: the tool's README, its dependency's changelog, and the neighbouring page's numbers. | Every comparable page in this namespace (openwpm, traffic_files, pagegraph, webxray, foxhound) needed three to five review passes, and every pass found blocking defects. So the absence of findings was evidence about the review, not about the page. The 2026-09-03 run therefore ran the full review layer over the published pages. Every published corpus **figure** held up: the script reproduces byte for byte, an independent full-text probe finds the same 10 papers, and every clause of every per-paper claim survived a clause-by-clause check. Two things that came out of the first pass's own **script** did not — its quote-check summary ("2 exact" was really 1 verbatim and 1 case-only) and a sliding-window match that passed a mutated quote — so "the arithmetic was fine, only the reading was wrong" would be too flattering. What was wrong in bulk was the reading: the tool's README, its dependency's changelog, an open bug in its issue tracker, and the neighbouring page's numbers. |
| |
| ===== 2. Populations, denominators and every query ===== | ===== 2. Populations, denominators and every query ===== |
| | [[https://doi.org/10.1145/3366423.3380113|ahmad2020_apophanies]] | Read in ''paper.cols.txt''. Verified verbatim: "a qualitative comparative evaluation of a selected set of **eight** crawling tools", "there were **932 unique domains** (of which 30 domains were removed as they returned SOA records) which were crawled by each of our workers", and "the choice of crawler has a critical impact on the data generated". The page quotes the 932 as the paper words it rather than saying "over 932", because 30 were then dropped. | | | [[https://doi.org/10.1145/3366423.3380113|ahmad2020_apophanies]] | Read in ''paper.cols.txt''. Verified verbatim: "a qualitative comparative evaluation of a selected set of **eight** crawling tools", "there were **932 unique domains** (of which 30 domains were removed as they returned SOA records) which were crawled by each of our workers", and "the choice of crawler has a critical impact on the data generated". The page quotes the 932 as the paper words it rather than saying "over 932", because 30 were then dropped. | |
| | [[https://www.usenix.org/conference/usenixsecurity23/presentation/sanchez-rola|sanchezrola2023_rods]] | Read in ''paper.cols.txt''. Verified verbatim, including the resolution the page leans on: "we analyzed a random sample of 1, 000 phishing websites using both techniques. In total, we found 728 websites calling fingerprint-related APIs. Our crawler was able to detect and log all invocations, while OpenWPM missed calls in 348 (47.8%) of the websites. The main reason behind this result lies in the fact that OpenWPM monitors a smaller set of fingerprinting APIs ... In this experiment 48 different types of APIs calls were detected, out of which Open-WPM supported only 36 ... After this update, both our crawler and OpenWPM were able to detect all the 2092 calls performed on the test set". Publishing only the 47.8% would have inverted the paper's own conclusion. | | | [[https://www.usenix.org/conference/usenixsecurity23/presentation/sanchez-rola|sanchezrola2023_rods]] | Read in ''paper.cols.txt''. Verified verbatim, including the resolution the page leans on: "we analyzed a random sample of 1, 000 phishing websites using both techniques. In total, we found 728 websites calling fingerprint-related APIs. Our crawler was able to detect and log all invocations, while OpenWPM missed calls in 348 (47.8%) of the websites. The main reason behind this result lies in the fact that OpenWPM monitors a smaller set of fingerprinting APIs ... In this experiment 48 different types of APIs calls were detected, out of which Open-WPM supported only 36 ... After this update, both our crawler and OpenWPM were able to detect all the 2092 calls performed on the test set". Publishing only the 47.8% would have inverted the paper's own conclusion. | |
| | [[https://github.com/duckduckgo/tracker-radar-collector|duckduckgo/tracker-radar-collector]] | Clone at ''8b64006...''; ''main'' at ''4ad2472b...'' (2026-08-24), and ''git diff --stat 8b64006..HEAD'' is ''.github/workflows/asana-sync.yml'' (+25) and one deleted test line (-2), reproduced independently by two passes. Read ''crawler.js'', ''crawlerConductor.js'', ''constants.js'', ''cli/crawl-cli.js'', ''cli/crawlConfig.js'', ''cli/metadataFile.js'', ''browser/*'', ''helpers/*'' and all nine collectors. {[duckduckgo_trc_2026]} | | | [[https://github.com/duckduckgo/tracker-radar-collector|duckduckgo/tracker-radar-collector]] | Clone at ''8b64006...''; ''main'' at ''4ad2472b...'' (2026-08-24), and ''git diff %%--stat%% 8b64006..HEAD'' is ''.github/workflows/asana-sync.yml'' (+25) and one deleted test line (-2), reproduced independently by two passes. Read ''crawler.js'', ''crawlerConductor.js'', ''constants.js'', ''cli/crawl-cli.js'', ''cli/crawlConfig.js'', ''cli/metadataFile.js'', ''browser/*'', ''helpers/*'' and all nine collectors. {[duckduckgo_trc_2026]} | |
| | [[https://github.com/duckduckgo/tracker-radar|duckduckgo/tracker-radar]] | ''pushed_at'' 2026-09-02, so genuinely current; the pinned ''a1d894db...'' (2026-08-12) is a real ancestor two commits behind HEAD. All four generated files present and populated: ''domain_map.json'' 10.4 MB, ''entity_map.json'' 4.7 MB, ''domain_summary.json'' 16.3 MB, ''data_by_site.json'' 20.1 MB. ''docs/DATA_MODEL.md'' matches the page's field description including ''exampleSites''. {[duckduckgo_tracker_radar_2026]} | | | [[https://github.com/duckduckgo/tracker-radar|duckduckgo/tracker-radar]] | ''pushed_at'' 2026-09-02, so genuinely current; the pinned ''a1d894db...'' (2026-08-12) is a real ancestor two commits behind HEAD. All four generated files present and populated: ''domain_map.json'' 10.4 MB, ''entity_map.json'' 4.7 MB, ''domain_summary.json'' 16.3 MB, ''data_by_site.json'' 20.1 MB. ''docs/DATA_MODEL.md'' matches the page's field description including ''exampleSites''. {[duckduckgo_tracker_radar_2026]} | |
| | [[https://github.com/duckduckgo/tracker-radar-detector|duckduckgo/tracker-radar-detector]] | ''"archived": false, "pushed_at": "2025-03-31T15:00:37Z"''. The pinned ''0f6eb7db...'' **is** HEAD — no commit in roughly seventeen months, no deprecation notice. ''config.json'' line 2 is ''"minSites": 2,'', used at ''src/trackers/classes/crawl.js:72''; ''keepFirstParty'' and ''treatCnameAsFirstParty'' confirmed in ''config.json'' and ''site.js:152,174''. {[duckduckgo_tracker_radar_detector_2026]} | | | [[https://github.com/duckduckgo/tracker-radar-detector|duckduckgo/tracker-radar-detector]] | ''"archived": false, "pushed_at": "2025-03-31T15:00:37Z"''. The pinned ''0f6eb7db...'' **is** HEAD — no commit in roughly seventeen months, no deprecation notice. ''config.json'' line 2 is ''"minSites": 2,'', used at ''src/trackers/classes/crawl.js:72''; ''keepFirstParty'' and ''treatCnameAsFirstParty'' confirmed in ''config.json'' and ''site.js:152,174''. {[duckduckgo_tracker_radar_detector_2026]} | |
| * The 10/1,120 figure is adoption in seven selected venues under an extraction query. It is not field prevalence, commercial usage, or a share of all web measurement. | * The 10/1,120 figure is adoption in seven selected venues under an extraction query. It is not field prevalence, commercial usage, or a share of all web measurement. |
| * The 2025--2026 slice is provisional and cannot support an end-of-year trend. | * The 2025--2026 slice is provisional and cannot support an end-of-year trend. |
| * Whether DuckDuckGo has said anything publicly about ''tracker-radar-detector''`s dormancy. No archive flag, no README notice, nothing either way. The page states the dormancy as an observed fact and draws no conclusion about intent. | * Whether DuckDuckGo has said anything publicly about ''tracker-radar-detector'''s dormancy. No archive flag, no README notice, nothing either way. The page states the dormancy as an observed fact and draws no conclusion about intent. |
| * Whether ''screenshots'', ''trace'' and the fork-only capabilities behave as described. Those four schema rows are checked against source only; ''requests'', ''cookies'', ''apis'', ''targets'' and ''cookiepopups'' were checked against real output. | * Whether ''screenshots'' and ''trace'' behave as described, and whether the fork-only capabilities do. Those rows are checked against source only; ''requests'', ''cookies'', ''apis'', ''targets'' and ''cookiepopups'' were checked against real output. |
| * The extraction's ''crawlConfig'', ''ethics'' and ''artifacts'' objects **share one evidence quote per paper**, so no single key in the ''crawlConfig'' survey is independently quote-checked. The page says so in a box rather than presenting the row totals as verified per-key counts. | * The extraction's ''crawlConfig'', ''ethics'' and ''artifacts'' objects **share one evidence quote per paper**, so no single key in the ''crawlConfig'' survey is independently quote-checked. The page says so in a box rather than presenting the row totals as verified per-key counts. |
| * No performance benchmark was run. One two-site crawl proves the tool starts and what it emits; it says nothing about throughput, failure rates at scale, or bot detection. | * No performance benchmark was run. One two-site crawl proves the tool starts and what it emits; it says nothing about throughput, failure rates at scale, or bot detection. |
| Five passes over the **published** pages, each told explicitly that the author's context might not be exhaustive and to verify from primary sources. Every pass was handed the page text, the provenance text, the report script, its real output, the recall probe and its output, and the two neighbouring pages. Findings are recorded whether accepted or rejected: a rejection is the only record of whether a reviewer earns its slot. | Five passes over the **published** pages, each told explicitly that the author's context might not be exhaustive and to verify from primary sources. Every pass was handed the page text, the provenance text, the report script, its real output, the recall probe and its output, and the two neighbouring pages. Findings are recorded whether accepted or rejected: a rejection is the only record of whether a reviewer earns its slot. |
| |
| ==== 10.1 Coordinator's own pass (''opus''), by running the tool ==== | ==== 10.1 Coordinator's own pass (opus), by running the tool ==== |
| |
| ^ # ^ Finding ^ Action ^ | ^ # ^ Finding ^ Action ^ |
| | X1 | **Blocking.** "the README documents a default maximum page-load window of 30 seconds" — the code fallback in ''crawler.js'' is ''options.maxLoadTimeMs'' or-else ''60000''. The page published a README error as a default | **accepted**; the page now states 60 s and makes the divergence the pitfall | | | X1 | **Blocking.** "the README documents a default maximum page-load window of 30 seconds" — the code fallback in ''crawler.js'' is ''options.maxLoadTimeMs'' or-else ''60000''. The page published a README error as a default | **accepted**; the page now states 60 s and makes the divergence the pitfall | |
| | X2 | The page says values need added instrumentation, but ''set-cookie'' is in ''RequestCollector''`s nine-header default safelist, so HTTP-set cookie values, ''HttpOnly'', ''Secure'', ''SameSite'' and ''Expires'' are already in the default output — shown in real output from wikipedia.org | **accepted**; the cookie row and the schema table now say so, and the safelist is named | | | X2 | The page says values need added instrumentation, but ''set-cookie'' is in ''RequestCollector'''s nine-header default safelist, so HTTP-set cookie values, ''HttpOnly'', ''Secure'', ''SameSite'' and ''Expires'' are already in the default output — shown in real output from wikipedia.org | **accepted**; the cookie row and the schema table now say so, and the safelist is named | |
| | X3 | ''expires'' is milliseconds (''Math.floor(cdpDate * 1000)''), and ''-1'' becomes ''undefined'' so the key is **absent**; ''sameSite'' likewise. Real output has 5 keys on one cookie and 6 in the union | **accepted**, both stated | | | X3 | ''expires'' is milliseconds (''Math.floor(cdpDate * 1000)''), and ''-1'' becomes ''undefined'' so the key is **absent**; ''sameSite'' likewise. Real output has 5 keys on one cookie and 6 in the union | **accepted**, both stated | |
| | X4 | "The CLI can use a fresh context" — there is no such flag; ''LocalChrome'' mkdtemps a profile unconditionally. Meanwhile the README offers a ''browserContext'' option that exists only in the README | **accepted**; the page now says a fresh temp profile is unconditional and cannot be turned off from the CLI | | | X4 | "The CLI can use a fresh context" — there is no such flag; ''LocalChrome'' mkdtemps a profile unconditionally. Meanwhile the README offers a ''browserContext'' option that exists only in the README | **accepted**; the page now says a fresh temp profile is unconditional and cannot be turned off from the CLI | |
| | X5 | The CLI cannot be pointed at a browser you already have (no ''--executable-path''; ''PUPPETEER_EXECUTABLE_PATH'' ignored), and Chrome for Testing has no linux-arm64 build, so the CLI cannot start on aarch64 | **accepted**; new install section with the real error | | | X5 | The CLI cannot be pointed at a browser you already have (no ''%%--executable-path%%''; ''PUPPETEER_EXECUTABLE_PATH'' ignored), and Chrome for Testing has no linux-arm64 build, so the CLI cannot start on aarch64 | **accepted**; new install section with the real error | |
| | X6 | No published command, no %%<file>%% block, no evidence the tool was ever run — unlike every sibling crawler page | **accepted**; ''--help'', a runnable script and real output are now published | | | X6 | No published command, no %%<file>%% block, no evidence the tool was ever run — unlike every sibling crawler page | **accepted**; ''%%--help%%'', a runnable script and real output are now published | |
| | X7 | The OpenWPM comparison rests on "TRC = Puppeteer/CDP vs OpenWPM = Selenium/Firefox", but TRC has ''--selenium-hub'' and a ''selenium-webdriver'' dependency | **accepted in part**; pass (c) established that Selenium only provisions a remote **Chrome** and CDP is still the instrument, so the conclusion stands and the sentence was corrected rather than the row | | | X7 | The OpenWPM comparison rests on "TRC = Puppeteer/CDP vs OpenWPM = Selenium/Firefox", but TRC has ''%%--selenium-hub%%'' and a ''selenium-webdriver'' dependency | **accepted in part**; pass (c) established that Selenium only provisions a remote **Chrome** and CDP is still the instrument, so the conclusion stands and the sentence was corrected rather than the row | |
| | X8 | ''targets'' on a real crawl is one row, ''{"type":"page","url":"about:blank"}'' — the URL is captured at attach time | **accepted**, stated in the schema table | | | X8 | ''targets'' on a real crawl is one row, ''{"type":"page","url":"about:blank"}'' — the URL is captured at attach time | **accepted**, stated in the schema table | |
| | X9 | The ''CollectorData'' typedef is in ''helpers/collectorsList.js'' and omits ''trace'' while declaring ''elements'' | **accepted**; the page now names the file and says the typedef is not the authority | | | X9 | The ''CollectorData'' typedef is in ''helpers/collectorsList.js'' and omits ''trace'' while declaring ''elements'' | **accepted**; the page now names the file and says the typedef is not the authority | |
| | X12 | The index's "caps concurrency at 38" is a README claim removed from the code in 2025-04-16 | **accepted**; the index bullet was rewritten | | | X12 | The index's "caps concurrency at 38" is a README claim removed from the code in 2025-04-16 | **accepted**; the index bullet was rewritten | |
| | X13 | No pitfalls, install, papers-to-read-first or related-pages section, unlike every sibling | **accepted**, all four added | | | X13 | No pitfalls, install, papers-to-read-first or related-pages section, unlike every sibling | **accepted**, all four added | |
| | X15 | ''metadata.json''`s ''config'' writes every field with a falsy-coalescing default to ''undefined'', so ''filterOutFirstParty: false'' produces no key; and ''projectVersion'' has been ''1.0.0'' since the initial commit | **accepted**, both stated | | | X15 | ''metadata.json'''s ''config'' writes every field with a falsy-coalescing default to ''undefined'', so ''filterOutFirstParty: false'' produces no key; and ''projectVersion'' has been ''1.0.0'' since the initial commit | **accepted**, both stated | |
| | X16 | A real ''optOut'' run detected a Sourcepoint CMP and never acted on it (''started'', ''succeeded'' both false) while three cookies were set | **accepted**; "detection is not action" is now a pitfall and the checklist asks for the success rate | | | X16 | A real ''optOut'' run detected a Sourcepoint CMP and never acted on it (''started'', ''succeeded'' both false) while three cookies were set | **accepted**; "detection is not action" is now a pitfall and the checklist asks for the success rate | |
| | X18 | Headless is a hard-coded ''const VISUAL_DEBUG = false'' with no flag; statefulness is a module-only choice. So the 8-of-10 "not stated" in the ''crawlConfig'' survey is mostly "nothing to state", not under-reporting | **accepted**; the survey table now says which kind of silence it is | | | X18 | Headless is a hard-coded ''const VISUAL_DEBUG = false'' with no flag; statefulness is a module-only choice. So the 8-of-10 "not stated" in the ''crawlConfig'' survey is mostly "nothing to state", not under-reporting | **accepted**; the survey table now says which kind of silence it is | |
| X14 (the revision-log timing of the first run) and X17 (the script repairs) are recorded in §1.1 and §10.2 rather than as page changes. X19 is §6.2. | X14 (the revision-log timing of the first run) and X17 (the script repairs) are recorded in §1.1 and §10.2 rather than as page changes. X19 is §6.2. |
| |
| ==== 10.2 Figures against the script (''sonnet'') ==== | ==== 10.2 Figures against the script (sonnet) ==== |
| |
| Re-ran the report independently and confirmed byte-identity with both the committed output and the %%<file text>%% block on the published provenance page (one trailing blank line apart). Hand-verified every per-year denominator, numerator and percentage, the 6/4 window split, the 21/11 paper-level figures, the ''9 = 4 + 5'' string audit, and that all 10 TRC papers are inside CRAWLED. Confirmed no shelled-out or back-calculated denominator anywhere in the script. | Re-ran the report independently and confirmed byte-identity with both the committed output and the %%<file text>%% block on the published provenance page (one trailing blank line apart). Hand-verified every per-year denominator, numerator and percentage, the 6/4 window split, the 21/11 paper-level figures, the ''9 = 4 + 5'' string audit, and that all 10 TRC papers are inside CRAWLED. Confirmed no shelled-out or back-calculated denominator anywhere in the script. |
| | R | It listed "30 s ''maxLoadTimeMs'' verified directly at the pinned commit" among things it checked as **correct** | **rejected**. It read the README, not ''crawler.js''. Same error as the page, and as pass (b) | | | R | It listed "30 s ''maxLoadTimeMs'' verified directly at the pinned commit" among things it checked as **correct** | **rejected**. It read the README, not ''crawler.js''. Same error as the page, and as pass (b) | |
| |
| ==== 10.3 Citations and quotes (''sonnet'') ==== | ==== 10.3 Citations and quotes (sonnet) ==== |
| |
| Checked all 14 published keys, the render on both pages, a DOI-and-title duplicate scan, BibTeX hygiene, every author list and DOI against a primary source, all 10 quotes by hand, and every clause of every row of the ten-paper table. | Checked all 14 published keys, the render on both pages, a DOI-and-title duplicate scan, BibTeX hygiene, every author list and DOI against a primary source, all 10 quotes by hand, and every clause of every row of the ten-paper table. |
| | R | It confirmed the 30-second load window "in README lines 70/90" among its correct list | **rejected**, same reason as §10.2 | | | R | It confirmed the 30-second load window "in README lines 70/90" among its correct list | **rejected**, same reason as §10.2 | |
| |
| ==== 10.4 External currency (''sonnet'') ==== | ==== 10.4 External currency (sonnet) ==== |
| |
| Fetched rather than recalled: three fresh clones, ''npm ci'', a real ''--help'', a real crawl, the Node release schedule, the npm registry and all three repository links. | Fetched rather than recalled: three fresh clones, ''npm ci'', a real ''%%--help%%'', a real crawl, the Node release schedule, the npm registry and all three repository links. |
| |
| ^ # ^ Finding ^ Action ^ | ^ # ^ Finding ^ Action ^ |
| | R | It graded the Consent-O-Matic problem **PLAUSIBLE** on the basis that ''grep'' finds no mention in the current source | **not accepted as the final grading**: pass (d) read the dependency's changelog and found the removal, which makes it CONFIRMED. Two passes reached different confidence on one fact because one looked for absence and the other for the removal event | | | R | It graded the Consent-O-Matic problem **PLAUSIBLE** on the basis that ''grep'' finds no mention in the current source | **not accepted as the final grading**: pass (d) read the dependency's changelog and found the removal, which makes it CONFIRMED. Two passes reached different confidence on one fact because one looked for absence and the other for the removal event | |
| |
| ==== 10.5 The OpenWPM comparison (''sonnet'') ==== | ==== 10.5 The OpenWPM comparison (sonnet) ==== |
| |
| Read the openwpm page in full, cloned OpenWPM (v0.36.0, HEAD ''61d285f''), and checked every cell of the comparison table against both. | Read the openwpm page in full, cloned OpenWPM (v0.36.0, HEAD ''61d285f''), and checked every cell of the comparison table against both. |
| | R | It called the index's 21 "no contradiction" because each number is correct for its own rule | **partly rejected**: correct about the rules, wrong that the labelling is harmless. The row is labelled after the crawler and the prose attributed the growth to the crawler. See §2.2 | | | R | It called the index's 21 "no contradiction" because each number is correct for its own rule | **partly rejected**: correct about the rules, wrong that the labelling is harmless. The row is labelled after the crawler and the prose attributed the growth to the crawler. See §2.2 | |
| |
| ==== 10.6 Generic pass (''fable'', no checklist) ==== | ==== 10.6 Generic pass (fable, no checklist) ==== |
| |
| //This pass was still running when the page was first saved on 2026-09-03; this section is filled in the following revision. If it still says this, the pass did not return and nobody noticed — treat the page as three-quarters reviewed.// | It returned after the pages were already published, and it justified its slot: **twelve findings on a page four focused passes had cleared**, including the one boundary the whole page is about. Every finding was verified against the paper or the repository before being accepted, and every one was accepted. |
| | |
| | ^ # ^ Finding ^ Action ^ |
| | | 1 | **One of the "10 TRC users" did not run TRC.** {[aziz2024_johnny]} is the single ''browser-automation'' tuple, and its crawler is its own: //"We used custom scripts, written in Python and JavaScript, to drive and instrument an instance of Chrome using the Chrome DevTools Protocol"//. The extraction files ''"custom scripts" [crawler-framework/produced]'' and ''"Chrome DevTools Protocol" [browser-automation/used]'' beside the TRC tuple. The count is correct under the written rule, but "TRC users", "The papers use it as a base" and the "nothing to state" argument about ''headless''/''statefulness'' were all applied to a paper that had every choice TRC does not give you | **accepted**, verified verbatim. The page now says the 10 is nine TRC-driven crawls plus one module reuse, names the paper, and warns against reading the ''crawlConfig'' silences as TRC defaults for that row. This is the best finding of the review: a page whose thesis is "the count that is easy to get wrong" had a boundary inside its own count | |
| | | 2 | "every one of those seven wrote it" is false for at least two of the seven | **accepted**. The coordinator had already softened this to a disjunction between passes; the pass's per-paper detail is better and replaced it. Verified individually: {[senol2022_leaky]} ("We modify TRC to efficiently discover inner pages"), {[moti2024_targeted]} and {[zagi2025_referrer]} (each wrote a ''LinkCollector''), {[senol2024_double]} (regex-matched candidate links, up to fifteen on a second pass) — four wrote it; {[roongta2025_sheep]} selected inner pages into a pool ("for websites with over 10 inner pages, we included three additional inner pages… The final website pool consisted of 4,710 web pages"); {[dimova2021cname]} says only "a preliminary crawl of 20 pages per website" and never says how they were found; {[aziz2024_johnny]}'s traversal is in its own crawler | |
| | | 3 | **A documented, still-open TRC bug that two of the ten papers cite was absent from the page.** Issue #77, "Early browser API accesses and function calls are missed" | **accepted**, and it is the most useful thing added to the page. Verified through the GitHub API on 2026-09-03: ''"state": "open"'', created 2022-09-26 by ''asumansenol'' — an author of two of the ten papers — last touched 2023-03-06, four comments, maintainers describing "a race condition between API collection script setting things up and scripts on the page already running", noting it makes TRC's own API-collection integration test flaky, and ending "We're exploring different options to fix this". {[senol2024_double]} footnote 6, verified verbatim: //"Although TRC already has the capability to intercept JavaScript API calls, we introduced a separate collector due to a known TRC bug that causes it to miss the initial function calls, documented in a public GitHub issue"//. So a partial ''callStats'' has a second cause besides configuration, and the page's advice did not cover it | |
| | | 4 | This page's own headline self-assessment contradicted its §6: "the corpus arithmetic held up completely" against "both halves of that were wrong" about the first pass's quote-check summary | **accepted**, §1.1 and §11 both reworded. An overstatement about our own rigour is exactly what a provenance page is for catching, and it was in the second paragraph | |
| | | 5 | "Only two fields are not reproducible" — ''time'' is a measured duration and ''remoteIPAddress'' is anycast, and §12 of this page already said so | **accepted**, now four fields, and the two pages agree | |
| | | 6 | The ''cmps[]'' field list omits ''final'' — which is the **first** key in the coordinator's own published real output | **accepted**, added. A field list transcribed from a typedef while the real output sat two sections away | |
| | | 7 | {[sanchezrola2023_rods]}'s crawler is described as "CDP-breakpoint"; the paper says "a custom instrumentation based on the Chrome debugging protocol", and ''breakpoint''/''hook''/''wrap''/''overrid''/''getter'' have zero hits in it | **accepted**, now "CDP-instrumented Chromium crawler" | |
| | | 8 | Five rendered apostrophes are backticks, from ''%%''x''%%`s'' | **accepted**, all five fixed on both pages | |
| | | 9 | The CLI error block's ''...'' is above the success banner, but ''crawl-cli.js'' prints the banner **before** ''reporters.cleanup'' emits the start/finish/duration lines and the tally — so the elided lines fall below it, and "note the last three lines" pointed at the ellipsis | **accepted**, block and prose corrected | |
| | | 10 | The two pages disagreed on which schema keys were checked against real output (four versus five) and on how many scripts there are (four versus five) | **accepted**, both aligned | |
| | | 11 | **The published smoke script did not fail loud on a timed-out crawl** — it wrote the JSON and printed "wrote" regardless of ''data.timeout'', against this page's own lesson that a success message is not a success signal | **accepted**, and mutation-tested rather than read; see below | |
| | | 12 | "it predates TRC" is loose, and "over 932 popular domains" is ambiguous | **accepted**; now "its crawls (April 2019) predate TRC's first commit (2020-02-27)", and the coordinator had already re-quoted the 932 | |
| | |
| | ==== 10.7 Mutation-testing the guard that finding 11 added ==== |
| | |
| | A guard that never fires asserts nothing, so the new line was tested in both directions instead of read. The first attempt was misleading and is worth recording: forcing ''maxLoadTimeMs: 1'' makes ''crawl()'' throw a ''TimeoutError'' outright, because ''maxTotalTimeMs = maxLoadTimeMs * 2 + collectorExtraTimeMs'' kills the whole crawl before the inner path runs — no result, no file, guard never reached. That would have "passed" while proving nothing. |
| | |
| | The reachable path needs ''maxLoadTimeMs'' large enough that ''maxTotalTimeMs'' still covers the 2.5 s settle. ''crawler.js'' catches a navigation ''TimeoutError'' at line 338, sends ''Page.stopLoading'', sets ''timeout = true'' at line 349, and **carries on collecting**, returning a partial result at line 380. Reproduced with a local server serving a page whose image handler sleeps 12 s, and ''maxLoadTimeMs: 5000'': |
| | |
| | <code> |
| | # with the guard: |
| | [trc] crawling http://127.0.0.1:8099/ |
| | [log] Page navigation timeout |
| | if (data.timeout) throw new Error(`${u} hit the crawler's timeout path; capture is partial`); |
| | Error: http://127.0.0.1:8099/ hit the crawler's timeout path; capture is partial |
| | --- files written: (none) |
| | |
| | # same run with the guard line deleted: |
| | [trc] crawling http://127.0.0.1:8099/ |
| | [trc] wrote /tmp/trcout7/http_127_0_0_1_8099_.json |
| | /tmp/trcout7/http_127_0_0_1_8099_.json timeout = True | requests: 2 |
| | </code> |
| | |
| | So the guard does real work: without it the script announces success over a two-request partial capture. The published block and ''scripts/trc_smoke.mjs'' were then re-synchronised and the amended script re-run against the two live sites to confirm it still completes normally. |
| | |
| | ==== 10.8 What the generic pass checked and found correct ==== |
| | |
| | Worth reading before re-checking anything. It independently reproduced the rendered-DOM audit on both pages (all anchors resolve, zero real red links — it correctly identified that the one ''wikilink2'' string on this page is literal text inside a code block — no list item containing preformatted text, no uneven table rows, all four WRAP boxes rendering as plugins, the footnote rendering, and the citekey-to-reference counts on both pages); confirmed the ''%%--help%%'' block is byte-identical to the real command at the pinned commit; and re-verified some thirty repository facts including the user-agent constants, the viewports, ''VISUAL_DEBUG'', the 60000/2500 defaults, the UA and browser-close gating, the unconditional ''downloadChrome'', the ''size'' coercion and its clickhouse comment, the ''saveHeaders'' default, the ''responseReceivedExtraInfo'' listener that makes ''set-cookie'' real, ''filterListMatched'' being written once and only read, and 82 commits since 2026-01-01 (so "actively developed" holds). It also established that ''example.com'' currently returns **none** of the nine safelisted headers, so the empty ''responseHeaders'' in the published output is correct rather than a capture failure — a check nobody else thought to make. On the index page it confirmed the diff contains only the three intended edits and that the 9/1/1 footnote split matches the reconcile output. |
| |
| ===== 11. Judgement calls ===== | ===== 11. Judgement calls ===== |
| |
| * **Reviewed and rewrote rather than replaced.** The first pass's corpus work was sound and its structure was reasonable. Every figure it published survived; roughly half its non-corpus prose did not. The page kept its skeleton and gained an install section, a pitfalls box, three new subsections and two new citations. | * **Reviewed and rewrote rather than replaced.** The first pass's corpus work was sound and its structure was reasonable. Every **figure** it published survived — though not everything its script printed about those figures — and roughly half its non-corpus prose did not. The page kept its skeleton and gained an install section, a pitfalls box, three new subsections and two new citations. |
| * **Edited a neighbour.** Fixing ''programming:crawler''`s 21-row label and its "38 crawlers" bullet is outside a strict reading of "review this page", but publishing a contradiction and leaving the wrong side live would have been worse. ''tool_fold.mjs'' was deliberately **not** changed (§2.2). | * **Edited a neighbour.** Fixing ''programming:crawler'''s 21-row label and its "38 crawlers" bullet is outside a strict reading of "review this page", but publishing a contradiction and leaving the wrong side live would have been worse. ''tool_fold.mjs'' was deliberately **not** changed (§2.2). |
| * **Did not aggregate the measured results.** ''detection[]'' has 61 prevalence strings across the ten papers. Three are published, each with the denominator read out of the paper. A "TRC papers find X% of sites do Y" number would have been the most quotable thing on the page and the least defensible. | * **Did not aggregate the measured results.** ''detection[]'' has 61 prevalence strings across the ten papers. Three are published, each with the denominator read out of the paper. A "TRC papers find X% of sites do Y" number would have been the most quotable thing on the page and the least defensible. |
| * **Published a real crawl of two live sites and one CMP site.** Three page loads of public homepages, no login, no interaction beyond the consent action the collector performs. The scraped button text from that run is described but not published, because it is third-party page content. | * **Published a real crawl of two live sites and one CMP site.** Three page loads of public homepages, no login, no interaction beyond the consent action the collector performs. The scraped button text from that run is described but not published, because it is third-party page content. |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| |
