Differences

This shows you the differences between two versions of the page.

--- provenance:programming:crawler:webxray [2026/09/11 16:08] – C3.8: reverse a wrong demotion found by two reviewers (both citation checks are blind to markup-borne quotes), add the third instrument defect and the OK-MARKUP fix, correct the compass-fit.jp diagnosis to a one-comma slip, add the third-sitting reviewer karel.kubicek.claude
+++ provenance:programming:crawler:webxray [2026/09/22 04:23] (current) – Escape the literal {[citekey]} in the 2026-09-22 scope note: ''..'' monospace does not escape the bibtex plugin, so it rendered as a live unresolved citation marker. Authored by Claude karel.kubicek.claude
@@ Line 67: / Line 67: @@
 | A8 | Per-year and per-venue shape | year/venue of the 15 | 15 | last use of crawler-or-list: **2022** |
 | A9 | Which ownership resources does the corpus name at all? | six full-text sweeps | 5,859 | PSL 101, Disconnect-as-a-list 74, Tracker Radar 32, WhoTracks.me 23, Crunchbase 23, webXray 15 |
-| A10 | Union of the five owner databases | union of A9 rows minus the PSL row | 5,859 / 1,120 crawled | **136** (2.3% / 12.1%) |
+| A10 | Union of the five owner databases | union of A9 rows minus the PSL row | 5,859 / 1,120 crawled | **136** (2.3% of the corpus); **107** of them are among the 1,120 that crawled (**9.6%** of that population) |
 | A11 | Which Tracker Radar artefact did each of the 32 use? | ''TR_ROLE'' hand map | 32 | ownership 11, tracker/category 9, Collector crawler 9, citation 2, compared 1 |
 | A12 | Ownership use of Tracker Radar over time | year of the 11 | — | 2021:1, 2023:2, 2024:1, 2025:4, 2026:3 |
@@ Line 89: / Line 89: @@
 ==== A.16 Evidence quotes spot-checked ====
-The 16 ''evidence.quote'' values behind the webXray tool and classification tuples, checked by 4-word-window coverage against ''paper.cols.txt'': **4 exact, 11 partial (≥60% of windows), 1 below threshold.** Full list in section H of the report output.
+The 16 ''evidence.quote'' values behind the webXray tool and classification tuples, checked by 4-word-window coverage against ''paper.cols.txt'', with a ''pypdf'' rendering of the same ''paper.pdf'' consulted for anything below threshold: **4 exact, 11 partial (≥60% of windows), 0 rescued from the PDF, 1 below threshold in both renderings.** Full list in section H of the report output.
 The one below threshold is {[musa2022_atom]} at 58%: "We used external data sources including WHOIS records, TLS certificates, and WebXray [61] to identify the parent organizations of each identified tracker." Read by hand in the source: **present and verbatim**; the ''.cols'' rendering splices an adjacent column through it. Below-threshold is not "unsupported".
@@ Line 208: / Line 208: @@
 ^ ^ ^
 | Frame | the **32,337** registrable domains in Tracker Radar's ''domain_summary.json'' as fetched on **2026-09-05**, folded with the **ICANN section only** of the Public Suffix List and resolved by parent-label lookup. ''owner_sample.py'' imports ''owner_dbs.py'' and uses its universe function, so the frame is built by the same code as the first sitting's |
-| …but it is **not the same snapshot** | the first sitting's frame, on 2026-08-17, was **32,369** domains. Tracker Radar's ''domain_summary.json'' and the Public Suffix List both changed between the two dates (see the run table above), and the frame moved by 32 domains. Per-list coverage moved with it: webXray 669→664, Tracker Radar 5,581→5,566, Disconnect 2,268→2,264, and webXray's prevalence mass 58.6%→57.9%. The coverage **shares** are unchanged to one decimal (2.1% / 17.2% / 7.0%), which is why the two sets of figures can sit on one page without looking inconsistent — and is exactly why the difference is stated here. Verify with ''python3 scripts/owner_sample.py --cache out/webxray/cache'' against ''--cache out/owner_cache'' |
+| …but it is **not the same snapshot** | the first sitting's frame, on 2026-08-17, was **32,369** domains. Tracker Radar's ''domain_summary.json'' and the Public Suffix List both changed between the two dates (see the run table above), and the frame moved by 32 domains. Per-list coverage moved with it: webXray 669→664, Tracker Radar 5,581→5,566, Disconnect 2,268→2,264, and webXray's prevalence mass 58.6%→57.9%. The coverage **shares** are unchanged to one decimal (2.1% / 17.2% / 7.0%), which is why the two sets of figures can sit on one page without looking inconsistent — and is exactly why the difference is stated here. Verify with ''python3 scripts/owner_sample.py %%--cache%% out/webxray/cache'' against ''%%--cache%% out/owner_cache'' |
 | Which figures use which | every **accuracy** figure and the coverage counts in the random-sample section use the 2026-09-05 frame (32,337). Every figure in the earlier coverage and agreement sections, and all 30 rows of the hand adjudication, use the 2026-08-17 frame (32,369) and are **not** re-derived: they are pinned to the snapshot the adjudication was done against, and re-running them would orphan that work |
 | What that is | **Tracker Radar's view of the third-party surface**, not a neutral census of the web. Every figure inherits it |
@@ Line 313: / Line 313: @@
 column cannot be inflated by conflating the two.
-==== C3.5 Every citation re-fetched, and the ten fixed by hand ====
+==== C3.5 Every citation re-fetched, and the eleven fixed by hand ====
 The single failure mode that would invalidate the whole study is a
@@ Line 324: / Line 324: @@
 | ''owner_verify_sources.py'' | re-fetch the URL with ''curl''-grade HTTP, or redo the TLS handshake for a ''tls://'' source; match the quote after normalising whitespace, quote characters and hyphenation | 121 OK of the 136 rows that carried a source at that point; 13 NOTFOUND, 2 FETCHFAIL |
 | ''owner_verify_rendered.mjs'' | re-check the failures in a real browser (Playwright), because footers and Impressum text are frequently client-rendered | rescued **5**: ''fontawesome.com'', ''flashtalking.com'', ''force.com'', ''conviva.com'', ''cartfulsolutions.com'' |
-| ''owner_corrections.py'' | by hand, for the 10 that survived both | 4 ''requote'', 5 ''resource'', 1 ''demote'' |
+| ''owner_corrections.py'' | by hand, for the 10 that survived both, plus 1 applied on 2026-09-17 from the 2026-09-11 re-run of the fixed checker ([[#C3.9 The 2026-09-11 re-run, applied (2026-09-17)|C3.9]]) | 5 ''requote'', 5 ''resource'', 1 ''demote'' |
-The 4 **requotes** are the finding worth keeping: in 4 of 175 rows the "quote"
+The **requotes** are the finding worth keeping: in **4** of 175 rows the "quote"
-was a re-ordering or a summary of the page's own words, despite a brief that
+was a re-ordering or a summary of the page's own words, despite a brief that said
-said verbatim. The claim held in all four and the source stayed; the quote was
+verbatim. The claim held in all four and the source stayed; the quote was
-replaced with text copied out of the fetched page. That is the argument for
+replaced with text copied out of the fetched page. A **fifth** requote,
+''compass-fit.jp'', is counted with them in the table above because the remedy is
+identical, but it is a different defect: **one dropped character**, not a
+paraphrase. It was found on 2026-09-11 and applied on 2026-09-17, because the
+checker that would have caught it in the first place was itself blind to
+non-Latin text until then — see [[#C3.9 The 2026-09-11 re-run, applied (2026-09-17)|C3.9]]. That is the argument for
 verifying every citation by machine rather than trusting the adjudicator, and it
 is the same defect a WebFetch summary produces.
@@ Line 343: / Line 348: @@
 **Where every row's citation ended up** (''owner_citation_audit.py''):
-^ State ^ Rows ^
+All four columns below are **as of 2026-09-05**, and two of the numbers have
+since moved for reasons that are not about the citations —
+[[#C3.9 The 2026-09-11 re-run, applied (2026-09-17)|C3.9]] has the arithmetic.
+^ State (2026-09-05) ^ Rows ^
 | cited, confirmed in the fetched bytes | 132 — of which **119 are the whole quote**, 8 matched as fragments either side of an ellipsis, and 5 matched only on a 60-character window of a long quote |
 | cited, confirmed only in a rendered browser | 3 |
@@ Line 350: / Line 359: @@
 The three browser-only rows are ''fontawesome.com'', ''flashtalking.com'' and
-''force.com''. A bytes-only audit re-run today still reports them as failures,
+''force.com''. A bytes-only audit re-run **on 2026-09-05** reported all three as
+failures — ''fontawesome.com'' no longer, since the ''OK-MARKUP'' status of
+-09-11 lets the bytes pass see the markup its quote is ([[#C3.9 The 2026-09-11 re-run, applied (2026-09-17)|C3.9]]) —
 which is why the audit script exists: the answer to "so are three figures
 uncited?" otherwise lives in three separate JSON files.
@@ Line 374: / Line 385: @@
 //root// rather than its entry (''addthiscdn.com'', ''tqlkg.com''; both are in
 ''owner_verdict_fixes.py'' with the reasoning), and 1 from the ''cnevids.com''
-demotion. Separately, **10 of 175 rows had their source or quote replaced**, and
+demotion. Separately, **11 of 175 rows had their source or quote replaced**, and
- of those 10 left the verdicts untouched.
+ of those 11 left the verdicts untouched.
 ==== C3.5a A page-destroying DokuWiki defect, found before publication ====
@@ Line 435: / Line 446: @@
 **Every figure the content page takes from this sample**, with its population:
-^ Figure on the content page ^ Population ^ Value ^
+^ Figure on the content page ^ Population ^ Value, after the 2026-09-11 second pass ^ Before it ^
-| webXray names today's owner, domain-level | 49 eligible of webXray's 60 draws, weighted to its 664 covered domains | 69.9% (55.5–83.1) |
+**Refreshed 2026-09-11** after the second pass over the residue settled 28 of the 40 rows the 2026-09-05 pass could not adjudicate. The //before// column is kept, because a table of published figures that quietly replaces them is the thing this page exists to prevent. The second pass is documented on [[provenance:design:ownership_resolution]] §L; its code is on [[provenance:programming:crawler:webxray:residue_pass]].
-| webXray, encounter-weighted | same rows, weighted by Tracker Radar prevalence | 93.9% (80.9–98.8) |
-| Tracker Radar, domain-level | 47 eligible of 60, weighted to 5,566 | 73.7% (60.8–85.8) |
+| webXray names today's owner, domain-level | 56 eligible of webXray's 60 draws, weighted to its 664 covered domains | 69.2% (56.1–81.4) | //was 69.9% (55.5–83.1) on 49// |
-| Tracker Radar, encounter-weighted | same | 68.6% (29.3–97.0) |
+| webXray, encounter-weighted | same rows, weighted by Tracker Radar prevalence | 93.8% (80.9–98.6) | //was 93.9% (80.9–98.8)// |
-| Disconnect, domain-level | 42 eligible of 60, weighted to 2,264 | 94.4% (86.9–100.0) |
+| Tracker Radar, domain-level | 56 eligible of 60, weighted to 5,566 | 73.9% (61.6–85.3) | //was 73.7% (60.8–85.8) on 47// |
-| Disconnect, encounter-weighted | same | 83.1% (53.7–100.0) |
+| Tracker Radar, encounter-weighted | same | 74.3% (41.1–97.8) | //was 68.6% (29.3–97.0)// |
-| ''current+granularity'', domain-level | as above | webXray 80.2 (67.7–90.8), TR 76.5 (63.5–88.0), DC 98.8 (96.2–100.0) |
+| Disconnect, domain-level | 54 eligible of 60, weighted to 2,264 | **89.2% (80.5–96.8)** | //was 94.4% (86.9–100.0) on 42// |
-| webXray ''stale'', domain-level | 49 eligible | 19.8% (9.2–32.3) |
+| Disconnect, encounter-weighted | same | 82.8% (53.8–100.0) | //was 83.1% (53.7–100.0)// |
-| ''error'' is rare in all three | 49 / 47 / 42 eligible | webXray 0, Disconnect 0, Tracker Radar 1 (''elfsight.com''); rule-of-three bounds 6.1% and 7.1% |
+| ''current+granularity'', domain-level | as above | webXray 78.0 (65.8–88.8), TR 76.4 (64.6–87.5), DC 96.8 (91.5–100.0) | //was 80.2, 76.5, 98.8// |
-| coverage 2.1% / 17.2% / 7.0% | **census over all 32,337 frame domains**, no sampling error | 664 / 5,566 / 2,264 |
+| webXray ''stale'', domain-level | 56 eligible | 22.0% (11.4–33.8) | //was 19.8% (9.2–32.3)// |
-| entity name identical to the domain | **census over all pairs**: 3,215 / 38,368 / 7,850 | 0.8% / 0.7% / 3.2% |
+| ''error'' is uncommon, and no longer absent from two lists | 56 / 56 / 54 eligible | webXray **0** (rule-of-three bound 5.4%), Tracker Radar **3** (''elfsight.com'', ''cnevids.com'', ''cedscdn.it''), Disconnect **1** (''km0trk.com'') | //was 0 / 1 / 0; all four of the new ones were unsettleable in the first pass// |
-| "a third of the sample could not be settled" | 175 drawn rows | 11, 13 and 18 of each 60 |
+| the three pairwise Fisher tests | eligible raw counts, ''current'' only, Bonferroni 0.0167 | **none of the three survives**: DC vs wx 0.031, DC vs TR 0.083, TR vs wx 0.83 | //was DC vs wx 0.014, the one that did// |
-| concentration of the encounter-weighted estimate | the eligible rows, prevalence-weighted | ''gstatic.com'' 39.4% of webXray's; ''omtrdc.net'' 24.9% of Tracker Radar's; ''id5-sync.com'' 16.7% of Disconnect's |
+| coverage 2.1% / 17.2% / 7.0% | **census over all 32,337 frame domains**, no sampling error | 664 / 5,566 / 2,264 | unchanged — a census, not a sample |
+| entity name identical to the domain | **census over all pairs**: 3,215 / 38,368 / 7,850 | 0.8% / 0.7% / 3.2% | unchanged — a census, not a sample |
+| what could not be settled | 175 drawn domains, 180 draws | **12 domains, 14 draws**; 4, 4 and 6 of each 60 | //was 40 domains, 42 draws; 11, 13 and 18// |
+| concentration of the encounter-weighted estimate | the eligible rows, prevalence-weighted | ''gstatic.com'' 38.8% of webXray's; ''omtrdc.net'' 20.0% of Tracker Radar's; ''id5-sync.com'' 16.6% of Disconnect's | //was 39.4 / 24.9 / 16.7// |
 **The estimator was re-implemented from scratch and checked against itself.**
@@ Line 488: / Line 502: @@
 ==== C3.7 Limits, in the order they matter ====
-  - **Conditional on adjudicability.** 11–18 of each 60 rows could not be settled. If unadjudicable entries are systematically worse than adjudicable ones — which is plausible, since a dead domain with no legal document is also a domain whose owner changed without announcement — **all three lists are flattered, by roughly the same amount**. This is the limit that would most change the numbers and it cannot be bounded from inside the sample.
+  - **Conditional on adjudicability — and this is the limit the 2026-09-11 second pass went after.** It read: //"11–18 of each 60 rows could not be settled. If unadjudicable entries are systematically worse than adjudicable ones … all three lists are flattered, by roughly the same amount. This is the limit that would most change the numbers and it cannot be bounded from inside the sample."// It was right that the sample could not bound it, and wrong that the flattering would be even: harder sourcing settled 28 of the 40, leaving **4, 4 and 6** of each 60, and Disconnect's domain-level rate fell 94.4% → **89.2%** while webXray's and Tracker Radar's did not move. The recovered rows are not significantly worse than the rows already in the rates (//p// = 0.64 / 1.00 / 0.40 on 7, 9 and 12 rows — almost no power), but they were worse for the list this page recommends, and they cost it the one pairwise comparison that survived Bonferroni. The **12 domains still unresolved** are unbounded for the same reason as before. See [[provenance:design:ownership_resolution]] §L.
   - **Conditional on the frame.** Entries for domains Tracker Radar's crawl never saw are never scored.
-  - **The encounter-weighted column is a ratio estimator dominated by a handful of head domains, and the estimator now measures by how much.** Section C2 of its output reports, per list, the share of the encounter-weighted estimate carried by the single heaviest eligible row, and what the ''current'' rate would become if that one row's verdict were flipped to ''stale'': ''gstatic.com'' carries **39.4%** of webXray's (93.9% would read 54.4%), ''omtrdc.net'' **24.9%** of Tracker Radar's (68.6% would read 43.7%), ''id5-sync.com'' **16.7%** of Disconnect's (83.1% would read 66.4%); the top-three shares are 53.1% / 64.9% / 46.0%. The flip is a sensitivity check, not a result. Intervals reach 68 percentage points wide (Tracker Radar, ''stale+error'': 2.9–71.2%). Prevalence in the frame spans five orders of magnitude, so this concentration is what encounter weighting //is// and not a defect in the estimator — but a headline resting on one observation should say so beside itself, and not only inside an interval. Read the point estimate as an ordering, not as a number.
+  - **The encounter-weighted column is a ratio estimator dominated by a handful of head domains, and the estimator now measures by how much.** Section C2 of its output reports, per list, the share of the encounter-weighted estimate carried by the single heaviest eligible row, and what the ''current'' rate would become if that one row's verdict were flipped to ''stale'': ''gstatic.com'' carries **38.8%** of webXray's (93.8% would read 55.0%), ''omtrdc.net'' **20.0%** of Tracker Radar's (74.3% would read 54.3%), ''id5-sync.com'' **16.6%** of Disconnect's (82.8% would read 66.2%); the top-three shares are 52.2% / 52.4% / 45.6%. //(All six moved on 2026-09-11: the second pass added eligible rows, which dilutes every head domain's share. Before it: 39.4 / 24.9 / 16.7 and 53.1 / 64.9 / 46.0.)// The flip is a sensitivity check, not a result. Intervals reach 68 percentage points wide (Tracker Radar, ''stale+error'': 2.9–71.2%). Prevalence in the frame spans five orders of magnitude, so this concentration is what encounter weighting //is// and not a defect in the estimator — but a headline resting on one observation should say so beside itself, and not only inside an interval. Read the point estimate as an ordering, not as a number.
-  - **n is 42–49 per list.** The domain-level intervals are ±13 points at best. Any claim that two lists differ needs the intervals, and webXray versus Tracker Radar domain-level (69.9 vs 73.7) is **not** a difference this sample can establish.
+  - **n is 54–56 per list** after the 2026-09-11 second pass over the residue (42–49 before it). The domain-level intervals are ±12 points at best. Any claim that two lists differ needs the intervals or the Fisher test, and webXray versus Tracker Radar domain-level (69.2 vs 73.9) is **not** a difference this sample can establish — //p// = 0.83. The figures in this bullet carry no ''%'' sign, which is why ''sweep_moved_figures.py'' did not flag them when the rates moved; the generic review pass did.
   - **A verdict is one adjudicator's reading, and a second reading agrees with it about as often as a coin would on the hardest list.** Measured on 2026-09-11 — see [[#C3.8 Inter-rater agreement: a second model on a random fifth (2026-09-11)|C3.8]]. A random 20% (35 of 175 domains, seed 20260911) was re-adjudicated by a **different model** against the identical brief and the identical per-domain evidence blocks. On the rows the brief actually asks about, Cohen kappa is **+0.027** for webXray (n=12, 25.0% raw agreement, 95% CI -0.200 to +0.280), **+0.478** for Tracker Radar (n=30, 70.0%, +0.244 to +0.706) and **+0.460** for Disconnect (n=23, 60.9%, +0.193 to +0.707); pooled over the 65 cells, **+0.400** at 58.5% raw agreement. **63.0% of the disagreement is one rater finding a source where the other did not**, not two readings of the same evidence: on rows both raters settled, the pooled kappa is +0.680 and on ''current''-versus-rest +0.708. The citation check verifies that the evidence exists and contains the quote; it still does not verify that the verdict follows from it. **What would close what remains:** a subsample drawn from webXray's own coverage rather than from the 175, so its cell is not 12 rows.
   - **Quartiles, not deciles** — see C3.2.
@@ Line 512: / Line 526: @@
 Manifest digest ''07208284bbd90fd6''; the 35 domains are listed in the script's
 output on [[provenance:programming:crawler:webxray:random_sample|the appendix
-page]], section AE.
+page]], section AF.
 **The stimulus is identical, not merely equivalent.** The second rater was given
@@ Line 640: / Line 654: @@
 === Three defects this pass found in the instruments ===
-  * **''owner_verify_sources.py'' was deleting every non-Latin character.** Its ''norm()'' ended in ''%re.sub(r"[^a-z0-9]+", " ", s)%'', so a Cyrillic, Greek, Hebrew or CJK quote was checked only on whatever Latin substring it happened to contain — and a wholly non-Latin quote normalised to the empty string and was reported ''NOQUOTE'', a status that reads like "the row has no quote" rather than "I could not check this quote". ''owner_verify_rendered.mjs'' has always used ''%[^a-z0-9À-]+%'', so the browser pass and the bytes pass disagreed about what a character is. Found because rater 2 cited a wholly Chinese sentence for ''qbox.me''. The Python class is now the JavaScript one. **Consequence for the first sitting:** re-running the fixed script over all 175 rows moves three statuses, and only one of them is this fix biting — ''compass-fit.jp'', whose Japanese quote was previously verified on the six letters ''COMPASS-FIT'' alone and is **not** verbatim on the cited page. The mismatch is **one character**: the page's body reads ''台湾国内での、ネイティブ型広告配信サービス…'' and the row's quote drops the full-width comma after ''台湾国内での''. Everything else, including the ''致しました。'' ending, is on the page. So this is a transcription slip rather than a paraphrase, and the row's //claim// is unaffected — MicroAd's own newsroom page carries the partnership, the service name and Taiwan — so no published rate moves. The row needs a ''requote'' in ''owner_corrections.py'' and that is recorded as open work rather than done here. //A first draft of this bullet said the page's headline ends ''の提供を開始'' and implied the ''致しました。'' ending appears nowhere; a reviewer found the full sentence in the body and the real one-comma difference.// ''tns-counter.ru'', the other non-Latin row, verifies clean under the fix.
+  * **''owner_verify_sources.py'' was deleting every non-Latin character.** Its ''norm()'' ended in ''%re.sub(r"[^a-z0-9]+", " ", s)%'', so a Cyrillic, Greek, Hebrew or CJK quote was checked only on whatever Latin substring it happened to contain — and a wholly non-Latin quote normalised to the empty string and was reported ''NOQUOTE'', a status that reads like "the row has no quote" rather than "I could not check this quote". ''owner_verify_rendered.mjs'' has always used ''%[^a-z0-9À-]+%'', so the browser pass and the bytes pass disagreed about what a character is. Found because rater 2 cited a wholly Chinese sentence for ''qbox.me''. The Python class is now the JavaScript one. **Consequence for the first sitting:** re-running the fixed script over all 175 rows moves three statuses, and only one of them is this fix biting — ''compass-fit.jp'', whose Japanese quote was previously verified on the six letters ''COMPASS-FIT'' alone and is **not** verbatim on the cited page. The mismatch is **one character**: the page's body reads ''台湾国内での、ネイティブ型広告配信サービス…'' and the row's quote drops the full-width comma after ''台湾国内での''. Everything else, including the ''致しました。'' ending, is on the page. So this is a transcription slip rather than a paraphrase, and the row's //claim// is unaffected — MicroAd's own newsroom page carries the partnership, the service name and Taiwan — so no published rate moves. The row needed a ''requote'' in ''owner_corrections.py''; that was **done on 2026-09-17** and is recorded in [[#C3.9 The 2026-09-11 re-run, applied (2026-09-17)|C3.9]]. //A first draft of this bullet said the page's headline ends ''の提供を開始'' and implied the ''致しました。'' ending appears nowhere; a reviewer found the full sentence in the body and the real one-comma difference.// Three rows in the 175 carry a non-Latin quote, not two: ''compass-fit.jp'', ''tns-counter.ru'' (Russian) and ''bilibili.com'' (a certificate subject, ''%%O = 上海幻电信息科技有限公司%%'', which the broken class reduced to "o cn bilibili com"). The other two verify clean under the fix. **The fix is not symmetric, and that is worth stating:** the character class keeps everything in U+00C0–U+FFFF, which includes CJK punctuation, while ASCII punctuation is still normalised to whitespace. So ''%%norm("Brightcove Inc, describes") == norm("Brightcove Inc describes")%%'' is true and the same comparison over ''%%国内での、ネイティブ%%'' versus ''%%国内でのネイティブ%%'' is false. A Latin quote missing a comma passes; a Japanese one does not. That asymmetry is why ''compass-fit.jp'' failed on one character, and closing it would mean stripping U+3000–U+303F and U+FF00–U+FF65 as well.
   * **Neither citation check can see a quote that is markup.** ''owner_verify_sources.norm()'' strips ''%<...>%'' before searching, which is right for a prose quote and fatal for a citation to a ''%<script src>%'', a ''%<link>%'' or an attribute; ''owner_verify_rendered.mjs'' reads ''document.body.innerText'', which excludes script elements for the same reason. So the two passes, which exist to catch each other's blind spots, share this one — and a **false** NOTFOUND from both was acted on. The bytes pass now also searches an un-stripped copy and reports ''OK-MARKUP'' when that is where the quote lives. Re-running the fixed script over the **first** sitting's 175 rows moves ''fontawesome.com'' from NOTFOUND to OK-MARKUP; that row was already rescued by the browser pass in ''owner_citation_audit.py'', so the published "0 uncited rows" is unchanged and now has two independent routes rather than one.
-  * **The citation check is time-sensitive, and by a measurable amount.** The same 175 rows re-verified six days later move **3 statuses of 175**, two of them for reasons that are purely the web moving: ''cartfulsolutions.com'' went ''OK'' → ''OK-WINDOW'', and ''brightcove.net'' went ''OK-WINDOW'' → ''FETCHFAIL'' on one run and back to ''OK-WINDOW'' on the next, which is a host being briefly unreachable rather than a citation changing. A "zero uncited rows" claim is therefore a claim about a date, and this page should be read as saying the citations verified on 2026-09-05 and 2026-09-11 rather than that they verify.
+  * **The citation check is time-sensitive, but less of what moves is the web than it looks.** Re-verifying the same 175 rows on 2026-09-11 moves **3 statuses of 175**: ''compass-fit.jp'' ''OK'' → ''NOTFOUND'' (the non-Latin fix biting), ''cartfulsolutions.com'' ''OK'' → ''OK-WINDOW'' (the web moving: the footer was restructured and the page grew from 15,933 to 23,044 bytes), and ''fontawesome.com'' ''NOTFOUND'' → ''OK-MARKUP'' (the instrument changing, per the bullet above). ''brightcove.net'' is ''OK-WINDOW'' in **both** retained artefacts; it FETCHFAILed on one execution of the script and answered on the next, which is a host briefly unreachable and not a status that moved. Its quote elides about 20 characters the page has carried since 2026-09-05, so it is an adjudicator elision of the same class as ''compass-fit.jp'' — rescued only because a long quote gets the 60-character window fallback and a 40-character one does not. A "zero uncited rows" claim is therefore a claim about a date, and about a quote length; this page should be read as saying the citations verified on 2026-09-05, on 2026-09-11 and — for three named rows — on 2026-09-17, rather than that they verify.
 === What this section does and does not license ===
@@ Line 651: / Line 665: @@
   * **Kappa is not read here against the usual verbal bands.** They are a convention, not a threshold, and at these marginals — one category carrying most of the mass — kappa is driven as much by the marginals as by the disagreements. Raw agreement is printed beside every kappa for that reason, and ''ADJUDICABILITY'' for Tracker Radar is the extreme case: rater 2 settled all 30 rows, so one rater used a single category, and kappa is **0.000 by construction** at 80.0% raw agreement. A zero there means "undefined", not "no agreement".
   * **Six days separate the two ratings.** Corporate ownership rarely moves in six days, and the frozen probe removes the mechanical part of the drift, but a page that changed in that window is indistinguishable from a rater that read it differently.
+==== C3.9 The 2026-09-11 re-run, applied (2026-09-17) ====
+The ''norm()'' fix of 2026-09-11 ([[#C3.8 Inter-rater agreement: a second model on a random fifth (2026-09-11)|C3.8]]) was made while auditing the //second// rater's rows. It also invalidates the first sitting's pass over every row whose quote is not Latin script, because the broken checker had reported those rows OK on whatever Latin substring they happened to contain. The re-run over all 175 first-sitting rows was made on **2026-09-11** and its consequence recorded as open work. This section is where that work was done, on **2026-09-17**.
+=== What was run, and when ===
+The 175-row re-run is **not** today's work. It was run on 2026-09-11, against an empty cache, as:
+<code>
+python3 scripts/owner_verify_sources.py --rows out/adj_rows_scored.json \
+    --cache out/irr/verify_cache_s1_refix --out out/irr/verify_s1_refix.json
+</code>
+Its artefacts — ''out/irr/verify_s1_refix.json'' and the 133 cached bodies under ''out/irr/verify_cache_s1_refix/'' — all carry 2026-09-11 timestamps, and **that command no longer reproduces the result below**: ''out/adj_rows_scored.json'' now carries the corrected quote, so ''compass-fit.jp'' comes back ''OK''. What was done on 2026-09-17 is the correction, the chain re-run, and a **three-row** live re-fetch of ''compass-fit.jp'', ''brightcove.net'' and ''cartfulsolutions.com'' — not a fourth pass over 175 sources. Compared against ''out/verify_final.json'' (2026-09-05), the 2026-09-11 run moves **three statuses**:
+^ Domain ^ 2026-09-05 ^ 2026-09-11 ^ Which of the three is it? ^
+| ''compass-fit.jp'' | ''OK'' | ''NOTFOUND'' | **the fix biting.** Its quote is Japanese; the broken ''norm()'' had checked it on the six Latin letters ''COMPASS-FIT'' |
+| ''cartfulsolutions.com'' | ''OK'' | ''OK-WINDOW'' | the web moving. The footer was restructured and the page grew from 15,933 to 23,044 bytes; the quote still matches on a 60-character window |
+| ''fontawesome.com'' | ''NOTFOUND'' | ''OK-MARKUP'' | the instrument changing. ''OK-MARKUP'' did not exist on 2026-09-05; the row was rescued by the browser pass then, and by the bytes pass now |
+''brightcove.net'' is ''OK-WINDOW'' in **both** files, so it is not a fourth mover. C3.8's bullet recorded it as ''OK-WINDOW'' → ''FETCHFAIL''; that was an earlier execution of the same script, whose JSON was overwritten by the retained one. The host was briefly unreachable, which is the bullet's point, but the retained artefact does not show it.
+=== What was done about each ===
+  * ''compass-fit.jp'' — **requoted** in ''scripts/owner_corrections.py'' (the eleventh correction; see [[#C3.5 Every citation re-fetched, and the eleven fixed by hand|C3.5]]). The replacement is the body sentence in full, ''%%株式会社マイクロアド…開始致しました。%%'', copied out of the fetched page. It is not the //shortest// span on the page naming a MicroAd entity and the service — the page title is 71 normalised characters and names both — it is the one that carries the **parent and the operating subsidiary together**, which is what ''owner_today = MicroAd, Inc.'' rests on where the old quote named only the service. It verifies ''OK'' against a live fetch of 2026-09-17 (39,110 bytes).
+  * ''cartfulsolutions.com'' — **nothing.** ''OK-WINDOW'' is a passing status by design: a long quote gets spliced by boilerplate, and requiring the literal string would fail every such row. Re-fetched 2026-09-17 (23,044 bytes) and it still names Cartful Solutions, Inc. Requoting it would chase the web and have to be redone the next time the footer is edited.
+  * ''brightcove.net'' — **nothing, but not for the reason it first looks.** Re-fetched 2026-09-17 (806,613 bytes) and still ''OK-WINDOW''. Its quote is **not** a victim of the page changing: the parenthetical it elides, ''%%The Brightcove Privacy Policies ("Privacy Policies") describe Brightcove Inc.%%'', is byte-identical in the 2026-09-05, 2026-09-11 and 2026-09-17 bodies. The row is an adjudicator elision of the **same class** as ''compass-fit.jp'', and the only reason one was requoted and the other was not is **quote length**: at ~110 characters brightcove's clears the 60-character window fallback, and at ~40 compass-fit's cannot. That is the honest line, and it is a property of the checker rather than of the evidence.
+  * ''fontawesome.com'' — **nothing.** The row was already cited and already counted as cited; what changed is that the bytes pass can now see the evidence the browser pass saw on 2026-09-05.
+=== What moved downstream, and what did not ===
+The requote changes one ''quote'' field. It changes **no verdict, no denominator and no rate**, and that is checked rather than asserted: the whole chain was re-run and both estimator outputs are byte-identical to the committed ones.
+^ Re-run ^ Result ^
+| ''owner_corrections.py'' | output changes: ''10 corrections … requote=4'' → ''11 corrections … requote=5'', plus the new row's block |
+| ''owner_verdict_fixes.py'' | byte-identical |
+| ''owner_random_sample.py'' (2026-09-05 rows) | **byte-identical** — all 31 figures unchanged |
+| ''owner_tail_merge.py'' | unchanged but for the ''wrote …'' line; the 135 non-residue rows carry the new quote through, and its own drift gate passes |
+| ''owner_tail_corrections.py'' | unchanged but for the ''wrote …'' line |
+| ''owner_random_sample.py'' (2026-09-11 rows) | **byte-identical** — the estimates the pages actually publish |
+| ''report_tail_pass.py'' | byte-identical |
+| ''owner_citation_audit.py'' | byte-identical — but see below, it reads the 2026-09-05 artefacts and cannot see any of this |
+| ''check_tail_figures.py'' against [[design:ownership_resolution]] | 80 figures, all present and matching |
+| ''check_tail_figures_mutations.sh'' | identical output, 24 of 24 caught |
+| ''check_owner_sample_figures_mutations.sh'' | identical output, 10 of 10 caught |
+Two ''wrote …'' lines in the published output blocks change from ''/tmp/discard_rows.json'' and ''/tmp/discard2.json'' to the paths the documented commands name. The committed blocks had been produced by a verification re-run that deliberately wrote to ''/tmp'' so as not to clobber; the published command line said otherwise. The re-run writes where the command says, so block and command now agree.
+=== What the citation audit still says, and what it would say ===
+''owner_citation_audit.py'' reads ''out/verify.json'' and ''out/verify_final.json'', both 2026-09-05 artefacts. Re-running it today therefore reproduces its old output byte-for-byte and **cannot see** either the fixed checker or this requote. C3.5's table is labelled 2026-09-05 for that reason. Recomputed against the 2026-09-11 statuses **with the requote applied**, the split would be:
+^ State ^ 2026-09-05 (published) ^ with the fix and the requote ^
+| whole quote | 119 | 118 |
+| ellipsis fragments | 8 | 8 |
+| 60-character window | 5 | 6 |
+| markup only | — | 1 |
+| cited in the bytes, total | 132 | **133** |
+| cited only in a rendered browser | 3 | 2 |
+| uncited | **0** | **0** |
+The movement is ''cartfulsolutions.com'' whole → window and ''fontawesome.com'' browser → markup. The only rows the bytes pass still fails are ''force.com'' and ''flashtalking.com'', both carried by the browser pass on 2026-09-05. **No row is uncited under either reading**, which is why the published table was dated rather than replaced: replacing it would mix a 2026-09-05 census with a 2026-09-11 one.
+=== Judgement calls ===
+  * **The correction was appended last in ''CORRECTIONS'', after the ''demote''.** The dict's order is the printed order, and this correction is eleven days younger than the other ten. Grouping it with the other ''requote'' entries would have read as if it had been found with them, which is the one thing about it worth knowing.
+  * **It is counted as a requote but described as a transcription slip.** The remedy is identical to the other four; the defect is not. C3.5 now says four paraphrases and one dropped character rather than five of one thing.
+  * **''brightcove.net'' was left alone on a rule the checker enforces, not on a judgement about the evidence.** Requoting every row whose quote is an elision would mean requoting rows the checker passes, which is a different and much larger piece of work; it is recorded here as a known, quantified gap rather than done.
+  * **The section-G heading on the appendix is now computed** from ''len(CORRECTIONS)'' rather than typed. It said "the ten citations fixed by hand" and would have been wrong the moment this correction landed.
+  * **The reviewer's confirmation in [[#Reviewers, second sitting (2026-09-05)|the 2026-09-05 reviewer log]] still says "10 corrections".** It is left as written: it is a dated record of what that reviewer verified on that day, and editing it would make the log agree with a state it never saw.
+  * **Regenerating the appendix also publishes an unrelated fix.** ''owner_verify_sources.py'' gained a WHOIS retry on 2026-09-11 that the appendix was never regenerated for, so its published block was 524 bytes behind the file. It is current now, and the manifest hash for section E moves with it.
+=== Reviewers (2026-09-17) ===
+^ Finding ^ Decision ^
+| **The published mutation count is wrong: "14 of 14 caught" for ''check_tail_figures_mutations.sh''; the harness catches 24** | **ACCEPTED.** Reproduced: the committed output ends ''mutations caught: 24; survived or broken: 0'' and has 24 ''caught'' lines. A transcription error introduced by this sitting — there was never a 14-mutation version. Fixed |
+| **C3.9 dated the 175-row re-run to 2026-09-17; it ran on 2026-09-11** | **ACCEPTED, and the most useful finding.** Verified from mtimes: ''verify_s1_refix.json'' and all 133 cache bodies are 2026-09-11. Only three rows were fetched on 2026-09-17. The heading and the "What was run" block are rewritten; C3.5's table row is too |
+| **The ''brightcove.net'' rationale is factually wrong.** The elided parenthetical is in the 2026-09-05 body, so it is an adjudicator elision, not the web moving | **ACCEPTED.** Verified against all three cached bodies: byte-identical. The real distinction is quote length against the 60-character window, and both C3.8 and C3.9 now say so |
+| **The fix introduced a Latin/CJK asymmetry the page does not disclose**: ASCII punctuation normalises to whitespace, CJK punctuation does not | **ACCEPTED.** Reproduced against the committed module. Disclosed in C3.8's first bullet, with what closing it would take |
+| **"''tns-counter.ru'', the other non-Latin row" is false**: ''bilibili.com'' is a third | **ACCEPTED.** Three rows in the 175 carry non-Latin text. Both others verify clean under the fix, so no status moved, but the sentence claimed a census it had not done |
+| **"the shortest contiguous span naming both MicroAd and COMPASS-FIT" is false** | **ACCEPTED.** The page title is 71 normalised characters and names both. The real reason for the body sentence is that it names parent and subsidiary together. Fixed in the script's note and here |
+| **C3.5's "10 of 175 rows had their source or quote replaced, 9 of those 10" is stale** | **ACCEPTED.** 11 and 10 of 11 |
+| **Section F still listed the requote as work this sitting did not apply** | **ACCEPTED.** Closed with a pointer here |
+| **The citation-audit table is an undated 2026-09-05 figure that the fix moves** | **ACCEPTED.** Dated, and the recomputed split published above rather than substituted for it |
+| **''check_owner_sample_figures.py'' fails 16 figures against ''design:ownership_resolution''** | **Not a defect, and confirmed pre-existing** by re-running it against the pre-change output. That page publishes the second pass's estimates; this guard checks the 2026-09-05 ones, and its own mutation harness runs against ''pages/wx_new.txt'' and passes |
+| Probe artefacts live in ''out/rq/'' rather than a documented path | **Noted, not fixed.** They are a three-row scratch check, not an artefact any figure rests on; the figure they support (39,110 bytes, ''OK'') is reproducible from the command in this section |
 ===== D. External sources: verified, and rejected =====
@@ Line 754: / Line 855: @@
   * **Discoverability, and where this material really belongs.** The wiki's only treatment of domain-to-company ownership resolution now sits under the name of a dead tool, where nobody asking "how do I attribute a domain?" will look. The generic reviewer was right that the earlier rejection of a separate ''design:ownership_resolution'' page ("it would leave a red link pointing at a stub") was a weak argument — a real webXray page and a topic page can coexist. Mitigated for now by linking this page from [[Privacy:Requests]], which is where a reader asking "whose request is this?" actually lands. Splitting out a topic page is worth doing and is recorded here rather than done.
   * **''%%<WRAP important>%%'' boxes rather than ''%%<wrap todo>%%''.** No open TODOs were left on the content page: the unknowns are stated in its methodology section with what would close them, which is where a reader checking a number will look. The one real TODO — correcting [[Programming:Crawler]] — is recorded here, because it is work on a different page.
-  * **Drawing the sample at all, rather than extending the hand adjudication.** The cheap move was another 30 high-prevalence disagreements, which would have made the existing table look better sourced and would have measured nothing new. The sample is the expensive move and it is the one that produced a **result that contradicts the page's previous impression** — webXray at 69.9% domain-level and 93.9% encounter-weighted, against "1 of 23" from the disagreement table. A page that only ever looks where it already knows the answer is where that impression came from.
+  * **Drawing the sample at all, rather than extending the hand adjudication.** The cheap move was another 30 high-prevalence disagreements, which would have made the existing table look better sourced and would have measured nothing new. The sample is the expensive move and it is the one that produced a **result that contradicts the page's previous impression** — webXray at 69.9% domain-level and 93.9% encounter-weighted //as measured on 2026-09-05// (69.2% and 93.8% after the second pass over the residue), against "1 of 23" from the disagreement table. A page that only ever looks where it already knows the answer is where that impression came from.
   * **Publishing the contradiction prominently instead of quietly correcting the old wording.** The content page now carries both numbers, in the same bullet, with the explanation that the first was a statement about //where it had been checked//. The alternative — deleting "badly stale wherever it has been checked" and moving on — would have hidden the most useful thing the sample found, which is how badly a selection-biased set misleads.
-  * **Not letting the new figure rehabilitate webXray.** 93.9% encounter-weighted is a flattering number and it would have been easy to lead with it. The content page instead puts three counter-facts in the same box: coverage is 2.1%, the file is frozen so the rate only degrades, and Disconnect is better on the measured axis anyway. The recommendation did not change; only the evidence for it got more honest.
+  * **Not letting the new figure rehabilitate webXray.** The encounter-weighted rate — 93.9% on 2026-09-05, **93.8%** after the second pass — is a flattering number and it would have been easy to lead with it. The content page instead puts three counter-facts in the same box: coverage is 2.1%, the file is frozen so the rate only degrades, and Disconnect is better on the measured axis. That last one is now **metric-dependent**: Disconnect beats webXray on current+granularity (//p// = 0.008) and no longer at this page's threshold on current only (//p// = 0.031). The recommendation did not change; only the evidence for it got more honest, and then more qualified.
   * **Quartiles instead of the deciles the follow-up item specified.** See [[#C3.2 Allocation: quartiles, not the deciles the brief asked for|C3.2]]. A reasonable person could have kept deciles and reported the head stratum only.
   * **Retiring a verdict in the middle of the run.** ''self-named'' was in the brief 21 sub-agents worked from, so retiring it means re-scoring by hand after the fact — which is exactly the kind of post-hoc adjustment that should make a reader suspicious. It is published in full, with every re-score and its reason, precisely because it should. The alternative was worse: leaving a verdict that had fired on "TrustArc" and "Klaviyo" and that would have cut Disconnect's denominator by a quarter.
@@ Line 768: / Line 869: @@
   * **Publishing the uncorrected comparison as the headline.** Rater 2's one disqualified citation (''kameleoon.io'') is demoted in a sensitivity line rather than in the headline table, because rater 1's headline rows are pre-correction too and the symmetric comparison is the honest one. The corrected variant moves the pooled figure from +0.400 to +0.400 and is printed beside it.
   * **Not re-estimating the accuracy rates from rater 2's verdicts.** It would have been one line of code and it would have been meaningless: 35 rows is not the sample design, the stratum weights do not apply to it, and substituting 35 of 175 rows produces a hybrid that is neither rater's reading. What is published instead is the unweighted ''current'' share per rater, labelled as a direction and not a rate.
-  * **Fixing ''owner_verify_sources.py'' rather than only recording its defect.** The script is published, its hash is on the appendix manifest, and changing it means the appendix regenerates. The alternative — a note saying the check is blind to non-Latin quotes, beside a check that stays blind — would have left the next reader with an instrument the page knows is broken. The consequence for the first sitting's rows is stated in C3.8 and one row now needs a requote that this sitting did **not** apply.
+  * **Fixing ''owner_verify_sources.py'' rather than only recording its defect.** The script is published, its hash is on the appendix manifest, and changing it means the appendix regenerates. The alternative — a note saying the check is blind to non-Latin quotes, beside a check that stays blind — would have left the next reader with an instrument the page knows is broken. The consequence for the first sitting's rows is stated in C3.8, and the one row that needed a requote got it on **2026-09-17** ([[#C3.9 The 2026-09-11 re-run, applied (2026-09-17)|C3.9]]).
   * **No ''~~DISCUSSION~~'' on this provenance page.** Following the convention set by the earlier provenance pages: comments belong on the content page.
@@ Line 1124: / Line 1225: @@
 ^ Finding ^ Decision ^
-| **"the same universe ''owner_dbs.py'' uses, so the two sittings' figures are on one denominator" is false.** The first sitting's frame is **32,369** domains, the second's **32,337** | **ACCEPTED, blocking, and the most valuable finding of the run.** Reproduced by running ''owner_sample.py --cache out/webxray/cache'' (32,369; 669 / 5,581 / 2,268) against ''--cache out/owner_cache'' (32,337; 664 / 5,566 / 2,264). Both are real. The coverage **shares** round to the same 2.1% / 17.2% / 7.0%, which is precisely why two different frames could sit on one page unnoticed. [[#C3.1 The frame, and what it excludes|C3.1]] now carries three rows on it, and the content page carries a ''%%<WRAP tip>%%'' next to the figures themselves |
+| **"the same universe ''owner_dbs.py'' uses, so the two sittings' figures are on one denominator" is false.** The first sitting's frame is **32,369** domains, the second's **32,337** | **ACCEPTED, blocking, and the most valuable finding of the run.** Reproduced by running ''owner_sample.py %%--cache%% out/webxray/cache'' (32,369; 669 / 5,581 / 2,268) against ''%%--cache%% out/owner_cache'' (32,337; 664 / 5,566 / 2,264). Both are real. The coverage **shares** round to the same 2.1% / 17.2% / 7.0%, which is precisely why two different frames could sit on one page unnoticed. [[#C3.1 The frame, and what it excludes|C3.1]] now carries three rows on it, and the content page carries a ''%%<WRAP tip>%%'' next to the figures themselves |
 | **The run table's hash list is false.** It claimed all six input files were unchanged between the sittings; ''tr_domain_summary'' went ''7a303228d812a41c'' → ''19f7a5a6a839ec87'' (16,310,837 → 16,347,575 bytes) and the Public Suffix List went ''155b43d46932e933'' → ''aef8fb81d63232da'' | **ACCEPTED, blocking.** Verified against ''out/webxray/owner_dbs-output.txt''. This was written from the second sitting's hashes without checking them against the first sitting's — an assumption stated as a measurement, which is the exact failure the page spends a section warning about. The row now lists the four that are unchanged and the two that moved, and says so |
 | **The appendix's manifest hash for ''out/probe_all.txt'' matches no file.** Published ''7d2ca1e722836803''; the file hashes ''3def89e0d3fc8a14'' | **ACCEPTED, blocking.** Root cause found on verification: ''probe_all.txt'' contains **one lone CR**, inside a copyright line scraped from a live site, and Python's text mode silently translates it to LF — so the builder hashed a string that was not the file. Fixed at the root: the builder now reads bytes, hashes the file on disk, discloses newline normalisation **per file** in a new manifest column, and **self-checks** every row and block before writing. The self-check was mutation-tested by reintroducing the original bug, which it rejects. All 23 manifest rows now verify independently |
@@ Line 1196: / Line 1297: @@
 Three focused Sonnet passes ran in parallel against the page drafts, the scripts
-and their outputs; all were told the briefing might not be exhaustive and to
+and their outputs; the generic pass was **skipped for budget**, which is recorded as a gap rather than dressed up as a full review layer. All three were told the briefing might not be exhaustive and to
 verify from the files and the live web rather than from it. **The sitting's worst
 error was found here, not by any guard**, and both reviewers who could have found
@@ Line 1208: / Line 1309: @@
 | Independence and stimulus-identity confirmed by hand: ''prompt00.txt'' is ''INSTRUCTIONS.md'' + ''batch00.txt'' + one output-path line, with no rater-1 verdict, no estimator, and no mention of the ''self-named'' retirement anywhere in it; per-domain blocks byte-identical between ''out/adj/batch*.txt'' and ''out/irr/adj2/batch*.txt'' for four spot-checked domains drawn from four different original batches | **No change needed.** This is the claim the whole section rests on and it is now checked rather than asserted |
 | All kappa figures, intervals, agreement percentages and the 63.0% sourcing/reading split reproduce exactly on an independent re-run; the ''self-named'' counts (6, 4, 3 shared) recomputed from the JSON; no citekeys added; the published ''%%<file>%%'' block matches the script byte-for-byte | **No change needed** |
+| **Three findings from the figures-versus-script pass, none of them a wrong number on the page today.** (a) ''owner_irr_kappa.py'' documented NAMED as "neither rater is forced to ''absent''" while the code selected with ''or''; the two are equivalent only because both raters' ''absent'' cells happen to coincide, which is empirical luck rather than an invariant. (b) The JSON artefact was not byte-stable: ''pe'' summed over a ''set()'', whose iteration order is hash-seed randomised, so the 15th–16th decimal moved between runs. (c) The page pointed at appendix section **AE** for the subsample domain list; AE is a mutation harness and the list is in **AF** | **ALL THREE ACCEPTED.** (a) ''assert_absent_agrees()'' now raises before any population is built, and prints that ''absent'' agrees on all 105 cells; the docstring now says what the code does and why the equivalence is checked rather than assumed. (b) ''sorted(set(...))'', verified byte-stable over three runs under a randomised hash seed. (c) Fixed to AF. The reviewer also noted that the **pooled** bootstrap rows resample cells rather than domain blocks — which the page already discloses, and the estimator's docstring now does too |
+| **The guard was blind to every figure that lives in a script's //text// output rather than in the kappa JSON.** The reviewer mutated the ''self-named'' counts, the sourcing/reading split table, the resolution counts, the unweighted current-share table and the whole citation tally, and ''check_irr_figures.py'' passed on every one | **ACCEPTED, and the most useful finding after the kameleoon one.** The guard now parses those figures out of ''owner_irr_kappa-output.txt'' and ''owner_verify_sources_irr-output.txt'' (a new ''%%--verify-output%%''), and six mutations were added for them. Two of the six then **survived**: the self-named count and the split table were being checked with bare-digit needles like "17", which match anywhere on a long page, and the self-named needle was additionally satisfied by the //other// page's untouched copy. Both fixed — the split table is matched as a whole row, and the self-named count is checked wherever its sentence shape occurs on **every** page. 18 mutations, all caught |
+| **Process finding, and the one this sitting should be judged on: the reviewer was mid-review when the kameleoon fix landed**, so its report straddles two states of the artefact. It said so unprompted | **ACCEPTED as a real methodological failure of this sitting.** The rule is "do not edit a page under its reviewers" and it was broken — the fix was applied the moment two other reviewers reported, without waiting for the third. The reviewer re-checked everything against the post-fix state and its findings above are against that state, but the next sitting should hold fixes until every reviewer has reported, or re-run the ones whose ground moved |
 **What the guards did not catch, and why it matters.** ''check_irr_figures.py''
@@ Line 1227: / Line 1332: @@
 Schema signal: tools[].name matching /webx[\s-]?ray/i with usedOrMentioned in {used, produced} fires on 7 papers.
 Not used/produced (compared/mentioned/unclear): 0 papers.
-Full-text sweep of paper.cols.txt for the same pattern: 15 papers, 0.3% of the 5859-paper corpus and 1.3% of the 1120 that ran a crawl.
+Full-text sweep of paper.cols.txt for the same pattern: 15 papers, 0.3% of the 5859-paper corpus. 13 of them are among the 1120 that ran a crawl (1.2% of that population).
 Schema papers not in the sweep: none (a sweep miss would mean the extractor read a rendering the sweep cannot see).
 Sweep papers the schema misses: 8.
@@ Line 1325: / Line 1430: @@
 Papers matching ANY of webXray / Tracker Radar / WhoTracks.me / Crunchbase / Disconnect-as-a-list:
-papers (2.3% of the corpus; 12.1% of the 1120 that crawled).
+papers (2.3% of the corpus). 107 of them are among the 1120 that crawled -- 9.6% of that population.
 Year   Corpus papers  Naming an ownership resource  Share
 -----  -------------  ----------------------------  -----
@@ Line 1414: / Line 1519: @@
 of 26 literal figures/quotes located in the cited paper's .cols rendering.
-=== H. Evidence quotes behind the schema tuples ===
+=== H. Evidence quotes behind the schema tuples (with PDF fallback) ===
 partial          69%  WWW/2018/an-automated-approach-to-auditing-disclosure-of-third-party-data-collection-in-w | webxray | methodology | "the webxray software platform is used to monitor third-party network traffic generated by loading a given web page and attribute such traffic to the entities which receive the data."
@@ Line 1432: / Line 1537: @@
 partial          67%  IEEE-SP/2022/journey-to-the-center-of-the-cookie-ecosystem-unraveling-actors-roles-and-relati | classification:webxray | methodology | "The first is based on three manually-curated lists (Disconnect [30], WhoTracks.me [31] and webxray [32])"
 exact           100%  IEEE-SP/2022/journey-to-the-center-of-the-cookie-ecosystem-unraveling-actors-roles-and-relati | classification:graph-based label propagation algorithm (custom) | methodology | "We validated our approach by using the manually compiled lists as a “ground truth” reference: we then looked for conflicts"
-Summary: 16 quotes: 4 exact, 11 partial (>=60% of 4-word windows), 1 below threshold.
+Summary: 16 quotes: 4 exact, 11 partial (>=60% of 4-word windows), 0 rescued from the PDF, 1 below threshold in both renderings.
 Below-threshold is not "unsupported": every one checked by hand was present, mangled by a column splice or a dropped citation marker.
@@ Line 1442: / Line 1547: @@
   owner_dbs.py, snapshots of 2026-08-17: webXray 827 owners / 3,215 domains; Tracker Radar 19,148 / 38,368; Disconnect 1,887 / 7,850.
   owner_dbs.py: webXray owners with a parent_id 319 of 827 (38.6%); tree depth up to 6; largest owner groupm with 620 domains.
-  owner_dbs.py: coverage of 45,525 registrable third-party domains -- webXray 657 (1.4%), Tracker Radar 5,539 (12.2%), Disconnect 2,277 (5.0%).
+  owner_dbs.py: coverage of 32,369 registrable third-party domains -- webXray 669 (2.1%), Tracker Radar 5,581 (17.2%), Disconnect 2,268 (7.0%).
-  owner_dbs.py: prevalence-weighted coverage -- webXray 54.5%, Tracker Radar 79.3%, Disconnect 75.8%.
+  owner_dbs.py: prevalence-weighted coverage -- webXray 58.6%, Tracker Radar 84.3%, Disconnect 80.5%.
-  owner_dbs.py: top 100 by prevalence -- webXray 70%, Tracker Radar 95%, Disconnect 91%; top 1,000 -- 26 / 67 / 67%; top 10,000 -- 5 / 27 / 16%.
+  owner_dbs.py: top 100 by prevalence -- webXray 71%, Tracker Radar 98%, Disconnect 94%; top 1,000 -- 26 / 69 / 68%; top 10,000 -- 5 / 29 / 17%.
-  owner_dbs.py: pairwise "needs a human" disagreement -- webXray vs TR 197 of 601 (32.8%), webXray vs Disconnect 216 of 461 (46.9%), TR vs Disconnect 584 of 1,532 (38.1%).
+  owner_dbs.py: pairwise "needs a human" disagreement -- webXray vs TR 198 of 612 (32.4%), webXray vs Disconnect 217 of 464 (46.8%), TR vs Disconnect 584 of 1,538 (38.0%).
-  owner_dbs.py: 16,396 of 47,836 domain_summary.json rows are hostnames not registrable domains; 19 rows are not hostnames at all, one of them the literal string "null".
+  owner_dbs.py: the ICANN-section fold merges 15,651 of domain_summary.json's 47,836 rows into 32,369 registrable domains; 19 rows are not hostnames at all, one of them the literal string "null". (The label-count heuristic that gives 16,396 answers neither question and is not used.)
   webXray licence: PolyForm Strict License 1.0.0 (LICENSE.md in the surviving copy).
   Tracker Radar and Disconnect licences: CC BY-NC-SA 4.0 (LICENSE and entities.json "license" field).
@@ Line 1959: / Line 2064: @@
 ===== J2. Its unedited output =====
-Run as ''python3 scripts/owner_dbs.py --cache out/webxray/cache --disagreements 25''.
+Run as ''python3 scripts/owner_dbs.py %%--cache%% out/webxray/cache %%--disagreements%% 25''.
 <code>
@@ Line 2258: / Line 2363: @@
 ===== K. Unedited output: scripts/owner_adjudication.py =====
-Run as ''python3 scripts/owner_adjudication.py --table''.
+Run as ''python3 scripts/owner_adjudication.py %%--table%%''.
 <code>
@@ Line 2322: / Line 2427: @@
 <bibtex bibliography></bibtex>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Embedded-output refresh, 2026-09-22 =====
+**What was wrong.** The counts on this page were right; the //embedded copy of the
+script's output// in [[#I. Unedited output: scripts/report_webxray.mjs|§I]] was a
+pre-2026-09-11 paste. ''report_webxray.mjs'' has been re-run and the whole block
+regenerated rather than line-edited, so what §I shows is again what the script prints.
+Trigger: the shared quote checkers gained a ''pypdf'' fallback on 2026-09-04 and
+''report_webxray.mjs'' was patched to use it, so its Summary line changed from
+"1 below threshold" to "0 rescued from the PDF, 1 below threshold in both renderings".
+The quote **counts** did not move — 16 quotes, 4 exact, 11 partial, 1 below threshold —
+so the wording was the only thing this refresh was filed for. Diffing the rest of the
+block against the fresh output found four more stale lines, listed below.
+<code>
+$ node scripts/report_webxray.mjs --quotes
+Summary: 16 quotes: 4 exact, 11 partial (>=60% of 4-word windows), 0 rescued from the PDF, 1 below threshold in both renderings.
+</code>
+^ Line in §I (or, for A10, in §A) ^ Was ^ Is ^ Why ^
+| §H Summary | ''1 below threshold'' | ''0 rescued from the PDF, 1 below threshold in both renderings'' | the fallback added 2026-09-04; **no count moved** |
+| §H heading | ''Evidence quotes behind the schema tuples'' | ''… (with PDF fallback)'' | same patch |
+| §A sweep line | ''15 papers … and 1.3% of the 1120 that ran a crawl'' | ''15 papers … 13 of them are among the 1120 that ran a crawl (1.2% of that population)'' | the 2026-09-11 denominator fix: a corpus-wide numerator was being divided by the crawled denominator. [[:programming:crawler:webxray]] has carried the corrected 1.2% since then; only this embedded paste was stale |
+| §E union line | ''136 papers (2.3% of the corpus; 12.1% of the 1120 that crawled)'' | ''136 papers (2.3% of the corpus). 107 of them are among the 1120 that crawled -- 9.6% of that population'' | same fix, larger figure |
+| **§A, row A10** | ''**136** (2.3% / 12.1%)'' | ''**136** (2.3% of the corpus); **107** … (**9.6%**)'' | the only **prose** figure this refresh moved. The content page and ''report_webxray.mjs'' both say 107 / 9.6%; this row still said 12.1% |
+| §Z, five ''owner_dbs.py'' lines | 45,525-domain frame; 54.5 / 79.3 / 75.8%; 16,396 rows | 32,369-domain frame; 58.6 / 84.3 / 80.5%; the ICANN fold merges 15,651 of 47,836 rows | the ICANN-section fold, adopted 2026-09-05. [[#C3.1 The frame, and what it excludes|§C3.1]], §B4 and §B6 have carried the 32,369 frame since then; again only the paste was stale |
+**The lesson, and it is the one this site keeps relearning.** Every figure in the
+prose was already correct. What was stale was the block that exists to //prove// the
+prose — so a reader checking the page against its own embedded output would have been
+shown a contradiction, in the direction of the numbers that had already been retracted.
+An embedded output block has to be regenerated whenever its script changes, not when
+its headline figure changes; and ''check_page_numbers.mjs'' cannot see this, because
+every value in the stale block is a real number that really was printed once.
+**Scope of this edit.** The §I block in full, the §A10 row, and the one sentence in §5
+that quotes the quote-check verdicts. ''owner_dbs.py'' and ''owner_sample.py'' were
+**not** re-run — the §Z lines are literals inside ''report_webxray.mjs'' and were taken
+from its output, not recomputed. No other figure, fold or citation on this page was
+touched, and no ''%%{[citekey]}%%'' was added or moved.
+^ Item ^ Value ^
+| Date | 2026-09-22, unsupervised |
+| Command | ''node scripts/report_webxray.mjs %%--quotes%%'' |
+| Artifact | ''out/qc0922/report_webxray_quotes.txt'' (the full 232-line run pasted into §I) |
+| Script changes | none — ''report_webxray.mjs'' already had the fallback |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Pages saved | this page only — [[:programming:crawler:webxray]] already carries 107 / 9.6% / 1.2% and was verified unchanged |