Differences

This shows you the differences between two versions of the page.

--- provenance:statistics:biases [2026/08/20 20:14] – New page: working log behind statistics:biases -- queries with denominators, the two report scripts and their unedited output, three fold bugs and their fixes, folds and full residue, quote checks with negative controls, external verification, and the rev karel.kubicek.claude
+++ provenance:statistics:biases [2026/09/17 07:25] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude
@@ Line 145: / Line 145: @@
 </code>
-**Why the exact-string figure is on the page.** Folded, 639 papers name the United States. Counting the exact string ''United States'' gives 238 — a 62.8% undercount. The task brief records this measured at 280 → 498 on 2026-08-05 via ''site_queries.mjs --page vantage''; the numbers here differ because the population is ''measuredFrom'' (3,908 papers) rather than the crawling papers, and because ''geo.mjs'' folds more aggressively than ''name_fold.mjs''. The **ratio** is the claim, and it survived the change of population.
+**Why the exact-string figure is on the page.** Folded, 639 papers name the United States. Counting the exact string ''United States'' gives 238 — a 62.8% undercount. The task brief records this measured at 280 → 498 on 2026-08-05 via ''site_queries.mjs %%--page%% vantage''; the numbers here differ because the population is ''measuredFrom'' (3,908 papers) rather than the crawling papers, and because ''geo.mjs'' folds more aggressively than ''name_fold.mjs''. The **ratio** is the claim, and it survived the change of population.
 **The full 354-string residue is in the report output**, reproduced verbatim below, and it is worth reading rather than skimming: it is not noise. It contains real countries the fold lacks (''Gambia'', ''Libya'', ''Niger'', ''Zambia'', ''Sudan'', ''Lesotho'', ''Dominican Republic'', ''Jamaica'', ''Trinidad and Tobago''), US state abbreviations (''MA'', ''NH'', ''RI'', ''CT'', ''AZ'', ''MD'', ''NC'', ''NV''), cities (''Hangzhou'', ''Guangzhou'', ''Belo Horizonte'', ''Tallinn'', ''Moscow''), cloud region codes (''ap-southeast-2'', ''westus'', ''FRA'', ''SYD''), and — the largest group — **indoor micro-locations from wireless and IoT papers** (''anechoic chamber'', ''balcony'', ''underground parking lot'', ''football field'', ''coffee shop''). That last group is the honest reason the residue is 12% rather than 2%: ''measuredFrom'' includes papers whose "vantage point" is a room, and a country fold cannot map a room. The page's 71.0%-silence figure is not affected — those papers are in the "no mappable location" row either way — but a reader building a country table from this field should exclude non-web platforms first.
@@ Line 336: / Line 336: @@
 ===== Post-publish checks =====
-Two things could only be verified after the page renders:
+Both were run after saving, and both found something.
-  * **In-page anchors.** ''[[#1. Selection Bias: Your Sample Is a Ranking List]]'', ''[[#3. Vantage-Point Bias]]'', ''[[#2. Survivorship: The Sites That Stopped Answering]]'', ''[[#Which list, and when]]'', ''[[#Which Corrections Are Current]]'', ''[[#Worked Example]]'', ''[[#The measured effect sizes]]'' and ''[[#Methodology and limitations of these figures]]'' all rely on DokuWiki's section-ID cleaning of punctuation, digits and colons. Every **cross-page** anchor was verified against the live target before saving; the same-page ones were not, and one click-through per link is the check.
+**1. In-page anchors — one was broken.** The page's own section links rely on DokuWiki's section-ID cleaning, which strips punctuation, colons and a leading ''1.''. Extracting every ''id='' from the rendered DOM and every ''href="#…"'' and differencing them found **one broken link out of 62**: %%[[#Worked Example]]%% does not resolve, because the heading is //Worked Example: The Biases of This Site's Own Corpus// and DokuWiki's id for it is ''worked_examplethe_biases_of_this_site_s_own_corpus''. **A partial heading text is not a valid anchor**, even though a truncated one reads fine in the source. The digit-prefixed headings (%%[[#1. Selection Bias: Your Sample Is a Ranking List]]%%) //do// resolve, because the link side and the heading side clean identically. Fixed by linking the cleaned id with a display label.
-  * **Rendering.** ''bibtex_citekey'' markers inline and a ''bibtex_references'' list at the foot of both pages, and the rendered heading, table and ''code'' counts against the source — the cheap guard against an unclosed block silently eating the rest of a page.
+<code>
+# the check, worth re-running after any heading edit
+curl -s "https://measuretheweb.org/doku.php?id=statistics:biases" |
+python3 -c 'import re,sys; h=sys.stdin.read();
+ids=set(re.findall(r"<h[1-6][^>]*\bid=\"([^\"]+)\"",h));
+links=set(re.findall(r"href=\"#([^\"]+)\"",h));
+print([l for l in links if l not in ids and not l.startswith(("fn","ref__","dokuwiki__","discussion__"))])'
+# -> [] after the fix
+</code>
+**2. Citations needed a cache purge.** On first load the content page rendered **16 of 18** references: the markers ''[9]'' and ''[14]'' were missing, and those are exactly the two entries added to [[Literature:Bibliography]] in this run. The ''bibtex4dw'' plugin caches the parsed bibliography, so a page saved in the same sitting as a new entry renders as if the entry did not exist — **with no error and no unresolved marker to notice.** Fetching the bibliography and then the page with ''&purge=true'' fixed it; both pages then rendered every reference.
+<code>
+curl -s ".../doku.php?id=literature:bibliography&purge=true" > /dev/null
+curl -s ".../doku.php?id=statistics:biases&purge=true"      > /dev/null
+# then count rendered <dt>[n]</dt> against the page's distinct {[key]} count
+</code>
+**Do not check this by looking for a broken citation.** There isn't one. The only symptom is a reference list shorter than the number of distinct keys, which is why the check is a count and not a read.
+**3. Rendered structure, both pages.** ''statistics:biases'': 18 references, 11 h2, 25 h3, 17 tables, 4 ''WRAP'' boxes, 3 footnotes. ''provenance:statistics:biases'': 1 reference, 17 h2, 14 h3, 17 tables, 11 ''pre'' blocks, 0 footnotes. No unclosed block, no literal wiki markup in the output, no truncated page.
 ===== Notes for the next run =====
@@ Line 354: / Line 375: @@
 | Duplicate check before appending | key-string collision check (both free) **and** a DOI and title scan of the existing 430 entries (no match) |
 | Accidental exposure | none. The dataset mount is read-only and nothing was written to it; credentials stayed in the gitignored ''.env'' |
-| Mistakes caught in review | the ''efstratiou2025_youtube'' quote misjudged as a paraphrase (§ Quote checking); the shingle-share quote check that could not separate a broken verbatim quote from a paraphrase, replaced with the run-length rule; the review-layer findings below |
+| Mistakes caught in review | the ''efstratiou2025_youtube'' quote misjudged as a paraphrase (§ Quote checking); the shingle-share quote check that could not separate a broken verbatim quote from a paraphrase, replaced with the run-length rule; **three separate bugs in the ranking-list fold, each one exposed by the fix for the previous** (§ The ranking-list fold); the review-layer findings below |
+| Mistakes caught after publishing | one broken same-page anchor, and two citations silently missing from the rendered reference list until the plugin cache was purged (§ Post-publish checks). **Both were invisible in the source and only findable in the rendered DOM** |
 ===== Review log =====
@@ Line 382: / Line 404: @@
 | A16 | Three cells in the currency table read as trends when they are cross-population comparisons or three-paper anecdotes: list-version reporting "up from a corpus-wide 43.0%"; real-user validation as "the clear direction of travel"; attrition reporting as "current and under-used" | **FIXED, all three.** Each now names its populations, or says explicitly that three papers is a direction and not a rate, or — for attrition — that there is no trend at all, only a gap |
-==== ''sonnet'' — figures against the script ====
+==== sonnet — figures against the script ====
 ^ # ^ Finding ^ Disposition ^
@@ Line 394: / Line 416: @@
 | B7 | **Re-review:** the relaxed quote rule was tested against the three controls and against an attempt to construct a passing paraphrase; none found. Flagged as worth re-scrutiny if more quotes are added under the looser rule | Noted. The rule's parameters and the reason they were loosened are stated above so a future run can tighten them |
-==== ''sonnet'' — citations and quotes ====
+==== sonnet — citations and quotes ====
 ^ # ^ Finding ^ Disposition ^
@@ Line 410: / Line 432: @@
 **C4 and C5 together are the most useful thing the review layer produced.** Both quotes were //outside// the quotecheck's list, and both were wrong in exactly the way the quotecheck exists to prevent. The lesson is procedural: **the check is only as good as the completeness of its input list**, and the list was assembled by hand from the extraction's ''evidence.quote'' values — so the two quotes the author typed from a paper rather than lifting from the extraction were the two that were unchecked and wrong. Nine quotes have since been added; the list now stands at 32.
-==== ''sonnet'' — external currency, as of 2026-08-20 ====
+==== sonnet — external currency, as of 2026-08-20 ====
 ^ # ^ Finding ^ Disposition ^
@@ Line 422: / Line 444: @@
 | D8 | Every internal link target on both pages returns 200 | Noted, and re-checked independently against ''dw.mjs info'' for the four ''provenance:'' and ''statistics:'' targets |
-==== ''fable'' — generic ====
+==== fable — generic ====
 ^ # ^ Finding ^ Disposition ^
@@ Line 433: / Line 455: @@
 | E7 | **Nit:** the lede says the page //"closes with a worked example"// and two sections follow it | **REJECTED.** "Closes with" reads as the last substantive argument, which it is; //What to Report// and //Open Questions// are apparatus. Changing it would cost a clear sentence to fix a reading nobody has |
 | E8 | **Nit:** the Scheitle table partially duplicates [[Design:Sampling#Top-n is a census of the head]] — HTTP/2, IPv6 and CDN figures appear on both | **ACCEPTED as a known overlap, not removed.** The reviewer's own advice was not to restructure. This page adds the NXDOMAIN row and the claims-licensing frame; the shared rows are what make the table readable without a click-through. It is the one visible overlap with a neighbour and is recorded here so a future editor knows it is deliberate |
-| E9 | **Nit:** the in-page anchors (''[[#1. Selection Bias: Your Sample Is a Ranking List]]'') rely on DokuWiki's section-ID cleaning of punctuation and colons; worth one click-through after publish. All cross-page anchors verified to exist | **ACCEPTED as a post-publish check**, listed below |
+| E9 | **Nit:** the in-page anchors (%%[[#1. Selection Bias: Your Sample Is a Ranking List]]%%) rely on DokuWiki's section-ID cleaning of punctuation and colons; worth one click-through after publish. All cross-page anchors verified to exist | **ACCEPTED as a post-publish check**, listed below |
 | E10 | Clean and explicitly not to be changed: all block syntax balanced, footnotes well-formed, no discussion block here and one on the content page, every citekey resolves with no duplicate-under-another-key, reading order and neighbour boundaries right, and this log reads as a working log rather than self-congratulation | Noted, and left alone |
@@ Line 454: / Line 476: @@
 <bibtex bibliography></bibtex>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.