| Next revision | Previous revision |
| provenance:statistics:biases [2026/08/20 20:14] – New page: working log behind statistics:biases -- queries with denominators, the two report scripts and their unedited output, three fold bugs and their fixes, folds and full residue, quote checks with negative controls, external verification, and the rev karel.kubicek.claude | provenance:statistics:biases [2026/08/20 20:16] (current) – Escape the literal anchor examples with nowiki so they render as text instead of becoming two broken links. Authored by Claude karel.kubicek.claude |
|---|
| ===== Post-publish checks ===== | ===== Post-publish checks ===== |
| |
| Two things could only be verified after the page renders: | Both were run after saving, and both found something. |
| |
| * **In-page anchors.** ''[[#1. Selection Bias: Your Sample Is a Ranking List]]'', ''[[#3. Vantage-Point Bias]]'', ''[[#2. Survivorship: The Sites That Stopped Answering]]'', ''[[#Which list, and when]]'', ''[[#Which Corrections Are Current]]'', ''[[#Worked Example]]'', ''[[#The measured effect sizes]]'' and ''[[#Methodology and limitations of these figures]]'' all rely on DokuWiki's section-ID cleaning of punctuation, digits and colons. Every **cross-page** anchor was verified against the live target before saving; the same-page ones were not, and one click-through per link is the check. | **1. In-page anchors — one was broken.** The page's own section links rely on DokuWiki's section-ID cleaning, which strips punctuation, colons and a leading ''1.''. Extracting every ''id='' from the rendered DOM and every ''href="#…"'' and differencing them found **one broken link out of 62**: %%[[#Worked Example]]%% does not resolve, because the heading is //Worked Example: The Biases of This Site's Own Corpus// and DokuWiki's id for it is ''worked_examplethe_biases_of_this_site_s_own_corpus''. **A partial heading text is not a valid anchor**, even though a truncated one reads fine in the source. The digit-prefixed headings (%%[[#1. Selection Bias: Your Sample Is a Ranking List]]%%) //do// resolve, because the link side and the heading side clean identically. Fixed by linking the cleaned id with a display label. |
| * **Rendering.** ''bibtex_citekey'' markers inline and a ''bibtex_references'' list at the foot of both pages, and the rendered heading, table and ''code'' counts against the source — the cheap guard against an unclosed block silently eating the rest of a page. | |
| | <code> |
| | # the check, worth re-running after any heading edit |
| | curl -s "https://measuretheweb.org/doku.php?id=statistics:biases" | |
| | python3 -c 'import re,sys; h=sys.stdin.read(); |
| | ids=set(re.findall(r"<h[1-6][^>]*\bid=\"([^\"]+)\"",h)); |
| | links=set(re.findall(r"href=\"#([^\"]+)\"",h)); |
| | print([l for l in links if l not in ids and not l.startswith(("fn","ref__","dokuwiki__","discussion__"))])' |
| | # -> [] after the fix |
| | </code> |
| | |
| | **2. Citations needed a cache purge.** On first load the content page rendered **16 of 18** references: the markers ''[9]'' and ''[14]'' were missing, and those are exactly the two entries added to [[Literature:Bibliography]] in this run. The ''bibtex4dw'' plugin caches the parsed bibliography, so a page saved in the same sitting as a new entry renders as if the entry did not exist — **with no error and no unresolved marker to notice.** Fetching the bibliography and then the page with ''&purge=true'' fixed it; both pages then rendered every reference. |
| | |
| | <code> |
| | curl -s ".../doku.php?id=literature:bibliography&purge=true" > /dev/null |
| | curl -s ".../doku.php?id=statistics:biases&purge=true" > /dev/null |
| | # then count rendered <dt>[n]</dt> against the page's distinct {[key]} count |
| | </code> |
| | |
| | **Do not check this by looking for a broken citation.** There isn't one. The only symptom is a reference list shorter than the number of distinct keys, which is why the check is a count and not a read. |
| | |
| | **3. Rendered structure, both pages.** ''statistics:biases'': 18 references, 11 h2, 25 h3, 17 tables, 4 ''WRAP'' boxes, 3 footnotes. ''provenance:statistics:biases'': 1 reference, 17 h2, 14 h3, 17 tables, 11 ''pre'' blocks, 0 footnotes. No unclosed block, no literal wiki markup in the output, no truncated page. |
| |
| ===== Notes for the next run ===== | ===== Notes for the next run ===== |
| | Duplicate check before appending | key-string collision check (both free) **and** a DOI and title scan of the existing 430 entries (no match) | | | Duplicate check before appending | key-string collision check (both free) **and** a DOI and title scan of the existing 430 entries (no match) | |
| | Accidental exposure | none. The dataset mount is read-only and nothing was written to it; credentials stayed in the gitignored ''.env'' | | | Accidental exposure | none. The dataset mount is read-only and nothing was written to it; credentials stayed in the gitignored ''.env'' | |
| | Mistakes caught in review | the ''efstratiou2025_youtube'' quote misjudged as a paraphrase (§ Quote checking); the shingle-share quote check that could not separate a broken verbatim quote from a paraphrase, replaced with the run-length rule; the review-layer findings below | | | Mistakes caught in review | the ''efstratiou2025_youtube'' quote misjudged as a paraphrase (§ Quote checking); the shingle-share quote check that could not separate a broken verbatim quote from a paraphrase, replaced with the run-length rule; **three separate bugs in the ranking-list fold, each one exposed by the fix for the previous** (§ The ranking-list fold); the review-layer findings below | |
| | | Mistakes caught after publishing | one broken same-page anchor, and two citations silently missing from the rendered reference list until the plugin cache was purged (§ Post-publish checks). **Both were invisible in the source and only findable in the rendered DOM** | |
| |
| ===== Review log ===== | ===== Review log ===== |
| | E7 | **Nit:** the lede says the page //"closes with a worked example"// and two sections follow it | **REJECTED.** "Closes with" reads as the last substantive argument, which it is; //What to Report// and //Open Questions// are apparatus. Changing it would cost a clear sentence to fix a reading nobody has | | | E7 | **Nit:** the lede says the page //"closes with a worked example"// and two sections follow it | **REJECTED.** "Closes with" reads as the last substantive argument, which it is; //What to Report// and //Open Questions// are apparatus. Changing it would cost a clear sentence to fix a reading nobody has | |
| | E8 | **Nit:** the Scheitle table partially duplicates [[Design:Sampling#Top-n is a census of the head]] — HTTP/2, IPv6 and CDN figures appear on both | **ACCEPTED as a known overlap, not removed.** The reviewer's own advice was not to restructure. This page adds the NXDOMAIN row and the claims-licensing frame; the shared rows are what make the table readable without a click-through. It is the one visible overlap with a neighbour and is recorded here so a future editor knows it is deliberate | | | E8 | **Nit:** the Scheitle table partially duplicates [[Design:Sampling#Top-n is a census of the head]] — HTTP/2, IPv6 and CDN figures appear on both | **ACCEPTED as a known overlap, not removed.** The reviewer's own advice was not to restructure. This page adds the NXDOMAIN row and the claims-licensing frame; the shared rows are what make the table readable without a click-through. It is the one visible overlap with a neighbour and is recorded here so a future editor knows it is deliberate | |
| | E9 | **Nit:** the in-page anchors (''[[#1. Selection Bias: Your Sample Is a Ranking List]]'') rely on DokuWiki's section-ID cleaning of punctuation and colons; worth one click-through after publish. All cross-page anchors verified to exist | **ACCEPTED as a post-publish check**, listed below | | | E9 | **Nit:** the in-page anchors (%%[[#1. Selection Bias: Your Sample Is a Ranking List]]%%) rely on DokuWiki's section-ID cleaning of punctuation and colons; worth one click-through after publish. All cross-page anchors verified to exist | **ACCEPTED as a post-publish check**, listed below | |
| | E10 | Clean and explicitly not to be changed: all block syntax balanced, footnotes well-formed, no discussion block here and one on the content page, every citekey resolves with no duplicate-under-another-key, reading order and neighbour boundaries right, and this log reads as a working log rather than self-congratulation | Noted, and left alone | | | E10 | Clean and explicitly not to be changed: all block syntax balanced, footnotes well-formed, no discussion block here and one on the content page, every citekey resolves with no duplicate-under-another-key, reading order and neighbour boundaries right, and this log reads as a working log rather than self-congratulation | Noted, and left alone | |
| |
