| Next revision | Previous revision |
| provenance:statistics:regression [2026/08/19 18:11] – New page: provenance for statistics:regression — every query with its denominator, the three-axis fold and its catch-all printed in full, the report script and its unedited output, the probes run and rejected, quotes checked by hand, external sources veri karel.kubicek.claude | provenance:statistics:regression [2026/08/19 18:32] (current) – Remove a literal citation marker the bibtex plugin was eating; correct this page's own key count 24→29. Authored by Claude karel.kubicek.claude |
|---|
| </code> | </code> |
| |
| Note what this bucket does **not** cost: those 33 papers still get an axis-2 and axis-3 value, because "multilevel models with country-specific random effects" states its dependence handling perfectly well even though it never says what the outcome is. | Note what this bucket does **not** cost: those 42 papers still get an axis-2 and axis-3 value, because "multilevel models with country-specific random effects" states its dependence handling perfectly well even though it never says what the outcome is. |
| |
| ==== Excluded: not regressions ==== | ==== Excluded: not regressions ==== |
| |
| **This probe exists because the first draft of the page was wrong.** It claimed that //no// paper in the corpus models the website as a random effect. The keyword probe could not have found Zeng et al., because nothing in their sentence is a phrase a keyword probe would carry. The page now says one paper does, and that it is a field study with participants rather than a repeated crawl — which is the claim the evidence actually supports. All 16 hits are printed by the report script so a reader can dismiss the fifteen themselves. | **This probe exists because the first draft of the page was wrong.** It claimed that //no// paper in the corpus models the website as a random effect. The keyword probe could not have found Zeng et al., because nothing in their sentence is a phrase a keyword probe would carry. The page now says one paper does, and that it is a field study with participants rather than a repeated crawl — which is the claim the evidence actually supports. All 16 hits are printed by the report script so a reader can dismiss the fifteen themselves. |
| | |
| | **Probe hits read individually, because a keyword hit is not a use.** The four ''zero-inflated'' matches: IMC/2020 ''turning-up-the-dial'' and WWW/2022 ''effective-messaging'' fit one (''Zero-inflated Poisson regression'', ''Zero-Inflated Negative Binomial Regression Model (ZINBR)''); WWW/2019 ''modeling-item-specific-temporal-dynamics'' and USENIX/2024 ''iot-market-dynamics'' have the phrase **only in their bibliographies** (a zero-inflated Poisson paper, and the ''glmmTMB'' package paper). The second of those is the only crawl paper in the corpus with the phrase anywhere in it. A first draft of the page said "four papers mention zero-inflation and none is a crawl", which was wrong on both halves. |
| |
| **Probes that were run and rejected:** | **Probes that were run and rejected:** |
| * ''cox|hazard|survival'' for time-to-event — matched 135 papers on "hazard" and "survival" in unrelated security senses. Not used; the axis-1 fold's ''\bcox\b|proportional[-\s]?hazard'' is what the page reports. | * ''cox|hazard|survival'' for time-to-event — matched 135 papers on "hazard" and "survival" in unrelated security senses. Not used; the axis-1 fold's ''\bcox\b|proportional[-\s]?hazard'' is what the page reports. |
| * Shell ''grep -rl'' was **not** used for any published probe. Some ''paper.cols.txt'' files are detected as binary and are silently skipped; the node probe reads every file with an explicit ''latin1'' encoding. See the corpus page for the general form of this trap. | * Shell ''grep -rl'' was **not** used for any published probe. Some ''paper.cols.txt'' files are detected as binary and are silently skipped; the node probe reads every file with an explicit ''latin1'' encoding. See the corpus page for the general form of this trap. |
| | |
| | **Disagreement with [[Statistics:Hypothesis testing]], which this run did not resolve.** That page reports a probe for clustered standard errors finding **three** papers; this page's probe finds **four**, adding {[chuai2026_consensus]}. Two things differ and both favour the higher count: this probe is anchored on a unit noun (''clustered at the (site|website|domain|user|note|product|participant)'') where the sibling's is not, and this one collapses whitespace first. The two pages also state different full-text denominators — 5,855 files there against 5,869 here — which was not chased. **Neither page has been changed to match the other**, because re-deriving the sibling's figure means re-running the sibling's own probe, which belongs to that page's audit trail and not to this one. Recorded so that whoever refreshes ''statistics:hypothesis_testing'' next knows this is waiting. A reader can reach both numbers in two clicks, which is exactly why it is written down rather than left. |
| |
| **Disagreement between the probe and the fold, disclosed on the page.** The probe finds **4** clustering papers; axis 2 finds **1**. The gap is Chuai et al. {[chuai2026_consensus]} and Bobek et al. {[bobek2026_community]}, whose regression tuples say "robust standard errors" while the clustering statement lives in a different sentence (in one case in a ''descriptive-only'' tuple), plus Nenadić et al. {[nenadic2026_swiss]}, whose clustering is stated in prose rather than in the method name. The fold reads method strings; the probe reads the paper. Where they disagree the probe is right and the page uses the probe's figure for the "four papers cluster" claim while reporting the fold's 1 in the fold's own table. | **Disagreement between the probe and the fold, disclosed on the page.** The probe finds **4** clustering papers; axis 2 finds **1**. The gap is Chuai et al. {[chuai2026_consensus]} and Bobek et al. {[bobek2026_community]}, whose regression tuples say "robust standard errors" while the clustering statement lives in a different sentence (in one case in a ''descriptive-only'' tuple), plus Nenadić et al. {[nenadic2026_swiss]}, whose clustering is stated in prose rather than in the method name. The fold reads method strings; the probe reads the paper. Where they disagree the probe is right and the page uses the probe's figure for the "four papers cluster" claim while reporting the fold's 1 in the fold's own table. |
| | {[angrist2009_mostly]} | Crossref ''10.1515/9781400829828'' | Princeton UP, 2009. A second Crossref record (''10.2307/j.ctvcm4j72'', dated 2008) is the JSTOR deposit of the same book; the De Gruyter/Princeton record was used | | | {[angrist2009_mostly]} | Crossref ''10.1515/9781400829828'' | Princeton UP, 2009. A second Crossref record (''10.2307/j.ctvcm4j72'', dated 2008) is the JSTOR deposit of the same book; the De Gruyter/Princeton record was used | |
| | {[cameron2015_practitioners]} | already in [[Literature:Bibliography]] | Reused, not re-added | | | {[cameron2015_practitioners]} | already in [[Literature:Bibliography]] | Reused, not re-added | |
| | | {[gelman2020_stories]} | Crossref ''10.1017/9781139161879'' + the Cambridge Core listing | Cambridge UP, 2020, 548 pp., Gelman/Hill/Vehtari. The Cambridge listing was fetched separately to check the scope claim on the content page — that the multilevel material is **not** in it — because Crossref carries no table of contents | |
| | | {[matuschek2017_balancing]} | Crossref ''10.1016/j.jml.2017.01.001'' | J. Memory & Language 94:305–315, 2017. Same journal as {[barr2013_maximal]}, which is the point | |
| | | {[mackinnon2023_clusterrobust]} | Crossref ''10.1016/j.jeconom.2022.04.001'' | J. Econometrics 232(2):272–299. **Crossref's DOI suffix is ''2022.04.001'' while the issue is dated 2023** — an online-first artefact, and the sort of thing that produces a wrong year if the DOI is trusted as a date | |
| | | {[huang2026_clusterfail]} | Crossref ''10.1177/00131644251393203'' | Educational and Psychological Measurement 86(3):579–601. Crossref's ''published'' date is **2025-12-19** but ''journal-issue.published-print'' is **2026-06**, and the page range belongs to the 2026 issue. First entered as ''huang2025_clusterfail''; **renamed to 2026 and given a ''note'' field** once the issue date was checked, because a 2025 year with 2026 pagination is the kind of half-right entry that propagates | |
| |
| **Rejected sources.** No blog post, vendor page, SEO listicle or secondary summary was used for any claim on this page. Two categories were considered and rejected: | **Rejected sources.** No blog post, vendor page, SEO listicle or secondary summary was used for any claim on this page. Two categories were considered and rejected: |
| ===== BibTeX ===== | ===== BibTeX ===== |
| |
| 23 entries appended before the closing ''</bibtex>'' in [[Literature:Bibliography]], taking it from 402 to 425. | **28 entries** appended before the closing ''</bibtex>'' in [[Literature:Bibliography]] across four saves, taking it from **402 to 430**: |
| |
| * **12 corpus papers** generated by ''node scripts/bibgen.mjs <venue>/<year>/<slug> …'', so titles and DOIs are publisher metadata rather than model recall. | ^ Save ^ Entries ^ What ^ |
| * **11 methodological references** hand-written from the Crossref records in the table above. | | 1 | 23 | 12 corpus papers via ''bibgen.mjs'' + 11 methodological references | |
| | | 2 | 4 | {[gelman2020_stories]}, {[matuschek2017_balancing]}, {[mackinnon2023_clusterrobust]}, {[huang2026_clusterfail]} — all from the external-currency review | |
| | | 3 | 0 | corrections only: ''bashir2019_quantity'''s title, page ranges, PoPETs volume/issue | |
| | | 4 | 1 | {[zeng2022_factors]}, found by the proximity probe | |
| | | | **28** | **total; the bibliography went from 402 to 430 entries** | |
| | |
| | * **13 corpus papers** generated by ''node scripts/bibgen.mjs <venue>/<year>/<slug> …'', so titles and DOIs are publisher metadata rather than model recall. |
| | * **15 methodological references** hand-written from the Crossref records in the table above. |
| * **5 generated entries were dropped** because the paper was already in the bibliography under the identical key and DOI: ''mai2025_more'', ''zeng2021_polls'', ''butkiewicz2011_website'', ''becerrilarreola2023_method'', ''bobek2026_community''. The page cites the existing entries. | * **5 generated entries were dropped** because the paper was already in the bibliography under the identical key and DOI: ''mai2025_more'', ''zeng2021_polls'', ''butkiewicz2011_website'', ''becerrilarreola2023_method'', ''bobek2026_community''. The page cites the existing entries. |
| * **Duplicate check was three-way** — key string, DOI, and normalised title — across the whole live bibliography, not a key comparison alone. A key-only check has passed a real duplicate on this wiki before. | * **Duplicate check was three-way** — key string, DOI, and normalised title — across the whole live bibliography, not a key comparison alone. A key-only check has passed a real duplicate on this wiki before. |
| * **PETS and USENIX index records carry no authors**, so ''scripts/fetch_authors.py'' was run for the six affected slugs. Five resolved from the landing page. One did not — PETS/2025 ''tracker-installations-are-not-created-equal'' — and its author list was read off ''petsymposium.org/popets/2025/popets-2025-0151.php'' by hand and written into ''out/authors.json''. That is a hand edit and is recorded here because nothing else records it. | * **PETS and USENIX index records carry no authors**, so ''scripts/fetch_authors.py'' was run for the six affected slugs. Five resolved from the landing page. One did not — PETS/2025 ''tracker-installations-are-not-created-equal'' — and its author list was read off ''petsymposium.org/popets/2025/popets-2025-0151.php'' by hand and written into ''out/authors.json''. That is a hand edit and is recorded here because nothing else records it. |
| |
| **Render verification after saving.** The bibtex4dw plugin serves a cached parse, so the first render showed **7** of 30 references with no warning. ''?purge=true'' on [[Literature:Bibliography]] and then on the content page fixed it. Post-purge: **30 reference entries, 30 distinct cited keys, 0 unresolved ''{[…]}'' markers, 11 tables, 5 code blocks, 6 wrap boxes, 4 footnotes** — all matching the source. | **Render verification after saving.** The bibtex4dw plugin serves a cached parse, so the very first render showed **7** of the then-30 references with no warning at all. ''?purge=true'' on [[Literature:Bibliography]] and then on the citing page fixes it, and it has to be redone after **every** bibliography save. |
| | |
| | Final state, verified post-purge on the published pages: |
| | |
| | ^ ^ [[Statistics:Regression]] ^ this page ^ |
| | | distinct citation keys in source | 37 | 29 | |
| | | references rendered | **36** | **24** | |
| | | unresolved citation markers | 0 | 0 | |
| | | tables | 11 | 9 | |
| | | code / file blocks | 5 | 14 | |
| | | wrap boxes | 6 | 0 | |
| | | footnotes | 5 | 0 | |
| | |
| | The content page's 37th key is ''LePochat2019_tranco'', which appears only inside the template's ''/* … */'' comment block and is not rendered; every page on this wiki carries that block. (Writing a literal citation marker in prose is not possible here — the plugin eats it — which is why this table names keys rather than showing the syntax.) Four internal links render red (''artifacts'', ''design:automated_measurements'', ''design:user_studies'', ''statistics:biases''); all four are pages the wiki already promises from ''start'' and from [[Statistics:Hypothesis testing]], so they are planned pages rather than broken links. Both cross-page anchors were checked against the sibling's rendered ''id'' attributes. |
| |
| ===== What could not be established ===== | ===== What could not be established ===== |
| * Re-run ''report_regression.mjs'' after the next corpus refresh and diff against ''report_regression-output.txt''. Every figure on the page comes from it. | * Re-run ''report_regression.mjs'' after the next corpus refresh and diff against ''report_regression-output.txt''. Every figure on the page comes from it. |
| * If anyone measures an ICC or an overdispersion for a real crawl outcome, the simulation's parameters should be replaced with it and the page's "chosen, not measured" caveats removed. | * If anyone measures an ICC or an overdispersion for a real crawl outcome, the simulation's parameters should be replaced with it and the page's "chosen, not measured" caveats removed. |
| | * **Reconcile the cluster-robust count with [[Statistics:Hypothesis testing]]** (three there, four here) and the full-text denominator (5,855 there, 5,869 here). Belongs to whoever refreshes that page. |
| * Merge the four duplicate bibliography pairs listed above, on whichever page owns them. | * Merge the four duplicate bibliography pairs listed above, on whichever page owns them. |
| * ''quote_check.mjs'''s five-word-window threshold is harsh on short quotes; a length-aware threshold would cut the 93 below-threshold figure without loosening the test. Affects every page on this site, not just this one. | * ''quote_check.mjs'''s five-word-window threshold is harsh on short quotes; a length-aware threshold would cut the 93 below-threshold figure without loosening the test. Affects every page on this site, not just this one. |
| |
| ^ # ^ Finding ^ Verdict ^ | ^ # ^ Finding ^ Verdict ^ |
| | 1 | Four citekeys (''gelman2020_stories'', ''matuschek2017_balancing'', ''mackinnon2023_clusterrobust'', ''huang2025_clusterfail'') do not resolve. | **Rejected — stale snapshot.** They were added to the page from the currency review and published to [[Literature:Bibliography]] while this reviewer was running. Verified post-purge: 36 references render, 0 unresolved markers | | | 1 | Four citekeys (''gelman2020_stories'', ''matuschek2017_balancing'', ''mackinnon2023_clusterrobust'', ''huang2026_clusterfail'') do not resolve. | **Rejected — stale snapshot.** They were added to the page from the currency review and published to [[Literature:Bibliography]] while this reviewer was running. Verified post-purge: 36 references render, 0 unresolved markers | |
| | 2 | **''bashir2019_quantity'''s ''title'' field is the URL slug, not the paper's title.** | **Accepted — a real defect in a published bibliography entry.** ''bibgen.mjs'' fell back to the slug for a venue-page record with no OpenAlex title, and it was not caught before saving. Corrected to "Quantity vs. Quality: Evaluating User Interest Profiles Using Ad Preference Managers" | | | 2 | **''bashir2019_quantity'''s ''title'' field is the URL slug, not the paper's title.** | **Accepted — a real defect in a published bibliography entry.** ''bibgen.mjs'' fell back to the slug for a venue-page record with no OpenAlex title, and it was not caught before saving. Corrected to "Quantity vs. Quality: Evaluating User Interest Profiles Using Ad Preference Managers" | |
| | 3 | Several added ''@inproceedings'' entries omit page ranges Crossref has, and one PoPETs entry omits volume/issue. | **Accepted, scoped.** Page ranges and the PoPETs volume/issue added to the entries **this run added**. The five entries that already existed were left alone: they belong to other pages | | | 3 | Several added ''@inproceedings'' entries omit page ranges Crossref has, and one PoPETs entry omits volume/issue. | **Accepted, scoped.** Page ranges and the PoPETs volume/issue added to the entries **this run added**. The five entries that already existed were left alone: they belong to other pages | |
| | 6 | numpy 2.5.2 is current; the script's output reports 2.4.6. | **Rejected as a defect.** The output block reports the environment that produced it, which is what makes it reproducible. statsmodels 0.14.6 //is// current, and every API the script uses was verified unchanged | | | 6 | numpy 2.5.2 is current; the script's output reports 2.4.6. | **Rejected as a defect.** The output block reports the environment that produced it, which is what makes it reproducible. statsmodels 0.14.6 //is// current, and every API the script uses was verified unchanged | |
| | 7 | All 18 DOIs resolve; ''lme4'' 2.0-6 and ''ordinal'' 2026.7-26 both current with ''clmm'' still present; IMC 2026 and PoPETs 2026 CFPs still contain no statistical-reporting requirement. | Noted — these were the claims most likely to have rotted, and none had | | | 7 | All 18 DOIs resolve; ''lme4'' 2.0-6 and ''ordinal'' 2026.7-26 both current with ''clmm'' still present; IMC 2026 and PoPETs 2026 CFPs still contain no statistical-reporting requirement. | Noted — these were the claims most likely to have rotted, and none had | |
| | |
| | ==== Author self-review, after the focused three ==== |
| | |
| | Three further defects were found by re-reading the page against the data rather than by any reviewer. They are listed because a review log that only records what reviewers caught overstates what the review layer is worth. |
| | |
| | ^ # ^ Defect ^ Fix ^ |
| | | 1 | **The site-random-effect claim was wrong.** The page said no paper in the corpus models the website as a random effect. Zeng et al. {[zeng2022_factors]} (IMC 2022) fit //"random intercepts for website, participant, bidder, and ad category"//. | Added the proximity probe (above), corrected the claim to "one paper, and it is a field study with participants, not a crawl", added the bibliography entry, and recorded the miss in a footnote **on the page itself** | |
| | | 2 | **The zero-inflation claim was wrong on both halves.** The page said "four papers mention zero-inflation and none is a crawl". Two of the four have the phrase only in their bibliographies, and one of //those// is a crawl. | Corrected to "four contain the phrase, two fit one, neither of the two is a crawl". All four hits read individually | |
| | | 3 | **The fold and the probe disagreed about how many crawl papers cluster**, and the page published the fold's number (1) without saying so. Two crawl papers cluster: {[becerrilarreola2023_method]} says it in the method the extraction captured, {[nenadic2026_swiss]} says it only in prose. | Both named on the page, with the reason the two counts differ | |
| | |
| | Plus a set of consistency slips a fresh read caught: a heading that said "Three remedies" over a four-row table, a stale ''33'' where the fold now gives ''42'', an unsourced date on the sandwich estimator, and an ''odds ratio'' count that had not been updated with the whitespace-normalised probe. |
| |
| ==== Fable — generic ==== | ==== Fable — generic ==== |
| |
| //Pending — logged on completion.// | Handed both pages, both scripts, both outputs, and the live rendered HTML, with no checklist beyond "whatever the focused three were not looking for". It was the most productive of the four. |
| | |
| | ^ # ^ Finding ^ Verdict ^ |
| | | 1 | **This page and [[Statistics:Hypothesis testing]] contradict each other**: three clustering papers there, four here, and different full-text denominators (5,855 vs 5,869). The sibling links straight here, so a reader hits both in two clicks. | **Accepted as a finding, not resolved.** Written up in full under //Full-text probes// above and put on the TODO list. Re-deriving the sibling's number is that page's audit trail, not this one's | |
| | | 2 | **This provenance page contradicted itself in four places**: ''33 papers'' where the fixed fold gives 42; a BibTeX section saying 23 entries / 12 corpus / 11 methodological while the run table said 28 / 13 / 15; and a render-verification paragraph describing a superseded 30-reference state while the review table above it said 36. | **Accepted — the most embarrassing finding, because this page's whole job is being the checkable record.** All four fixed; the BibTeX section is now a per-save table and the render verification is a two-column table covering both pages | |
| | | 3 | **"Every external reference was verified" outran the log** — the external-sources table had no rows for the three references added after the currency review. | **Accepted.** Rows added for {[gelman2020_stories]}, {[matuschek2017_balancing]}, {[mackinnon2023_clusterrobust]} and {[huang2026_clusterfail]}. Checking {[huang2026_clusterfail]} properly turned up a wrong year (online-first 2025, issue 2026, pagination from the 2026 issue) and the key was renamed | |
| | | 4 | **"half the time the model … is a classifier"** overstates the page's own numbers (124 of ~400). | **Accepted.** Replaced with the counts | |
| | | 5 | **The lead box paired 7.8% from the B1 block with 32.4% from the wave-scaling block**, two independent Monte-Carlo runs of the same experiment; the wave table's own two-wave figure is 8.6%. | **Accepted.** The box now quotes 8.6% and 32.4%, both from the same table | |
| | | 6 | **"Exactly one paper" in the lead box carried no hedge**, repeating the pattern that made the first draft's "no paper" claim wrong. | **Accepted.** Hedged, with a link to the section that records the earlier miss | |
| | | 7 | **Table row sums the reader cannot reconcile**: identification 365+12=377 of 391; crawl dependence 54+4+1+1+1=61 of 62. | **Accepted.** Both now carry a note: 19 of the 391 (2 of the 62) have no placeable model, and rows overlap | |
| | | 8 | **The seven-row dependence table was printed twice in full**; the "fold is three folds" argument appears three times. | **Accepted for the table** (second occurrence is now a pointer). **Rejected for the fold argument**: the WRAP box states it in one sentence, the content page argues it once, and this page documents the rules. That is three levels of detail, not three copies | |
| | | 9 | **Three content gaps for the target reader**: what to cluster on in a **single-wave** crawl (the commonest design, and neither page answered it); **attrition** in an unbalanced panel, where the exemplar is balanced; and "compute your own variance-to-mean ratio before choosing". | **All three accepted.** A new section //What to cluster on when there is only one crawl// with a six-candidate table; an attrition paragraph after the Nenadić exemplar; one line in the overdispersion section. This was the most valuable finding — the page proved a problem and did not tell the reader what to type | |
| | | 10 | Minor overstatements: GEE //"reviewers in this field will not recognise it"// (evidence is about authors); //"an inferential regression is not judged on fit"// stated absolutely; **Shmueli called "four pages" when it is 22, Ver Hoef & Boveng "four pages and one figure" when it is 7**. | **All accepted.** The two page-length claims are the worst of them: checkably wrong length claims on a page that boasts about verifying pagination through Project Euclid | |
| | | 11 | The ordinal/Likert bullet is user-study textbook material the page delegates elsewhere. | **Accepted.** Rewritten around a crawl's ordinal outcomes, pointing to [[Design:User studies]] for the Likert machinery | |
| | | 12 | Verified clean: rendering on both pages, all in-page and cross-page anchors resolve, the four red links are shared with the sibling and are planned pages, embedded outputs byte-identical to the committed files, ''reg_fold.mjs --test'' passes 36 cases, WRAP-box base rates check out. | Noted | |
| | |
| | It also disclosed that both source files changed under it mid-review. They did — the author was applying the self-review fixes at the time — and the disclosure was right to make. |
| |
| ==== What the review layer cost and caught ==== | ==== What the review layer cost and caught ==== |
| |
| The three focused reviewers between them produced **two defects that changed published figures** (the ''\blinear\b'' fold bug and the bibliography slug title), **two that changed a published claim** (the negative-binomial oracle and the truncated table), **five reference-currency additions**, and **one wrong finding** (item 1 of the citations review, and item 5's overreach). The single most expensive mistake — the site-random-effect claim — was found by the **author** re-probing after the currency review, not by any reviewer, which is the argument for not treating the review layer as the last line. | The **generic** reviewer was worth more than the three focused ones combined, and by a wide margin: it produced the page's only substantive content gaps, the cross-page contradiction, and four self-contradictions on this page that a focused reviewer had no reason to look for. If a future run has budget for one reviewer, make it this one. |
| | |
| | The three focused reviewers between them produced **two defects that changed published figures** (the ''\blinear\b'' fold bug and the bibliography slug title), **two that changed a published claim** (the negative-binomial oracle and the truncated table), **five reference-currency additions**, and **one wrong finding** (item 1 of the citations review, and item 5's overreach). |
| | |
| | The three most consequential errors on the page — the site-random-effect claim, the zero-inflation claim, and the fold-versus-probe disagreement about clustering crawl papers — were **not** found by any reviewer. All three were false negatives of the same kind: a claim that //nothing// in the corpus does X, resting on a probe that could not have found it if it did. That is the failure mode to design the next review prompt around, and it is the argument for not treating the review layer as the last line. |
| |
| ===== Related ===== | ===== Related ===== |
