| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:statistics:hypothesis_testing [2026/09/04 08:12] – RETRACT the 'quotes in the PDF and in none of the stored renderings' section — all are in paper.cols.txt; the negative came from a grep wrapper with -I. Refresh the quote check (541/118/216), close the TODO, fix <wrap>. Authored by Claude karel.kubicek.claude | provenance:statistics:hypothesis_testing [2026/09/17 07:25] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude |
|---|
| |
| ^ Item ^ Value ^ | ^ Item ^ Value ^ |
| | Date | 2026-08-13 | | | Date | 2026-08-13. **One section of this page was retracted and the quote-check figures refreshed on 2026-09-04** — see //Correction pass// at the foot. Everything else stands as written on 2026-08-13. | |
| | Corpus | ''data/extract/run1'', 5,859 extracted papers, 7 venues, 2010–2026 | | | Corpus | ''data/extract/run1'', 5,859 extracted papers, 7 venues, 2010–2026 | |
| | Page status | **New page.** ''statistics:hypothesis_testing'' was a red link promised from ''start'' ("[[Statistics:Hypothesis testing]] suitable for web measurements") and from [[Statistics:Pvalue corrections]] ("choosing the test whose //p//-values you are about to correct"). Nothing on the wiki covered it | | | Page status | **New page.** ''statistics:hypothesis_testing'' was a red link promised from ''start'' ("[[Statistics:Hypothesis testing]] suitable for web measurements") and from [[Statistics:Pvalue corrections]] ("choosing the test whose //p//-values you are about to correct"). Nothing on the wiki covered it | |
| ''participants'', not ''humanAnnotation'' — crowdworkers labelling data are annotators, not subjects, and conflating them double-counts Mechanical Turk. | ''participants'', not ''humanAnnotation'' — crowdworkers labelling data are annotators, not subjects, and conflating them double-counts Mechanical Turk. |
| |
| ===== The fold: ''test_fold.mjs'' ===== | ===== The fold: test_fold.mjs ===== |
| |
| ''statistics[].method'' is free text and agrees run-to-run on ~20% of exact strings, so it is never aggregated by exact string. **798 distinct strings describe about forty procedures.** The fold is an ordered rule list; first match wins; anything matching no rule lands in ''residue'' and is printed. | ''statistics[].method'' is free text and agrees run-to-run on ~20% of exact strings, so it is never aggregated by exact string. **798 distinct strings describe about forty procedures.** The fold is an ordered rule list; first match wins; anything matching no rule lands in ''residue'' and is printed. |
| </code> | </code> |
| |
| **118 of the 334 are defects in the stored text, not in the extraction.** They are quotes the ''paper.cols.txt'' check cannot locate and an independent rendering of the same PDF finds. The corpus-wide version of this measurement, its method and its limits are on [[provenance:literature:corpus]]; the short version is that ''decolumn.mjs'' skips the two-column repair on any page whose gutter it cannot find, and a quote on such a page is spliced to the neighbouring column every ten words or so. | **118 of the 334 are defects in the stored text, not in the extraction.** They are quotes the ''paper.cols.txt'' check cannot locate and an independent rendering of the same PDF finds. The same defect measured corpus-wide is on [[provenance:literature:corpus]] — **under a different and much stricter rule, so the two percentages are not comparable.** This script's threshold is 60% of a quote's 5-word windows; that page scores every quote as //exact// / //8-word run// / //fragmented// / //not found//, and a quote below this script's threshold is usually still an //8-word run// there. The mechanism is the same in both: ''decolumn.mjs'' skips the two-column repair on any page whose gutter it cannot find, and a quote on such a page is spliced to the neighbouring column every ten words or so. |
| |
| **Below threshold is not "unsupported".** Five below-threshold quotes were chased by hand and **all five are present**: | **Below threshold is not "unsupported".** Five below-threshold quotes were chased by hand and **all five are present**: |
| Every quote published on the content page was located by hand. | Every quote published on the content page was located by hand. |
| |
| ==== RETRACTED, 2026-09-04: the Mai et al. quotes are in ''paper.cols.txt'' ==== | ==== RETRACTED, 2026-09-04: the Mai et al. quotes are in paper.cols.txt ==== |
| |
| This section previously said that four of Mai et al.'s methodology quotes were present in the stored ''paper.pdf'' and in **none** of ''paper.cols.txt'', ''paper.norm.txt'' or ''paper.txt'', and drew three consequences from it. **The claim was false and the consequences with it.** It is kept here rather than deleted because the way it was got wrong is worth more than the claim was. | This section previously said that four of Mai et al.'s methodology quotes were present in the stored ''paper.pdf'' and in **none** of ''paper.cols.txt'', ''paper.norm.txt'' or ''paper.txt'', and drew three consequences from it. **The claim was false and the consequences with it.** The original text is **summarised rather than reproduced** below, and kept rather than deleted, because the way it was got wrong is worth more than the claim was. |
| |
| What is actually true, checked on 2026-09-04 in Python rather than through the shell: | What is actually true, checked on 2026-09-04 in Python rather than through the shell: |
| | A8 | The bug table below contained a backslash-escaped pipe inside a table cell. **DokuWiki does not treat a backslash-escaped pipe as an escape** — the pipe still delimits, so the row rendered with 5 cells in a 3-column table and broke the table from there down | **FIXED.** Found by a cell-count consistency check over every table on both pages, not by reading. **And then it happened again**: the row you are reading originally quoted the offending character and broke this table too. The check caught that as well. Do not write a pipe inside a DokuWiki table cell, not even to talk about one | | | A8 | The bug table below contained a backslash-escaped pipe inside a table cell. **DokuWiki does not treat a backslash-escaped pipe as an escape** — the pipe still delimits, so the row rendered with 5 cells in a 3-column table and broke the table from there down | **FIXED.** Found by a cell-count consistency check over every table on both pages, not by reading. **And then it happened again**: the row you are reading originally quoted the offending character and broke this table too. The check caught that as well. Do not write a pipe inside a DokuWiki table cell, not even to talk about one | |
| |
| ==== ''sonnet'' — figures against the script ==== | ==== sonnet — figures against the script ==== |
| |
| Re-ran ''test_fold.mjs'' (146/146), ''report_hypothesis_testing.mjs'', ''ht_quotecheck.mjs'' and the demo, confirmed the scripts were byte-identical to the frozen copies, and checked every table cell. | Re-ran ''test_fold.mjs'' (146/146), ''report_hypothesis_testing.mjs'', ''ht_quotecheck.mjs'' and the demo, confirmed the scripts were byte-identical to the frozen copies, and checked every table cell. |
| | R1-7 | Running ''check_page_numbers.mjs'' against the **provenance** page reports many unaccounted numbers, all of them historical or external (DOI fragments, the before/after fold values like 212 → 213, byte counts) | **REJECTED as a defect.** The guard is built to audit a content page against its report; a working log that deliberately records superseded numbers is out of its scope. Noted here so the next run does not "fix" it | | | R1-7 | Running ''check_page_numbers.mjs'' against the **provenance** page reports many unaccounted numbers, all of them historical or external (DOI fragments, the before/after fold values like 212 → 213, byte counts) | **REJECTED as a defect.** The guard is built to audit a content page against its report; a working log that deliberately records superseded numbers is out of its scope. Noted here so the next run does not "fix" it | |
| |
| ==== ''sonnet'' — citations and quotes ==== | ==== sonnet — citations and quotes ==== |
| |
| ^ # ^ Finding ^ Disposition ^ | ^ # ^ Finding ^ Disposition ^ |
| | R2-5 | Divine et al. 2018, Rochon 2012 and Campbell 2007 were assessed from abstracts and search results, not primary full text (paywalled) | **ACCEPTED as a limit**, recorded in //What could not be established//. Divine et al.'s thesis is in its title, so the risk is low, but it is not zero | | | R2-5 | Divine et al. 2018, Rochon 2012 and Campbell 2007 were assessed from abstracts and search results, not primary full text (paywalled) | **ACCEPTED as a limit**, recorded in //What could not be established//. Divine et al.'s thesis is in its title, so the risk is low, but it is not zero | |
| |
| ==== ''sonnet'' — external currency, as of 2026-08-13 ==== | ==== sonnet — external currency, as of 2026-08-13 ==== |
| |
| ^ # ^ Finding ^ Disposition ^ | ^ # ^ Finding ^ Disposition ^ |
| | R3-7 | No dead or renamed links | No change | | | R3-7 | No dead or renamed links | No change | |
| |
| ==== ''fable'' — generic ==== | ==== fable — generic ==== |
| |
| Run on the post-fix snapshot ''out/review_ht/page_r2.txt''. It found the single most consequential item of the whole review, and its closing observation is the one to keep. | Run on the post-fix snapshot ''out/review_ht/page_r2.txt''. It found the single most consequential item of the whole review, and its closing observation is the one to keep. |
| |
| ^ What ^ Disposition ^ | ^ What ^ Disposition ^ |
| | The //"quotes are in the PDF and in none of the stored renderings"// section | **RETRACTED.** Both quoted sentences are in ''paper.cols.txt''; ''Friedman'', ''Conover'', ''omnibus'', ''172.47'' and all ten authors are in all three renderings. The section is kept, marked retracted, with the ugrep transcript that explains the error | | | The //"quotes are in the PDF and in none of the stored renderings"// section | **RETRACTED.** Both quoted sentences are in ''paper.cols.txt''; ''Friedman'', ''Conover'', ''omnibus'', ''172.47'' and all ten authors are in all three renderings. The section is kept, marked retracted, with the shell transcript that explains the error | |
| | The claim that ''paper.cols.txt'' drops five of the ten authors | **RETRACTED.** All ten are in its header | | | The claim that ''paper.cols.txt'' drops five of the ten authors | **RETRACTED.** All ten are in its header | |
| | The claim that the dataset's unlocatable rate //"is partly an artefact of text extraction"// | **UPHELD, and now measured** — but on other papers, not this one. 73.1% corpus-wide | | | The claim that the dataset's unlocatable rate //"is partly an artefact of text extraction"// | **UPHELD, and now measured** — but on other papers, not this one. 73.2% of the corpus's unlocatable quotes are present in an independent rendering, under the four-verdict rule described on [[provenance:literature:corpus]] and **not** under this page's threshold | |
| | %%<wrap todo>%% in //What could not be established// | **FIXED to %%<WRAP todo>%%.** The lowercase form renders a ''<span>'', which cannot hold a list: the rendered DOM had all eight bullets as one paragraph of literal asterisks. **Four** other pages still have it and are logged as a follow-up — ''provenance:practices:ethics'', ''provenance:practices:notifying_websites'', ''provenance:practices:public_relations'' and ''provenance:statistics:pvalue_corrections''. ((An earlier draft said //eight//, from ''grep -l "<wrap "'' over a site export; that also matches pages which merely mention the tag inside ''%%…%%''. ''check_wrap.mjs'' gives five, including this page.)) | | | %%<wrap todo>%% in //What could not be established// | **FIXED to %%<WRAP todo>%%.** The lowercase form renders a ''<span>'', which cannot hold a list: the rendered DOM had all eight bullets as one paragraph of literal asterisks. **Four** other pages still have it and are logged as a follow-up — ''provenance:practices:ethics'', ''provenance:practices:notifying_websites'', ''provenance:practices:public_relations'' and ''provenance:statistics:pvalue_corrections''. ((An earlier draft said //eight//, from ''grep -l "<wrap "'' over a site export; that also matches pages which merely mention the tag inside ''%%…%%''. ''check_wrap.mjs'' gives five, including this page.)) | |
| | The quote-check block | **REFRESHED.** ''exact'' 931 unchanged; ''partial'' 540 → **541** and ''below threshold'' 335 → 334 because ''ht_quotecheck.mjs'' was reading the wrong file for one OCR-repaired paper; the 334 then splits into 118 //rescued from the PDF// and **216** //below threshold in both// | | | The quote-check block | **REFRESHED.** ''exact'' 931 unchanged; ''partial'' 540 → **541** and ''below threshold'' 335 → 334 because ''ht_quotecheck.mjs'' was reading the wrong file for one OCR-repaired paper; the 334 then splits into 118 //rescued from the PDF// and **216** //below threshold in both// | |
| |
| <bibtex bibliography></bibtex> | <bibtex bibliography></bibtex> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| |
