| Both sides previous revisionPrevious revision | |
| provenance:statistics:hypothesis_testing [2026/09/04 08:12] – RETRACT the 'quotes in the PDF and in none of the stored renderings' section — all are in paper.cols.txt; the negative came from a grep wrapper with -I. Refresh the quote check (541/118/216), close the TODO, fix <wrap>. Authored by Claude karel.kubicek.claude | provenance:statistics:hypothesis_testing [2026/09/04 08:29] (current) – Review pass: flag the retraction from the run table at the top, state that this page's threshold and the corpus-wide four-verdict rule are not comparable, correct 73.1 to 73.2, and drop the 'ugrep' attribution. Authored by Claude karel.kubicek.claude |
|---|
| |
| ^ Item ^ Value ^ | ^ Item ^ Value ^ |
| | Date | 2026-08-13 | | | Date | 2026-08-13. **One section of this page was retracted and the quote-check figures refreshed on 2026-09-04** — see //Correction pass// at the foot. Everything else stands as written on 2026-08-13. | |
| | Corpus | ''data/extract/run1'', 5,859 extracted papers, 7 venues, 2010–2026 | | | Corpus | ''data/extract/run1'', 5,859 extracted papers, 7 venues, 2010–2026 | |
| | Page status | **New page.** ''statistics:hypothesis_testing'' was a red link promised from ''start'' ("[[Statistics:Hypothesis testing]] suitable for web measurements") and from [[Statistics:Pvalue corrections]] ("choosing the test whose //p//-values you are about to correct"). Nothing on the wiki covered it | | | Page status | **New page.** ''statistics:hypothesis_testing'' was a red link promised from ''start'' ("[[Statistics:Hypothesis testing]] suitable for web measurements") and from [[Statistics:Pvalue corrections]] ("choosing the test whose //p//-values you are about to correct"). Nothing on the wiki covered it | |
| </code> | </code> |
| |
| **118 of the 334 are defects in the stored text, not in the extraction.** They are quotes the ''paper.cols.txt'' check cannot locate and an independent rendering of the same PDF finds. The corpus-wide version of this measurement, its method and its limits are on [[provenance:literature:corpus]]; the short version is that ''decolumn.mjs'' skips the two-column repair on any page whose gutter it cannot find, and a quote on such a page is spliced to the neighbouring column every ten words or so. | **118 of the 334 are defects in the stored text, not in the extraction.** They are quotes the ''paper.cols.txt'' check cannot locate and an independent rendering of the same PDF finds. The same defect measured corpus-wide is on [[provenance:literature:corpus]] — **under a different and much stricter rule, so the two percentages are not comparable.** This script's threshold is 60% of a quote's 5-word windows; that page scores every quote as //exact// / //8-word run// / //fragmented// / //not found//, and a quote below this script's threshold is usually still an //8-word run// there. The mechanism is the same in both: ''decolumn.mjs'' skips the two-column repair on any page whose gutter it cannot find, and a quote on such a page is spliced to the neighbouring column every ten words or so. |
| |
| **Below threshold is not "unsupported".** Five below-threshold quotes were chased by hand and **all five are present**: | **Below threshold is not "unsupported".** Five below-threshold quotes were chased by hand and **all five are present**: |
| ==== RETRACTED, 2026-09-04: the Mai et al. quotes are in ''paper.cols.txt'' ==== | ==== RETRACTED, 2026-09-04: the Mai et al. quotes are in ''paper.cols.txt'' ==== |
| |
| This section previously said that four of Mai et al.'s methodology quotes were present in the stored ''paper.pdf'' and in **none** of ''paper.cols.txt'', ''paper.norm.txt'' or ''paper.txt'', and drew three consequences from it. **The claim was false and the consequences with it.** It is kept here rather than deleted because the way it was got wrong is worth more than the claim was. | This section previously said that four of Mai et al.'s methodology quotes were present in the stored ''paper.pdf'' and in **none** of ''paper.cols.txt'', ''paper.norm.txt'' or ''paper.txt'', and drew three consequences from it. **The claim was false and the consequences with it.** The original text is **summarised rather than reproduced** below, and kept rather than deleted, because the way it was got wrong is worth more than the claim was. |
| |
| What is actually true, checked on 2026-09-04 in Python rather than through the shell: | What is actually true, checked on 2026-09-04 in Python rather than through the shell: |
| |
| ^ What ^ Disposition ^ | ^ What ^ Disposition ^ |
| | The //"quotes are in the PDF and in none of the stored renderings"// section | **RETRACTED.** Both quoted sentences are in ''paper.cols.txt''; ''Friedman'', ''Conover'', ''omnibus'', ''172.47'' and all ten authors are in all three renderings. The section is kept, marked retracted, with the ugrep transcript that explains the error | | | The //"quotes are in the PDF and in none of the stored renderings"// section | **RETRACTED.** Both quoted sentences are in ''paper.cols.txt''; ''Friedman'', ''Conover'', ''omnibus'', ''172.47'' and all ten authors are in all three renderings. The section is kept, marked retracted, with the shell transcript that explains the error | |
| | The claim that ''paper.cols.txt'' drops five of the ten authors | **RETRACTED.** All ten are in its header | | | The claim that ''paper.cols.txt'' drops five of the ten authors | **RETRACTED.** All ten are in its header | |
| | The claim that the dataset's unlocatable rate //"is partly an artefact of text extraction"// | **UPHELD, and now measured** — but on other papers, not this one. 73.1% corpus-wide | | | The claim that the dataset's unlocatable rate //"is partly an artefact of text extraction"// | **UPHELD, and now measured** — but on other papers, not this one. 73.2% of the corpus's unlocatable quotes are present in an independent rendering, under the four-verdict rule described on [[provenance:literature:corpus]] and **not** under this page's threshold | |
| | %%<wrap todo>%% in //What could not be established// | **FIXED to %%<WRAP todo>%%.** The lowercase form renders a ''<span>'', which cannot hold a list: the rendered DOM had all eight bullets as one paragraph of literal asterisks. **Four** other pages still have it and are logged as a follow-up — ''provenance:practices:ethics'', ''provenance:practices:notifying_websites'', ''provenance:practices:public_relations'' and ''provenance:statistics:pvalue_corrections''. ((An earlier draft said //eight//, from ''grep -l "<wrap "'' over a site export; that also matches pages which merely mention the tag inside ''%%…%%''. ''check_wrap.mjs'' gives five, including this page.)) | | | %%<wrap todo>%% in //What could not be established// | **FIXED to %%<WRAP todo>%%.** The lowercase form renders a ''<span>'', which cannot hold a list: the rendered DOM had all eight bullets as one paragraph of literal asterisks. **Four** other pages still have it and are logged as a follow-up — ''provenance:practices:ethics'', ''provenance:practices:notifying_websites'', ''provenance:practices:public_relations'' and ''provenance:statistics:pvalue_corrections''. ((An earlier draft said //eight//, from ''grep -l "<wrap "'' over a site export; that also matches pages which merely mention the tag inside ''%%…%%''. ''check_wrap.mjs'' gives five, including this page.)) | |
| | The quote-check block | **REFRESHED.** ''exact'' 931 unchanged; ''partial'' 540 → **541** and ''below threshold'' 335 → 334 because ''ht_quotecheck.mjs'' was reading the wrong file for one OCR-repaired paper; the 334 then splits into 118 //rescued from the PDF// and **216** //below threshold in both// | | | The quote-check block | **REFRESHED.** ''exact'' 931 unchanged; ''partial'' 540 → **541** and ''below threshold'' 335 → 334 because ''ht_quotecheck.mjs'' was reading the wrong file for one OCR-repaired paper; the 334 then splits into 118 //rescued from the PDF// and **216** //below threshold in both// | |
