| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:privacy:age_assurance [2026/09/15 17:08] – Refresh the quote-guard span counts and the embedded output after the second review round. Authored by Claude karel.kubicek.claude | provenance:privacy:age_assurance [2026/09/15 17:21] (current) – Second-round review log, the de-hyphenation-as-someone-else's-defect mistake, three stale figures corrected, and the completed neighbour backlinks. Authored by Claude karel.kubicek.claude |
|---|
| | 11 candidates from the title-and-summary probe in ''scripts/gap_probe_roadmap.mjs'' | The 11 are really **10** — the probe has no word boundary before ''age'' (below) — and **3** of them are in the derived population: //Easy As Child's Play//, //Tales from the Porn// and the CCS 2022 kids'-apps poster. **27.3% precision** against the 11. A full-text probe finds **38** candidates and **5** population papers, so the title probe's **recall is 3 of 5 (60%)**: it misses {[west2024_picture]} and {[alomar2022_developers]}, whose titles say nothing about age. | | | 11 candidates from the title-and-summary probe in ''scripts/gap_probe_roadmap.mjs'' | The 11 are really **10** — the probe has no word boundary before ''age'' (below) — and **3** of them are in the derived population: //Easy As Child's Play//, //Tales from the Porn// and the CCS 2022 kids'-apps poster. **27.3% precision** against the 11. A full-text probe finds **38** candidates and **5** population papers, so the title probe's **recall is 3 of 5 (60%)**: it misses {[west2024_picture]} and {[alomar2022_developers]}, whose titles say nothing about age. | |
| | "most are children's-privacy and COPPA-compliance work" | Confirmed, and sized: **36** papers name COPPA in ''legal[]'', against **5** in the age-assurance population, and exactly **1** paper is in both. | | | "most are children's-privacy and COPPA-compliance work" | Confirmed, and sized: **36** papers name COPPA in ''legal[]'', against **5** in the age-assurance population, and exactly **1** paper is in both. | |
| | "the one squarely on it is //Easy As Child's Play//" | Confirmed. It is the only ''OBJECT'' verdict in the audit, and it has 194 phrase matches against the runner-up's 22. | | | "the one squarely on it is //Easy As Child's Play//" | Confirmed. It is the only ''OBJECT'' verdict in the audit, and it dominates the probe: the matched-form counts in the script output below are driven by it. | |
| | "the 2019 IMC porn-ecosystem paper is adjacent and useful for the web case" | Confirmed, and it turned out to be the **only** web-platform measurement of age gates in the corpus, and the source of the page's vantage-point argument. | | | "the 2019 IMC porn-ecosystem paper is adjacent and useful for the web case" | Confirmed, and it turned out to be the **only** web-platform measurement of age gates in the corpus, and the source of the page's vantage-point argument. | |
| | "the regulatory surface is moving years ahead of the published measurement" | Confirmed with nine dated primary sources, 2025-01-16 to 2026-09-11. | | | "the regulatory surface is moving years ahead of the published measurement" | Confirmed with nine dated primary sources, 2025-01-16 to 2026-09-11. | |
| |
| One thing the queue did not anticipate and the page now leads on: **six papers hit age assurance as an obstacle to a measurement about something else**, and that is the most common way it appears in the corpus. Four of those six are from 2025–2026. | One thing the queue did not anticipate and the page now leads on: **six papers hit age assurance as an obstacle to a measurement about something else** — more than measure it, though fewer than the 20 that merely mention it. Five of those six are from 2025–2026. |
| |
| ===== The population ===== | ===== The population ===== |
| | ''SECTION'' | a section of the paper measures age assurance in the wild | 4 | | | ''SECTION'' | a section of the paper measures age assurance in the wild | 4 | |
| | ''OBSTACLE'' | age assurance is a constraint on, or a treatment in, the method | 6 | | | ''OBSTACLE'' | age assurance is a constraint on, or a treatment in, the method | 6 | |
| | ''MENTION'' | background, related work, regulation text, or a category label | 19 | | | ''MENTION'' | background, related work, regulation text, a category label, or //age estimation// as the name of an unrelated ML task | 20 | |
| | ''ARTEFACT'' | a de-columning artefact, or a cited title in the bibliography only | 8 | | | ''ARTEFACT'' | a de-columning artefact (''voltage signal''), or a cited title in the bibliography only | 7 | |
| |
| ''OBJECT'' + ''SECTION'' = the **5**-paper population the page uses. **13.2% precision** against the candidate set. | ''OBJECT'' + ''SECTION'' = the **5**-paper population the page uses. **13.2% precision** against the candidate set. |
| // is not the 'verification' in a needle typed on a keyboard, and a true quote | // is not the 'verification' in a needle typed on a keyboard, and a true quote |
| // scores as a fabrication. | // scores as a fabrication. |
| const LIGATURES = [[/\ufb00/g, 'ff'], [/\ufb01/g, 'fi'], [/\ufb02/g, 'fl'], [/\ufb03/g, 'ffi'], [/\ufb04/g, 'ffl']]; | const LIGATURES = [[/\ufb00/g, 'ff'], [/\ufb01/g, 'fi'], [/\ufb02/g, 'fl'], [/\ufb03/g, 'ffi'], [/\ufb04/g, 'ffl'], |
| | // U+2011 NON-BREAKING HYPHEN is not U+002D. Ofcom's PDF writes "third\u2011party" |
| | // and a needle typed with a plain hyphen misses it, for the same reason the |
| | // ligatures do. Fold the whole dash block to a plain hyphen. |
| | [/[\u2010-\u2015]/g, '-']]; |
| const delig = (s) => LIGATURES.reduce((t, [re, r]) => t.replace(re, r), s); | const delig = (s) => LIGATURES.reduce((t, [re, r]) => t.replace(re, r), s); |
| const norm = (s) => delig(s).replace(//g, '').replace(/-\n/g, '').replace(/\s+/g, ' '); | const norm = (s) => delig(s).replace(//g, '').replace(/-\n/g, '').replace(/\s+/g, ' '); |
| 'USENIX/2023/glitchhiker-uncovering-vulnerabilities-of-image-signal-transmission-with-iemi': 'ARTEFACT', | 'USENIX/2023/glitchhiker-uncovering-vulnerabilities-of-image-signal-transmission-with-iemi': 'ARTEFACT', |
| 'NDSS/2026/peering-inside-the-black-box-long-range-and-scalable-model-architecture-snooping-via-gpu-electromagnetic-side-channel': 'ARTEFACT', | 'NDSS/2026/peering-inside-the-black-box-long-range-and-scalable-model-architecture-snooping-via-gpu-electromagnetic-side-channel': 'ARTEFACT', |
| // No paper.cols.txt in the mount, so this one was NOT reachable by the probe; | // Its five matches are all "age estimation" as the name of a downstream |
| // it is here because the title probe on the roadmap reached it. Judged from | // face-attribute ML task ("deep learning tasks such as age estimation, |
| // title and summary only — say so rather than pretend it was read. | // attribute recognition, expression analysis"), not an age check. A first |
| 'PETS/2026/gan-invert-unveiling-vulnerabilities-in-privacy-preserving-facial-transformation': 'ARTEFACT', | // draft filed this as ARTEFACT and claimed the paper had no full text in the |
| | // mount, on the strength of a mistyped slug; the file is there and the probe |
| | // did read it. MENTION is the honest bucket. |
| | 'PETS/2026/gan-invert-unveiling-vulnerabilities-in-privacy-preserving-facial-transformation': 'MENTION', |
| })); | })); |
| |
| console.log(table(['year', 'candidates', 'measures it', 'obstructed by it'], yearRows)); | console.log(table(['year', 'candidates', 'measures it', 'obstructed by it'], yearRows)); |
| console.log('* 2025-2026 are provisional venue-years — see literature:corpus.'); | console.log('* 2025-2026 are provisional venue-years — see literature:corpus.'); |
| | // A share of candidates means nothing without the corpus's own share for the |
| | // same years: the corpus grew, so "more mentions lately" is partly arithmetic. |
| | const recentCand = rows.filter((r) => r.p.year >= 2025).length; |
| | const recentCorpus = P.filter((p) => p.year >= 2025).length; |
| | console.log(`\n2025-2026: ${recentCand} of ${tightKeys.size} candidates (${pct(recentCand, tightKeys.size)})` + |
| | ` against ${recentCorpus} of ${P.length} papers in the corpus (${pct(recentCorpus, P.length)})`); |
| | console.log(` of those ${recentCand} candidates, ${rows.filter((r) => r.p.year >= 2025 && (r.v === 'OBJECT' || r.v === 'SECTION')).length} measure age assurance` + |
| | ` and ${rows.filter((r) => r.p.year >= 2025 && r.v === 'OBSTACLE').length} were obstructed by it`); |
| const venueRows = [...new Set(P.map((p) => p.venue))] | const venueRows = [...new Set(P.map((p) => p.venue))] |
| .map((v) => [v, rows.filter((r) => r.p.venue === v).length, | .map((v) => [v, rows.filter((r) => r.p.venue === v).length, |
| verdict papers | verdict papers |
| -------- ------ | -------- ------ |
| MENTION 19 | MENTION 20 |
| ARTEFACT 8 | ARTEFACT 7 |
| OBSTACLE 6 | OBSTACLE 6 |
| SECTION 4 | SECTION 4 |
| MENTION 2026 IEEE-SP zkFuzz: Foundation and Framework for Effective Fuzzing of Zero-Knowledge Circuits. | MENTION 2026 IEEE-SP zkFuzz: Foundation and Framework for Effective Fuzzing of Zero-Knowledge Circuits. |
| MENTION 2026 PETS Chatbot Confessions:~Large-Scale Analysis of Private Data Disclosure in Shared AI Chatbo | MENTION 2026 PETS Chatbot Confessions:~Large-Scale Analysis of Private Data Disclosure in Shared AI Chatbo |
| | MENTION 2026 PETS GAN-Invert: Unveiling Vulnerabilities in Privacy-Preserving Facial Transformations |
| ARTEFACT 2019 WWW Demographic Inference and Representative Population Estimates from Multilingual Social M | ARTEFACT 2019 WWW Demographic Inference and Representative Population Estimates from Multilingual Social M |
| ARTEFACT 2020 PETS Illuminating the Dark or how to recover what should not be seen in FE-based classifiers | ARTEFACT 2020 PETS Illuminating the Dark or how to recover what should not be seen in FE-based classifiers |
| ARTEFACT 2025 NDSS SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song | ARTEFACT 2025 NDSS SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song |
| ARTEFACT 2026 NDSS Peering Inside the Black-Box: Long-Range and Scalable Model Architecture Snooping via GP | ARTEFACT 2026 NDSS Peering Inside the Black-Box: Long-Range and Scalable Model Architecture Snooping via GP |
| ARTEFACT 2026 PETS GAN-Invert: Unveiling Vulnerabilities in Privacy-Preserving Facial Transformations | |
| |
| POPULATION (measures age assurance) = 5 papers of the 38 candidates = 13.2% precision | POPULATION (measures age assurance) = 5 papers of the 38 candidates = 13.2% precision |
| 2026 * 6 0 1 | 2026 * 6 0 1 |
| * 2025-2026 are provisional venue-years — see literature:corpus. | * 2025-2026 are provisional venue-years — see literature:corpus. |
| | |
| | 2025-2026: 17 of 38 candidates (44.7%) against 1185 of 5859 papers in the corpus (20.2%) |
| | of those 17 candidates, 1 measure age assurance and 5 were obstructed by it |
| |
| venue candidates measures it | venue candidates measures it |
| // is not the 'verification' in a needle typed on a keyboard, and a true quote | // is not the 'verification' in a needle typed on a keyboard, and a true quote |
| // scores as a fabrication. | // scores as a fabrication. |
| const LIGATURES = [[/\ufb00/g, 'ff'], [/\ufb01/g, 'fi'], [/\ufb02/g, 'fl'], [/\ufb03/g, 'ffi'], [/\ufb04/g, 'ffl']]; | const LIGATURES = [[/\ufb00/g, 'ff'], [/\ufb01/g, 'fi'], [/\ufb02/g, 'fl'], [/\ufb03/g, 'ffi'], [/\ufb04/g, 'ffl'], |
| | // U+2011 NON-BREAKING HYPHEN is not U+002D. Ofcom's PDF writes "third\u2011party" |
| | // and a needle typed with a plain hyphen misses it, for the same reason the |
| | // ligatures do. Fold the whole dash block to a plain hyphen. |
| | [/[\u2010-\u2015]/g, '-']]; |
| const delig = (s) => LIGATURES.reduce((t, [re, r]) => t.replace(re, r), s); | const delig = (s) => LIGATURES.reduce((t, [re, r]) => t.replace(re, r), s); |
| const norm = (s) => delig(s).replace(//g, '').replace(/-\n/g, '').replace(/\s+/g, ' ') | const norm = (s) => delig(s).replace(//g, '').replace(/-\n/g, '').replace(/\s+/g, ' ') |
| 'risk-based, flexible, tech-neutral and future-proof': 'aa/ico_joint.txt', | 'risk-based, flexible, tech-neutral and future-proof': 'aa/ico_joint.txt', |
| 'laid before the end of the year, and the changes should be implemented in Spring 2027': 'aa/gov_factsheet.txt', | 'laid before the end of the year, and the changes should be implemented in Spring 2027': 'aa/gov_factsheet.txt', |
| | 'should be implemented in Spring 2027': 'aa/gov_factsheet.txt', |
| 'serious doubts': 'aa/ofcom2026.txt', | 'serious doubts': 'aa/ofcom2026.txt', |
| | 'Almost all analysed pornography services relied exclusively on third-party vendors, with only one analysed pornography service using an in-house solution': 'aa/ofcom2026.txt', |
| | 'receive minimal outcome signals from third-party age assurance providers': 'aa/ofcom2026.txt', |
| | 'facial age estimation and photo ID matching the most commonly deployed': 'aa/ofcom2026.txt', |
| })); | })); |
| |
| [ n/a] X% of sites have an age gate — the page's own example of a meaningless quantity | [ n/a] X% of sites have an age gate — the page's own example of a meaningless quantity |
| [ n/a] I am over 18 — the page describing what a button says, not quoting a paper | [ n/a] I am over 18 — the page describing what a button says, not quoting a paper |
| | [ ok ] facial age estimation and photo ID matching the most commonly deployed — aa/ofcom2026.txt |
| [ ok ] open banking, photo ID matching, facial age estimation, mobile network — aa/ofcom.txt | [ ok ] open banking, photo ID matching, facial age estimation, mobile network — aa/ofcom.txt |
| [ ok ] methods including self-declaration of age and online payments which do — aa/ofcom.txt | [ ok ] methods including self-declaration of age and online payments which do — aa/ofcom.txt |
| [ ok ] only investigate a subset of the top-50 most popular pornographic webs — vallina2019_porn, paper.cols.txt | [ ok ] only investigate a subset of the top-50 most popular pornographic webs — vallina2019_porn, paper.cols.txt |
| [ ok ] if our automatic crawler manages to bypass the mechanism, a child coul — vallina2019_porn, paper.cols.txt | [ ok ] if our automatic crawler manages to bypass the mechanism, a child coul — vallina2019_porn, paper.cols.txt |
| | [ ok ] we did not find any instance of AgeID being deployed during our study — vallina2019_porn, paper.cols.txt |
| [ ok ] we evaluate the effectiveness of age verification. We find that age ve — west2024_picture, paper.pdf (pypdf) | [ ok ] we evaluate the effectiveness of age verification. We find that age ve — west2024_picture, paper.pdf (pypdf) |
| [ ok ] trivially bypassed by providing a birth year — alomar2022_developers, paper.cols.txt | [ ok ] trivially bypassed by providing a birth year — alomar2022_developers, paper.cols.txt |
| [ n/a] Continue — same | [ n/a] Continue — same |
| [ n/a] Accept — same | [ n/a] Accept — same |
| | [ ok ] in 8 languages — vallina2019_porn, paper.cols.txt |
| [ ok ] heavily relies on taint analysis, which might not be perfect — yao2025_easy, paper.cols.txt | [ ok ] heavily relies on taint analysis, which might not be perfect — yao2025_easy, paper.cols.txt |
| | [ ok ] Almost all analysed pornography services relied exclusively on third-p — aa/ofcom2026.txt |
| | [ ok ] receive minimal outcome signals from third-party age assurance provide — aa/ofcom2026.txt |
| | [ ok ] the most common default languages in our list of pornographic websites — vallina2019_porn, paper.cols.txt |
| [ n/a] Sites with an age gate — same | [ n/a] Sites with an age gate — same |
| [ ok ] with roughly half of the app promotions not in compliance with host ap — zhao2023_mobile, paper.cols.txt | [ ok ] with roughly half of the app promotions not in compliance with host ap — zhao2023_mobile, paper.cols.txt |
| [ ok ] when creating an account, minors below 13 can enter a false birth date — aa/ec_meta.txt | [ ok ] when creating an account, minors below 13 can enter a false birth date — aa/ec_meta.txt |
| [ ok ] laid before the end of the year, and the changes should be implemented — aa/gov_factsheet.txt | [ ok ] laid before the end of the year, and the changes should be implemented — aa/gov_factsheet.txt |
| | [ ok ] should be implemented in Spring 2027 — aa/gov_factsheet.txt |
| [ ok ] serious doubts — aa/ofcom2026.txt | [ ok ] serious doubts — aa/ofcom2026.txt |
| |
| 42 spans: 8 not-a-quote, 13 external, 21 paper; 0 failed, 2 located in a paper other than the nearest citekey | 49 spans: 8 not-a-quote, 17 external, 24 paper; 0 failed, 2 located in a paper other than the nearest citekey |
| control ok: a fabricated span is not located | control ok: a fabricated span is not located |
| control ok: a real span resolves to its own paper, not to the other key in the block | control ok: a real span resolves to its own paper, not to the other key in the block |
| * **Any web-side prevalence figure after 2019 //from the literature//.** //Tales from the Porn// is seven years old, predates the UK duty, the EU blueprint and every US state statute, its 20% is a figure for click-through interstitials — a category the current UK rules explicitly exclude — and its denominator is at most fifty hand-checked sites. There is no more recent web measurement in these seven venues. There //is// a more recent deployment measurement: Ofcom's, obtained by statutory compulsion over 32 services and not reproducible by anyone else. The page now leads on that contrast rather than on the absence. | * **Any web-side prevalence figure after 2019 //from the literature//.** //Tales from the Porn// is seven years old, predates the UK duty, the EU blueprint and every US state statute, its 20% is a figure for click-through interstitials — a category the current UK rules explicitly exclude — and its denominator is at most fifty hand-checked sites. There is no more recent web measurement in these seven venues. There //is// a more recent deployment measurement: Ofcom's, obtained by statutory compulsion over 32 services and not reproducible by anyone else. The page now leads on that contrast rather than on the absence. |
| * **Recall of either published detector against an independently drawn sample.** {[yao2025_easy]} does report an error rate — 100 apps sampled from each side of GUARD's own output, hand-verified, 3 false positives and 2 false negatives — but a sample drawn from the classifier's own positives bounds precision, not recall over the population. {[vallina2019_porn]} reports no evaluation of its age-verification detector at all and abandoned it for manual checking. The page says this rather than the flat "neither reports recall" an earlier draft carried. | * **Recall of either published detector against an independently drawn sample.** {[yao2025_easy]} does report an error rate — 100 apps sampled from each side of GUARD's own output, hand-verified, 3 false positives and 2 false negatives — but a sample drawn from the classifier's own positives bounds precision, not recall over the population. {[vallina2019_porn]} reports no evaluation of its age-verification detector at all and abandoned it for manual checking. The page says this rather than the flat "neither reports recall" an earlier draft carried. |
| * **One candidate was judged from title and summary only.** ''PETS/2026/gan-invert-unveiling-vulnerabilities-in-privacy-preserving-facial-transformation'' has no ''paper.cols.txt'' in the mount (one of four such papers in the whole corpus), so the full-text probe never reached it; it is in the verdict map as ''ARTEFACT'' on the basis of its title and abstract, which are about inverting privacy-preserving face transformations, not about age. If that judgement is wrong the population is 5, not 6, by at most one. | * //(An earlier draft had a fourth entry here, claiming one candidate was judged from title and summary only because its full text was missing. That was false — see the mistakes list below.)// |
| |
| ===== Judgement calls ===== | ===== Judgement calls ===== |
| - **The roadmap's own committed probe has the same defect** and is the reason //F-BLEAU: Fast Black-Box Leakage Estimation// is one of its 11 age-assurance candidates. | - **The roadmap's own committed probe has the same defect** and is the reason //F-BLEAU: Fast Black-Box Leakage Estimation// is one of its 11 age-assurance candidates. |
| - **The first quote check failed on a true quote.** {[west2024_picture]}'s contribution sentence is spliced across a column boundary in all three ''.txt'' renderings; ''pypdf'' has it verbatim. The checker was rewritten to try four renderings and to print which one located each needle, and a fabricated-needle control was added so the check cannot pass vacuously. | - **The first quote check failed on a true quote.** {[west2024_picture]}'s contribution sentence is spliced across a column boundary in all three ''.txt'' renderings; ''pypdf'' has it verbatim. The checker was rewritten to try four renderings and to print which one located each needle, and a fabricated-needle control was added so the check cannot pass vacuously. |
| - **The bibliography cache served a stale parse.** After appending 11 entries and saving the page, 11 of 20 references rendered as allocated-but-empty numbers while both sources looked perfect. Purging ''literature/bibliography?purge=true'' and then the page fixed it; the rendered reference count was then checked against the distinct marker count (20 = 20, 42 markers, 42 ''bibtex_citekey'' spans). | - **The bibliography cache served a stale parse.** After appending 11 entries and saving the page, 11 of 20 references rendered as allocated-but-empty numbers while both sources looked perfect. Purging ''literature/bibliography?purge=true'' and then the page fixed it. The check that matters is **distinct citekeys against rendered references**: 20 = 20. The marker-to-span ratio is not a check — the plugin emits **two** ''bibtex_citekey'' spans per marker, so the page's 55 markers render as 110 spans, and an earlier note here that read "42 markers, 42 spans" was counting one of them with a regex that only matched the opening span. |
| - **ofcom.org.uk is unreachable from this sandbox** (403 to curl, to WebFetch, and to a full headless-Chromium context). The January 2025 quotes come from an Internet Archive capture; the July 2026 report was obtained from the UK government's asset host, which is not blocked. The page says so in both footnotes rather than implying a direct read. | - **ofcom.org.uk is unreachable from this sandbox** (403 to curl, to WebFetch, and to a full headless-Chromium context). The January 2025 quotes come from an Internet Archive capture; the July 2026 report was obtained from the UK government's asset host, which is not blocked. The page says so in both footnotes rather than implying a direct read. |
| | - **A de-hyphenation artefact was published as a defect in someone else's paper.** The artefacts table asserted that the repository URL printed in {[moti2024_targeted]} 404s. It does not. The paper prints ''https://github.com/targeted-and-troublesome/'', broken across a line at ''targeted-and-''; ''paper.cols.txt'', ''paper.norm.txt'' and the extraction's ''artifacts.codeUrl'' all rejoin it as ''targeted-andtroublesome'', which is what 404s. Only ''paper.txt'', which preserves the line break, shows the truth. The generic reviewer found it by reading all three renderings. **The tooling's own artefact was published as their error**, which is the second time in this run that an ENOENT or a mangled string was read as a fact about someone else's work. |
| | - **A mistyped slug produced a published falsehood about the corpus itself.** An early context pull used ''…privacy-preserving-facial-transformation**s**'' where the directory is singular. The ENOENT was read as "this paper has no full text", and that became an ''ARTEFACT'' verdict, a script comment, a provenance bullet and a limitation on the content page saying one candidate had been judged from its title alone. The file is there, 102 KB; the probe read it; its five matches are //age estimation// as the name of a face-attribute ML task. The generic reviewer found it by checking the claim against the script's **own printed list** of the four papers without full text — GAN-Invert is not on it. The verdict is now ''MENTION'' (20/7) and the limitation is deleted. Nothing about the population changed, but four separate places had repeated the same unchecked inference. |
| - **The page's most important figures were the ones a first draft got wrong, and neither was caught by a guard.** The 2019 age-verification percentages were published against the wrong denominator — the paper's 6,843-site corpus and "six vantage points", where the section itself is a hand check of at most fifty sites in four countries — because the quote-check located the sentence and nothing checked what the sentence was a share //of//. And the whole 2026 UK deployment picture was missing, because the page was written from the corpus and the corpus stops at seven academic venues. Both came from reviewers. | - **The page's most important figures were the ones a first draft got wrong, and neither was caught by a guard.** The 2019 age-verification percentages were published against the wrong denominator — the paper's 6,843-site corpus and "six vantage points", where the section itself is a hand check of at most fifty sites in four countries — because the quote-check located the sentence and nothing checked what the sentence was a share //of//. And the whole 2026 UK deployment picture was missing, because the page was written from the corpus and the corpus stops at seven academic venues. Both came from reviewers. |
| |
| ==== Review log ==== | ==== Review log ==== |
| |
| Four reviewers, each told explicitly that the author's context may not be exhaustive, and each handed the page text, the report script, its output and this provenance draft. The three focused passes ran in parallel on the pre-review draft; everything below was then applied in one pass, and the generic pass read the result. | Four reviewers, each told explicitly that the author's context may not be exhaustive, and each handed the page text, the report script, its output and this provenance draft. The three focused passes ran in parallel on the pre-review draft; their findings were applied in one pass, and all three were then re-run against the corrected pages. The generic pass ran last — and its first finding was a process one: **the page was republished twice while it was reading**, so it reviewed a moving target and had to mark which of its findings the in-flight edits already fixed. That is a real cost and it is recorded rather than tidied away; the next page should freeze the source while the generic pass runs. |
| |
| ^ # ^ Pass ^ Finding ^ Disposition ^ | ^ # ^ Pass ^ Finding ^ Disposition ^ |
| | 3 | external currency, re-run | The Australian day of effect is still unreachable: ''/text'', ''/downloadPdf'' and ''/contentDocuments'' all 404, the public page is a client-rendered SPA with an empty body, and ''esafety.gov.au'' and ''infrastructure.gov.au'' fail at the network layer. | **Rejection stands.** Still not asserted. | | | 3 | external currency, re-run | The Australian day of effect is still unreachable: ''/text'', ''/downloadPdf'' and ''/contentDocuments'' all 404, the public page is a client-rendered SPA with an empty body, and ''esafety.gov.au'' and ''infrastructure.gov.au'' fail at the network layer. | **Rejection stands.** Still not asserted. | |
| | 3 | external currency, re-run | Nothing supersedes the Ofcom report or the Meta preliminary finding; the Chrome stage reading and all page figures re-confirmed against the primary sources. | **Accepted as a pass.** | | | 3 | external currency, re-run | Nothing supersedes the Ofcom report or the Meta preliminary finding; the Chrome stage reading and all page figures re-confirmed against the primary sources. | **Accepted as a pass.** | |
| | |
| | And the generic pass, which read the page after all of the above: |
| | |
| | ^ Finding ^ Disposition ^ |
| | | **The "one candidate judged from title only" limitation is false** and the script's own output contradicts it. | **Accepted**, and it is the worst defect of the run — see the mistakes list above. | |
| | | The probe is described on the page as **five phrases**; it is fourteen forms, and six of the 38 candidates enter only through the extra ones. A reader reproducing "the five phrases" gets about 31. | **Accepted.** The page now says fourteen and points at the pattern here. | |
| | | **The 2019 detector is not English-only** — it searches its five keywords //"in 8 languages"//, chosen as the most common default languages in that corpus. | **Accepted.** A flat factual error, corrected in the detection table and the language bullet. The paper's claim to //"manually validate the accuracy of our method in Section 7.2"// is also now stated precisely: §7.2 replaces the detector with a manual check and reports no error rate. | |
| | | The page said the CCS poster is //the only mechanism taxonomy derived from observation// while also crediting GUARD's six categories with six of the eight rows in its own mechanism table. | **Accepted.** Scoped to children's apps. | |
| | | //"The first three are the ones that exist at scale"// is contradicted by the page's own Ofcom section (facial age estimation and photo ID matching most commonly deployed). | **Accepted.** Both halves now name their population and the sentence no longer asserts a general ranking. | |
| | | Six unsupported negatives, including //"the field's most-cited web figure"//, //"the corpus contains no general guidance"// on adult-content crawls (there is: {[vallina2019_porn]} §8), and //"the ordering below is the one the field and the regulators both use"//. | **Accepted, all six.** Each is now bounded to what was actually checked. | |
| | | //"the mentions are growing and the measurements are not"// generalises past //n// = 1–2 per year, and the 17-of-38 has no base rate beside it. | **Accepted.** 44.7% of candidates against 20.2% of the corpus is now printed, and the second half is labelled a small-//n// observation. | |
| | | **The page names no artefacts.** GUARD's code, the 500-labels repository, the 2,004-site child-directed list and DiffAudit's release all exist; a methods page that says "do not rebuild one" should list them. | **Accepted, and it turned up a live defect in a paper.** A new section gives all four with their HTTP status, and checking them found that the URL printed in {[moti2024_targeted]} — ''github.com/targeted-andtroublesome/'' — **404s**: the org is ''targeted-and-troublesome'' and the list is ''urls/kids_websites.csv'' in ''targeted-and-troublesome-crawler'', 2,004 distinct URLs, which matches the paper exactly. The same section records that {[vallina2019_porn]}'s population construction cannot be repeated because Alexa's Adult category was retired in 2022. | |
| | | //"the rule that separates a gate from a cookie banner, a paywall and a login wall"// is demanded and never sketched; and //"all four are dismissable overlays"// is wrong about paywalls and login walls. | **Accepted.** The features to build such a rule from are now listed, with the honest note that nobody has published one. | |
| | | The page claims //"a verbatim-quote check of every per-paper number"//; eleven numbers sit outside every needle and a mutation of 14% to 24% passed both guards. The reviewer hand-verified all eleven and they are correct. | **Accepted.** The methodology section now names the eleven figures that are hand-checked rather than guarded. | |
| | | Missing practical point in the ethics section: verification vendors expose sandbox modes. | **Rejected for now.** No primary source was reached for which vendors do, and the page does not name vendors at all; asserting it would be exactly the vendor-marketing claim the source policy on this page rejects. Recorded here so the next run can close it. | |
| | | No neighbouring page links back to this one. | **Accepted and done**, as a separate edit after the page was frozen for the re-review: one line each in [[:design:blocking_and_geodifference]], [[:design:crawling_location]], [[:privacy:consent]] and [[:practices:ethics]]. No figures on those pages were touched. | |
| | | Several phrases repeat (//at most fifty// five times, //one of its two case studies// three times). | **Partly accepted.** Two instances trimmed; the rest carry the caveat in places a reader may arrive at directly. | |
| | | Nits: {[west2024_picture]} uses Frida rather than the camera; the 5,855-apps / 5,855-papers coincidence; an unsupported causal claim about the six obstacle papers; the unused //"we did not find any instance of AgeID being deployed"//. | **All four accepted.** | |
| | |
| | Both passes were re-run once more against the frozen pages. What they found the second time: |
| | |
| | ^ Finding ^ Disposition ^ |
| | | **The artefacts section published a defect in someone else's paper that does not exist.** {[moti2024_targeted]} prints its repository URL correctly; ''paper.cols.txt'', ''paper.norm.txt'' and the extraction all de-hyphenate it across a line break into a URL that 404s. | **Accepted — the worst finding of the second round**, and recorded in the mistakes list above. The row now states the correct URL, its HTTP 200, and the artefact that caused the confusion. | |
| | | The Zenodo record for GUARD is titled //Drexel-SePAL/AgeScope: v1.0.1-1//, so searching for "GUARD" will not find it. | **Accepted.** The row now says so. | |
| | | Three unbounded negatives introduced by the new artefacts section — //"there is no dataset"//, //"nobody has published one"//, and an ordering sentence attributing a sort to Ofcom that Ofcom does not perform. | **Accepted, all three.** | |
| | | The closing argument said a crawl against adult sites would measure //the smaller half//, which nothing on the page sizes, and asserted the Spring 2027 date the fact sheet hedges. | **Accepted.** Both softened to what the sources support. | |
| | | The intro's verdict clause attached //"the name of an unrelated ML task"// to the 7 ''ARTEFACT'' papers, where it describes the 20th ''MENTION''. | **Accepted.** Reworded. | |
| | | The methodology section's list of eleven hand-checked figures is incomplete — the artefact file sizes, Ofcom's 25%-to-43% and 64-of-100, the survey base of 50 and several paraphrased denominators are in the same position. | **Accepted, and the enumeration was abandoned.** It was incomplete twice; the page now states the class instead, which is both shorter and true. | |
| | | Three stale numbers in this provenance page: the "most common way it appears in the corpus" phrasing, an unsourced "194 phrase matches", and a marker-to-span identity that was never consistent. | **Accepted, all three.** The last one is worth keeping in mind: the bibtex plugin emits **two** spans per marker, so marker count and span count are not a check. | |
| | | Scripts reproduce their committed outputs byte-for-byte; the verdict split, 44.7%/20.2%, the four artefact URLs, 55,481 bytes, 2,004 distinct URLs and the 2025-02-04 push date all independently verified. | **Accepted as a pass.** | |
| | | The two ''WARN'' spans are correctly attributed; the log did not name them. | **Accepted.** They are //"Despite being rated as 17+…"// (nearest key ''vallina2019_porn'', found in ''yao2025_easy'') and //"seven out of 20 apps…"// (nearest key ''ardi2023_prevalence'', found in ''moti2025_whispertest''). In both, the previous bullet's trailing citekey is closer in characters than the bullet's own key at its end. A third ''WARN'' would be a new thing to look at. | |
| |
| |
