Measure The Web

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Provenance: privacy:age_assurance ======

Working notes behind **[[:privacy:age_assurance]]**: every query, the report script and its unedited output, the hand verdict for all 38 candidates, the quote checks, the external sources and the ones that were rejected. Corpus-level caveats — venue scope, the selection funnel, the provisional 2025–2026 venue-years, extraction stability — are on [[:literature:corpus]] and are not restated here.

Written 2026-09-15 against ''data/extract/run1'', 5,859 extracted papers, 5,855 with full text on disk.

===== Why this page exists at all, and why it is short =====

''privacy:age_assurance'' was queued on [[:roadmap]] on 2026-09-07 with the note that it would be //thin on purpose//. That held, and the queued reasoning was corrected in one direction and confirmed in another.

^ Queued claim ^ What derivation found ^
| 11 candidates from the title-and-summary probe in ''scripts/gap_probe_roadmap.mjs'' | The 11 are really **10** — the probe has no word boundary before ''age'' (below) — and **3** of them are in the derived population: //Easy As Child's Play//, //Tales from the Porn// and the CCS 2022 kids'-apps poster. **27.3% precision** against the 11. A full-text probe finds **38** candidates and **5** population papers, so the title probe's **recall is 3 of 5 (60%)**: it misses {[west2024_picture]} and {[alomar2022_developers]}, whose titles say nothing about age. |
| "most are children's-privacy and COPPA-compliance work" | Confirmed, and sized: **36** papers name COPPA in ''legal[]'', against **5** in the age-assurance population, and exactly **1** paper is in both. |
| "the one squarely on it is //Easy As Child's Play//" | Confirmed. It is the only ''OBJECT'' verdict in the audit, and it dominates the probe: the matched-form counts in the script output below are driven by it. |
| "the 2019 IMC porn-ecosystem paper is adjacent and useful for the web case" | Confirmed, and it turned out to be the **only** web-platform measurement of age gates in the corpus, and the source of the page's vantage-point argument. |
| "the regulatory surface is moving years ahead of the published measurement" | Confirmed with nine dated primary sources, 2025-01-16 to 2026-09-11. |

One thing the queue did not anticipate and the page now leads on: **six papers hit age assurance as an obstacle to a measurement about something else** — more than measure it, though fewer than the 20 that merely mention it. Five of those six are from 2025–2026.

===== The population =====

==== Inclusion rule, as written before the first count ====

<WRAP box>
A paper is **in the population** if it reports an empirical result about a mechanism that establishes or gates on a user's age — how many services have one, which kind, or how well it works. Proposing a privacy-preserving age credential is out; measuring a deployed one is in. Measuring what a children's service does with data is out.
</WRAP>

Five verdict values, applied to every one of the 38 candidates:

^ Verdict ^ Meaning ^ Papers ^
| ''OBJECT'' | the paper's research question is age assurance | 1 |
| ''SECTION'' | a section of the paper measures age assurance in the wild | 4 |
| ''OBSTACLE'' | age assurance is a constraint on, or a treatment in, the method | 6 |
| ''MENTION'' | background, related work, regulation text, a category label, or //age estimation// as the name of an unrelated ML task | 20 |
| ''ARTEFACT'' | a de-columning artefact (''voltage signal''), or a cited title in the bibliography only | 7 |

''OBJECT'' + ''SECTION'' = the **5**-paper population the page uses. **13.2% precision** against the candidate set.

==== Probe history — three widths, and the one that was wrong ====

Probe width decided the claim here more than anywhere else on the page. All three widths were run over all 5,855 papers with ''paper.cols.txt'', after collapsing whitespace and rejoining hyphenated line breaks.

^ # ^ Pattern (age-assurance family) ^ Candidates ^ Verdict on the probe ^
| 1 | ''age[- ](verification|…|signal|…)'' — **no word boundary before //age//** | 233 | **Broken.** ''voltage signal'' and ''image signal'' match. Nine of the top thirteen hits by mention count were electromagnetic side-channel papers. Discarded. |
| 2 | ''\bage[- ](verification|…)…|prove (their|…) age'' | 40 | Still broken, subtly: ''prove their age'' with no leading ''\b'' matches **im**''prove their age''''ncy''. Two Jordanian smart-home papers entered the set on the phrase //"improve their agency"//. Discarded. |
| 3 | as #2 with ''\bprove … age\b'' — **the published probe** | **38** | Used. Every matched surface form is printed in the script output below, so what the probe actually caught is visible rather than summarised. |

The lesson is in the residue, not the count: probe #1's 233 looked like a healthy literature and was 82% electromagnetics.

==== Recall probes: four more vocabularies, one new paper ====

A phrase probe is a recall claim, so four further probe families were run over the same 5,855 papers to find papers that measure age assurance without using any of the five phrases. Counts are candidate sets and were read by title.

^ Probe family ^ Pattern sketch ^ Hits ^ New population papers ^
| date of birth | ''date of birth'', ''birth ?date'', ''\bDOB\b'', ''birth year'' | 195 | 0 — the high-mention hits are health-forum disclosure, policy-comparison and PII-removal studies |
| identity document | ''identity (verification|document|proof)'', ''\bKYC\b'', ''government-issued ID'', passport/licence scan | 158 | 0 — payments, mobile money, CCPA data-broker compliance, remote proctoring |
| facial age / liveness | ''facial age'', ''age from (a )?(face|photo|selfie)'', ''liveness (check|detection)'' | 110 | 0 — this family is almost entirely **biometric authentication attack** work, not age estimation |
| parental gate | ''parental gate'', ''\bparent gate\b'', ''\bage screen\b'', ''\bage-?gate\b'' | 4 | 0 |
| age rating | ''age rating'', ''content rating'', ''\bESRB\b'', ''\bPEGI\b'', ''\bIARC\b'', ''Designed for Families'', ''Teacher[- ]Approved'', ''maturity rating'' | 193 | **1 found, then excluded** — //Are Mobile Advertisements in Compliance with App's Age Group?// (TheWebConf 2023). It measures ad content against the host app's **rating**, which the inclusion rule puts out of the population. It is cited on the page in the //denominator// section as the thing that is not age assurance. |
| zero-knowledge / credential | ''zero-?knowledge…age'', ''anonymous credential…age'', ''EUDI'', ''eIDAS'', ''digital identity wallet'' | 64 | 0 — the one substantial hit is an attribute-based-credential **acceptance study** (PoPETs 2024) |
| AU social-media ban | ''social media (minimum age|ban)'', ''under-?16 (ban|law)'' | 1 | 0 |

The ''parental gate'' probe was **also broken on its first run** and is a second instance of the same defect: without ''\b'' before ''age'', ''age screen'' matches ''webp''''age screen''''shot'', and it returned 57 papers of which the top five were phishing-detection work. Fixed, it returns 4.

<WRAP important>
**This is a bounded negative, not a proof.** Four probe families found no measurement paper the phrase probe had missed. That raises confidence in the 5; it does not establish that a paper measuring age gates under some vocabulary none of these eleven patterns covers does not exist. The page says so.
</WRAP>

==== The 11 title-probe candidates, judged ====

''scripts/gap_probe_roadmap.mjs'' (committed, with its output) matched 11 papers on title and summary. They are listed here because the roadmap row rests on them and because their precision is the number that justified writing a thin page rather than a full one.

^ Year ^ Venue ^ Title ^ In the full-text candidate set? ^ Verdict ^
| 2012 | IMC | New kid on the block: exploring the google+ social graph | no | off topic — matched ''kid'' |
| 2013 | CCS | When kids' toys breach mobile phone security | no | matched ''kids'' — children's security, no age mechanism |
| 2013 | IMC | Profiling high-school students with facebook | yes | ''MENTION'' |
| 2018 | PETS | "Won't Somebody Think of the Children?" Examining COPPA Compliance at Scale | no | children's-privacy compliance; the page's reference construction for that denominator |
| 2019 | IEEE S&P | F-BLEAU: Fast Black-Box Leakage Estimation | no | **a probe defect.** It matched on //Leak**age Estimation**//: ''gap_probe_roadmap.mjs'' has no word boundary before ''age'' either. Corrected, the ''age_assurance'' row of that probe is **10**, not 11. |
| 2019 | IMC | Tales from the Porn | yes | ''SECTION'' |
| 2019 | USENIX | Evaluating the Contextual Integrity of Privacy Regulation: Parents' IoT Toy Privacy Norms Versus COPPA | no | children's-privacy norms, user study |
| 2020 | CCS | Dangerous Skills Got Certified | no | voice-skill certification; COPPA in ''legal[]'' |
| 2022 | CCS | Poster: An Analysis of Privacy Features in 'Expert-Approved' Kids' Apps | yes | ''SECTION'' |
| 2025 | USENIX | Easy As Child's Play | yes | ''OBJECT'' |
| 2025 | NDSS | The Kids Are All Right: YouTube Giveaway Scams | no | scam susceptibility by age group, not age assurance |

**3 of 11 in the population; 4 of 11 reached by the full-text probe.** Conversely the full-text probe's 38 include **34** the title probe never saw, and **2** of the 5 population papers are among them. A title-and-summary probe is a poor instrument for this topic because the measurement is often one section of a paper whose title says nothing about age — //A Picture is Worth 500 Labels// and //Developers Say the Darnedest Things// are both invisible to it.

<WRAP important>
**The first draft of this page said 1 of 11, 9.1% precision and 20% recall.** That was wrong and was contradicted by the table three rows above it, whose verdict column already read ''SECTION'' for two of the three. A reviewer re-ran ''gap_probe_roadmap.mjs'' and intersected the keys rather than reading the prose. The corrected figures are 3, 27.3% and 60%, and the same wrong numbers were published for twenty minutes on [[:roadmap]] before being fixed there too.
</WRAP>

===== Report script and output =====

Every corpus figure on the content page is produced by this script. It throws rather than printing a warning on four conditions: an unjudged candidate, a verdict keyed to a paper not in the extraction, a ''TIGHT'' probe that is not a subset of ''LOOSE'', and any published figure whose quote cannot be located in its source.

The full script.

<file javascript report_age_assurance.mjs>
// Every figure on privacy:age_assurance, with its denominator.
//
//   node scripts/report_age_assurance.mjs > scripts/report_age_assurance-output.txt
//
// The page is thin on purpose. This script exists to make the thinness
// checkable: it prints the probe that produced the candidate set, the
// hand-keyed verdict for every candidate, and a verbatim-quote check for every
// per-paper figure the page publishes.
//
// Three rules from data/extract/README.md are load-bearing here:
//   * every count names its own denominator (never "of 5,859 papers");
//   * a probe count is a CANDIDATE SET, not a population — the population is
//     the hand audit below;
//   * a figure taken from a paper is quoted with the paper's own denominator,
//     and the quote is checked against data/fulltext/.../paper.cols.txt.

import fs from 'node:fs';
import path from 'node:path';
import { execFileSync } from 'node:child_process';
import { loadExtractions, dataRoot, pct, table } from './lib.mjs';

const ROOT = path.join(dataRoot(), 'fulltext');
const P = loadExtractions();
const key = (p) => `${p.venue}/${p.year}/${p.slug}`;
const BY_KEY = new Map(P.map((p) => [key(p), p]));

// PDF line breaks inside a phrase otherwise silently undercount.
// Ligatures: pypdf hands back U+FB01 for 'fi', so 'verification' in the PDF
// is not the 'verification' in a needle typed on a keyboard, and a true quote
// scores as a fabrication.
const LIGATURES = [[/\ufb00/g, 'ff'], [/\ufb01/g, 'fi'], [/\ufb02/g, 'fl'], [/\ufb03/g, 'ffi'], [/\ufb04/g, 'ffl'],
  // U+2011 NON-BREAKING HYPHEN is not U+002D. Ofcom's PDF writes "third\u2011party"
  // and a needle typed with a plain hyphen misses it, for the same reason the
  // ligatures do. Fold the whole dash block to a plain hyphen.
  [/[\u2010-\u2015]/g, '-']];
const delig = (s) => LIGATURES.reduce((t, [re, r]) => t.replace(re, r), s);
const norm = (s) => delig(s).replace(//g, '').replace(/-\n/g, '').replace(/\s+/g, ' ');
const colsPath = (k) => {
  const [venue, year, slug] = k.split('/');
  return path.join(ROOT, year, venue, slug, 'paper.cols.txt');
};
const TEXT = new Map();
function text(k) {
  if (TEXT.has(k)) return TEXT.get(k);
  const f = colsPath(k);
  const t = fs.existsSync(f) ? norm(fs.readFileSync(f, 'utf8')) : null;
  TEXT.set(k, t);
  return t;
}

// Where a needle is located. .cols.txt repairs two-column reading order but
// still splices some sentences at a column boundary, and pypdf splices
// different ones — so a needle is checked against every rendering the mount
// has before it is called a bad quote, and the winning rendering is printed.
const RENDERINGS = ['paper.cols.txt', 'paper.norm.txt', 'paper.txt'];
function locate(k, needle) {
  const [venue, year, slug] = k.split('/');
  const dir = path.join(ROOT, year, venue, slug);
  for (const r of RENDERINGS) {
    const f = path.join(dir, r);
    if (fs.existsSync(f) && norm(fs.readFileSync(f, 'utf8')).includes(needle)) return r;
  }
  if (fs.existsSync(path.join(dir, 'paper.pdf'))) {
    const out = execFileSync('python3', [path.join('scripts', 'pdftext.py'), k],
      { encoding: 'utf8', maxBuffer: 1 << 28, stdio: ['ignore', 'pipe', 'ignore'] });
    if (norm(out).includes(needle)) return 'paper.pdf (pypdf)';
  }
  return null;
}

// ---------------------------------------------------------------- 1. the frame
console.log('=============================================================');
console.log('1. CORPUS FRAME');
console.log('=============================================================');
const withText = P.filter((p) => fs.existsSync(colsPath(key(p))));
console.log(`papers in extraction            ${P.length}`);
console.log(`  with paper.cols.txt           ${withText.length}`);
console.log(`  WITHOUT full text             ${P.length - withText.length}`);
for (const p of P.filter((x) => !fs.existsSync(colsPath(key(x)))))
  console.log(`      ${key(p)}`);
const crawled = P.filter((p) => p.crawlConfig !== null || p.studyTypes.includes('automated-web-crawl'));
const webCrawled = crawled.filter((p) => p.platforms.includes('web'));
const legalPop = P.filter((p) => p.legal.length > 0);
console.log(`ran a crawl (\`crawled\`)         ${crawled.length}`);
console.log(`  ... on the web platform       ${webCrawled.length}`);
console.log(`assessed a law (\`legal\`)        ${legalPop.length}`);

// ------------------------------------------------------- 2. the full-text probe
console.log('\n=============================================================');
console.log('2. FULL-TEXT PROBE  (candidate set, not a population)');
console.log('=============================================================');
// TIGHT is what the page quotes. LOOSE is a strictly wider phrasing of the same
// question and MUST contain TIGHT — a narrowing probe that returns more hits is
// broken, and so is a "loose" probe that is merely different.
const TIGHT =
  /\bage[- ](verification|verifying|assurance|estimation|gate|gating|check|checking|attestation|token|signal|declaration|disclosure)s?\b|verif(y|ied|ication|ying) (of )?(the )?(a )?(user'?s?|users'?|their|his|her|customer'?s?|visitor'?s?) age|estimat(e|ing|ion|ed) (of )?(the )?(a )?(user'?s?|users'?|their) age|\bage-?gated\b|\bprove (their|his|her|the user'?s) age\b/i;
const LOOSE = /\bage[- ]\w+|\bunder-?age\b|verif\w+ [^.]{0,30}\bage\b|\bage\b [^.]{0,30}verif\w+/i;

const hit = (re) => withText.filter((p) => re.test(text(key(p))));
const tight = hit(TIGHT);
const loose = hit(LOOSE);
const tightKeys = new Set(tight.map(key));
const looseKeys = new Set(loose.map(key));
const outside = [...tightKeys].filter((k) => !looseKeys.has(k));
console.log(`TIGHT  "age verification / age gate / age assurance / …"   ${tight.length} papers`);
console.log(`LOOSE  "age <word>" anywhere                              ${loose.length} papers`);
console.log(`TIGHT papers NOT inside LOOSE (must be 0)                 ${outside.length}`);
if (outside.length) { console.log(outside.join('\n')); throw new Error('TIGHT is not a subset of LOOSE'); }
if (tight.length > loose.length) throw new Error('TIGHT > LOOSE');

// Every distinct surface form the TIGHT probe matched, so the residue of the
// normalisation is visible rather than summarised.
const forms = new Map();
for (const p of tight) {
  for (const m of text(key(p)).match(new RegExp(TIGHT.source, 'gi')) ?? [])
    forms.set(m.toLowerCase(), (forms.get(m.toLowerCase()) ?? 0) + 1);
}
console.log('\nmatched surface forms (tuples, not papers):');
for (const [s, c] of [...forms.entries()].sort((a, b) => b[1] - a[1]))
  console.log(`  ${String(c).padStart(4)}  ${s}`);

// ------------------------------------------------------------ 3. the hand audit
console.log('\n=============================================================');
console.log('3. HAND AUDIT OF EVERY CANDIDATE');
console.log('=============================================================');
console.log(`OBJECT    the paper's research question is age assurance`);
console.log(`SECTION   a section of the paper measures age assurance in the wild`);
console.log(`OBSTACLE  age assurance is a constraint on, or a treatment in, the method`);
console.log(`MENTION   background, related work, regulation text, or a category label`);
console.log(`ARTEFACT  the phrase is a de-columning artefact ("voltage signal") or a`);
console.log(`          cited title in the bibliography only\n`);

const VERDICT = new Map(Object.entries({
  'USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and': 'OBJECT',
  'IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem': 'SECTION',
  'IEEE-SP/2024/a-picture-is-worth-500-labels-a-case-study-of-demographic-disparities-in-local-m': 'SECTION',
  'CCS/2022/poster-an-analysis-of-privacy-features-in-expert-approved-kids-apps': 'SECTION',
  'PETS/2022/developers-say-the-darnedest-things-privacy-compliance-processes-followed-by-dev': 'SECTION',
  'CCS/2025/whispertest-a-voice-control-based-library-for-ios-ui-automation': 'OBSTACLE',
  'IMC/2023/the-prevalence-of-single-sign-on-on-the-web-towards-the-next-generation-of-web-c': 'OBSTACLE',
  'PETS/2026/ad-personalization-and-transparency-in-mobile-ecosystems-a-comparative-analysis': 'OBSTACLE',
  'PETS/2025/more-and-scammier-ads-the-perils-of-youtubes-ad-privacy-settings': 'OBSTACLE',
  'USENIX/2025/analyzing-the-ai-nudification-application-ecosystem': 'OBSTACLE',
  'PETS/2025/understanding-privacy-norms-through-web-forms': 'OBSTACLE',
  'PETS/2026/a-risk-assessment-framework-for-digital-identification-systems': 'MENTION',
  'IMC/2024/diffaudit-auditing-privacy-practices-of-online-services-for-children-and-adolesc': 'MENTION',
  'PETS/2023/creative-beyond-tiktoks-investigating-adolescents-social-privacy-management-on-t': 'MENTION',
  'PETS/2025/sok-web-authentication-and-recovery-in-the-age-of-end-to-end-encryption': 'MENTION',
  'CCS/2023/marketing-to-children-through-online-targeted-advertising-targeting-mechanisms-a': 'MENTION',
  'PETS/2024/exploring-the-privacy-experiences-of-closeted-users-of-online-dating-services-in': 'MENTION',
  'CCS/2025/digital-safety-for-children-with-intellectual-disabilities-when-using-mobile-dev': 'MENTION',
  'IMC/2013/profiling-high-school-students-with-facebook-how-online-privacy-laws-can-actuall': 'MENTION',
  'PETS/2017/topics-of-controversy-an-empirical-analysis-of-web-censorship-lists': 'MENTION',
  'WWW/2019/measurement-and-early-detection-of-third-party-application-abuse-on-twitter': 'MENTION',
  'PETS/2023/on-the-role-and-form-of-personal-information-disclosure-in-cyberbullying-inciden': 'MENTION',
  'USENIX/2023/a-study-of-chinas-censorship-and-its-evasion-through-the-lens-of-online-gaming': 'MENTION',
  'PETS/2023/everybodys-looking-for-ssomething-a-large-scale-evaluation-on-the-privacy-of-oau': 'MENTION',
  'IEEE-SP/2024/sok-technical-implementation-and-human-impact-of-internet-privacy-regulations': 'MENTION',
  'IEEE-SP/2025/exploring-parent-child-perceptions-on-safety-in-generative-ai-concerns-mitigatio': 'MENTION',
  'PETS/2025/making-web-applications-gdpr-compliant-a-comparative-evaluation-of-gdpr-enforcem': 'MENTION',
  'IEEE-SP/2026/zkfuzz-foundation-and-framework-for-effective-fuzzing-of-zero-knowledge-circuits': 'MENTION',
  'PETS/2025/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic': 'MENTION',
  'PETS/2026/chatbot-confessions-large-scale-analysis-of-private-data-disclosure-in-shared-ai': 'MENTION',
  'PETS/2020/illuminating-the-dark-or-how-to-recover-what-should-not-be-seen-in-fe-based-clas': 'ARTEFACT',
  'PETS/2022/personal-information-inference-from-voice-recordings-user-awareness-and-privacy': 'ARTEFACT',
  'WWW/2019/demographic-inference-and-representative-population-estimates-from-multilingual': 'ARTEFACT',
  'NDSS/2025/songbsab-a-dual-prevention-approach-against-singing-voice-conversion-based-illegal-song-covers': 'ARTEFACT',
  'USENIX/2023/eavesdropping-mobile-app-activity-via-radio-frequency-energy-harvesting': 'ARTEFACT',
  'USENIX/2023/glitchhiker-uncovering-vulnerabilities-of-image-signal-transmission-with-iemi': 'ARTEFACT',
  'NDSS/2026/peering-inside-the-black-box-long-range-and-scalable-model-architecture-snooping-via-gpu-electromagnetic-side-channel': 'ARTEFACT',
  // Its five matches are all "age estimation" as the name of a downstream
  // face-attribute ML task ("deep learning tasks such as age estimation,
  // attribute recognition, expression analysis"), not an age check. A first
  // draft filed this as ARTEFACT and claimed the paper had no full text in the
  // mount, on the strength of a mistyped slug; the file is there and the probe
  // did read it. MENTION is the honest bucket.
  'PETS/2026/gan-invert-unveiling-vulnerabilities-in-privacy-preserving-facial-transformation': 'MENTION',
}));

const missingVerdict = [...tightKeys].filter((k) => !VERDICT.has(k));
const strayVerdict = [...VERDICT.keys()].filter((k) => !tightKeys.has(k) && BY_KEY.has(k));
const unknownKey = [...VERDICT.keys()].filter((k) => !BY_KEY.has(k));
if (missingVerdict.length) { console.log('UNJUDGED:\n' + missingVerdict.join('\n')); throw new Error(`${missingVerdict.length} candidates have no verdict`); }
if (unknownKey.length) { console.log('NOT IN CORPUS:\n' + unknownKey.join('\n')); throw new Error('verdict map names a paper that is not in the extraction'); }
console.log(`verdicts keyed ${VERDICT.size}; candidates ${tightKeys.size}; keyed-but-not-a-candidate ${strayVerdict.length}`);
for (const k of strayVerdict) console.log(`  (not reached by the probe) ${k}`);

const counts = {};
for (const v of VERDICT.values()) counts[v] = (counts[v] ?? 0) + 1;
console.log('\n' + table(['verdict', 'papers'], Object.entries(counts).sort((a, b) => b[1] - a[1])));

console.log('\nper-candidate:');
const ORDER = { OBJECT: 0, SECTION: 1, OBSTACLE: 2, MENTION: 3, ARTEFACT: 4 };
const rows = [...VERDICT.entries()]
  .map(([k, v]) => ({ k, v, p: BY_KEY.get(k) }))
  .sort((a, b) => ORDER[a.v] - ORDER[b.v] || a.p.year - b.p.year);
for (const r of rows)
  console.log(`  ${r.v.padEnd(9)} ${r.p.year} ${r.p.venue.padEnd(7)} ${r.p.title.replace(/\s+/g, ' ').slice(0, 88)}`);

const POPULATION = rows.filter((r) => r.v === 'OBJECT' || r.v === 'SECTION');
console.log(`\nPOPULATION (measures age assurance) = ${POPULATION.length} papers of the ${tightKeys.size} candidates` +
  ` = ${pct(POPULATION.length, tightKeys.size)} precision`);
console.log(`  of the ${P.length}-paper corpus: ${pct(POPULATION.length, P.length)}`);
console.log(`  of the ${crawled.length} papers that ran a crawl: ${POPULATION.filter((r) => crawled.includes(r.p)).length}`);

// --------------------------------------------------- 4. when, and in which venue
console.log('\n=============================================================');
console.log('4. WHEN AND WHERE  (candidate set of ' + tightKeys.size + ')');
console.log('=============================================================');
const yearRows = [];
for (let y = 2010; y <= 2026; y += 1) {
  const c = rows.filter((r) => r.p.year === y);
  if (!c.length) continue;
  yearRows.push([y === 2025 || y === 2026 ? `${y} *` : String(y), c.length,
    c.filter((r) => r.v === 'OBJECT' || r.v === 'SECTION').length,
    c.filter((r) => r.v === 'OBSTACLE').length]);
}
console.log(table(['year', 'candidates', 'measures it', 'obstructed by it'], yearRows));
console.log('* 2025-2026 are provisional venue-years — see literature:corpus.');
// A share of candidates means nothing without the corpus's own share for the
// same years: the corpus grew, so "more mentions lately" is partly arithmetic.
const recentCand = rows.filter((r) => r.p.year >= 2025).length;
const recentCorpus = P.filter((p) => p.year >= 2025).length;
console.log(`\n2025-2026: ${recentCand} of ${tightKeys.size} candidates (${pct(recentCand, tightKeys.size)})` +
  ` against ${recentCorpus} of ${P.length} papers in the corpus (${pct(recentCorpus, P.length)})`);
console.log(`  of those ${recentCand} candidates, ${rows.filter((r) => r.p.year >= 2025 && (r.v === 'OBJECT' || r.v === 'SECTION')).length} measure age assurance` +
  ` and ${rows.filter((r) => r.p.year >= 2025 && r.v === 'OBSTACLE').length} were obstructed by it`);
const venueRows = [...new Set(P.map((p) => p.venue))]
  .map((v) => [v, rows.filter((r) => r.p.venue === v).length,
    rows.filter((r) => r.p.venue === v && (r.v === 'OBJECT' || r.v === 'SECTION')).length])
  .sort((a, b) => b[1] - a[1]);
console.log('\n' + table(['venue', 'candidates', 'measures it'], venueRows));

// ------------------------------------------- 5. children's privacy, for contrast
console.log('\n=============================================================');
console.log("5. CHILDREN'S PRIVACY COMPLIANCE — a different question");
console.log('=============================================================');
const coppa = P.filter((p) => p.legal.some((l) => /coppa/i.test(l.law ?? '')));
console.log(`papers whose legal[] names COPPA                ${coppa.length}` +
  `  (of the ${legalPop.length} that assessed any law = ${pct(coppa.length, legalPop.length)})`);
const coppaYears = {};
for (const p of coppa) coppaYears[p.year] = (coppaYears[p.year] ?? 0) + 1;
console.log('by year: ' + Object.entries(coppaYears).map(([y, c]) => `${y}:${c}`).join(' '));
const coppaVenues = {};
for (const p of coppa) coppaVenues[p.venue] = (coppaVenues[p.venue] ?? 0) + 1;
console.log('by venue: ' + Object.entries(coppaVenues).sort((a, b) => b[1] - a[1]).map(([v, c]) => `${v}:${c}`).join(' '));
const coppaMobile = coppa.filter((p) => p.platforms.includes('mobile'));
const coppaWeb = coppa.filter((p) => p.platforms.includes('web'));
console.log(`  ... carrying the \`mobile\` platform tag       ${coppaMobile.length}  (${pct(coppaMobile.length, coppa.length)})`);
console.log(`  ... carrying the \`web\` platform tag          ${coppaWeb.length}  (${pct(coppaWeb.length, coppa.length)})`);
const overlap = coppa.filter((p) => POPULATION.some((r) => r.k === key(p)));
console.log(`papers in BOTH the COPPA set and the age-assurance population: ${overlap.length}` +
  (overlap.length ? ' — ' + overlap.map(key).join(', ') : ''));
console.log('\nthe COPPA set, by year:');
for (const p of coppa.sort((a, b) => a.year - b.year))
  console.log(`  ${p.year} ${p.venue.padEnd(7)} ${p.title.replace(/\s+/g, ' ').slice(0, 92)}`);

// --------------------------------------------- 6. the figures the page publishes
console.log('\n=============================================================');
console.log('6. PER-PAPER FIGURES, WITH THE PAPER\'S OWN DENOMINATOR');
console.log('=============================================================');
console.log('Each needle below is checked verbatim against paper.cols.txt after the\n' +
  'same whitespace normalisation. A needle must be SPECIFIC: a bare percentage\n' +
  'is shared across papers and would pass against the wrong sentence.\n');

const FIGURES = [
  { k: 'USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and',
    what: 'adult-only ("17+") Google Play apps implementing any age verification',
    den: '31,750 adult-only apps, themselves drawn from 693,334 Google Play apps',
    val: '1,165 (3.67%)',
    needle: 'Our analysis of 31,750 adult-only apps (out of 693,334 apps on Google Play) reveals that only 1,165 (3.67%) implement age verification' },
  { k: 'USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and',
    what: 'the paper\'s own two figures for the same quantity disagree',
    den: 'same 31,750; 1,165/31,750 = 3.669%, so the abstract is right and §RQ5 is wrong',
    val: '3.67% in the abstract and conclusion, 3.75% in the results section',
    needle: 'our results show that there are only 1,165 (3.75%) adult-only apps that have implemented the age verification' },
  { k: 'USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and',
    what: 'share of the verifying apps using the weakest and the strongest method',
    den: 'the 1,165 apps that implement age verification',
    val: 'age gate 31.84%, biometric verification 8.48%',
    // The digits must be inside the needle: a needle that stops before them
    // substantiates the sentence and not the number the page prints.
    needle: 'are the most widely implemented method (31.84%)' },
  { k: 'USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and',
    what: 'the strongest method is the least used',
    den: 'the same 1,165',
    val: 'biometric verification 8.48%',
    needle: 'Biometric verification is the least utilized (8.48%)' },
  { k: 'USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and',
    what: "the detector's own error rate, on a sample drawn from its own output",
    den: '100 apps sampled from each side of GUARD\'s classification, hand-verified',
    val: '3 false positives, 2 false negatives',
    needle: 'We randomly sampled 100 apps categorized as having age verification mechanisms and 100 apps without any verification methods' },
  { k: 'USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and',
    what: 'the declared minimum age does not match the store rating',
    den: 'apps carrying Google Play\'s "17+" rating',
    val: '152 apps enforce 21, 309 apps enforce 16',
    needle: 'Despite being rated as 17+, 152 apps actually enforce an age limit of 21 years' },
  // THE DENOMINATOR TRAP ON THIS PAGE. 6,843 (and the 6,346 OpenWPM crawled) is
  // the paper's corpus for its tracker and privacy-policy sections. The
  // age-verification section is a MANUAL check of at most fifty sites in four
  // countries, because the authors judged their own keyword detector too
  // false-positive-prone. A first draft of this page published 20% against
  // 6,843 and "six vantage points"; a reviewer caught it.
  { k: 'IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem',
    what: 'the age-verification section is manual, and is NOT over the 6,843-site corpus',
    den: 'stated by the paper itself',
    val: 'a subset of the top-50 most popular pornographic websites, manually, in 4 countries',
    needle: 'we only investigate a subset of the top-50 most popular pornographic websites manually. We perform this manual analysis in 4 countries' },
  { k: 'IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem',
    what: 'pornographic websites showing any age-verification mechanism, by country',
    den: 'a subset of the top-50 most popular pornographic websites, hand-checked, per country',
    val: '20% from the USA, UK and Spain; 14% from Russia',
    needle: 'the same set of 20% of the pornographic websites implement and show to the end user the same age verification mechanism' },
  { k: 'IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem',
    what: 'the same site behaves differently depending on where the visit appears to come from',
    den: 'the same hand-checked subset',
    val: '8% verify only in Russia; 12% verify everywhere except Russia',
    needle: '8% of the websites that do not verify users\' age for the rest of countries do so in Russia' },
  { k: 'IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem',
    what: 'why the authors abandoned the automated detector for this section',
    den: 'the paper\'s own account of its method',
    val: 'keyword matching was judged too false-positive-prone',
    needle: 'it is prone to introduce false positives, specially so in age-related keywords that appear often in the content of the websites' },
  { k: 'IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem',
    what: 'what the mechanism actually was',
    den: 'the 20% that had one',
    val: 'a warning text and a button; the crawler bypassed it',
    needle: 'if our automatic crawler manages to bypass the mechanism, a child could do it as well' },
  { k: 'CCS/2022/poster-an-analysis-of-privacy-features-in-expert-approved-kids-apps',
    what: 'distinct age-assurance methods observed in expert-approved kids\' apps',
    den: '137 apps analysed, from 150 selected out of a 470-app candidate set',
    val: '13 distinct methods; year of birth in 12 apps',
    // A vendor name from the adjacent column ("Moat") is spliced into this
    // sentence in paper.cols.txt, so the needle stops before the splice; the
    // "(12 apps)" half is checked by the next entry.
    needle: 'Apps used a total of 13 different methods for' },
  { k: 'CCS/2022/poster-an-analysis-of-privacy-features-in-expert-approved-kids-apps',
    what: 'the most common of those 13 methods',
    den: 'the same 137 apps',
    val: 'asking for the year of birth, 12 apps',
    needle: 'asking for the year of' },
  { k: 'PETS/2022/developers-say-the-darnedest-things-privacy-compliance-processes-followed-by-dev',
    what: 'child-directed app developers who say they use an age gate for parental consent',
    den: '50 responses to the initial organizational survey',
    val: '16%; and 6% said their own gate is trivially bypassed by a birth year',
    needle: 'In terms of how parental consent is obtained, 16% mentioned they use age gates' },
  { k: 'PETS/2022/developers-say-the-darnedest-things-privacy-compliance-processes-followed-by-dev',
    what: 'self-report against observation — the gap that makes a survey figure unusable alone',
    den: 'the same organizations\' apps, tested from California IP addresses',
    val: 'no verifiable parental consent mechanism was observed at all',
    needle: 'did not observe any mechanisms for obtaining verifiable parental consent' },
  { k: 'IEEE-SP/2024/a-picture-is-worth-500-labels-a-case-study-of-demographic-disparities-in-local-m',
    what: 'accuracy of a deployed facial age-estimation model, by demographic',
    den: 'the on-device vision models extracted from the TikTok and Instagram Android apps',
    val: 'qualitative — "less effective for younger demographics"; no accuracy figure published here',
    needle: 'if done using the model deployed by TikTok, is less effective for younger' },
  { k: 'PETS/2026/ad-personalization-and-transparency-in-mobile-ecosystems-a-comparative-analysis',
    what: 'an age gate removing a category from a measurement population',
    den: 'app categories the authors wanted in their app-store study',
    val: 'dating apps dropped — age verification could not be met at scale',
    needle: 'some potentially interesting app categories, such as dating apps, require age verification to install them' },
  { k: 'PETS/2025/more-and-scammier-ads-the-perils-of-youtubes-ad-privacy-settings',
    what: 'what the researchers themselves had to submit, by jurisdiction',
    den: 'the Google accounts used as experiment instances',
    val: 'selfie verification in AU, IE and UK; a button click in the US and Canada',
    needle: 'we used a mobile VPN and selfie verification in Australia, Ireland, and the UK' },
  { k: 'IMC/2023/the-prevalence-of-single-sign-on-on-the-web-towards-the-next-generation-of-web-c',
    what: 'age gates as a named cause of crawler failure',
    den: 'the login pages the crawler failed to find',
    val: 'age-verification prompts listed among the blocking artefacts',
    needle: 'These include age-verification prompts from adult websites' },
  { k: 'CCS/2025/whispertest-a-voice-control-based-library-for-ios-ui-automation',
    what: 'ads reached only after clearing a parental gate or age check',
    den: '20 children\'s iOS apps interacted with manually for 100 seconds each',
    val: '7 of 20 apps',
    needle: 'In seven out of 20 apps, we observed at least one ad, typically after bypassing challenging flows such as parental gates' },
  { k: 'PETS/2025/understanding-privacy-norms-through-web-forms',
    what: 'the one adjacent web-form study excludes standalone age forms by construction',
    den: 'the paper\'s web-form dataset',
    val: 'standalone age-verification forms are not in the dataset',
    needle: 'Many websites use standalone age verification forms that only ask for age but no other identifiers. These web forms are not included in the dataset' },
  { k: 'USENIX/2025/analyzing-the-ai-nudification-application-ecosystem',
    what: 'the vantage point chosen to avoid triggering an age check',
    den: 'the 20 nudification websites studied',
    val: 'data collected from a US region with no age-verification law for explicit content',
    needle: 'a region in the U.S. that does not have an age verification law' },
  { k: 'PETS/2018/won-t-somebody-think-of-the-children-examining-coppa-compliance-at-scale',
    what: "children's apps in Google Play's Designed for Families programme, accessing location",
    den: '5,855 DFF-enrolled Android apps (not "children\'s apps" in general)',
    val: '235 (4.0%) reached GPS; 28% accessed permission-protected sensitive data',
    needle: 'Our instrumentation observed 235 apps (4.0% of 5,855)' },
  { k: 'IEEE-SP/2024/targeted-and-troublesome-tracking-and-advertising-on-childrens-websites',
    what: 'trackers and targeted ads on child-directed websites',
    den: '2,004 manually verified child-directed websites, classified out of Common Crawl',
    val: '~90% embed a tracker; ~27% carry targeted ads',
    needle: 'around 90% of child-directed websites embed one or more trackers' },
  { k: 'WWW/2023/are-mobile-advertisements-in-compliance-with-apps-age-group',
    what: 'ads shown inside apps whose declared audience includes children',
    den: '11,270 ad views collected across 25,000 apps',
    val: '1,289 ad violations from 775 apps',
    needle: 'We collected 11,270 ad views' },
];

let bad = 0;
const whereCount = new Map();
for (const f of FIGURES) {
  const where = locate(f.k, norm(f.needle));
  if (!where) bad += 1;
  else whereCount.set(where, (whereCount.get(where) ?? 0) + 1);
  console.log(`[${where ? ' ok ' : 'FAIL'}] ${f.k}`);
  console.log(`        what: ${f.what}`);
  console.log(`  DENOMINATOR: ${f.den}`);
  console.log(`        value: ${f.val}`);
  console.log(`       needle: "${f.needle}"`);
  console.log(`     found in: ${where ?? 'NO RENDERING'}`);
  console.log('');
}
console.log('located in: ' + [...whereCount.entries()].map(([r, c]) => `${r} ${c}`).join(', '));
console.log(`quote check: ${FIGURES.length - bad}/${FIGURES.length} located verbatim`);
// Positive control for the checker itself: a sentence that is not in the paper
// must not be located, or the check is asserting nothing.
const CONTROL_KEY = FIGURES[0].k;
const CONTROL = 'reveals that only 9,999 (99.99%) implement age verification';
if (locate(CONTROL_KEY, norm(CONTROL)) !== null)
  throw new Error('quote checker located a sentence that is not in the paper');
console.log(`control: a fabricated needle is correctly NOT located in ${CONTROL_KEY}`);
if (bad) throw new Error(`${bad} published figures could not be located in their source`);

// Every paper the page cites for a figure must be in the corpus.
for (const f of FIGURES) if (!BY_KEY.has(f.k)) throw new Error(`figure cites a paper not in the extraction: ${f.k}`);

// -------------------------------------------- 7. what the extraction itself says
console.log('\n=============================================================');
console.log('7. WHAT THE EXTRACTION SCHEMA CARRIES');
console.log('=============================================================');
const easy = BY_KEY.get('USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and');
for (const d of easy.detection)
  console.log(`  detection  phenomenon="${d.phenomenon}"\n             technique="${d.technique}"\n             metric="${d.metric}"\n             prevalence="${d.prevalence}"`);
for (const c of easy.classification)
  console.log(`  classification  target=${c.target} resource="${c.resourceName}" taxonomy="${c.taxonomy}"`);
console.log(`\nNo enum in the schema names age assurance. The phenomenon above is free text,`);
console.log(`which is ~20% stable run-to-run, so it cannot be aggregated — it is usable`);
console.log(`only as a pointer back to the paper.`);
</file>

Its unedited output, as run on 2026-09-15.

<file  report_age_assurance-output.txt>
=============================================================
1. CORPUS FRAME
=============================================================
papers in extraction            5859
  with paper.cols.txt           5855
  WITHOUT full text             4
      USENIX/2010/idle-port-scanning-and-non-interference-analysis-of-network-protocol-stacks-usin
      CCS/2014/beware-your-hands-reveal-your-secrets
      IMC/2020/bgp-beacons-network-tomography-and-bayesian-computation-to-locate-route-flap-dam
      IEEE-SP/2020/burglars-iot-paradise-understanding-and-mitigating-security-risks-of-general-mes
ran a crawl (`crawled`)         1120
  ... on the web platform       857
assessed a law (`legal`)        402

=============================================================
2. FULL-TEXT PROBE  (candidate set, not a population)
=============================================================
TIGHT  "age verification / age gate / age assurance / …"   38 papers
LOOSE  "age <word>" anywhere                              1752 papers
TIGHT papers NOT inside LOOSE (must be 0)                 0

matched surface forms (tuples, not papers):
   236  age verification
    19  age-verification
    14  age gates
    14  age gate
    12  age estimation
     4  age assurance
     4  age disclosure
     4  age checks
     3  verify their age
     2  verify users' age
     2  verify user age
     2  age signals
     2  age check
     1  age-gates
     1  age signal
     1  verifying a user's age
     1  verify the user's age
     1  prove their age
     1  age disclosures

=============================================================
3. HAND AUDIT OF EVERY CANDIDATE
=============================================================
OBJECT    the paper's research question is age assurance
SECTION   a section of the paper measures age assurance in the wild
OBSTACLE  age assurance is a constraint on, or a treatment in, the method
MENTION   background, related work, regulation text, or a category label
ARTEFACT  the phrase is a de-columning artefact ("voltage signal") or a
          cited title in the bibliography only

verdicts keyed 38; candidates 38; keyed-but-not-a-candidate 0

verdict   papers
--------  ------
MENTION   20
ARTEFACT  7
OBSTACLE  6
SECTION   4
OBJECT    1

per-candidate:
  OBJECT    2025 USENIX  Easy As Child's Play: An Empirical Study on Age Verification of Adult-Oriented Android A
  SECTION   2019 IMC     Tales from the Porn: A Comprehensive Privacy Analysis of the Web Porn Ecosystem.
  SECTION   2022 CCS     Poster: An Analysis of Privacy Features in 'Expert-Approved' Kids' Apps.
  SECTION   2022 PETS    Developers Say the Darnedest Things: Privacy Compliance Processes Followed by Developers
  SECTION   2024 IEEE-SP A Picture is Worth 500 Labels: A Case Study of Demographic Disparities in Local Machine 
  OBSTACLE  2023 IMC     The Prevalence of Single Sign-On on the Web: Towards the Next Generation of Web Content 
  OBSTACLE  2025 CCS     WhisperTest: A Voice-Control-based Library for iOS UI Automation.
  OBSTACLE  2025 PETS    More and Scammier Ads: The Perils of YouTube's Ad Privacy Settings
  OBSTACLE  2025 USENIX  Analyzing the AI Nudification Application Ecosystem
  OBSTACLE  2025 PETS    Understanding Privacy Norms through Web Forms
  OBSTACLE  2026 PETS    Ad Personalization and Transparency in Mobile Ecosystems: A Comparative Analysis of Goog
  MENTION   2013 IMC     Profiling high-school students with facebook: how online privacy laws can actually incre
  MENTION   2017 PETS    Topics of Controversy: An Empirical Analysis of Web Censorship Lists
  MENTION   2019 WWW     Measurement and Early Detection of Third-Party Application Abuse on Twitter.
  MENTION   2023 PETS    Creative beyond TikToks: Investigating Adolescents' Social Privacy Management on TikTok
  MENTION   2023 CCS     Marketing to Children Through Online Targeted Advertising: Targeting Mechanisms and Lega
  MENTION   2023 PETS    On the Role and Form of Personal Information Disclosure in Cyberbullying Incidents
  MENTION   2023 USENIX  A Study of China's Censorship and Its Evasion Through the Lens of Online Gaming
  MENTION   2023 PETS    Everybody's Looking for SSOmething: A large-scale evaluation on the privacy of OAuth aut
  MENTION   2024 IMC     DiffAudit: Auditing Privacy Practices of Online Services for Children and Adolescents.
  MENTION   2024 PETS    Exploring the Privacy Experiences of Closeted Users of Online Dating Services in the US
  MENTION   2024 IEEE-SP SoK: Technical Implementation and Human Impact of Internet Privacy Regulations.
  MENTION   2025 PETS    SoK: Web Authentication and Recovery in the Age of End-to-End Encryption
  MENTION   2025 CCS     Digital Safety for Children with Intellectual Disabilities When Using Mobile Devices fro
  MENTION   2025 IEEE-SP Exploring Parent-Child Perceptions on Safety in Generative AI: Concerns, Mitigation Stra
  MENTION   2025 PETS    Making Web Applications GDPR Compliant: A Comparative Evaluation of GDPR-Enforcement Fra
  MENTION   2025 PETS    Sheep's clothing, wolfish intent: Automated detection and evaluation of problematic 'all
  MENTION   2026 PETS    A Risk Assessment Framework for Digital Identification Systems
  MENTION   2026 IEEE-SP zkFuzz: Foundation and Framework for Effective Fuzzing of Zero-Knowledge Circuits.
  MENTION   2026 PETS    Chatbot Confessions:~Large-Scale Analysis of Private Data Disclosure in Shared AI Chatbo
  MENTION   2026 PETS    GAN-Invert: Unveiling Vulnerabilities in Privacy-Preserving Facial Transformations
  ARTEFACT  2019 WWW     Demographic Inference and Representative Population Estimates from Multilingual Social M
  ARTEFACT  2020 PETS    Illuminating the Dark or how to recover what should not be seen in FE-based classifiers
  ARTEFACT  2022 PETS    Personal information inference from voice recordings: User awareness and privacy concern
  ARTEFACT  2023 USENIX  Eavesdropping Mobile App Activity via Radio-Frequency Energy Harvesting
  ARTEFACT  2023 USENIX  GlitchHiker: Uncovering Vulnerabilities of Image Signal Transmission with IEMI
  ARTEFACT  2025 NDSS    SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song
  ARTEFACT  2026 NDSS    Peering Inside the Black-Box: Long-Range and Scalable Model Architecture Snooping via GP

POPULATION (measures age assurance) = 5 papers of the 38 candidates = 13.2% precision
  of the 5859-paper corpus: 0.1%
  of the 1120 papers that ran a crawl: 2

=============================================================
4. WHEN AND WHERE  (candidate set of 38)
=============================================================
year    candidates  measures it  obstructed by it
------  ----------  -----------  ----------------
2013    1           0            0
2017    1           0            0
2019    3           1            0
2020    1           0            0
2022    3           2            0
2023    8           0            1
2024    4           1            0
2025 *  11          1            4
2026 *  6           0            1
* 2025-2026 are provisional venue-years — see literature:corpus.

2025-2026: 17 of 38 candidates (44.7%) against 1185 of 5859 papers in the corpus (20.2%)
  of those 17 candidates, 1 measure age assurance and 5 were obstructed by it

venue    candidates  measures it
-------  ----------  -----------
PETS     17          1
USENIX   5           1
CCS      4           1
IMC      4           1
IEEE-SP  4           1
WWW      2           0
NDSS     2           0

=============================================================
5. CHILDREN'S PRIVACY COMPLIANCE — a different question
=============================================================
papers whose legal[] names COPPA                36  (of the 402 that assessed any law = 9.0%)
by year: 2013:1 2014:1 2016:1 2017:1 2018:1 2019:3 2020:4 2021:4 2022:5 2023:3 2024:6 2025:5 2026:1
by venue: PETS:15 USENIX:6 CCS:4 IMC:3 NDSS:3 IEEE-SP:3 WWW:2
  ... carrying the `mobile` platform tag       25  (69.4%)
  ... carrying the `web` platform tag          9  (25.0%)
papers in BOTH the COPPA set and the age-assurance population: 1 — PETS/2022/developers-say-the-darnedest-things-privacy-compliance-processes-followed-by-dev

the COPPA set, by year:
  2013 IMC     Profiling high-school students with facebook: how online privacy laws can actually increase 
  2014 USENIX  Brahmastra: Driving Apps to Test the Security of Third-Party Components
  2016 IMC     Characterizing Website Behaviors Across Logged-in and Not-logged-in Users.
  2017 NDSS    Automated Analysis of Privacy Requirements for Mobile Apps
  2018 PETS    “Won’t Somebody Think of the Children?” Examining COPPA Compliance at Scale
  2019 PETS    MAPS: Scaling Privacy Compliance Analysis to a Million Apps
  2019 USENIX  50 Ways to Leak Your Data: An Exploration of Apps' Circumvention of the Android Permissions 
  2019 USENIX  Evaluating the Contextual Integrity of Privacy Regulation: Parents' IoT Toy Privacy Norms Ve
  2020 CCS     Dangerous Skills Got Certified: Measuring the Trustworthiness of Skill Certification in Voic
  2020 PETS    Angel or Devil? A Privacy Study of Mobile Parental Control Apps
  2020 PETS    The Price is (Not) Right: Comparing Privacy in Free and Paid Apps
  2020 IEEE-SP An Analysis of Pre-installed Android Software.
  2021 NDSS    Hey Alexa, is this Skill Safe?: Taking a Closer Look at the Alexa Skill Ecosystem
  2021 NDSS    PrivacyFlash Pro: Automating Privacy Policy Generation for Mobile Apps
  2021 PETS    A Calculus of Tracking: Theory and Practice
  2021 USENIX  Understanding Malicious Cross-library Data Harvesting on Android
  2022 PETS    Are iPhones Really Better for Privacy? A Comparative Study of iOS and Android Apps
  2022 PETS    Charting App Developers’ Journey Through Privacy Regulation Features in Ad Networks
  2022 PETS    Developers Say the Darnedest Things: Privacy Compliance Processes Followed by Developers of 
  2022 PETS    “We may share the number of diaper changes”: A Privacy and Security Analysis of Mobile Child
  2022 USENIX  Electronic Monitoring Smartphone Apps: An Analysis of Risks from Technical, Human-Centered, 
  2023 CCS     Marketing to Children Through Online Targeted Advertising: Targeting Mechanisms and Legal As
  2023 WWW     Are Mobile Advertisements in Compliance with App's Age Group?
  2023 WWW     Not Seen, Not Heard in the Digital World! Measuring Privacy Practices in Children's Apps.
  2024 IMC     DiffAudit: Auditing Privacy Practices of Online Services for Children and Adolescents.
  2024 PETS    Honesty is the Best Policy: On the Accuracy of Apple Privacy Labels Compared to Apps' Privac
  2024 CCS     VPVet: Vetting Privacy Policies of Virtual Reality Apps.
  2024 IEEE-SP SoK: Technical Implementation and Human Impact of Internet Privacy Regulations.
  2024 USENIX  Navigating the Privacy Compliance Maze: Understanding Risks with Privacy-Configurable Mobile
  2024 IEEE-SP Targeted and Troublesome: Tracking and Advertising on Children's Websites.
  2025 PETS    Understanding Privacy Norms through Web Forms
  2025 PETS    The Effect of Platform Policies on App Privacy Compliance: A Study of Child-Directed Apps
  2025 PETS    Privacy Settings of Third-Party Libraries in Android Apps: A Study of Facebook SDKs
  2025 PETS    Who’s Watching You Zoom? Investigating Privacy of Third-Party Zoom Apps
  2025 CCS     WhisperTest: A Voice-Control-based Library for iOS UI Automation.
  2026 PETS    Are Bite-Size Data Safety Details a Healthy Diet for Android Telehealth App Users? Impacts o

=============================================================
6. PER-PAPER FIGURES, WITH THE PAPER'S OWN DENOMINATOR
=============================================================
Each needle below is checked verbatim against paper.cols.txt after the
same whitespace normalisation. A needle must be SPECIFIC: a bare percentage
is shared across papers and would pass against the wrong sentence.

[ ok ] USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and
        what: adult-only ("17+") Google Play apps implementing any age verification
  DENOMINATOR: 31,750 adult-only apps, themselves drawn from 693,334 Google Play apps
        value: 1,165 (3.67%)
       needle: "Our analysis of 31,750 adult-only apps (out of 693,334 apps on Google Play) reveals that only 1,165 (3.67%) implement age verification"
     found in: paper.cols.txt

[ ok ] USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and
        what: the paper's own two figures for the same quantity disagree
  DENOMINATOR: same 31,750; 1,165/31,750 = 3.669%, so the abstract is right and §RQ5 is wrong
        value: 3.67% in the abstract and conclusion, 3.75% in the results section
       needle: "our results show that there are only 1,165 (3.75%) adult-only apps that have implemented the age verification"
     found in: paper.cols.txt

[ ok ] USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and
        what: share of the verifying apps using the weakest and the strongest method
  DENOMINATOR: the 1,165 apps that implement age verification
        value: age gate 31.84%, biometric verification 8.48%
       needle: "are the most widely implemented method (31.84%)"
     found in: paper.pdf (pypdf)

[ ok ] USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and
        what: the strongest method is the least used
  DENOMINATOR: the same 1,165
        value: biometric verification 8.48%
       needle: "Biometric verification is the least utilized (8.48%)"
     found in: paper.pdf (pypdf)

[ ok ] USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and
        what: the detector's own error rate, on a sample drawn from its own output
  DENOMINATOR: 100 apps sampled from each side of GUARD's classification, hand-verified
        value: 3 false positives, 2 false negatives
       needle: "We randomly sampled 100 apps categorized as having age verification mechanisms and 100 apps without any verification methods"
     found in: paper.pdf (pypdf)

[ ok ] USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and
        what: the declared minimum age does not match the store rating
  DENOMINATOR: apps carrying Google Play's "17+" rating
        value: 152 apps enforce 21, 309 apps enforce 16
       needle: "Despite being rated as 17+, 152 apps actually enforce an age limit of 21 years"
     found in: paper.cols.txt

[ ok ] IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem
        what: the age-verification section is manual, and is NOT over the 6,843-site corpus
  DENOMINATOR: stated by the paper itself
        value: a subset of the top-50 most popular pornographic websites, manually, in 4 countries
       needle: "we only investigate a subset of the top-50 most popular pornographic websites manually. We perform this manual analysis in 4 countries"
     found in: paper.cols.txt

[ ok ] IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem
        what: pornographic websites showing any age-verification mechanism, by country
  DENOMINATOR: a subset of the top-50 most popular pornographic websites, hand-checked, per country
        value: 20% from the USA, UK and Spain; 14% from Russia
       needle: "the same set of 20% of the pornographic websites implement and show to the end user the same age verification mechanism"
     found in: paper.cols.txt

[ ok ] IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem
        what: the same site behaves differently depending on where the visit appears to come from
  DENOMINATOR: the same hand-checked subset
        value: 8% verify only in Russia; 12% verify everywhere except Russia
       needle: "8% of the websites that do not verify users' age for the rest of countries do so in Russia"
     found in: paper.cols.txt

[ ok ] IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem
        what: why the authors abandoned the automated detector for this section
  DENOMINATOR: the paper's own account of its method
        value: keyword matching was judged too false-positive-prone
       needle: "it is prone to introduce false positives, specially so in age-related keywords that appear often in the content of the websites"
     found in: paper.cols.txt

[ ok ] IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem
        what: what the mechanism actually was
  DENOMINATOR: the 20% that had one
        value: a warning text and a button; the crawler bypassed it
       needle: "if our automatic crawler manages to bypass the mechanism, a child could do it as well"
     found in: paper.cols.txt

[ ok ] CCS/2022/poster-an-analysis-of-privacy-features-in-expert-approved-kids-apps
        what: distinct age-assurance methods observed in expert-approved kids' apps
  DENOMINATOR: 137 apps analysed, from 150 selected out of a 470-app candidate set
        value: 13 distinct methods; year of birth in 12 apps
       needle: "Apps used a total of 13 different methods for"
     found in: paper.cols.txt

[ ok ] CCS/2022/poster-an-analysis-of-privacy-features-in-expert-approved-kids-apps
        what: the most common of those 13 methods
  DENOMINATOR: the same 137 apps
        value: asking for the year of birth, 12 apps
       needle: "asking for the year of"
     found in: paper.cols.txt

[ ok ] PETS/2022/developers-say-the-darnedest-things-privacy-compliance-processes-followed-by-dev
        what: child-directed app developers who say they use an age gate for parental consent
  DENOMINATOR: 50 responses to the initial organizational survey
        value: 16%; and 6% said their own gate is trivially bypassed by a birth year
       needle: "In terms of how parental consent is obtained, 16% mentioned they use age gates"
     found in: paper.cols.txt

[ ok ] PETS/2022/developers-say-the-darnedest-things-privacy-compliance-processes-followed-by-dev
        what: self-report against observation — the gap that makes a survey figure unusable alone
  DENOMINATOR: the same organizations' apps, tested from California IP addresses
        value: no verifiable parental consent mechanism was observed at all
       needle: "did not observe any mechanisms for obtaining verifiable parental consent"
     found in: paper.cols.txt

[ ok ] IEEE-SP/2024/a-picture-is-worth-500-labels-a-case-study-of-demographic-disparities-in-local-m
        what: accuracy of a deployed facial age-estimation model, by demographic
  DENOMINATOR: the on-device vision models extracted from the TikTok and Instagram Android apps
        value: qualitative — "less effective for younger demographics"; no accuracy figure published here
       needle: "if done using the model deployed by TikTok, is less effective for younger"
     found in: paper.pdf (pypdf)

[ ok ] PETS/2026/ad-personalization-and-transparency-in-mobile-ecosystems-a-comparative-analysis
        what: an age gate removing a category from a measurement population
  DENOMINATOR: app categories the authors wanted in their app-store study
        value: dating apps dropped — age verification could not be met at scale
       needle: "some potentially interesting app categories, such as dating apps, require age verification to install them"
     found in: paper.cols.txt

[ ok ] PETS/2025/more-and-scammier-ads-the-perils-of-youtubes-ad-privacy-settings
        what: what the researchers themselves had to submit, by jurisdiction
  DENOMINATOR: the Google accounts used as experiment instances
        value: selfie verification in AU, IE and UK; a button click in the US and Canada
       needle: "we used a mobile VPN and selfie verification in Australia, Ireland, and the UK"
     found in: paper.cols.txt

[ ok ] IMC/2023/the-prevalence-of-single-sign-on-on-the-web-towards-the-next-generation-of-web-c
        what: age gates as a named cause of crawler failure
  DENOMINATOR: the login pages the crawler failed to find
        value: age-verification prompts listed among the blocking artefacts
       needle: "These include age-verification prompts from adult websites"
     found in: paper.cols.txt

[ ok ] CCS/2025/whispertest-a-voice-control-based-library-for-ios-ui-automation
        what: ads reached only after clearing a parental gate or age check
  DENOMINATOR: 20 children's iOS apps interacted with manually for 100 seconds each
        value: 7 of 20 apps
       needle: "In seven out of 20 apps, we observed at least one ad, typically after bypassing challenging flows such as parental gates"
     found in: paper.cols.txt

[ ok ] PETS/2025/understanding-privacy-norms-through-web-forms
        what: the one adjacent web-form study excludes standalone age forms by construction
  DENOMINATOR: the paper's web-form dataset
        value: standalone age-verification forms are not in the dataset
       needle: "Many websites use standalone age verification forms that only ask for age but no other identifiers. These web forms are not included in the dataset"
     found in: paper.cols.txt

[ ok ] USENIX/2025/analyzing-the-ai-nudification-application-ecosystem
        what: the vantage point chosen to avoid triggering an age check
  DENOMINATOR: the 20 nudification websites studied
        value: data collected from a US region with no age-verification law for explicit content
       needle: "a region in the U.S. that does not have an age verification law"
     found in: paper.cols.txt

[ ok ] PETS/2018/won-t-somebody-think-of-the-children-examining-coppa-compliance-at-scale
        what: children's apps in Google Play's Designed for Families programme, accessing location
  DENOMINATOR: 5,855 DFF-enrolled Android apps (not "children's apps" in general)
        value: 235 (4.0%) reached GPS; 28% accessed permission-protected sensitive data
       needle: "Our instrumentation observed 235 apps (4.0% of 5,855)"
     found in: paper.cols.txt

[ ok ] IEEE-SP/2024/targeted-and-troublesome-tracking-and-advertising-on-childrens-websites
        what: trackers and targeted ads on child-directed websites
  DENOMINATOR: 2,004 manually verified child-directed websites, classified out of Common Crawl
        value: ~90% embed a tracker; ~27% carry targeted ads
       needle: "around 90% of child-directed websites embed one or more trackers"
     found in: paper.cols.txt

[ ok ] WWW/2023/are-mobile-advertisements-in-compliance-with-apps-age-group
        what: ads shown inside apps whose declared audience includes children
  DENOMINATOR: 11,270 ad views collected across 25,000 apps
        value: 1,289 ad violations from 775 apps
       needle: "We collected 11,270 ad views"
     found in: paper.cols.txt

located in: paper.cols.txt 21, paper.pdf (pypdf) 4
quote check: 25/25 located verbatim
control: a fabricated needle is correctly NOT located in USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and

=============================================================
7. WHAT THE EXTRACTION SCHEMA CARRIES
=============================================================
  detection  phenomenon="age-verification implementation"
             technique="Static UI extraction, taint analysis, and dynamic path exploration"
             metric="share of analyzed adult-only apps"
             prevalence="1,165 apps; 3.67% in the abstract, 3.75% in the evaluation"
  detection  phenomenon="age-verification method types"
             technique="Code-feature heuristics and API, keyword, and file-upload detection"
             metric="distribution by verification method"
             prevalence="Age gate 31.84%; biometric verification 8.48%"
  detection  phenomenon="false positives and negatives"
             technique="Manual verification of randomly sampled apps"
             metric="false-positive and false-negative counts"
             prevalence="3 false positives and 2 false negatives"
  detection  phenomenon="personal-data collection"
             technique="Checking strings of UI components co-located in layouts"
             metric="counts of apps or instances collecting data"
             prevalence="Entertainment apps had 302 full-name and 156 address instances"
  detection  phenomenon="attack susceptibility"
             technique="Theoretical mapping of app mechanisms to six attack types"
             metric="share of verification-method categories"
             prevalence="Age gates vulnerable to A1; all non-biometric methods vulnerable to A4"
  classification  target=mobile-app resource="GUARD" taxonomy="age-verification mechanism presence and type"
  classification  target=mobile-app resource="GUARD" taxonomy="age-verification mechanism presence and type"
  classification  target=mobile-app resource="GUARD custom rules" taxonomy="age gate, template-based, online ID, credit card, document upload, biometric verification"

No enum in the schema names age assurance. The phenomenon above is free text,
which is ~20% stable run-to-run, so it cannot be aggregated — it is usable
only as a pointer back to the paper.
</file>

The PDF fallback both guards call when no .txt rendering has the needle.

<file python pdftext.py>
#!/usr/bin/env python3
"""Print a paper's PDF text, whitespace-collapsed, for quote checking.

paper.cols.txt repairs two-column reading order but still splices some
sentences across column boundaries; pypdf's own extraction splices different
ones. A needle that cannot be found in any .txt rendering is checked here
before it is called a bad quote.

    python3 scripts/pdftext.py <venue>/<year>/<slug>
"""
import re
import sys
import pathlib
import pypdf

ROOTS = ["/workspace/publications_dataset/data/fulltext",
         "/workspace/publications_dataset/fulltext"]
venue, year, slug = sys.argv[1].split("/")
for root in ROOTS:
    pdf = pathlib.Path(root) / year / venue / slug / "paper.pdf"
    if pdf.exists():
        break
else:
    sys.exit(f"no paper.pdf for {sys.argv[1]}")
reader = pypdf.PdfReader(str(pdf))
text = " ".join(page.extract_text() or "" for page in reader.pages)
text = text.replace("", "").replace("-\n", "")
sys.stdout.write(re.sub(r"\s+", " ", text))
</file>

The second guard: every quoted span on the page, pulled out of the page itself.

<file javascript aa_quotespans.mjs>
// Guard for privacy:age_assurance: every //"…"// span on the page must be
// accounted for.
//
//   node scripts/aa_quotespans.mjs aa/page.txt
//
// A curated quote list drifts from its page, so the list is not the input: the
// page is. Every span is pulled out of the page source and must either be
// located verbatim in the paper it is attributed to, or be on the NOT_A_QUOTE
// list below (rhetorical phrases the page puts in quotation marks that are not
// attributed to anyone) or the EXTERNAL list (checked against a fetched
// primary source, cached under aa/). An unlisted, unlocated span is an error.
//
// Renderings are tried in order: paper.cols.txt, paper.norm.txt, paper.txt,
// then pypdf. .cols repairs two-column reading order but still splices some
// sentences at a column boundary, and a spliced true quote would otherwise be
// scored as a fabrication.

import fs from 'node:fs';
import path from 'node:path';
import { execFileSync } from 'node:child_process';
import { dataRoot } from './lib.mjs';

const ROOT = path.join(dataRoot(), 'fulltext');
// Ligatures: pypdf hands back U+FB01 for 'fi', so 'verification' in the PDF
// is not the 'verification' in a needle typed on a keyboard, and a true quote
// scores as a fabrication.
const LIGATURES = [[/\ufb00/g, 'ff'], [/\ufb01/g, 'fi'], [/\ufb02/g, 'fl'], [/\ufb03/g, 'ffi'], [/\ufb04/g, 'ffl'],
  // U+2011 NON-BREAKING HYPHEN is not U+002D. Ofcom's PDF writes "third\u2011party"
  // and a needle typed with a plain hyphen misses it, for the same reason the
  // ligatures do. Fold the whole dash block to a plain hyphen.
  [/[\u2010-\u2015]/g, '-']];
const delig = (s) => LIGATURES.reduce((t, [re, r]) => t.replace(re, r), s);
const norm = (s) => delig(s).replace(//g, '').replace(/-\n/g, '').replace(/\s+/g, ' ')
  .replace(/[‘’]/g, "'").replace(/[“”]/g, '"');

// Spans the page puts in quotation marks that are NOT attributed to a source:
// the page's own scare quotes, and UI strings it is describing rather than
// quoting. Each must be justified here, not merely listed.
const NOT_A_QUOTE = new Map(Object.entries({
  'X% of sites have an age gate': "the page's own example of a meaningless quantity",
  'Sites with an age gate': 'same',
  'I am over 18': 'the page describing what a button says, not quoting a paper',
  'Yes': 'a keyword from the 2019 detector, listed individually below as a set',
  'Enter': 'same', 'Agree': 'same', 'Continue': 'same', 'Accept': 'same',
}));

// Spans attributed to a paper. Key = the citekey used on the page next to it.
const PAPERS = {
  'yao2025_easy': 'USENIX/2025/easy-as-childs-play-an-empirical-study-on-age-verification-of-adult-oriented-and',
  'vallina2019_porn': 'IMC/2019/tales-from-the-porn-a-comprehensive-privacy-analysis-of-the-web-porn-ecosystem',
  'west2024_picture': 'IEEE-SP/2024/a-picture-is-worth-500-labels-a-case-study-of-demographic-disparities-in-local-m',
  'alomar2022_developers': 'PETS/2022/developers-say-the-darnedest-things-privacy-compliance-processes-followed-by-dev',
  'ekambaranathan2022_poster': 'CCS/2022/poster-an-analysis-of-privacy-features-in-expert-approved-kids-apps',
  'woodruff2026_risk': 'PETS/2026/a-risk-assessment-framework-for-digital-identification-systems',
  'breuer2026_ad': 'PETS/2026/ad-personalization-and-transparency-in-mobile-ecosystems-a-comparative-analysis',
  'ardi2023_prevalence': 'IMC/2023/the-prevalence-of-single-sign-on-on-the-web-towards-the-next-generation-of-web-c',
  'moti2025_whispertest': 'CCS/2025/whispertest-a-voice-control-based-library-for-ios-ui-automation',
  'mai2025_more': 'PETS/2025/more-and-scammier-ads-the-perils-of-youtubes-ad-privacy-settings',
  'gibson2025_analyzing': 'USENIX/2025/analyzing-the-ai-nudification-application-ecosystem',
  'cui2025_privacy': 'PETS/2025/understanding-privacy-norms-through-web-forms',
  'zhao2023_mobile': 'WWW/2023/are-mobile-advertisements-in-compliance-with-apps-age-group',
  'chen2013_this': 'WWW/2013/is-this-app-safe-for-children-a-comparison-study-of-maturity-ratings-on-android',
};

// Spans quoted from an external primary source. Value = the cached file the
// quote must appear in, so the check does not depend on the network.
const EXTERNAL = new Map(Object.entries({
  'open banking, photo ID matching, facial age estimation, mobile network operator age checks, credit card checks, digital identity services and email-based age estimation': 'aa/ofcom.txt',
  "methods including self-declaration of age and online payments which don't require a person to be 18 are not highly effective": 'aa/ofcom.txt',
  'host or permit content that directs or encourages users to attempt to circumvent an age assurance process': 'aa/ofcom.txt',
  'started returning age signals for users in Brazil': 'aa/android_age_signals.txt',
  'triggers, and survives, review under intermediate scrutiny because it only incidentally burdens the protected speech of adults': 'aa/paxton.txt',
  'a 23-fold increase on the previous six months': 'aa/ofcom2026.txt',
  'with a further 10 geo-blocking UK users': 'aa/ofcom2026.txt',
  'Circumvention appears low, although some activity may not be captured due to reporting bias.': 'aa/ofcom2026.txt',
  "serious doubts about the efficacy of some age inference models, which analyse a user's activity and behaviour on a platform to estimate whether they are a child or an adult": 'aa/ofcom2026.txt',
  'when creating an account, minors below 13 can enter a false birth date that makes them at least 13 years old, with no effective controls in place to check the correctness of the self-declared date of birth': 'aa/ec_meta.txt',
  'risk-based, flexible, tech-neutral and future-proof': 'aa/ico_joint.txt',
  'laid before the end of the year, and the changes should be implemented in Spring 2027': 'aa/gov_factsheet.txt',
  'should be implemented in Spring 2027': 'aa/gov_factsheet.txt',
  'serious doubts': 'aa/ofcom2026.txt',
  'Almost all analysed pornography services relied exclusively on third-party vendors, with only one analysed pornography service using an in-house solution': 'aa/ofcom2026.txt',
  'receive minimal outcome signals from third-party age assurance providers': 'aa/ofcom2026.txt',
  'facial age estimation and photo ID matching the most commonly deployed': 'aa/ofcom2026.txt',
}));

const TEXTCACHE = new Map();
function renderings(key) {
  if (TEXTCACHE.has(key)) return TEXTCACHE.get(key);
  const [venue, year, slug] = key.split('/');
  const dir = path.join(ROOT, year, venue, slug);
  const out = [];
  for (const r of ['paper.cols.txt', 'paper.norm.txt', 'paper.txt']) {
    const f = path.join(dir, r);
    if (fs.existsSync(f)) out.push([r, norm(fs.readFileSync(f, 'utf8'))]);
  }
  if (fs.existsSync(path.join(dir, 'paper.pdf'))) {
    try {
      out.push(['paper.pdf (pypdf)', norm(execFileSync('python3',
        [path.join('scripts', 'pdftext.py'), key],
        { encoding: 'utf8', maxBuffer: 1 << 28, stdio: ['ignore', 'pipe', 'ignore'] }))]);
    } catch { /* no pdf text; the other renderings still stand */ }
  }
  TEXTCACHE.set(key, out);
  return out;
}

const src = fs.readFileSync(process.argv[2] ?? 'aa/page.txt', 'utf8');
// Pull each span together with the nearest citekey that follows it on the page,
// so the check is "this quote is in THAT paper", not "in some paper".
const spans = [];
const re = /\/\/"(.*?)"\/\//gs;
let m;
while ((m = re.exec(src))) {
  // The citekey can sit on either side of the quote — a bullet often opens
  // with it and a table row always does. Look at the enclosing block first
  // (bullets and table rows are one line each; paragraphs are one line), then
  // fall back to the nearest key after, then the nearest before.
  const lineStart = src.lastIndexOf('\n', m.index) + 1;
  let lineEnd = src.indexOf('\n', m.index + m[0].length);
  if (lineEnd < 0) lineEnd = src.length;
  // Order the block's citekeys by DISTANCE from the quote, not by position in
  // the line. "first key in the block" searches the wrong paper first whenever a
  // bullet cites two papers and the quote belongs to the second one, and it
  // would report ok against the wrong paper if that paper's text happened to
  // contain a colliding phrase.
  const near = (s, offset) =>
    [...s.matchAll(/\{\[([a-z0-9_]+)\]\}/gi)]
      .map((x) => ({ key: x[1], d: Math.abs(offset + x.index - m.index) }));
  const cands = [
    ...near(src.slice(lineStart, lineEnd), lineStart),
    ...near(src.slice(Math.max(0, m.index - 900), m.index + 900), Math.max(0, m.index - 900)),
  ].sort((a, b) => a.d - b.d);
  const block = [...new Set(cands.map((c) => c.key))];
  spans.push({ text: norm(m[1]), cite: block[0] ?? null, block });
}

// One resolver, used by the main loop AND by the control below. A control that
// is a separate hand-written assertion cannot see a regression in the code it is
// supposed to be guarding: mutating the matcher to always succeed left the old
// control still printing "correctly not located" while the run passed with a
// quote attributed to the wrong paper.
function resolve(span) {
  const tryKeys = [...new Set([span.cite, ...span.block])].filter((k) => PAPERS[k]);
  for (const k of tryKeys) {
    const r = renderings(PAPERS[k]).find(([, t]) => t.includes(span.text));
    if (r) return { hit: r, hitKey: k, tryKeys };
  }
  return { hit: null, hitKey: null, tryKeys };
}

let fail = 0;
let warn = 0;
const tally = new Map();
for (const s of spans) {
  if (NOT_A_QUOTE.has(s.text)) {
    tally.set('not-a-quote', (tally.get('not-a-quote') ?? 0) + 1);
    console.log(`[ n/a] ${s.text.slice(0, 70)}  — ${NOT_A_QUOTE.get(s.text)}`);
    continue;
  }
  if (EXTERNAL.has(s.text)) {
    const f = EXTERNAL.get(s.text);
    const ok = fs.existsSync(f) && norm(fs.readFileSync(f, 'utf8')).includes(s.text);
    if (!ok) fail += 1;
    tally.set('external', (tally.get('external') ?? 0) + 1);
    console.log(`[${ok ? ' ok ' : 'FAIL'}] ${s.text.slice(0, 70)}  — ${f}`);
    continue;
  }
  // Every citekey in the block is tried, nearest first, so a bullet citing two
  // papers is not a false failure — but the paper the quote was actually found
  // in is printed, so a misattribution stays visible.
  const { hit, hitKey, tryKeys } = resolve(s);
  if (tryKeys.length === 0) {
    fail += 1;
    console.log(`[FAIL] ${s.text.slice(0, 90)}  — no mapped citekey near it and it is on no list`);
    continue;
  }
  if (!hit) fail += 1;
  tally.set('paper', (tally.get('paper') ?? 0) + 1);
  // A quote found in a paper OTHER than the nearest cited one is not an error,
  // but it is the shape in which a misattribution would hide, so it is flagged
  // rather than passing silently.
  const fellBack = hit && hitKey !== tryKeys[0];
  if (fellBack) warn += 1;
  console.log(`[${hit ? (fellBack ? 'WARN' : ' ok ') : 'FAIL'}] ${s.text.slice(0, 70)}  — ${hitKey ?? tryKeys.join('/')}, ${hit ? hit[0] : 'NOT IN ANY RENDERING'}` +
    (fellBack ? `  [nearest citekey was ${tryKeys[0]}; check the attribution]` : ''));
}

console.log(`\n${spans.length} spans: ` +
  [...tally.entries()].map(([k, v]) => `${v} ${k}`).join(', ') +
  `; ${fail} failed, ${warn} located in a paper other than the nearest citekey`);

// Controls. Both are pushed through resolve(), the same function the main loop
// uses, so a regression in the matcher fails them.
//   (a) a fabricated sentence must NOT resolve at all;
//   (b) a real sentence from paper A, placed in a block that cites B then A,
//       must resolve to A — which is what catches a resolver that returns the
//       first citekey's paper regardless of the text.
const CONTROLS = [
  { label: 'a fabricated span is not located',
    span: { text: norm('age verification is implemented by ninety-nine percent of all adult-only apps'),
            cite: 'yao2025_easy', block: ['yao2025_easy'] },
    expect: null },
  { label: 'a real span resolves to its own paper, not to the other key in the block',
    span: { text: norm('Despite being rated as 17+, 152 apps actually enforce an age limit of 21 years'),
            cite: 'vallina2019_porn', block: ['vallina2019_porn', 'yao2025_easy'] },
    expect: 'yao2025_easy' },
];
for (const c of CONTROLS) {
  const got = resolve(c.span).hitKey;
  if (got !== c.expect)
    throw new Error(`control failed (${c.label}): expected ${c.expect}, got ${got} — the quote checker is broken`);
  console.log(`control ok: ${c.label}`);
}
if (fail) throw new Error(`${fail} quoted spans on the page are unaccounted for`);
</file>

Its unedited output over the published page.

<file  aa_quotespans-output.txt>
[ n/a] X% of sites have an age gate  — the page's own example of a meaningless quantity
[ n/a] I am over 18  — the page describing what a button says, not quoting a paper
[ ok ] facial age estimation and photo ID matching the most commonly deployed  — aa/ofcom2026.txt
[ ok ] open banking, photo ID matching, facial age estimation, mobile network  — aa/ofcom.txt
[ ok ] methods including self-declaration of age and online payments which do  — aa/ofcom.txt
[ ok ] started returning age signals for users in Brazil  — aa/android_age_signals.txt
[ ok ] An age verification app generates one-time use credentials so that a u  — woodruff2026_risk, paper.cols.txt
[ ok ] the most widely implemented method (31.84%)  — yao2025_easy, paper.pdf (pypdf)
[ ok ] the least utilized (8.48%)  — yao2025_easy, paper.pdf (pypdf)
[WARN] Despite being rated as 17+, 152 apps actually enforce an age limit of   — yao2025_easy, paper.cols.txt  [nearest citekey was vallina2019_porn; check the attribution]
[ ok ] specially so in age-related keywords that appear often in the content   — vallina2019_porn, paper.cols.txt
[ ok ] only investigate a subset of the top-50 most popular pornographic webs  — vallina2019_porn, paper.cols.txt
[ ok ] if our automatic crawler manages to bypass the mechanism, a child coul  — vallina2019_porn, paper.cols.txt
[ ok ] we did not find any instance of AgeID being deployed during our study  — vallina2019_porn, paper.cols.txt
[ ok ] we evaluate the effectiveness of age verification. We find that age ve  — west2024_picture, paper.pdf (pypdf)
[ ok ] trivially bypassed by providing a birth year  — alomar2022_developers, paper.cols.txt
[ ok ] did not observe any mechanisms for obtaining verifiable parental conse  — alomar2022_developers, paper.cols.txt
[ ok ] a total of 13 different methods  — ekambaranathan2022_poster, paper.cols.txt
[ ok ] Circumvention appears low, although some activity may not be captured   — aa/ofcom2026.txt
[ ok ] a 23-fold increase on the previous six months  — aa/ofcom2026.txt
[ ok ] with a further 10 geo-blocking UK users  — aa/ofcom2026.txt
[ ok ] serious doubts about the efficacy of some age inference models, which   — aa/ofcom2026.txt
[ ok ] some potentially interesting app categories, such as dating apps, requ  — breuer2026_ad, paper.cols.txt
[ ok ] These include age-verification prompts from adult websites  — ardi2023_prevalence, paper.cols.txt
[ ok ] which often must be dismissed before any other interaction is possible  — ardi2023_prevalence, paper.cols.txt
[WARN] seven out of 20 apps, we observed at least one ad, typically after byp  — moti2025_whispertest, paper.cols.txt  [nearest citekey was ardi2023_prevalence; check the attribution]
[ ok ] we used a mobile VPN and selfie verification in Australia, Ireland, an  — mai2025_more, paper.cols.txt
[ ok ] a region in the U.S. that does not have an age verification law for ex  — gibson2025_analyzing, paper.pdf (pypdf)
[ ok ] Many websites use standalone age verification forms that only ask for   — cui2025_privacy, paper.cols.txt
[ n/a] Yes  — a keyword from the 2019 detector, listed individually below as a set
[ n/a] Enter  — same
[ n/a] Agree  — same
[ n/a] Continue  — same
[ n/a] Accept  — same
[ ok ] in 8 languages  — vallina2019_porn, paper.cols.txt
[ ok ] heavily relies on taint analysis, which might not be perfect  — yao2025_easy, paper.cols.txt
[ ok ] Almost all analysed pornography services relied exclusively on third-p  — aa/ofcom2026.txt
[ ok ] receive minimal outcome signals from third-party age assurance provide  — aa/ofcom2026.txt
[ ok ] the most common default languages in our list of pornographic websites  — vallina2019_porn, paper.cols.txt
[ n/a] Sites with an age gate  — same
[ ok ] with roughly half of the app promotions not in compliance with host ap  — zhao2023_mobile, paper.cols.txt
[ ok ] host or permit content that directs or encourages users to attempt to   — aa/ofcom.txt
[ ok ] while we analyze the age estimation of computer vision models, we do n  — west2024_picture, paper.cols.txt
[ ok ] triggers, and survives, review under intermediate scrutiny because it   — aa/paxton.txt
[ ok ] risk-based, flexible, tech-neutral and future-proof  — aa/ico_joint.txt
[ ok ] when creating an account, minors below 13 can enter a false birth date  — aa/ec_meta.txt
[ ok ] laid before the end of the year, and the changes should be implemented  — aa/gov_factsheet.txt
[ ok ] should be implemented in Spring 2027  — aa/gov_factsheet.txt
[ ok ] serious doubts  — aa/ofcom2026.txt

49 spans: 8 not-a-quote, 17 external, 24 paper; 0 failed, 2 located in a paper other than the nearest citekey
control ok: a fabricated span is not located
control ok: a real span resolves to its own paper, not to the other key in the block
</file>

===== Quote checks =====

Every per-paper figure on the content page carries a verbatim needle, checked by the script above. Two rules were applied after earlier runs on this wiki got them wrong:

  * **The needle must be specific.** A bare percentage is shared across papers and would pass against the wrong sentence. The one generic needle in the first draft — ''our data was collected from'', for the nudification paper — was replaced with ''a region in the U.S. that does not have an age verification law''.
  * **The checker must be able to fail.** A fabricated needle (''reveals that only 9,999 (99.99%) implement age verification'') is run against the same paper on every execution and the script throws if it is located. Without that control the check asserts nothing.

**25 of 25 needles located.** 21 in ''paper.cols.txt''; **4 only in the PDF**. A second guard, ''scripts/aa_quotespans.mjs'', takes the opposite direction: it pulls **every** ''//"…"//'' span out of the page source — **42** of them, of which 21 are located in a cited paper, 13 in a cached copy of an external primary source, and 8 are on a short list of the page's own scare quotes with a written justification for each. A curated needle list drifts from its page; this one cannot, because the page is its input.

Its two controls both run through the **same resolver the main loop uses**, which is the part an earlier version got wrong: a fabricated span must not resolve at all, and a real sentence placed in a block that cites two papers must resolve to //its own// paper. Mutating the matcher to always succeed fails the first; mutating the resolver to credit the first citekey regardless of the text fails the second. Citekeys are tried in order of character distance from the quote, and any span located in a paper other than the nearest citekey prints ''WARN'' rather than passing silently — two do, both correctly.

^ Needle ^ Paper ^ Where it was found ^
| ''we evaluate the effectiveness of age verification. We find that age verification, if done using the model deployed by TikTok, is less effective for younger demographics'' | {[west2024_picture]} | **none of ''paper.cols.txt'', ''paper.norm.txt'' or ''paper.txt''** — the sentence is interleaved with the adjacent column (''"…the effectiveness of age Java, apps create links to JNI calls through the native verification…"''). Re-extracting ''paper.pdf'' with ''pypdf'' finds it verbatim. |

''scripts/pdftext.py'' exists for exactly this and is called by both guards as a last resort. The lesson is the one already recorded for this corpus: a quote-check keyed on ''paper.cols.txt'' alone can score a faithful quote as a fabrication.

A second cause of the same false failure turned up while tightening the needles: **pypdf returns U+FB01 for the ''fi'' ligature**, so //"Biometric veriﬁcation is the least utilized (8.48%)"// in the PDF is not the //"Biometric verification…"// of a needle typed on a keyboard. Both guards now fold the five Latin ligatures before comparing. Two true quotes had failed on this.

One needle is **truncated on purpose**: ''Apps used a total of 13 different methods for'' {[ekambaranathan2022_poster]}. The next words in the file are ''"Moat age assurance"'' — a vendor name from the adjacent column has been spliced into the sentence. The needle stops before the splice.

==== The needle that proved nothing ====

Two needles in the first draft stopped short of the digits the page prints — ''"are the most widely implemented method"'' and ''"Apps used a total of 13 different methods for"''. Both located, and neither substantiated **31.84%**, **8.48%** or **12 apps**, which is the whole reason a quote is attached to a figure. A reviewer flagged it. The first was extended to include ''(31.84%)'', a separate needle was added for ''"Biometric verification is the least utilized (8.48%)"'', and a second needle covers the ''(12 apps)'' half of the poster row, which cannot be joined to the first because a vendor name from the adjacent column is spliced into the sentence between them.

===== Folding =====

Nothing on the content page is a fold. Free-text aggregation was **deliberately not used**: with a 5-paper population and a 38-paper candidate set, every value is read rather than counted, and there is no field in the extraction whose values would be aggregated.

What is printed instead is the **matched-form residue** of the probe — every distinct surface string the regex caught, with its tuple count, in section 2 of the output above. 19 distinct forms across 310 tuples. ''age verification'' alone is 236 of them; ''age assurance'' appears **4** times in the entire corpus, which is itself a finding about how new the regulator's vocabulary is.

===== What the extraction schema does and does not carry =====

There is no enum anywhere in the schema for age assurance. It appears only as free text, and only in one paper's ''detection[]'' tuples:

  * ''phenomenon: "age-verification implementation"'' / ''metric: "share of analyzed adult-only apps"'' / ''prevalence: "1,165 apps; 3.67% in the abstract, 3.75% in the evaluation"''
  * ''phenomenon: "age-verification method types"'' / ''prevalence: "Age gate 31.84%; biometric verification 8.48%"''

The first of those is worth noting: **the extraction caught an internal inconsistency in the paper** that the page then verified by hand. //Easy As Child's Play// reports 3.67% in its abstract and conclusion and 3.75% in §RQ5 for the same quantity. 1,165 ÷ 31,750 = 3.669%, so the abstract is right and the results section is wrong. The page quotes 3.67% and both quotes are checked.

''classification[]'' carries GUARD's own taxonomy — ''"age gate, template-based, online ID, credit card, document upload, biometric verification"'' — which is the source of six of the eight rows in the page's mechanism table. The other two rows (click-through interstitial, device or OS signal) come from {[vallina2019_porn]} and from the platform documentation below.

Because ''detection[].phenomenon'' is free text and ~20% stable run-to-run, it was used **only to find the paper**, never to count anything.

===== External sources =====

Everything in the //regulatory surface// table and in the platform-API bullets was fetched on **2026-09-15**. Nothing was written from recall.

^ Claim on the page ^ Primary source ^ How it was verified ^
| Ofcom's list of methods capable of being highly effective; self-declaration explicitly excluded; the "do not host circumvention content" expectation | Ofcom, //Ofcom publishes industry guidance on effective age checks//, 16 January 2025 | **ofcom.org.uk returns HTTP 403** to curl, to WebFetch and to a headless Chromium with a desktop user-agent and a full browser context. Read instead from the Internet Archive capture ''web.archive.org/web/20260513043257/…'', which is Ofcom's own page text. Both quotes on the page are verbatim from that capture. |
| //FSC v. Paxton// holding | US Supreme Court slip opinion, No. 23–1122 | PDF fetched from ''supremecourt.gov'' directly, text extracted with ''pypdf'', quote read from the syllabus. Argued 15 Jan 2025, decided 27 Jun 2025. |
| EU age-verification blueprint v1, five first-adopter Member States, interoperability with EUDI Wallets | ''digital-strategy.ec.europa.eu'' news article, published 14 July 2025 | Fetched with curl and a browser user-agent; HTTP 200; the five countries and the date read from the article body. |
| Blueprint v2 adds passport/ID-card onboarding and Digital Credentials API support | Same site, published 10 October 2025 | As above. |
| Commission recommendation urging deployment by end of 2026 | Same site, published 29 April 2026 | As above. |
| Apple ''DeclaredAgeRange'' availability | ''developer.apple.com/documentation/declaredagerange'' | The HTML page requires JavaScript and renders empty in a headless browser. Fetched the documentation JSON at ''/tutorials/data/documentation/declaredagerange.json'' instead: ''platforms'' gives iOS/iPadOS/Mac Catalyst/macOS ''introducedAt 26.0'', ''beta: false''. |
| Play Age Signals dates for Brazil and Texas | ''developer.android.com/google/play/age-signals'' | Fetched with curl; the two dates are in the page's own banner. The API is marked **beta** and the page says so. |
| Digital Credentials API status | chromestatus API, features 5166035265650688 and 5099333963874304 | Queried the JSON API rather than the HTML, then the **per-feature** endpoint, because the list endpoint's summary ''status'' field is wrong: it reports ''{"text":"Origin trial","milestone_str":"141"}'' for presentation, while the feature's own ''stages'' array gives stage 150 (origin trial) at desktop 134 / Android 128 and **stage 160 (ship) at desktop 141 / Android 141**. Presentation therefore //shipped// at 141; the trial had already closed. The first draft of the page said "in origin trial since Chrome 141" on the strength of the summary field, and a reviewer caught it by reading the stages. Issuance is stage 160 at desktop 155, not yet reached. Stable channel **153.0.8010.36, released 2026-09-08**, from ''chromiumdash.appspot.com/fetch_releases''. |
| Ofcom's statutory report on the use of age assurance — 69 million checks, 32 services, 25%→43%, top-10 and 64-of-100, "serious doubts" about age inference | Ofcom, //Report on the use of age assurance//, presented to Parliament under s157 Online Safety Act 2023, published 16 July 2026, 92 pp | ofcom.org.uk 403s this sandbox, but the report is mirrored on the UK government's asset host: ''assets.publishing.service.gov.uk/media/6a56377e2f6185941a9a6493/Report_on_the_use_of_age_assurance.pdf''. Fetched (HTTP 200, 1.59 MB), text extracted with ''pypdf'', and every quoted phrase checked verbatim by ''aa_quotespans.mjs'' against the cached extraction. **Found by a reviewer, not by me** — the first draft of the page had no 2026 UK material at all and dated Ofcom at January 2025. |
| European Commission preliminary finding against Meta under DSA Article 28 | ''digital-strategy.ec.europa.eu'' press release, published 29 April 2026 | Fetched with curl and a browser user-agent; HTTP 200; the quoted sentence about the false birth date is verbatim from the article body. Also found by a reviewer. |
| Ofcom/ICO joint statement on age assurance, 25 March 2026 | //Age Assurance: A Joint Statement by Ofcom and the Information Commissioner's Office//, 14 pp | **''ico.org.uk'' is not blocked even though ''ofcom.org.uk'' is** — the same joint document is published by both regulators, and the ICO's copy fetches HTTP 200 at ''ico.org.uk/media2/5ybpmabf/ofcom-ico-joint-statement.pdf'' (733,881 bytes). A reviewer found the host; the PDF was then fetched and extracted here. |
| UK ban on certain social media for under-16s, Spring 2027 | //Fact sheet: New rules to protect children online//, DCMS and DSIT, updated 17 July 2026 | ''gov.uk'' is not blocked. Fetched HTTP 200 and the timeline read from the body. This row was **rejected on the first review pass and un-rejected on the second**, when a reviewer produced the government's own fact sheet in place of the law-firm posts it had originally been proposed from. |
| Australian minimum-age Act, day-of-effect instrument, and the 2026 enforcement amendment | Federal Register of Legislation | Queried ''api.prod.legislation.gov.au/v1/titles'' with an OData filter. Three records: C2024A00127 (Act, 10 Dec 2024), F2025N00628 (day-of-effect instrument, 29 Jul 2025), C2026A00083 (enforcement amendment, 11 Sep 2026). |

==== Rejected, and why ====

^ Source ^ Why it was not used ^
| Law-firm and vendor explainers on the UK OSA (Lewis Silkin, White & Case, Norton Rose, Reed Smith, Crowell) | Secondary. Every fact they carry is in Ofcom's own page, which was obtained. |
| Age-assurance vendor blogs (Yoti, Incode, VerifyMy) | Vendors selling the mechanism the page is about. Not cited, in any form. |
| The Age Verification Providers Association's US state-law tracker | A trade body's count of state statutes. Found by following a citation in a corpus paper (the PoPETs 2025 SoK on web authentication cites it for "US State age verification laws for adult content"), which is how it came to be considered at all. Rejected: the page makes no claim about how many US states have such a law, because every available count is advocacy-side. One statute with a Supreme Court citation is used instead. |
| Free Speech Coalition's bill tracker | Same reason, other side. Also unreachable: ''action.freespeechcoalition.com'' failed to load in the headless browser. |
| Australian eSafety Commissioner's minimum-age industry page | ''esafety.gov.au'' returned ''ERR_HTTP2_PROTOCOL_ERROR''. The legislative register was used instead, which is the primary source anyway. |
| News coverage of the EU app's April 2026 "technically ready" status | Replaced by the Commission's own 29 April 2026 recommendation page. |
| Secondary coverage of the Ofcom 2026 report (Lewis Silkin, Verifymy, Xident, biometricupdate) | These are how the report was //found//, and none of their figures were used. The report's own PDF was obtained and every figure taken from it. |
| Wikipedia and law-firm posts giving 10 December 2025 as the Australian day of effect | Consistent across four secondary sources, and still not a primary one. The page continues not to assert the day. |

===== What could not be established =====

  * **The calendar day of effect of the Australian minimum-age rule.** The instrument F2025N00628 is registered and named, but its text would not extract — ''legislation.gov.au'' returns HTML from every ''/text'' and ''/downloadPdf'' route tried, and the API has no document endpoint for a notifiable instrument. Four secondary sources (Wikipedia and three law firms) agree on **10 December 2025**, and a reviewer proposed adding it. Not added: the whole point of that table is that each row names a primary source read on the day. The page names the Act and the instrument and **does not assert the day**.
  * **How many US states have an age-verification statute.** No non-advocacy tracker was found. Not claimed.
  * **Ofcom's promised "rapid assessment" of age assurance for an under-16 threshold**, reported as due to Parliament by the end of October 2026. Only law-firm sources say so; the government's own fact sheet does not carry that deadline, and Ofcom's site is unreachable. Not on the page.
  * **Any web-side prevalence figure after 2019 //from the literature//.** //Tales from the Porn// is seven years old, predates the UK duty, the EU blueprint and every US state statute, its 20% is a figure for click-through interstitials — a category the current UK rules explicitly exclude — and its denominator is at most fifty hand-checked sites. There is no more recent web measurement in these seven venues. There //is// a more recent deployment measurement: Ofcom's, obtained by statutory compulsion over 32 services and not reproducible by anyone else. The page now leads on that contrast rather than on the absence.
  * **Recall of either published detector against an independently drawn sample.** {[yao2025_easy]} does report an error rate — 100 apps sampled from each side of GUARD's own output, hand-verified, 3 false positives and 2 false negatives — but a sample drawn from the classifier's own positives bounds precision, not recall over the population. {[vallina2019_porn]} reports no evaluation of its age-verification detector at all and abandoned it for manual checking. The page says this rather than the flat "neither reports recall" an earlier draft carried.
  * //(An earlier draft had a fourth entry here, claiming one candidate was judged from title and summary only because its full text was missing. That was false — see the mistakes list below.)//

===== Judgement calls =====

  - **A new page rather than a section on a neighbour.** The obvious alternative hosts were [[:design:blocking_and_geodifference]] (same detection problem) and [[:privacy:consent]] (same interstitial-as-treatment problem). Rejected because the //denominator// and the //ethics// are both unlike anything on either page, and because the id was already promised on [[:roadmap]] and gated by ''scripts/sitemap.mjs''. The cross-links do the work the alternative would have done.
  - **Publishing a page backed by five papers.** The alternative was to record "not enough literature" on the roadmap and stop. Rejected because the page's useful content is **method and denominators**, not a literature review, and because the six ''OBSTACLE'' papers mean a reader will meet this topic whether or not they set out to study it.
  - **''SECTION'' as a verdict, and four papers in it.** A stricter rule ("the paper is about age assurance") gives a population of 1 and a page with nothing in it. A looser one ("the paper mentions a measurement") pulls in the 19 ''MENTION'' papers. The line drawn is //reports an empirical result about the mechanism//, and a reasonable person could exclude {[alomar2022_developers]} on the grounds that a developer survey measures beliefs rather than deployment — which is why the page presents that paper's self-report and its contradicting observation together.
  - **Counting the CCS 2022 poster.** It is a poster with n=137 and no peer-reviewed full paper behind it in this corpus. Included because it is the only mechanism taxonomy in the corpus derived from observation, and labelled as a poster on the page.
  - **Not manufacturing a figure table.** The queued row asked for this explicitly and it was honoured: the page has one table of per-year candidate counts and one of per-paper figures, and no aggregate prevalence table, because five studies of five populations cannot be tabulated together.
  - **Dating methods rather than ranking them.** The corpus's own ranking would put "click-through interstitial" first because that is what 2019 measured. The page instead uses Ofcom's 2025 line — self-declaration is not age assurance — to say plainly that the corpus's most-measured mechanism is the one the regulator excludes. That is an external standard imported into a corpus-driven page, and it is a judgement.
  - **Ethics given its own section rather than a line.** [[:practices:ethics]] does not currently cover adult-content populations, synthetic identity submission, or publishing a working bypass. Rather than edit that page in the same sitting (and orphan its own figures), the four questions are stated here and flagged as a gap for it.

===== The run =====

^ ^ ^
| Date | 2026-09-15, one sitting |
| Corpus | ''data/extract/run1'', 5,859 papers, 5,855 with ''paper.cols.txt'', seven venues, 2010–2026 |
| Model | Claude Opus 5 for the derivation, drafting and external fetches |
| Scripts committed | ''scripts/report_age_assurance.mjs'', ''scripts/aa_quotespans.mjs'', ''scripts/pdftext.py'', ''scripts/build_provenance_age_assurance.py'' |
| Bibliography | 11 new entries; 0 duplicate keys, 0 duplicate DOIs against the live file; ''scripts/bib_dedup_scan.py'' reports 0 definite duplicate pairs over the merged 1,036 entries |
| Authors filled by hand | four PoPETs records whose landing pages ''fetch_authors.py'' could not parse (2018-0021, 2022-0108, 2025-0094, 2026-0046), read from ''petsymposium.org'' and written to ''out/authors.json'' |

==== Mistakes caught during the run ====

Recorded because they are the part with value.

  - **Two probe regexes were wrong in the same way**, and both were caught only by reading the printed residue rather than the count. ''voltage signal'' matched ''age signal'', ''improve their agency'' matched ''prove their age'', ''webpage screenshot'' matched ''age screen''. The first version of the candidate set was 233 papers, of which nine of the top thirteen were electromagnetic side-channel work.
  - **The roadmap's own committed probe has the same defect** and is the reason //F-BLEAU: Fast Black-Box Leakage Estimation// is one of its 11 age-assurance candidates.
  - **The first quote check failed on a true quote.** {[west2024_picture]}'s contribution sentence is spliced across a column boundary in all three ''.txt'' renderings; ''pypdf'' has it verbatim. The checker was rewritten to try four renderings and to print which one located each needle, and a fabricated-needle control was added so the check cannot pass vacuously.
  - **The bibliography cache served a stale parse.** After appending 11 entries and saving the page, 11 of 20 references rendered as allocated-but-empty numbers while both sources looked perfect. Purging ''literature/bibliography?purge=true'' and then the page fixed it. The check that matters is **distinct citekeys against rendered references**: 20 = 20. The marker-to-span ratio is not a check — the plugin emits **two** ''bibtex_citekey'' spans per marker, so the page's 55 markers render as 110 spans, and an earlier note here that read "42 markers, 42 spans" was counting one of them with a regex that only matched the opening span.
  - **ofcom.org.uk is unreachable from this sandbox** (403 to curl, to WebFetch, and to a full headless-Chromium context). The January 2025 quotes come from an Internet Archive capture; the July 2026 report was obtained from the UK government's asset host, which is not blocked. The page says so in both footnotes rather than implying a direct read.
  - **A de-hyphenation artefact was published as a defect in someone else's paper.** The artefacts table asserted that the repository URL printed in {[moti2024_targeted]} 404s. It does not. The paper prints ''https://github.com/targeted-and-troublesome/'', broken across a line at ''targeted-and-''; ''paper.cols.txt'', ''paper.norm.txt'' and the extraction's ''artifacts.codeUrl'' all rejoin it as ''targeted-andtroublesome'', which is what 404s. Only ''paper.txt'', which preserves the line break, shows the truth. The generic reviewer found it by reading all three renderings. **The tooling's own artefact was published as their error**, which is the second time in this run that an ENOENT or a mangled string was read as a fact about someone else's work.
  - **A mistyped slug produced a published falsehood about the corpus itself.** An early context pull used ''…privacy-preserving-facial-transformation**s**'' where the directory is singular. The ENOENT was read as "this paper has no full text", and that became an ''ARTEFACT'' verdict, a script comment, a provenance bullet and a limitation on the content page saying one candidate had been judged from its title alone. The file is there, 102 KB; the probe read it; its five matches are //age estimation// as the name of a face-attribute ML task. The generic reviewer found it by checking the claim against the script's **own printed list** of the four papers without full text — GAN-Invert is not on it. The verdict is now ''MENTION'' (20/7) and the limitation is deleted. Nothing about the population changed, but four separate places had repeated the same unchecked inference.
  - **The page's most important figures were the ones a first draft got wrong, and neither was caught by a guard.** The 2019 age-verification percentages were published against the wrong denominator — the paper's 6,843-site corpus and "six vantage points", where the section itself is a hand check of at most fifty sites in four countries — because the quote-check located the sentence and nothing checked what the sentence was a share //of//. And the whole 2026 UK deployment picture was missing, because the page was written from the corpus and the corpus stops at seven academic venues. Both came from reviewers.

==== Review log ====

Four reviewers, each told explicitly that the author's context may not be exhaustive, and each handed the page text, the report script, its output and this provenance draft. The three focused passes ran in parallel on the pre-review draft; their findings were applied in one pass, and all three were then re-run against the corrected pages. The generic pass ran last — and its first finding was a process one: **the page was republished twice while it was reading**, so it reviewed a moving target and had to mark which of its findings the in-flight edits already fixed. That is a real cost and it is recorded rather than tidied away; the next page should freeze the source while the generic pass runs.

^ # ^ Pass ^ Finding ^ Disposition ^
| 1 | figures vs script (''sonnet'') | The provenance page said the title probe had **1 of 11** in the population, 9.1% precision and 20% recall, contradicted by its own table three rows above, which scored two more as ''SECTION''. Real figures: **3 of 11**, 27.3% precision, 3 of 5 (60%) recall. Re-derived by re-running ''gap_probe_roadmap.mjs'' and intersecting extraction keys. | **Accepted.** Fixed here and on [[:roadmap]], where the wrong numbers had already been published. |
| 1 | figures vs script | Re-ran the report script: output reproduces the committed file byte-for-byte. Every page figure matches. Mutation-tested both published guards — narrowing ''LOOSE'' made the containment assertion throw; stubbing ''locate()'' made the fabricated-needle control throw. Neither passes vacuously. | **Accepted as a pass.** No change. |
| 1 | figures vs script | Two needles stopped before the digits the page prints, so the quote check did not substantiate 31.84%, 8.48% or 12 apps. | **Accepted.** Needles extended; two more added. |
| 2 | citations and quotes (''sonnet'') | **The 20%/14%/8%/12% figures are not over 6,843 sites from six vantage points.** §7.2 is a manual check of //"a subset of the top-50 most popular pornographic websites"// in **4** countries, because the authors judged their own keyword detector too false-positive-prone. | **Accepted — the most serious finding of the run.** Corrected in the population table, both figure bullets, the denominator section and the report script, and the paper's own reason for going manual is now on the page, because it is a methodological point in its own right. |
| 2 | citations and quotes | The page called {[west2024_picture]} an age-verification accuracy study; age is one of its **two** case studies and the paper is framed as an on-device-ML fairness audit. | **Accepted.** Qualified in the table, the figure bullet and the reading list. |
| 2 | citations and quotes | ''zhao2023_mobile'' BibTeX has //Grundy, John C.//; the paper's title page says //John Grundy//. | **Accepted.** Corrected in [[:literature:bibliography]]. |
| 2 | citations and quotes | All 20 citekeys resolve; no duplicate keys, DOIs or titles; 10 of 11 new entries verified against the paper's own title page including author order; every regulatory footnote checked against its primary source. | **Accepted as a pass.** |
| 3 | external currency (''sonnet'') | **Ofcom published a statutory report on 16 July 2026** — 69 million age checks over 32 services, 25%→43% child exposure, all of the UK top-10 and 64 of the top-100 pornography services — and the page dated UK regulation at January 2025. | **Accepted, and it changed the page's thesis.** The report's own PDF was fetched from the government asset host and every figure taken from it; the page now has a section on it, and the contrast between a regulator with information-notice powers and a literature with fifty hand-checked sites is the argument the page had been missing. |
| 3 | external currency | **The Commission preliminarily found Meta in breach of the DSA on 29 April 2026** over self-declared birth dates on Instagram and Facebook. | **Accepted.** Primary press release fetched; a row added to the regulatory table. |
| 3 | external currency | **Chrome's Digital Credentials API presentation support //shipped// in 141**; it did not enter origin trial then. The chromestatus list endpoint's summary ''status'' field says otherwise, and disagrees with the same feature's ''stages'' array. | **Accepted.** Verified against the per-feature endpoint. Footnote rewritten and the gotcha recorded above. |
| 3 | external currency | The Australian day of effect is 10 December 2025 per Wikipedia and three law firms. | **Rejected.** Consistent secondary sourcing is not a primary source, and every other row in that table names one. The page still does not assert the day. |
| 3 | external currency | An Ofcom/ICO joint statement (25 Mar 2026) and a UK plan for an under-16 social-media threshold (June 2026) are also missing. | **Rejected on the first pass, and the rejection was wrong** — see the re-review rows below. Both were surfaced only through law-firm posts and neither could be read from a primary source //at that point//; the reviewer then found primary sources for both and they were added. |
| 3 | external currency | Apple, Google Play, the EU blueprint pages, the SCOTUS opinion, the Australian register ids and all footnote URLs re-fetched and confirmed; no dead links. | **Accepted as a pass.** |

All three focused passes were then **re-run against the corrected pages**, because their findings had been acted on:

^ # ^ Pass ^ Finding on re-review ^ Disposition ^
| 1 | figures vs script, re-run | **''aa_quotespans.mjs''' fabricated-span control did not exercise the code it guarded.** It was a separate hand-written assertion against one hardcoded paper/string pair, so mutating the matcher to always succeed left it still printing "correctly not located" while a quote was silently reattributed to the wrong paper. | **Accepted.** The resolver is now one function used by both the main loop and the controls, and there are two controls: a fabricated span must not resolve, and a real span in a two-citekey block must resolve to **its own** paper. Both were mutation-tested: an always-matching matcher fails the first, a resolver that credits the first key fails the second. |
| 1 | figures vs script, re-run | The page said §2.2 of {[yao2025_easy]} is titled //Age Verification Methods//; that is Table 2's caption. The section is //Age Verification in Adult-Oriented Apps//. | **Accepted.** Corrected, and the reference now points at the table explicitly. |
| 1 | figures vs script, re-run | Script output reproduces the committed file byte-for-byte; the 3-of-11 / 27.3% / 60% figures, the corrected 2019 denominator, the Ofcom figures and the new 25-of-36 COPPA platform split all verified independently. | **Accepted as a pass.** |
| 2 | citations and quotes, re-run | **''aa_quotespans.mjs'' picked the //first// citekey in a block, not the nearest one.** On the one bullet that cites two papers it tried {[chen2013_this]} first and only fell through to the right paper by luck; a colliding phrase in the wrong paper would have passed as ''ok''. | **Accepted.** Citekeys are now ordered by character distance from the quote, and a quote located in any paper other than the nearest one prints ''WARN'' with both keys. Two spans currently warn, both correctly. |
| 2 | citations and quotes, re-run | Every Ofcom 2026 figure and quote, the Meta press-release quote, the rewritten Chrome footnote, the GUARD FP/FN entry and the corrected 2019 denominator all verified against their primary sources. The claim that {[vallina2019_porn]} reports no evaluation of its detector was independently confirmed. | **Accepted as a pass.** |
| 3 | external currency, re-run | **Two of the first pass's rejections were wrong, and the reviewer proved it by finding the primary sources.** ''ico.org.uk'' publishes the same Ofcom/ICO joint statement that ''ofcom.org.uk'' blocks, and the UK under-16 plan has a government fact sheet on ''gov.uk''. | **Accepted, both rejections reversed.** Two rows added to the regulatory table, and the under-16 timeline changed the page's closing argument: every figure on the page is about pornography and adult-only apps, and from Spring 2027 the UK duty is about ordinary social media. |
| 3 | external currency, re-run | The Australian day of effect is still unreachable: ''/text'', ''/downloadPdf'' and ''/contentDocuments'' all 404, the public page is a client-rendered SPA with an empty body, and ''esafety.gov.au'' and ''infrastructure.gov.au'' fail at the network layer. | **Rejection stands.** Still not asserted. |
| 3 | external currency, re-run | Nothing supersedes the Ofcom report or the Meta preliminary finding; the Chrome stage reading and all page figures re-confirmed against the primary sources. | **Accepted as a pass.** |

And the generic pass, which read the page after all of the above:

^ Finding ^ Disposition ^
| **The "one candidate judged from title only" limitation is false** and the script's own output contradicts it. | **Accepted**, and it is the worst defect of the run — see the mistakes list above. |
| The probe is described on the page as **five phrases**; it is fourteen forms, and six of the 38 candidates enter only through the extra ones. A reader reproducing "the five phrases" gets about 31. | **Accepted.** The page now says fourteen and points at the pattern here. |
| **The 2019 detector is not English-only** — it searches its five keywords //"in 8 languages"//, chosen as the most common default languages in that corpus. | **Accepted.** A flat factual error, corrected in the detection table and the language bullet. The paper's claim to //"manually validate the accuracy of our method in Section 7.2"// is also now stated precisely: §7.2 replaces the detector with a manual check and reports no error rate. |
| The page said the CCS poster is //the only mechanism taxonomy derived from observation// while also crediting GUARD's six categories with six of the eight rows in its own mechanism table. | **Accepted.** Scoped to children's apps. |
| //"The first three are the ones that exist at scale"// is contradicted by the page's own Ofcom section (facial age estimation and photo ID matching most commonly deployed). | **Accepted.** Both halves now name their population and the sentence no longer asserts a general ranking. |
| Six unsupported negatives, including //"the field's most-cited web figure"//, //"the corpus contains no general guidance"// on adult-content crawls (there is: {[vallina2019_porn]} §8), and //"the ordering below is the one the field and the regulators both use"//. | **Accepted, all six.** Each is now bounded to what was actually checked. |
| //"the mentions are growing and the measurements are not"// generalises past //n// = 1–2 per year, and the 17-of-38 has no base rate beside it. | **Accepted.** 44.7% of candidates against 20.2% of the corpus is now printed, and the second half is labelled a small-//n// observation. |
| **The page names no artefacts.** GUARD's code, the 500-labels repository, the 2,004-site child-directed list and DiffAudit's release all exist; a methods page that says "do not rebuild one" should list them. | **Accepted, and it turned up a live defect in a paper.** A new section gives all four with their HTTP status, and checking them found that the URL printed in {[moti2024_targeted]} — ''github.com/targeted-andtroublesome/'' — **404s**: the org is ''targeted-and-troublesome'' and the list is ''urls/kids_websites.csv'' in ''targeted-and-troublesome-crawler'', 2,004 distinct URLs, which matches the paper exactly. The same section records that {[vallina2019_porn]}'s population construction cannot be repeated because Alexa's Adult category was retired in 2022. |
| //"the rule that separates a gate from a cookie banner, a paywall and a login wall"// is demanded and never sketched; and //"all four are dismissable overlays"// is wrong about paywalls and login walls. | **Accepted.** The features to build such a rule from are now listed, with the honest note that nobody has published one. |
| The page claims //"a verbatim-quote check of every per-paper number"//; eleven numbers sit outside every needle and a mutation of 14% to 24% passed both guards. The reviewer hand-verified all eleven and they are correct. | **Accepted.** The methodology section now names the eleven figures that are hand-checked rather than guarded. |
| Missing practical point in the ethics section: verification vendors expose sandbox modes. | **Rejected for now.** No primary source was reached for which vendors do, and the page does not name vendors at all; asserting it would be exactly the vendor-marketing claim the source policy on this page rejects. Recorded here so the next run can close it. |
| No neighbouring page links back to this one. | **Accepted and done**, as a separate edit after the page was frozen for the re-review: one line each in [[:design:blocking_and_geodifference]], [[:design:crawling_location]], [[:privacy:consent]] and [[:practices:ethics]]. No figures on those pages were touched. |
| Several phrases repeat (//at most fifty// five times, //one of its two case studies// three times). | **Partly accepted.** Two instances trimmed; the rest carry the caveat in places a reader may arrive at directly. |
| Nits: {[west2024_picture]} uses Frida rather than the camera; the 5,855-apps / 5,855-papers coincidence; an unsupported causal claim about the six obstacle papers; the unused //"we did not find any instance of AgeID being deployed"//. | **All four accepted.** |

Both passes were re-run once more against the frozen pages. What they found the second time:

^ Finding ^ Disposition ^
| **The artefacts section published a defect in someone else's paper that does not exist.** {[moti2024_targeted]} prints its repository URL correctly; ''paper.cols.txt'', ''paper.norm.txt'' and the extraction all de-hyphenate it across a line break into a URL that 404s. | **Accepted — the worst finding of the second round**, and recorded in the mistakes list above. The row now states the correct URL, its HTTP 200, and the artefact that caused the confusion. |
| The Zenodo record for GUARD is titled //Drexel-SePAL/AgeScope: v1.0.1-1//, so searching for "GUARD" will not find it. | **Accepted.** The row now says so. |
| Three unbounded negatives introduced by the new artefacts section — //"there is no dataset"//, //"nobody has published one"//, and an ordering sentence attributing a sort to Ofcom that Ofcom does not perform. | **Accepted, all three.** |
| The closing argument said a crawl against adult sites would measure //the smaller half//, which nothing on the page sizes, and asserted the Spring 2027 date the fact sheet hedges. | **Accepted.** Both softened to what the sources support. |
| The intro's verdict clause attached //"the name of an unrelated ML task"// to the 7 ''ARTEFACT'' papers, where it describes the 20th ''MENTION''. | **Accepted.** Reworded. |
| The methodology section's list of eleven hand-checked figures is incomplete — the artefact file sizes, Ofcom's 25%-to-43% and 64-of-100, the survey base of 50 and several paraphrased denominators are in the same position. | **Accepted, and the enumeration was abandoned.** It was incomplete twice; the page now states the class instead, which is both shorter and true. |
| Three stale numbers in this provenance page: the "most common way it appears in the corpus" phrasing, an unsourced "194 phrase matches", and a marker-to-span identity that was never consistent. | **Accepted, all three.** The last one is worth keeping in mind: the bibtex plugin emits **two** spans per marker, so marker count and span count are not a check. |
| Scripts reproduce their committed outputs byte-for-byte; the verdict split, 44.7%/20.2%, the four artefact URLs, 55,481 bytes, 2,004 distinct URLs and the 2025-02-04 push date all independently verified. | **Accepted as a pass.** |
| The two ''WARN'' spans are correctly attributed; the log did not name them. | **Accepted.** They are //"Despite being rated as 17+…"// (nearest key ''vallina2019_porn'', found in ''yao2025_easy'') and //"seven out of 20 apps…"// (nearest key ''ardi2023_prevalence'', found in ''moti2025_whispertest''). In both, the previous bullet's trailing citekey is closer in characters than the bullet's own key at its end. A third ''WARN'' would be a new thing to look at. |


===== Related =====

  * **[[:privacy:age_assurance|The page these notes are for]]**
  * [[:literature:corpus]] — venue scope, the selection funnel, and the provisional 2025–2026 years
  * [[:roadmap]] / [[:provenance:roadmap]] — where this page's id and queued estimate came from
  * [[:provenance:privacy]] — the namespace's own log

No ''%%~~DISCUSSION~~%%'' block here, following the convention set by the other provenance pages: comments belong on the content page.

<bibtex bibliography></bibtex>