Measure The Web

User Tools

Site Tools

Trace: • web_vulnerabilities • traffic_files • hypothesis_testing • ethics • existing_datasets • website_selection • regression
provenance:design:platforms

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
provenance:design:platforms [2026/08/27 20:51] – Add the generic reviewer's log (14 findings, 11 accepted outright, 3 partly); correct this page's own quote-check coverage overclaim and regenerate the stale script/output blocks (42 checks, all passing); record the framing, cross-page-fold and schema-lab karel.kubicek.claudeprovenance:design:platforms [2026/09/22 01:33] (current) – Embed the real scripts/auth145_audit.mjs in the provenance log, per the page's convention. Authored by Claude karel.kubicek.claude
Line 4752: Line 4752:
   - **A true finding was framed so it invited a false inference.** The headline box's "official researcher routes are almost entirely absent from this literature" is correct as a count — the generic reviewer re-ran it with wider probes and the absolutes held — but a fresh reader would have concluded that the programmes do not work, when part of the absence is mechanical: 2023-and-later launches, a provisional 2025–2026 slice, and the venues that publish most such work (ICWSM, CHI, FAccT, communications journals) excluded by construction. That caveat existed on the page, 280 lines below the claim. Moved into the box.   - **A true finding was framed so it invited a false inference.** The headline box's "official researcher routes are almost entirely absent from this literature" is correct as a count — the generic reviewer re-ran it with wider probes and the absolutes held — but a fresh reader would have concluded that the programmes do not work, when part of the absence is mechanical: 2023-and-later launches, a provisional 2025–2026 slice, and the venues that publish most such work (ICWSM, CHI, FAccT, communications journals) excluded by construction. That caveat existed on the page, 280 lines below the claim. Moved into the box.
   - **Three pages counted the same names and got different numbers, and none of them said so.** [[design:website_selection]] reports 463 papers for Alexa; this page reported 412; [[design:user_studies]] publishes 139 / 92 / 279 for Mechanical Turk against this page's 179. A subset appearing to exceed its superset is a signal that two folds disagree, not that one is broken. Reconciled and now printed by the report script: **486** papers name Alexa in a stated population source at all, **464** of those have a web-unit population tuple — which is what [[design:website_selection]] measures — and 412 is what survives our role rule. The content page carries the reconciliation and does not claim its fold is the better one.   - **Three pages counted the same names and got different numbers, and none of them said so.** [[design:website_selection]] reports 463 papers for Alexa; this page reported 412; [[design:user_studies]] publishes 139 / 92 / 279 for Mechanical Turk against this page's 179. A subset appearing to exceed its superset is a signal that two folds disagree, not that one is broken. Reconciled and now printed by the report script: **486** papers name Alexa in a stated population source at all, **464** of those have a web-unit population tuple — which is what [[design:website_selection]] measures — and 412 is what survives our role rule. The content page carries the reconciliation and does not claim its fold is the better one.
-  - **A schema label was published as an audited one.** The ''%%crawlConfig.authentication%%'' split (145 / 22 / 4 / 0) was stated as what papers "state", when [[programming:registration]] warns on this exact field that ''%%crawlConfig%%'' carries one evidence quote for the whole object so the label cannot be checked against it, and [[privacy:consent]] measured 19.4% false positives on ''%%consentAction%%'', another field of that same object. We did not audit the 145; the page now says so. This is the finding most likely to move a number if someone does the audit.+  - **A schema label was published as an audited one.** The ''%%crawlConfig.authentication%%'' split (145 / 22 / 4 / 0) was stated as what papers "state", when [[programming:registration]] warns on this exact field that ''%%crawlConfig%%'' carries one evidence quote for the whole object so the label cannot be checked against it, and [[privacy:consent]]'s complete audit found **279 of 313 ''no-interaction'' labels unsupported (89.1%)**, alongside **7 of 36 (19.4%)** false positives among labels claiming interaction. We did not audit the 145; the page now says so. This is the finding most likely to move a number if someone does the audit.
   - **No accidental exposure.** No credentials, participant data or unpublished material was written to the wiki. The only non-public thing touched was ''.env'' for the JSON-RPC credentials, read by ''scripts/dw.mjs''.   - **No accidental exposure.** No credentials, participant data or unpublished material was written to the wiki. The only non-public thing touched was ''.env'' for the JSON-RPC credentials, read by ''scripts/dw.mjs''.
  
Line 4811: Line 4811:
 ^ Finding ^ Verdict ^ Action ^ ^ Finding ^ Verdict ^ Action ^
 | The headline "almost entirely absent from this literature" is numerically right (it re-ran the probes wider and the absolutes held) but invites the inference that the programmes do not work; the mechanical explanation sat 280 lines below | **accepted** | caveat moved into the box | | The headline "almost entirely absent from this literature" is numerically right (it re-ran the probes wider and the absolutes held) but invites the inference that the programmes do not work; the mechanical explanation sat 280 lines below | **accepted** | caveat moved into the box |
-| The ''%%crawlConfig.authentication%%'' split republishes a schema read that [[programming:registration]] explicitly warns against on this field, with no audit and no caveat | **accepted — the highest-value finding** | caveat carried inline, with the sibling field's measured 19.4% false-positive rate; the 145 remain unaudited and the page says so |+| The ''%%crawlConfig.authentication%%'' split republishes a schema read that [[programming:registration]] explicitly warns against on this field, with no audit and no caveat | **accepted — the highest-value finding** | caveat carried inline, with the sibling field's measured 89.1% unsupported-label rate (and 19.4% interacting-label false-positive rate); the 145 remain unaudited and the page says so |
 | Cross-page count conflicts: Alexa 412 here vs 463 on [[design:website_selection]] (a subset exceeding its superset), and Mechanical Turk 179 here vs 139 / 92 / 279 on [[design:user_studies]] | **accepted** | reconciled in the report script and on the page: 486 name Alexa at all, 464 with a web-unit population, 412 after the role rule | | Cross-page count conflicts: Alexa 412 here vs 463 on [[design:website_selection]] (a subset exceeding its superset), and Mechanical Turk 179 here vs 139 / 92 / 279 on [[design:user_studies]] | **accepted** | reconciled in the report script and on the page: 486 name Alexa at all, 464 with a web-unit population, 412 after the role rule |
 | This page's quote-check header claimed to cover "every figure the content page attributes to a paper" while ~12 attributed figures were in no check at all, and the embedded script and output were stale | **accepted** | 16 checks added (42 total, all passing), both ''%%<file>%%'' blocks regenerated, and the overclaim recorded above rather than quietly fixed | | This page's quote-check header claimed to cover "every figure the content page attributes to a paper" while ~12 attributed figures were in no check at all, and the embedded script and output were stale | **accepted** | 16 checks added (42 total, all passing), both ''%%<file>%%'' blocks regenerated, and the overclaim recorded above rather than quietly fixed |
Line 4838: Line 4838:
  
 <bibtex bibliography></bibtex> <bibtex bibliography></bibtex>
 +
 +===== ConsentAction audit propagation, 2026-09-17 =====
 +
 +**What was wrong.** The page used the 19.4% interacting-side ''consentAction'' false-positive rate as its warning about the neighbouring authentication labels, even though the complete audit found a much larger no-interaction-side failure.
 +
 +**Command and real output.** ''node scripts/platforms_report.mjs'' → exit 0; its platform figures were unchanged. ''node scripts/consent_action_noninteraction_audit.mjs'' → **279/313 = 89.1%** of no-interaction labels unsupported; the interacting-side comparison remains **7/36 = 19.4%** false positives.
 +
 +**What changed.** [[design:platforms]] and this provenance page now cite the stronger 89.1% result alongside 19.4%. The **145-paper authentication split was not audited**; it is filed as a separate drain item.
 +
 +**Review.** One **model: 'sonnet'** figures-vs-script pass re-ran ''platforms_report.mjs'' and checked the authentication paragraph and the adjacent route/platform figures. No mismatch was found. No citation pass was needed: no citekey or quoted claim changed.
 +
 +**Rejected finding.** The 89.1% consentAction result was not used as evidence that authentication labels are wrong; it only motivates auditing the 145-paper authentication split next.
 +
 +===== Audit of the 145 crawlConfig.authentication='none' platform papers, 2026-09-22 =====
 +
 +**What was wrong.** [[design:platforms]] published the ''%%crawlConfig.authentication%%'' split for the 260 platform-subject papers with a crawl configuration — **145 (55.8%)** ''%%none%%'', 22 ''%%account-registration%%'', 4 ''%%manual-login%%'', 0 ''%%automated-login%%'' / ''%%sso%%'' — with the caveat that these were schema labels and that the 145 had not been audited, and filed the audit as the next item. This is that audit. The shape was right and the schema counts are unchanged; what was missing is that ''%%none%%'' is overwhelmingly an //inference from silence//, and that a measurable minority of the labels are contradicted by the paper.
 +
 +**Why the evidence quote cannot decide this.** ''%%crawlConfig%%'' is an object carrying **one** ''%%evidence.quote%%'' for statefulness, browsers, headless, consent action, interaction depth, authentication, languages and repeat visits together ([[programming:registration]], [[privacy:consent]]). So the audit did not read the quote as evidence; it read the **paper text** — ''%%paper.cols.txt%%'' and the raw ''%%paper.txt%%'' pypdf render, whitespace collapsed and de-hyphenated, because either rendering alone splices sentences. The sibling ''%%consentAction%%'' result (279 of 313 ''%%no-interaction%%'' labels unsupported, 89.1%) was **not** used as evidence that authentication labels are wrong; see the rejected finding below.
 +
 +**Command and real output.**
 +
 +<code>
 +$ node scripts/platforms_report.mjs          # unchanged, the schema split the page publishes
 +## The login wall: crawlConfig.authentication
 +  platform-subject papers with a crawlConfig object (260)
 +    none                145  55.8%
 +    not-stated           89  34.2%
 +    account-registration   22  8.5%
 +    manual-login          4  1.5%
 +    -> states a value: 171/260 = 65.8%
 +
 +$ node scripts/auth145_audit.mjs --verdicts
 +## Audit of the 145 platform-subject papers labelled crawlConfig.authentication='none'
 +   (denominator: the 145, not the 260 with a crawlConfig object and not the corpus)
 +  contradicted    6  4.1%
 +  affirmed       12  8.3%
 +  ambiguous       8  5.5%
 +  silent        119  82.1%
 +  total 145
 +</code>
 +
 +**How the candidate set was built, and what its recall rests on.** The 145 were re-derived inside ''%%scripts/auth145_audit.mjs%%'' with the same four signals ''%%platforms_report.mjs%%'' uses, and the script **throws** if the platform-subject-with-crawlConfig population is not 260 or the ''%%none%%'' count is not 145. Five paired probes then ran over the paper text (''%%--ctx%%''), each with a tight and a loose form, and the script **throws** if any tight probe returns more matches than its loose form in the same file — the first version did, on //Cookies That Give You Away//, which is why the guard exists. Stage 3 (''%%--buckets%%'') sorted all 145 into three candidate sets: papers with a first-person register/login sentence (**64**), papers with an explicit no-authentication sentence (**11**), and the residue where neither fires (**70**). **This sentence was wrong when first published, and a figures review caught it.** It said the pool read by hand was stage 3's A and C sets, 75 papers. It was not: the pool is the **union** of stage 2's strong first-person list and stage 3's A and C sets, and the review found the proof — //A View into YouTube View Fraud// carries a verdict below while sitting in stage 3's **B residue**, because stage 2 surfaced it and stage 3 did not. The script now computes that union itself and ''%%--verdicts%%'' **throws** if any adjudicated paper falls outside it (mutation-tested: disabling the stage 2 term makes it throw on exactly that paper). Its real output: **97 of 145** papers in the pool, **26** given a verdict, **71** read and left silent, and **48** that never surfaced a sentence to read. So the 119 **silent** are 71 papers read and found to say nothing about their own authentication posture plus 48 that no probe reached — not, as first published, 119 papers none of which were read. Recall therefore rests on the C probe being loose (it fires on any ''%%our ... account%%''), which is why it returned 64 candidates of which most were topical noise about the papers' //subjects// rather than their methods.
 +
 +**What changed.** [[design:platforms]]'s logged-out-surface bullet now carries the audit result in place of "We did not audit these 145". The schema counts (145 / 22 / 4 / 0, and the 55.8%) are **unchanged** — this is a precision audit published alongside the labels, in the same style as the Meta / Twitter-X / TikTok / Amazon subject-fold audits higher up this page. No number was hand-edited and no label was rewritten in ''%%extractions.jsonl%%''.
 +
 +**Two of the six read as contradicting their own object.** //A Large-scale Investigation into Geodifferences in Mobile Apps// (USENIX Security 2022) has a ''%%crawlConfig%%'' quote about the Selenium privacy-policy scraper while the study's actual download apparatus is ten Google Pixel 2 phones, one Google account per country, with an apk scraper that "requires the device's ID and Google account credentials". //Experimental Analyses of the Physical Surveillance Risks in Client-Side Content Scanning// (NDSS 2024) is scored ''%%none%%'' while its own ''%%crawlConfig.evidence.quote%%'' reads "we scrape user photographs from Instagram using an alias account" — the label contradicts the quote it was stored with.
 +
 +==== scripts/auth145_audit.mjs ====
 +
 +The committed file, verbatim.
 +
 +<file javascript auth145_audit.mjs>
 +// Audit of the 145 platform-subject papers labelled crawlConfig.authentication='none'.
 +//
 +// Filed by design:platforms itself: "We did not audit these 145. Treat the
 +// authentication shape as real and the precise share as unverified."
 +//
 +// The label cannot be checked against its own evidence quote, because
 +// crawlConfig carries ONE evidence quote for the whole object (see
 +// programming:registration). So this script does not read the quote as
 +// evidence. It reads the PAPER TEXT.
 +//
 +//   node scripts/auth145_audit.mjs            # probe + candidate list
 +//   node scripts/auth145_audit.mjs --ctx <n>  # printed context window (default 260)
 +//   node scripts/auth145_audit.mjs --control  # random sample of non-firing papers
 +//
 +// The paper population is re-derived with the same four signals
 +// platforms_report.mjs uses; the counts are asserted against that script's
 +// published output (260 with a crawlConfig object, 145 labelled 'none') and
 +// this script throws if either moves.
 +
 +import fs from 'node:fs';
 +import path from 'node:path';
 +import { loadExtractions, isSentinel, dataRoot } from './lib.mjs';
 +import { tagsOf, roleOf, familiesOf, ANY_PLATFORM } from './plat_fold.mjs';
 +
 +const argv = process.argv.slice(2);
 +const has = (f) => argv.includes(f);
 +const val = (f, d) => (argv.indexOf(f) >= 0 ? argv[argv.indexOf(f) + 1] : d);
 +const CTX = Number(val('--ctx', 260));
 +const pct = (n, d) => `${((100 * n) / d).toFixed(1)}%`;
 +
 +const rows = loadExtractions();
 +const key = (r) => `${r.venue}/${r.year}/${r.slug}`;
 +const SIGNALS = {
 +  title: (r) => [r.title],
 +  sourceList: (r) => (r.population || []).map((p) => p.sourceList),
 +  phenomenon: (r) => (r.detection || []).map((d) => d.phenomenon),
 +  tool: (r) =>
 +    (r.tools || [])
 +      .filter((t) => t.usedOrMentioned === 'used' || t.usedOrMentioned === 'produced')
 +      .map((t) => t.name),
 +};
 +
 +const perPaper = new Map();
 +for (const r of rows) {
 +  const k = key(r);
 +  const fams = new Map();
 +  for (const [sig, get] of Object.entries(SIGNALS)) {
 +    for (const raw of get(r)) {
 +      if (!raw || isSentinel(raw)) continue;
 +      const s = String(raw);
 +      if (sig === 'sourceList' && (r.participants || []).length > 0) continue;
 +      for (const { family, role } of tagsOf(s)) {
 +        if (role !== 'subject') continue;
 +        if (!fams.has(family)) fams.set(family, new Set());
 +        fams.get(family).add(sig);
 +      }
 +    }
 +  }
 +  if (fams.size) perPaper.set(k, fams);
 +}
 +const byKey = new Map(rows.map((r) => [key(r), r]));
 +const anySubject = [...perPaper.keys()].map((k) => byKey.get(k));
 +const platCrawled = anySubject.filter((r) => r.crawlConfig !== null);
 +const none = platCrawled.filter((r) => r.crawlConfig.authentication === 'none');
 +
 +// Guard: this script's population must be platforms_report.mjs's population.
 +if (platCrawled.length !== 260)
 +  throw new Error(`population drift: platform-subject papers with a crawlConfig object = ${platCrawled.length}, platforms_report.mjs publishes 260`);
 +if (none.length !== 145)
 +  throw new Error(`population drift: authentication='none' = ${none.length}, platforms_report.mjs publishes 145`);
 +
 +// ------------------------------------------------------------------ full text
 +// paper.cols.txt de-columns but leaves ~a quarter of pages spliced; paper.txt
 +// is the raw pypdf render. Read BOTH and probe the union, whitespace collapsed,
 +// because a PDF line break inside a phrase silently undercounts.
 +const collapse = (s) => s.replace(/­/g, '').replace(/-\s*\n\s*/g, '').replace(/\s+/g, ' ');
 +function textOf(r) {
 +  const dir = path.join(dataRoot(), 'fulltext', String(r.year), r.venue, r.slug);
 +  const out = [];
 +  for (const f of ['paper.cols.txt', 'paper.txt', 'paper.norm.txt']) {
 +    const p = path.join(dir, f);
 +    if (!fs.existsSync(p)) continue;
 +    const raw = fs.readFileSync(p, 'latin1');
 +    if (raw.length === 0) continue;
 +    out.push({ file: f, text: collapse(raw) });
 +  }
 +  return out;
 +}
 +
 +// ------------------------------------------------------------------- probes
 +// Two widths per concern. The tight probe must return a subset of the loose
 +// one; the script asserts tight <= loose per paper and prints both.
 +const PROBES = [
 +  ['created-account', /\b(creat(?:e|ed|ing)|register(?:ed|ing)?|set up|made)\s+(?:\w+\s+){0,4}(accounts?|profiles?|personas?|sock[- ]?puppets?|bot accounts?)\b/i,
 +                      /(account|profile|persona|sock)/i],
 +  ['logged-in',       /\b(logged[- ]?in|log(?:ged)?\s+into|sign(?:ed|ing)?[- ]in(?:to)?|authenticated (?:session|crawl|user|request)|login(?:s|ed)? (?:to|into)|after (?:the )?login)\b/i,
 +                      /(log|sign|authenticat)/i],
 +  ['credentials',     /\b(credentials?|username and password|password(?:ed)? account|session cookies?|auth(?:entication)? tokens?|access tokens?|OAuth)\b/i,
 +                      /(credential|password|cookie|token|oauth|username)/i],
 +  ['login-wall',      /\b(login[- ]wall|behind a login|requires? (?:a )?(?:login|an account|sign[- ]?up)|logged[- ]?out (?:view|surface|user))\b/i,
 +                      /(log|account|sign)/i],
 +  ['sso',             /\b(single sign[- ]on|SSO|sign in with (?:Google|Facebook|Apple)|social login|identity provider)\b/i,
 +                      /(sso|sign|single|identity provider)/i],
 +];
 +
 +const results = [];
 +for (const r of none) {
 +  const texts = textOf(r);
 +  const hits = {};
 +  let anyText = texts.length > 0 && texts.some((t) => t.text.length > 2000);
 +  for (const [name, tight, loose] of PROBES) {
 +    let nT = 0, nL = 0, sample = null;
 +    for (const { text } of texts) {
 +      const mT = text.match(new RegExp(tight.source, 'gi')) || [];
 +      const mL = text.match(new RegExp(loose.source, 'gi')) || [];
 +      if (mT.length > mL.length)
 +        throw new Error(`probe ${name} on ${key(r)} in ${'' + text.length}: tight ${mT.length} > loose ${mL.length} (loose must contain tight)`);
 +      nT = Math.max(nT, mT.length);
 +      nL = Math.max(nL, mL.length);
 +      if (!sample && mT.length) {
 +        const i = text.search(new RegExp(tight.source, 'i'));
 +        sample = text.slice(Math.max(0, i - CTX / 2), i + CTX).trim();
 +      }
 +    }
 +    hits[name] = { tight: nT, loose: nL, sample };
 +  }
 +  results.push({ r, hits, anyText, nfiles: texts.length });
 +}
 +
 +const fired = results.filter((x) => Object.values(x.hits).some((h) => h.tight > 0));
 +const quiet = results.filter((x) => !Object.values(x.hits).some((h) => h.tight > 0));
 +const notext = results.filter((x) => !x.anyText);
 +
 +console.log(`## Population`);
 +console.log(`  platform-subject papers with a crawlConfig object : ${platCrawled.length}`);
 +console.log(`  of those, crawlConfig.authentication = 'none'     : ${none.length}`);
 +console.log(`  full text readable for                            : ${results.length - notext.length}/${none.length}`);
 +if (notext.length) for (const x of notext) console.log(`    NO TEXT  ${key(x.r)}`);
 +
 +console.log(`\n## Probe totals over the 145 (papers, not matches)`);
 +for (const [name] of PROBES) {
 +  const t = results.filter((x) => x.hits[name].tight > 0).length;
 +  const l = results.filter((x) => x.hits[name].loose > 0).length;
 +  console.log(`  ${name.padEnd(16)} tight ${String(t).padStart(3)}   loose ${String(l).padStart(3)}`);
 +}
 +console.log(`\n  any tight probe fires : ${fired.length}/${none.length}`);
 +console.log(`  no tight probe fires  : ${quiet.length}/${none.length}`);
 +
 +if (has('--control')) {
 +  // Deterministic sample of the quiet papers, for the false-negative side.
 +  const s = quiet.slice().sort((a, b) => key(a.r).localeCompare(key(b.r)));
 +  const pick = [];
 +  for (let i = 0; i < s.length; i += Math.max(1, Math.floor(s.length / 20))) pick.push(s[i]);
 +  console.log(`\n## Control: ${pick.length} papers where no tight probe fired`);
 +  for (const x of pick) console.log(`  ${key(x.r)}  ${x.r.title}`);
 +  process.exit(0);
 +}
 +
 +console.log(`\n## Candidates (tight probe fired) — ${fired.length}`);
 +for (const x of fired.sort((a, b) => key(a.r).localeCompare(key(b.r)))) {
 +  const names = Object.entries(x.hits).filter(([, h]) => h.tight > 0).map(([n, h]) => `${n}:${h.tight}`);
 +  console.log(`\n--- ${key(x.r)}`);
 +  console.log(`    title: ${x.r.title}`);
 +  console.log(`    probes: ${names.join(' ')}`);
 +  console.log(`    crawlConfig.evidenceQuote: ${JSON.stringify(x.r.crawlConfig.evidenceQuote || null)}`);
 +  for (const [n, h] of Object.entries(x.hits)) if (h.sample) console.log(`    [${n}] ${h.sample}`);
 +}
 +
 +// ---------------------------------------------------------- stage 2: --method
 +// A tight probe fires on any mention of accounts or logins, including the
 +// paper's SUBJECT (spam accounts, credential theft). What decides the
 +// authentication label is what the AUTHORS did. Stage 2 keeps only sentences
 +// that carry a first-person / own-apparatus subject AND an auth term, and
 +// prints them in full for hand reading. Run over all 145, not just the 82,
 +// because sentence co-occurrence is a different probe, not a narrowing of
 +// stage 1.
 +if (has('--method')) {
 +  const FIRST = /\b(we|our|us|the authors?|this (?:paper|study|work)|i )\b/i;
 +  const APPARATUS = /\b(crawler?s?|scraper?s?|bots?|browser|instrumentation|framework|pipeline|measurement|collection|harvest)\b/i;
 +  const AUTH = /\b(account|log ?in|logged|sign(?:ed|ing)? ?in|authenticat|credential|password|session cookie|API key|access token|OAuth|sock ?puppet|persona|profile we|registered)\b/i;
 +  let printed = 0;
 +  for (const x of results.sort((a, b) => key(a.r).localeCompare(key(b.r)))) {
 +    const seen = new Set();
 +    const keep = [];
 +    for (const { text } of textOf(x.r)) {
 +      for (const sent of text.split(/(?<=[.!?])\s+(?=[A-Z(])/)) {
 +        if (sent.length < 30 || sent.length > 700) continue;
 +        if (!AUTH.test(sent)) continue;
 +        if (!FIRST.test(sent) && !APPARATUS.test(sent)) continue;
 +        const norm = sent.toLowerCase().replace(/[^a-z0-9]/g, '').slice(0, 90);
 +        if (seen.has(norm)) continue;
 +        seen.add(norm);
 +        keep.push(sent.trim());
 +      }
 +    }
 +    console.log(`\n=== ${key(x.r)}`);
 +    console.log(`    ${x.r.title}`);
 +    console.log(`    quote: ${JSON.stringify(x.r.crawlConfig.evidenceQuote || null)}`);
 +    if (!keep.length) console.log('    (no first-person auth sentence)');
 +    for (const s of keep.slice(0, 14)) console.log(`      * ${s}`);
 +    printed++;
 +  }
 +  console.log(`\n-- ${printed} papers printed`);
 +}
 +
 +// ------------------------------------------------------- stage 3: --buckets
 +// `none` and `not-stated` are BOTH available in the schema, so `none` is an
 +// affirmative claim: the crawler did not log in or register. Stage 3 sorts the
 +// 145 by what the paper actually says about its own crawl:
 +//   A  affirmed    — the paper states it did not authenticate
 +//   C  contradicted — the paper describes registering or logging in to collect data
 +//   B  silent      — neither; `none` was inferred from the paper saying nothing
 +// A and C are candidate sets, hand-read one by one; B is the residue and is
 +// printed in full so it can be counted, not assumed.
 +if (has('--buckets')) {
 +  // "does not take into account" / "accounts for 75% of the market" swamp any
 +  // probe that treats the bare word `account` as an authentication term: all 20
 +  // hits of the first version of AFFIRM were that. AFFIRM therefore requires a
 +  // login/authentication term, and excludes the two idioms explicitly.
 +  const IDIOM = /(take[ns]? |taking |taken )?into account|accounts? for\b|on account of/i;
 +  const AFFIRM = [
 +    /\b(without|never|not|no|did not|does not|neither|refrain\w* from)\b[^.]{0,90}\b(logg?(?:ed|ing)? ?in(?:to)?|log ?in|sign(?:ed|ing)? ?in|authenticat\w*|login wall|creat\w+ an account|registering|registration)\b/i,
 +    /\b(logged[- ]?out|unauthenticated|without (?:an? )?(?:account|login)|publicly available without)\b/i,
 +    /\bdo(?:es)? not (?:require|need)\b[^.]{0,40}\b(login|logging|account|authenticat\w*)\b/i,
 +  ];
 +  const CONTRA = [
 +    /\b(we|our)\b[^.]{0,120}?\b(creat\w+|register\w+|set ?up|sign(?:ed)? up|purchas\w+)\b[^.]{0,60}?\b(account|profile|persona|sock ?puppet)s?\b/i,
 +    /\b(we|our)\b[^.]{0,120}?\b(logg?ed (?:in|into)|log(?:ged)? in to|signed in(?:to)?|authenticat\w+)\b/i,
 +    /\bour\b[^.]{0,40}\b(accounts?|credentials?|logins?)\b/i,
 +    /\b(account|login) credentials?\b[^.]{0,60}\b(crawler|scraper|tool|script|download)/i,
 +  ];
 +  const bucket = { A: [], B: [], C: [] };
 +  for (const x of results) {
 +    const sents = [];
 +    for (const { text } of textOf(x.r))
 +      for (const s of text.split(/(?<=[.!?])\s+(?=[A-Z(])/)) if (s.length >= 30 && s.length <= 700) sents.push(s);
 +    const a = sents.filter((s) => !IDIOM.test(s) && AFFIRM.some((p) => p.test(s)));
 +    const c = sents.filter((s) => CONTRA.some((p) => p.test(s)));
 +    x.aHits = a; x.cHits = c;
 +    if (c.length) bucket.C.push(x);
 +    else if (a.length) bucket.A.push(x);
 +    else bucket.B.push(x);
 +  }
 +  console.log(`\n## Stage 3 candidate buckets over the 145 (probe output, NOT the audit result)`);
 +  console.log(`  C  paper has a first-person register/login sentence : ${bucket.C.length}`);
 +  console.log(`  A  paper has an explicit no-authentication sentence : ${bucket.A.length}`);
 +  console.log(`  B  residue, neither probe fires                     : ${bucket.B.length}`);
 +  console.log(`  total ${bucket.A.length + bucket.B.length + bucket.C.length} (must be 145)`);
 +  if (bucket.A.length + bucket.B.length + bucket.C.length !== 145) throw new Error('buckets do not sum to 145');
 +  for (const [name, set] of Object.entries(bucket)) {
 +    console.log(`\n######## BUCKET ${name} (${set.length})`);
 +    for (const x of set.sort((p, q) => key(p.r).localeCompare(key(q.r)))) {
 +      console.log(`\n=== ${key(x.r)}\n    ${x.r.title}`);
 +      console.log(`    quote: ${JSON.stringify(x.r.crawlConfig.evidence ? x.r.crawlConfig.evidence.quote : null)}`);
 +      const seen = new Set();
 +      for (const s of [...x.cHits.map((v) => ['C', v]), ...x.aHits.map((v) => ['A', v])].slice(0, 40)) {
 +        const n = s[1].toLowerCase().replace(/[^a-z0-9]/g, '').slice(0, 80);
 +        if (seen.has(n)) continue; seen.add(n);
 +        if (seen.size > 8) break;
 +        console.log(`      ${s[0]}> ${s[1]}`);
 +      }
 +    }
 +  }
 +}
 +
 +// ------------------------------------------------------- stage 4: --verdicts
 +// The audit result. Every verdict below was reached by READING the paper, not
 +// by reading crawlConfig.evidence.quote — that quote covers the whole
 +// crawlConfig object and cannot adjudicate this field (programming:registration).
 +//
 +// The candidate set that was read is stage 3's A and C buckets (the papers with
 +// any first-person sentence about accounts, logins or authentication, either
 +// direction). Papers in neither bucket are SILENT by construction: they contain
 +// no first-person statement about their own authentication posture, so `none`
 +// there is an inference from the paper saying nothing, not a reading of it.
 +//
 +//   affirmed     the paper states its collection was unauthenticated
 +//   contradicted the paper describes registering or using accounts FOR the
 +//                collection the crawlConfig object describes
 +//   ambiguous    the study used accounts, but the crawl the evidence quote
 +//                describes plausibly did not; a reasonable person could score
 +//                this either way
 +//   silent       no statement either way (the residue; not individually read)
 +const VERDICTS = {
 +  // ---- contradicted (6)
 +  'NDSS/2012/warningbird-detecting-suspicious-urls-in-twitter-stream':
 +    ['contradicted', 'Collection is the Twitter Streaming API under the authors’ own authenticated accounts: "To collect tweets, we use Twitter Streaming APIs. Our accounts have the Spritzer access role".'],
 +  'WWW/2013/i-know-the-shortened-urls-you-clicked-on-twitter-inference-attack-using-public-c':
 +    ['contradicted', '"To perform the attack, we create monitoring accounts that monitor messages from all followings of a target user" — and "We created a Twitter account and followed 29 Twitter users". The accounts are the collection apparatus.'],
 +  'USENIX/2022/a-large-scale-investigation-into-geodifferences-in-mobile-apps':
 +    ['contradicted', '"We set up these phones with one Google account per country" and "The apk scraper requires the device’s ID and Google account credentials". The crawlConfig quote describes only the Selenium privacy-policy sub-crawl.'],
 +  'USENIX/2022/exploring-the-unchartered-space-of-container-registry-typosquatting':
 +    ['contradicted', '"We register all of our accounts legally through Docker Hub and Google Container Registry"; pull counts are read from the authors’ own Docker Hub and Google Cloud Console, and 20,000 AWS accounts were spawned from a registered pilot account.'],
 +  'NDSS/2024/experimental-analyses-of-the-physical-surveillance-risks-in-client-side-content-scanning':
 +    ['contradicted', 'The crawlConfig evidence quote ITSELF says "we scrape user photographs from Instagram using an alias account". The label contradicts its own quote.'],
 +  'USENIX/2025/no-way-to-sign-out-unpacking-non-compliance-with-google-plays-app-account-deleti':
 +    ['contradicted', '"We registered around 300 accounts in total throughout the study. Each app was assigned two registered accounts"; "we manually registered, deleted, relogged in, and re-registered accounts".'],
 +  // ---- affirmed (12)
 +  'IMC/2010/detecting-and-characterizing-social-spam-campaigns':
 +    ['affirmed', '"Because access to regional networks is unauthenticated, we chose 8 regional networks … as targets for data collection."'],
 +  'WWW/2012/serf-and-turf-crowdturfing-for-fun-and-profit':
 +    ['affirmed', '"All information on both sites is publicly available, and neither site employs security measures to prevent crawling." The accounts the authors created were for a separate end-to-end campaign experiment, not the crawl.'],
 +  'USENIX/2014/towards-detecting-anomalous-user-behavior-in-online-social-networks':
 +    ['affirmed', '"We did not seek or receive any account credentials of any Facebook user."'],
 +  'IEEE-SP/2018/the-spyware-used-in-intimate-partner-violence':
 +    ['affirmed', '"We did not set up any user profile, and performed each search from a new browser session." Accounts were created only during manual app analysis, not for the search crawl.'],
 +  'WWW/2019/measuring-political-personalization-of-google-news-search':
 +    ['affirmed', '"we do not create or log-in to a Firefox account during training"; "We also do not create or log-in to a Google account".'],
 +  'USENIX/2020/the-tools-and-tactics-used-in-intimate-partner-surveillance-an-analysis-of-onlin':
 +    ['affirmed', '"Our data is from publicly available fora accessible on the Internet without authentication."'],
 +  'USENIX/2022/behind-the-tube-exploitative-monetization-of-content-on-youtube':
 +    ['affirmed', '"We considered only sites publicly accessible … and did not require registration or prior site activity to view information."'],
 +  'IEEE-SP/2024/no-easy-way-out-the-effectiveness-of-deplatforming-an-extremist-forum-to-suppres':
 +    ['affirmed', '"we refrain from scraping private and protected posts behind the login wall due to safety and legality concerns."'],
 +  'USENIX/2025/lost-in-the-mists-of-time-expirations-in-dns-footprints-of-mobile-apps':
 +    ['affirmed', '"We did not attempt to bypass any login or splash screens of apps."'],
 +  'PETS/2026/banned-books-analysis-of-censorship-on-amazon-com':
 +    ['affirmed', '"at no point in our methodology do we sign into any Amazon account"; "all data we collected was available publicly without being logged in".'],
 +  'PETS/2026/the-tcf-doesn-t-really-a-a-id-automatic-privacy-analysis-and-legal-compliance-of':
 +    ['affirmed', '"Our Selenium instance was not logged in our Google account while crawling" — which is also the crawlConfig evidence quote.'],
 +  'USENIX/2026/bridges-to-self-silent-web-to-app-tracking-on-mobile-via-localhost':
 +    ['affirmed', '"We did not use authenticated user accounts and limited interactions to what was necessary to observe default script behavior."'],
 +  // ---- ambiguous (8)
 +  'CCS/2010/spam-the-underground-on-140-characters-or-less':
 +    ['ambiguous', 'The URL crawler in the evidence quote is unauthenticated, but the Twitter monitoring used a "fake account to act as a spam trap".'],
 +  'CCS/2022/ready-raider-one-exploring-the-misuse-of-cloud-gaming-services':
 +    ['ambiguous', '"We registered all of our accounts legally through cloud gaming services" — but the crawlConfig quote is the Steam game-database crawler, which plausibly needed none.'],
 +  'CCS/2023/dont-leak-your-keys-understanding-measuring-and-exploiting-the-appsecret-leaks-i':
 +    ['ambiguous', '"we first registered two accounts, a victim account and an attack account" — for a single proof-of-concept attack, not for the keyword-seeded mini-program crawl in the quote.'],
 +  'USENIX/2018/o-single-sign-off-where-art-thou-an-empirical-analysis-of-single-sign-on-account':
 +    ['ambiguous', 'The 1M-site SSO-prevalence crawl only detects login/signup pages, but the RP takeover study "create[s] a new account using SSO" for each website.'],
 +  'USENIX/2022/ovrseen-auditing-network-traffic-and-privacy-policies-in-oculus-vr':
 +    ['ambiguous', 'The webstore crawl in the quote is unauthenticated; the network-traffic collection that is the paper’s main measurement used "our test OVR account".'],
 +  'WWW/2022/a-view-into-youtube-view-fraud':
 +    ['ambiguous', '"we generated 20 API keys (the maximum number of keys allowed by Google per account)" — an API credential tied to a Google account, which the schema’s "logged in or registered accounts" does not clearly cover.'],
 +  'PETS/2024/privacy-policies-on-the-fediverse-a-case-study-of-mastodon-instances':
 +    ['ambiguous', 'The 351-policy Selenium download was unauthenticated; one account was created afterwards to check a single instance’s cookie claim.'],
 +  'USENIX/2025/darkgram-a-large-scale-analysis-of-cybercriminal-activity-channels-on-telegram':
 +    ['ambiguous', '"we used the official Telegram API to collect 64,801 posts" — Telegram API access is bound to an account, but the paper never says so.'],
 +};
 +
 +// The candidate pool that was actually hand-read is the UNION of two probes,
 +// not stage 3 alone. Stage 2 (--method) prints, for all 145, the sentences that
 +// carry a first-person/own-apparatus subject AND an auth term; the "strong"
 +// subset of that list is what was read. Stage 3 (--buckets) adds its A and C
 +// sets. A figures review caught that WWW/2022/a-view-into-youtube-view-fraud
 +// carries a verdict while sitting in stage 3's B residue — it was surfaced by
 +// stage 2, not stage 3. The pool is therefore computed here, and --verdicts
 +// THROWS if any adjudicated paper is outside it, so the script can no longer
 +// claim a candidate-discovery process it does not reproduce.
 +const STRONG = [
 +  /\b(we|our)\b[^.]{0,140}?\b(creat\w*|register\w*|set up|signed up|sign up|made|use[d]?|operat\w*|control\w*|maintain\w*)\b[^.]{0,60}?\b(account|profile|persona|sock ?puppet|bot account)s?\b/i,
 +  /\b(we|our)\b[^.]{0,140}?\b(logged[- ]?in|log ?in|logged into|signed in|authenticat\w*|credential\w*|session cookie|access token|api key|oauth|bearer token)\b/i,
 +  /\b(logged[- ]?in|authenticated)\b[^.]{0,60}\b(crawl\w*|session|browser|account|user|state)\b/i,
 +  /\b(without|not)\b[^.]{0,40}\b(log ?in|logging in|account|authenticat\w*)\b/i,
 +];
 +function stage2Strong(x) {
 +  for (const { text } of textOf(x.r))
 +    for (const sent of text.split(/(?<=[.!?])\s+(?=[A-Z(])/)) {
 +      if (sent.length < 30 || sent.length > 700) continue;
 +      if (STRONG.some((p) => p.test(sent))) return true;
 +    }
 +  return false;
 +}
 +
 +if (has('--verdicts')) {
 +  const keys145 = new Set(none.map(key));
 +  for (const k of Object.keys(VERDICTS)) if (!keys145.has(k)) throw new Error(`verdict for a paper that is not in the 145: ${k}`);
 +  // Recompute the candidate pool and assert every verdict came out of it.
 +  const AF = [
 +    /\b(without|never|not|did not|does not|neither|refrain\w* from)\b[^.]{0,90}\b(logg?(?:ed|ing)? ?in(?:to)?|log ?in|sign(?:ed|ing)? ?in|authenticat\w*|login wall|creat\w+ an account|registering|registration)\b/i,
 +    /\b(logged[- ]?out|unauthenticated|without (?:an? )?(?:account|login)|publicly available without)\b/i,
 +  ];
 +  const CO = [
 +    /\b(we|our)\b[^.]{0,120}?\b(creat\w+|register\w+|set ?up|sign(?:ed)? up|purchas\w+)\b[^.]{0,60}?\b(account|profile|persona|sock ?puppet)s?\b/i,
 +    /\b(we|our)\b[^.]{0,120}?\b(logg?ed (?:in|into)|log(?:ged)? in to|signed in(?:to)?|authenticat\w+)\b/i,
 +    /\bour\b[^.]{0,40}\b(accounts?|credentials?|logins?)\b/i,
 +  ];
 +  const IDIOM2 = /(take[ns]? |taking |taken )?into account|accounts? for\b|on account of/i;
 +  const pool = new Set();
 +  for (const x of results) {
 +    let hit = stage2Strong(x);
 +    if (!hit)
 +      for (const { text } of textOf(x.r)) {
 +        for (const sent of text.split(/(?<=[.!?])\s+(?=[A-Z(])/)) {
 +          if (sent.length < 30 || sent.length > 700) continue;
 +          if (CO.some((p) => p.test(sent)) || (!IDIOM2.test(sent) && AF.some((p) => p.test(sent)))) { hit = true; break; }
 +        }
 +        if (hit) break;
 +      }
 +    if (hit) pool.add(key(x.r));
 +  }
 +  for (const k of Object.keys(VERDICTS))
 +    if (!pool.has(k)) throw new Error(`adjudicated paper outside the candidate pool the script computes: ${k}`);
 +  console.log(`\n## Candidate pool actually read (stage 2 strong list UNION stage 3 buckets A and C)`);
 +  console.log(`  papers in the pool          : ${pool.size}/145`);
 +  console.log(`  of those, given a verdict   : ${Object.keys(VERDICTS).length}`);
 +  console.log(`  of those, read and left silent: ${pool.size - Object.keys(VERDICTS).length}`);
 +  console.log(`  never surfaced a sentence   : ${145 - pool.size}`);
 +  const counts = { contradicted: 0, affirmed: 0, ambiguous: 0, silent: 0 };
 +  const rowsOut = [];
 +  for (const r of none) {
 +    const v = VERDICTS[key(r)];
 +    const verdict = v ? v[0] : 'silent';
 +    counts[verdict]++;
 +    if (v) rowsOut.push([verdict, key(r), r.title, v[1]]);
 +  }
 +  const total = Object.values(counts).reduce((a, b) => a + b, 0);
 +  if (total !== 145) throw new Error(`verdicts sum to ${total}, not 145`);
 +  console.log(`\n## Audit of the 145 platform-subject papers labelled crawlConfig.authentication='none'`);
 +  console.log(`   (denominator: the 145, not the 260 with a crawlConfig object and not the corpus)`);
 +  for (const k of ['contradicted', 'affirmed', 'ambiguous', 'silent'])
 +    console.log(`  ${k.padEnd(13)} ${String(counts[k]).padStart(3)}  ${pct(counts[k], 145)}`);
 +  console.log(`  total ${total}`);
 +  console.log(`\n## Per-paper evidence for every non-silent verdict`);
 +  for (const k of ['contradicted', 'affirmed', 'ambiguous'])
 +    for (const [v, kk, t, why] of rowsOut.filter((x) => x[0] === k))
 +      console.log(`\n  [${v}] ${kk}\n    ${t}\n    ${why}`);
 +}
 +
 +</file>
 +
 +==== The 26 adjudicated papers ====
 +
 +Generated from ''%%node scripts/auth145_audit.mjs --verdicts%%''. The other 119 of the 145 are **silent**: no first-person sentence about the study's own authentication posture in either direction.
 +
 +^ Verdict ^ Paper ^ Deciding sentence, from the paper text ^
 +| **contradicted** | //WarningBird: Detecting Suspicious URLs in Twitter Stream// (NDSS 2012) | Collection is the Twitter Streaming API under the authors’ own authenticated accounts: "To collect tweets, we use Twitter Streaming APIs. Our accounts have the Spritzer access role". |
 +| **contradicted** | //I know the shortened URLs you clicked on Twitter: inference attack using public click analytics and Twitter metadata// (WWW 2013) | "To perform the attack, we create monitoring accounts that monitor messages from all followings of a target user" — and "We created a Twitter account and followed 29 Twitter users". The accounts are the collection apparatus. |
 +| **contradicted** | //A Large-scale Investigation into Geodifferences in Mobile Apps// (USENIX 2022) | "We set up these phones with one Google account per country" and "The apk scraper requires the device’s ID and Google account credentials". The crawlConfig quote describes only the Selenium privacy-policy sub-crawl. |
 +| **contradicted** | //Exploring the Unchartered Space of Container Registry Typosquatting// (USENIX 2022) | "We register all of our accounts legally through Docker Hub and Google Container Registry"; pull counts are read from the authors’ own Docker Hub and Google Cloud Console, and 20,000 AWS accounts were spawned from a registered pilot account. |
 +| **contradicted** | //Experimental Analyses of the Physical Surveillance Risks in Client-Side Content Scanning// (NDSS 2024) | The crawlConfig evidence quote ITSELF says "we scrape user photographs from Instagram using an alias account". The label contradicts its own quote. |
 +| **contradicted** | //No Way to Sign Out? Unpacking Non-Compliance with Google Play's App Account Deletion Requirements// (USENIX 2025) | "We registered around 300 accounts in total throughout the study. Each app was assigned two registered accounts"; "we manually registered, deleted, relogged in, and re-registered accounts". |
 +| **affirmed** | //Detecting and characterizing social spam campaigns// (IMC 2010) | "Because access to regional networks is unauthenticated, we chose 8 regional networks … as targets for data collection." |
 +| **affirmed** | //Serf and turf: crowdturfing for fun and profit// (WWW 2012) | "All information on both sites is publicly available, and neither site employs security measures to prevent crawling." The accounts the authors created were for a separate end-to-end campaign experiment, not the crawl. |
 +| **affirmed** | //Towards Detecting Anomalous User Behavior in Online Social Networks// (USENIX 2014) | "We did not seek or receive any account credentials of any Facebook user." |
 +| **affirmed** | //Measuring Political Personalization of Google News Search// (WWW 2019) | "we do not create or log-in to a Firefox account during training"; "We also do not create or log-in to a Google account". |
 +| **affirmed** | //The Tools and Tactics Used in Intimate Partner Surveillance: An Analysis of Online Infidelity Forums// (USENIX 2020) | "Our data is from publicly available fora accessible on the Internet without authentication." |
 +| **affirmed** | //Behind the Tube: Exploitative Monetization of Content on YouTube// (USENIX 2022) | "We considered only sites publicly accessible … and did not require registration or prior site activity to view information." |
 +| **affirmed** | //No Easy Way Out: the Effectiveness of Deplatforming an Extremist Forum to Suppress Hate and Harassment// (IEEE-SP 2024) | "we refrain from scraping private and protected posts behind the login wall due to safety and legality concerns." |
 +| **affirmed** | //Banned Books: Analysis of Censorship on Amazon.com// (PETS 2026) | "at no point in our methodology do we sign into any Amazon account"; "all data we collected was available publicly without being logged in". |
 +| **affirmed** | //The TCF doesn’t really A(A)ID – Automatic Privacy Analysis and Legal Compliance of TCF-based Android Applications// (PETS 2026) | "Our Selenium instance was not logged in our Google account while crawling" — which is also the crawlConfig evidence quote. |
 +| **affirmed** | //The Spyware Used in Intimate Partner Violence// (IEEE-SP 2018) | "We did not set up any user profile, and performed each search from a new browser session." Accounts were created only during manual app analysis, not for the search crawl. |
 +| **affirmed** | //Lost in the Mists of Time: Expirations in DNS Footprints of Mobile Apps// (USENIX 2025) | "We did not attempt to bypass any login or splash screens of apps." |
 +| **affirmed** | //Bridges to Self: Silent Web-to-App Tracking on Mobile via Localhost// (USENIX 2026) | "We did not use authenticated user accounts and limited interactions to what was necessary to observe default script behavior." |
 +| **ambiguous** | //@spam: the underground on 140 characters or less// (CCS 2010) | The URL crawler in the evidence quote is unauthenticated, but the Twitter monitoring used a "fake account to act as a spam trap". |
 +| **ambiguous** | //O Single Sign-Off, Where Art Thou? An Empirical Analysis of Single Sign-On Account Hijacking and Session Management on the Web// (USENIX 2018) | The 1M-site SSO-prevalence crawl only detects login/signup pages, but the RP takeover study "create[s] a new account using SSO" for each website. |
 +| **ambiguous** | //Ready Raider One: Exploring the Misuse of Cloud Gaming Services// (CCS 2022) | "We registered all of our accounts legally through cloud gaming services" — but the crawlConfig quote is the Steam game-database crawler, which plausibly needed none. |
 +| **ambiguous** | //OVRseen: Auditing Network Traffic and Privacy Policies in Oculus VR// (USENIX 2022) | The webstore crawl in the quote is unauthenticated; the network-traffic collection that is the paper’s main measurement used "our test OVR account". |
 +| **ambiguous** | //A View into YouTube View Fraud// (WWW 2022) | "we generated 20 API keys (the maximum number of keys allowed by Google per account)" — an API credential tied to a Google account, which the schema’s "logged in or registered accounts" does not clearly cover. |
 +| **ambiguous** | //Don't Leak Your Keys: Understanding, Measuring, and Exploiting the AppSecret Leaks in Mini-Programs// (CCS 2023) | "we first registered two accounts, a victim account and an attack account" — for a single proof-of-concept attack, not for the keyword-seeded mini-program crawl in the quote. |
 +| **ambiguous** | //Privacy Policies on the Fediverse: A Case Study of Mastodon Instances// (PETS 2024) | The 351-policy Selenium download was unauthenticated; one account was created afterwards to check a single instance’s cookie claim. |
 +| **ambiguous** | //DarkGram: A Large-Scale Analysis of Cybercriminal Activity Channels on Telegram// (USENIX 2025) | "we used the official Telegram API to collect 64,801 posts" — Telegram API access is bound to an account, but the paper never says so. |
 +
 +**Review.** One **model: 'sonnet'** figures-vs-script pass re-ran ''%%platforms_report.mjs%%'', ''%%report_registration.mjs%%'' and ''%%scripts/auth145_audit.mjs%%'' in all four modes, and checked every figure in the rewritten bullet plus the figures either side of it (the 383/208/175 existing-dataset split above it, the 897 denominator, the 34.2% not-stated share below it). Every verdict count, percentage and table row matched. It raised **two findings, both accepted**: (1) the candidate-pool sentence above misdescribed which papers were read, fixed above with a new script-computed guard; (2) [[programming:interaction]] enumerated the 857 web crawls as 553 + 204 + 81 = **838**, silently dropping the 19 papers with no ''%%crawlConfig%%'' object that ''%%report_registration.mjs%%'' prints as a sixth bucket — that page now carries the 19 and the sum reconciles. It also flagged that the ''%%kumar2022_investigation%%'' quote on [[design:platforms]] was ellipsis-joined across two sentences; it has been replaced with the contiguous sentence the audit script itself verifies. One **model: 'sonnet'** citations pass ran, because the bullet added two ''%%{[citekey]}%%'' markers and several verbatim quotes. It verified both citekeys against the papers' author lists, titles, venues and years, and **all 26** table rows verbatim against the paper text, reading the PDF directly where a column-spliced render made a quote look non-contiguous and stripping NUL bytes from WarningBird's text where plain ''%%grep%%'' silently returns nothing. It independently confirmed from ''%%extractions.jsonl%%'' that the Instagram //"alias account"// line really is that record's own ''%%crawlConfig.evidence.quote%%'' under an ''%%authentication: none%%'' label. It found nothing wrong.
 +
 +**Rejected finding.** That the 89.1% ''%%consentAction%%'' unsupported-label rate is evidence the authentication labels are wrong. It is not, and it was not used as such: it is a different field with a different extraction failure mode, and the authentication verdicts above each rest on a sentence from the paper in question. The 89.1% is the reason the audit was prioritised, not an input to any verdict on this page.
 +
 +**Deliberately not concluded.** That the corrected split is 139 / 28 / 4 / 0. Six labels being contradicted does not license re-deriving the published counts by hand: the corrected value for each of the six is itself a judgement (WarningBird's authenticated Streaming API accounts are not obviously ''%%account-registration%%'' rather than ''%%automated-login%%''), and the **8 ambiguous** papers would move the result either way depending on whether "the crawler" means the study's apparatus or the specific crawl the evidence quote names. The schema counts stay; the audit sits beside them.
  
provenance/design/platforms.1787863909.txt.gz · Last modified: by karel.kubicek.claude