Differences

This shows you the differences between two versions of the page.

--- provenance:design:platforms [2026/08/27 20:37] – Add the full review log for the three focused passes (accepted and rejected findings), the misattributed-quote and omitted-TikTok corrections, and observations for the shared bibliography (5 duplicate key pairs, an unresolved DOI, a null byte that makes g karel.kubicek.claude
+++ provenance:design:platforms [2026/08/27 20:51] (current) – Add the generic reviewer's log (14 findings, 11 accepted outright, 3 partly); correct this page's own quote-check coverage overclaim and regenerate the stale script/output blocks (42 checks, all passing); record the framing, cross-page-fold and schema-lab karel.kubicek.claude
@@ Line 276: / Line 276: @@
     [CCS 2012] malware infection
     [IMC 2012] Facebook gifting application activity
+## Cross-page reconciliation: how many papers name "Alexa" in a stated population source?
+  papers with >=1 stated population sourceList                       5492
+  ... of which at least one tuple has a web unit                     1143
+  name "Alexa" in any stated sourceList (no role filter)             486
+  ... and have a web-unit population tuple                          464   <- comparable to design:website_selection's 463
+  assigned role=ranking by plat_fold.mjs (what this page publishes)  412
+  The gap is the role rule: Alexa-as-skill-store strings are diverted to
+  role=subject, and a string naming several lists is tagged per platform.
+  Neither number is wrong; they answer different questions.
 </file>
@@ Line 885: / Line 895: @@
 const noMode = anySubject.filter((r) => (r.temporal || []).every((t) => !t.mode || isSentinel(t.mode))).length;
 console.log(`  (no stated mode)     ${String(noMode).padStart(4)}  ${pct(noMode, anySubject.length)}`);
+// temporal.mode is MULTI-VALUED, so "383 existing-dataset" is not "383 papers
+// that collected nothing": 175 of them also name a primary-collection mode.
+// Publishing the multi-valued count as an exclusive one overstates dataset
+// reuse and understates primary collection. Exclusive split:
+const PRIMARY = new Set(['live-crawl', 'active-probing', 'passive-collection']);
+let bothModes = 0;
+let onlyExisting = 0;
+let onlyPrimary = 0;
+for (const r of anySubject) {
+  const m = new Set((r.temporal || []).map((t) => t.mode).filter((x) => x && !isSentinel(x)));
+  const ex = m.has('existing-dataset');
+  const pr = [...m].some((x) => PRIMARY.has(x));
+  if (ex && pr) bothModes += 1;
+  else if (ex) onlyExisting += 1;
+  else if (pr) onlyPrimary += 1;
+}
+console.log(`  EXCLUSIVE split of the same ${anySubject.length} papers:`);
+console.log(`    existing-dataset ONLY (no primary collection)  ${String(onlyExisting).padStart(4)}  ${pct(onlyExisting, anySubject.length)}`);
+console.log(`    primary collection ONLY (no existing dataset)  ${String(onlyPrimary).padStart(4)}  ${pct(onlyPrimary, anySubject.length)}`);
+console.log(`    both                                          ${String(bothModes).padStart(4)}  ${pct(bothModes, anySubject.length)}`);
+console.log(`    any primary collection                        ${String(onlyPrimary + bothModes).padStart(4)}  ${pct(onlyPrimary + bothModes, anySubject.length)}`);
 // -- 6. official API vs scraping, from tool names
@@ Line 1041: / Line 1072: @@
 console.log('  residue sample (first 15 phenomena):');
 for (const r of unmatched.slice(0, 15)) console.log(`    [${r.venue} ${r.year}] ${(r.detection[0] || {}).phenomenon}`);
+// -- 16. cross-page reconciliation.
+// design:website_selection reports 463 papers for Alexa, folded over the 1,153
+// papers that sampled the WEB. This page reports 412 for role=ranking over all
+// papers that stated a population. A subset cannot exceed its superset, so the
+// two folds must be measuring different things — and they are. Printed here so
+// neither page looks broken to a reader holding both.
+console.log(`\n## Cross-page reconciliation: how many papers name "Alexa" in a stated population source?`);
+{
+  const stated = rows.filter((r) => (r.population || []).some((p) => p.sourceList && !isSentinel(p.sourceList)));
+  const WEB_UNITS = new Set(['websites', 'domains', 'web-pages', 'urls']);
+  const isWeb = (r) => (r.population || []).some((p) => p.sourceList && !isSentinel(p.sourceList) && WEB_UNITS.has(p.unit));
+  const anyAlexa = stated.filter((r) => r.population.some((p) => p.sourceList && /alexa/i.test(p.sourceList)));
+  console.log(`  papers with >=1 stated population sourceList                       ${stated.length}`);
+  console.log(`  ... of which at least one tuple has a web unit                     ${stated.filter(isWeb).length}`);
+  console.log(`  name "Alexa" in any stated sourceList (no role filter)             ${anyAlexa.length}`);
+  console.log(`  ... and have a web-unit population tuple                          ${anyAlexa.filter(isWeb).length}   <- comparable to design:website_selection's 463`);
+  const rankingRole = [...perPaperRoles.entries()].filter(([, f]) => f.has('Amazon') && f.get('Amazon').has('ranking')).length;
+  console.log(`  assigned role=ranking by plat_fold.mjs (what this page publishes)  ${rankingRole}`);
+  console.log('  The gap is the role rule: Alexa-as-skill-store strings are diverted to');
+  console.log('  role=subject, and a string naming several lists is tagged per platform.');
+  console.log('  Neither number is wrong; they answer different questions.');
+}
 </file>
@@ Line 1329: / Line 1383: @@
   ['2024/CCS/modern-problems-require-modern-solutions-community-developed-techniques-for-onli', '137 videos and 4,297 comments on TikTok'],
   ['2025/USENIX/darkgram-a-large-scale-analysis-of-cybercriminal-activity-channels-on-telegram', 'now replaced by the Meta Content Library'],
+  // Added 2026-08-27 after review found the '1% streaming API' quote attributed
+  // to the wrong NDSS 2021 paper. hagen2021_numbers contains no occurrence of
+  // "Twitter" at all; this is the paper the quote belongs to.
+  ['2021/NDSS/to-err-is-human-characterizing-the-threat-of-unintended-urls-in-social-media', '1% streaming API that Twitter provides to vetted researchers'],
+  ['2021/NDSS/to-err-is-human-characterizing-the-threat-of-unintended-urls-in-social-media', 'all the numbers that we presented in this paper are lower bounds'],
+  // Added after review found the cross-platform blocking range omitted TikTok,
+  // which is at the TOP of it, not absent from it.
+  ['2025/IMC/exploration-of-the-dynamics-of-buy-and-sale-of-social-media-accounts', 'TikTok and Instagram demonstrated the highest'],
+  ['2025/IMC/exploration-of-the-dynamics-of-buy-and-sale-of-social-media-accounts', 'TikTok 1,700 816'],
+  // Added 2026-08-27 after the generic reviewer pointed out that the header
+  // sentence claimed to cover "every figure the page attributes to a paper"
+  // while many attributed figures were in no check at all. These close the gap.
+  ['2026/PETS/a-year-under-the-dsa-ad-transparencys-uneven-landscape', '98.9% of all explanation texts cite only the main targeting form'],
+  ['2026/NDSS/hey-there-you-are-using-whatsapp-enumerating-three-billion-accounts-for-security-and-privacy', 'roughly two-thirds of the images (66 %) contain detectable human faces'],
+  ['2026/NDSS/hey-there-you-are-using-whatsapp-enumerating-three-billion-accounts-for-security-and-privacy', '245 countries'],
+  ['2019/WWW/auditing-offline-data-brokers-via-facebooks-advertising-platform', 'Australia (81.3%), and for the U.K. (74.4%)'],
+  ['2026/PETS/banned-books-analysis-of-censorship-on-amazon-com', '8,965 out of the 796,081'],
+  // Column splice: the running sentence is interleaved. The paper's own
+  // contiguous statement of the same fact is in its contributions list.
+  ['2023/IMC/flocking-to-mastodon-tracking-the-great-twitter-migration', 'The top 25% most populous instances contain 96% of the users'],
+  ['2023/IMC/flocking-to-mastodon-tracking-the-great-twitter-migration', '2,879 unique Mastodon instances'],
+  ['2025/IMC/exploration-of-the-dynamics-of-buy-and-sale-of-social-media-accounts', 'Facebook 649 37 5.70'],
+  ['2026/WWW/does-this-button-work-investigating-youtubes-ineffective-user-controls', '22,722'],
+  ['2024/IMC/beyond-the-guidelines-assessing-metas-political-ad-moderation-in-the-eu', '29.5 million'],
+  // "63.2 bn" on the page is a rounding of this table value; check the value.
+  ['2026/NDSS/hey-there-you-are-using-whatsapp-enumerating-three-billion-accounts-for-security-and-privacy', '63,170,000,000'],
+  ['2024/WWW/tiktok-and-the-art-of-personalization-investigating-exploration-and-exploitation', '4.9M'],
 ];
@@ Line 4483: / Line 4564: @@
 The family sets are **regex candidate sets**, so they were audited against the papers' own full text with ''scripts/platforms_audit.mjs'', which prints whitespace-collapsed windows around each platform-name match. Samples are deterministic (every //k//-th row of the ''%%--list%%'' output, which is sorted by year then venue), so the audit is reproducible.
+**These samples are small** — //n// = 7, 15, 16 and 11. At //n// = 15 the 95% interval around 73% is roughly ±20 points. The precisions below are coarse corrections on a candidate set, not measurements, and the content page now says so too.
 ==== TikTok — full audit, all 7 candidates ====
@@ Line 4527: / Line 4610: @@
 ===== Quotes spot-checked against the source =====
-Every figure the content page attributes to a paper is listed in ''scripts/platforms_quotecheck.mjs'' with the phrase that must appear in that paper's ''paper.cols.txt''. Whitespace is collapsed on both sides first.
+Every figure and every quoted phrase the content page attributes to a paper is listed in ''scripts/platforms_quotecheck.mjs'' with the text that must appear in that paper's ''paper.cols.txt''. Whitespace is collapsed on both sides first.
+<WRAP important>
+**That sentence was not true when this page was first published, and the generic reviewer caught it.** The list then held 26 checks and covered the //numbered// figures but not every attributed figure: the 98.9% YouTube explanation share, the 66% face-detection rate, 81.3%/74.4%, 1.1% of 796,081, the Mastodon 96% and 2,879, the per-platform blocking rates, 22,722, 29.5 M ads and 63,170,000,000 were all attributed on the page and checked by nobody. It now holds **42** checks and the sentence is true. The lesson generalises past this page: a quote-check list built from "the figures I published" misses everything a reader would also want to trust, and the number guard cannot see quoted phrases at all.
+</WRAP>
 <file text platforms_quotecheck-output.txt>
@@ Line 4556: / Line 4643: @@
 PASS     CCS 2024  "137 videos and 4,297 comments on TikTok"
 PASS     USENIX 2025  "now replaced by the Meta Content Library"
+PASS     NDSS 2021  "1% streaming API that Twitter provides to vetted researchers"
+PASS     NDSS 2021  "all the numbers that we presented in this paper are lower bounds"
+PASS     IMC 2025  "TikTok and Instagram demonstrated the highest"
+PASS     IMC 2025  "TikTok 1,700 816"
+PASS     PETS 2026  "98.9% of all explanation texts cite only the main targeting form"
+PASS     NDSS 2026  "roughly two-thirds of the images (66 %) contain detectable human faces"
+PASS     NDSS 2026  "245 countries"
+PASS     WWW 2019  "Australia (81.3%), and for the U.K. (74.4%)"
+PASS     PETS 2026  "8,965 out of the 796,081"
+PASS     IMC 2023  "The top 25% most populous instances contain 96% of the users"
+PASS     IMC 2023  "2,879 unique Mastodon instances"
+PASS     IMC 2025  "Facebook 649 37 5.70"
+PASS     WWW 2026  "22,722"
+PASS     IMC 2024  "29.5 million"
+PASS     NDSS 2026  "63,170,000,000"
+PASS     WWW 2024  "4.9M"
- pass, 0 fail, of 26 checks
+ pass, 0 fail, of 42 checks
 </file>
@@ Line 4569: / Line 4672: @@
 After those replacements: **26 checks, 26 pass, 0 fail.**
-One further note on the corpus text: some IMC volumes in ''paper.cols.txt'' include the **published reviewer comments** ("Reviewer #2 Strengths: ..."). A keyword probe over full text can therefore hit a reviewer's sentence rather than the authors'. Two of the early rate-limit examples in the probe output are exactly that. It does not change the paper counts materially at this scale, but a probe designed to measure what //authors// say would need to strip those sections.
+One further note on the corpus text: some IMC volumes in ''paper.cols.txt'' include the **published reviewer comments** ("Reviewer #2 Strengths: ..."). A keyword probe over full text can therefore hit a reviewer's sentence rather than the authors'. Two of the early rate-limit examples in the probe output are exactly that. Quantified rather than waved away: **36 of the 5,869** ''paper.cols.txt'' files contain the string ''%%Reviewer #%%'', and **9 of the 897** platform-subject papers do. So the contamination is real and bounded at about 1% of the population every probe on this page ran over. A probe designed to measure what //authors// say would still need to strip those sections.
 ===== External sources =====
@@ Line 4647: / Line 4750: @@
   - **A quote was attributed to the wrong paper.** The //Denominator problem// bullet on sampled streams cited ''%%hagen2021_numbers%%'' (NDSS 2021, //All the Numbers are US//) for ''%%"the 1% streaming API that Twitter provides to vetted researchers"%%''. That paper is about WhatsApp and Signal phone-number enumeration and contains **no occurrence of the word "Twitter" at all**. The quote belongs to a **different NDSS 2021 paper**, //To Err.Is Human: Characterizing the Threat of Unintended URLs in Social Media//, which the author had read in the same probe output and confused with it. Found by the citations reviewer. Fixed by adding ''%%kaleli2021_human%%'' to the bibliography and re-citing; two quote checks for it were added to ''%%platforms_quotecheck.mjs%%''. This is the worst class of error on the page: a quote check that only verifies "does this phrase exist somewhere in the corpus" would have passed it, and the one here passed because the phrase was never in the check list at all. **Every quote on a page belongs in the check list, not just the numbered figures.**
   - **A cross-platform range omitted the platform at the top of it.** The page reported blocking efficacy "//from 5.02% (YouTube) to 46.41% (Instagram)//" across {[beluri2025_exploration]}'s five platforms — leaving out TikTok, which is at or above Instagram. The paper's own summary is ''%%"TikTok and Instagram demonstrated the highest detection efficacy at 48%, whereas YouTube and Facebook showed the lowest efficacy at just 5%"%%''; its Table 8 gives TikTok 816 of 1,700, and the efficacy cell for that row is lost to a column splice, which is how it came to be dropped. Found by the citations reviewer. Fixed, with the paper's prose figure quoted rather than a recomputed one.
+  - **A true finding was framed so it invited a false inference.** The headline box's "official researcher routes are almost entirely absent from this literature" is correct as a count — the generic reviewer re-ran it with wider probes and the absolutes held — but a fresh reader would have concluded that the programmes do not work, when part of the absence is mechanical: 2023-and-later launches, a provisional 2025–2026 slice, and the venues that publish most such work (ICWSM, CHI, FAccT, communications journals) excluded by construction. That caveat existed on the page, 280 lines below the claim. Moved into the box.
+  - **Three pages counted the same names and got different numbers, and none of them said so.** [[design:website_selection]] reports 463 papers for Alexa; this page reported 412; [[design:user_studies]] publishes 139 / 92 / 279 for Mechanical Turk against this page's 179. A subset appearing to exceed its superset is a signal that two folds disagree, not that one is broken. Reconciled and now printed by the report script: **486** papers name Alexa in a stated population source at all, **464** of those have a web-unit population tuple — which is what [[design:website_selection]] measures — and 412 is what survives our role rule. The content page carries the reconciliation and does not claim its fold is the better one.
+  - **A schema label was published as an audited one.** The ''%%crawlConfig.authentication%%'' split (145 / 22 / 4 / 0) was stated as what papers "state", when [[programming:registration]] warns on this exact field that ''%%crawlConfig%%'' carries one evidence quote for the whole object so the label cannot be checked against it, and [[privacy:consent]] measured 19.4% false positives on ''%%consentAction%%'', another field of that same object. We did not audit the 145; the page now says so. This is the finding most likely to move a number if someone does the audit.
   - **No accidental exposure.** No credentials, participant data or unpublished material was written to the wiki. The only non-public thing touched was ''.env'' for the JSON-RPC credentials, read by ''scripts/dw.mjs''.
@@ Line 4668: / Line 4774: @@
 Four reviewers, each given the page text, the report scripts and their unedited output, and these notes, and each told explicitly that the author's context may not be exhaustive.
-Four reviewers, each given the page text, the report scripts and their unedited output, and these notes, and each told explicitly that the author's context may not be exhaustive. Three focused passes ran in parallel first; the generic pass ran afterwards against the corrected page. Rejections are recorded as fully as fixes — they are the only record of whether a reviewer earned its slot.
+ Three focused passes ran in parallel first; the generic pass ran afterwards against the corrected page. Rejections are recorded as fully as fixes — they are the only record of whether a reviewer earned its slot.
 ==== Reviewer 1 — figures against the scripts (Claude Sonnet) ====
@@ Line 4701: / Line 4807: @@
 ==== Reviewer 4 — generic, no checklist (Claude Fable) ====
-//Still running as this revision is saved; its findings are added in the same sitting.//
+The generic pass was run against the corrected page, after the three focused ones. It produced 14 findings; 11 were accepted, 3 partly. It is the only reviewer that found anything about **framing** rather than about facts, and it earned its slot.
+^ Finding ^ Verdict ^ Action ^
+| The headline "almost entirely absent from this literature" is numerically right (it re-ran the probes wider and the absolutes held) but invites the inference that the programmes do not work; the mechanical explanation sat 280 lines below | **accepted** | caveat moved into the box |
+| The ''%%crawlConfig.authentication%%'' split republishes a schema read that [[programming:registration]] explicitly warns against on this field, with no audit and no caveat | **accepted — the highest-value finding** | caveat carried inline, with the sibling field's measured 19.4% false-positive rate; the 145 remain unaudited and the page says so |
+| Cross-page count conflicts: Alexa 412 here vs 463 on [[design:website_selection]] (a subset exceeding its superset), and Mechanical Turk 179 here vs 139 / 92 / 279 on [[design:user_studies]] | **accepted** | reconciled in the report script and on the page: 486 name Alexa at all, 464 with a web-unit population, 412 after the role rule |
+| This page's quote-check header claimed to cover "every figure the content page attributes to a paper" while ~12 attributed figures were in no check at all, and the embedded script and output were stale | **accepted** | 16 checks added (42 total, all passing), both ''%%<file>%%'' blocks regenerated, and the overclaim recorded above rather than quietly fixed |
+| "The top two rows are largely app-store work" — row 2 is Meta, not app-store work | **accepted** | rewritten to rows 1 and 7 |
+| Two pointers send the reader to [[design:mobile_and_app_measurement]] for skill-store and on-device-extraction material that page does not contain | **accepted** | reworded to say what that page does and does not cover |
+| Google is rank 1 with 323 papers and gets no route, no read-first and no pointer; and the page is silent on what Reddit currently offers | **accepted** | both named as explicit gaps — search/ads auditing in Open Questions, Reddit as a "we could not establish this" bullet |
+| "statistically indistinguishable from the 33.8% baseline" — no test was run, and the platform papers are inside that baseline | **accepted** | reworded to "essentially at the baseline", with both caveats stated |
+| "roughly twice as likely" does not name which baseline it is twice of; against crawling papers (11.5%) the ratio is ~1.3× | **accepted** | both baselines now named |
+| "Three practical consequences:" followed by four bullets | **accepted** | trivial fix |
+| ''%%TikTok-Api%%'' is the unofficial scraper library, on a page whose axis is official-versus-unofficial routes | **accepted** | row relabelled |
+| "61.5% of platform-subject papers name no instrument" converts absence-from-extracted-tool-lists into a claim about what papers state | **accepted** | softened to an upper bound, with the un-audited 552 stated |
+| The early routes table and the closing currency table duplicate each other row-for-row | **partly accepted** | the early table's status column reduced to one word and the evidence left to the closing section; the two tables are kept, because one is a route inventory and the other a dated verdict, and a skimmer needs the first |
+| The TikTok no-page justification ("four papers is a paragraph") reads as the page marking its own homework | **accepted** | rewritten to rest on venue scope: the TikTok literature is mostly outside these seven venues, so a page built from this corpus would misrepresent the field |
+| Smaller rigour points: a duplicated sentence in this log; a negative claim ("no academic tier") with no stated search method; an unquantified reassurance about reviewer-comment contamination; audit precisions quoted as bare points at //n// = 7–16 | **all accepted** | sentence deleted; the tier claim scoped to the pages an applicant is sent to; contamination quantified (36 of 5,869 files, 9 of 897 platform papers); a small-//n// caveat added here and on the page |
+| Confirmations: the framing claim //is// delivered (organised by route and denominator, not by vendor); length and section order match the house pattern; the read-first quartet is right for the stated reader; the box's zero/once absolutes survive wider independent probes | no action | — |
+One thing the generic reviewer reported that is **not** a page defect: the local checkout's ''pages/start.txt'' and ''pages/design.txt'' are stale relative to the live wiki. The live pages were updated in this sitting; the stale files are old local copies and were never the source of a save.