Differences

This shows you the differences between two versions of the page.

--- provenance:design:connected_tv [2026/09/12 17:48] – Review round: add §5b (the map guard was mutation-tested, flipping a tier letter exited 0; fixed with a summing invariant) and §15, the full review log with every finding and whether it was accepted or rejected. Correct the alias-fold size to 27 spellings karel.kubicek.claude
+++ provenance:design:connected_tv [2026/09/13 02:30] (current) – Quote check: eight needles added for the claims round 2 changed (41 quotes, 0 NOTFOUND); re-embed the checker and its output; section 7 counts and the new-needle table. Authored by Claude karel.kubicek.claude
@@ Line 44: / Line 44: @@
 | Q10 | ''population[].unit'' | the 35 | ''iot-devices'' 20, ''other'' 17, ''mobile-apps'' 5 |
 | Q11 | Named instruments, alias-folded, ''used'' only | the 35 | Wireshark 10, tcpdump 9, mitmproxy 4, Frida 3, adb 3 |
-| Q12 | Interception-evidence probes | 13 Tier A / 35 A+B | router/AP 11/30, mitm 10/22, DNS 5/12, HDMI 6/9, remote-control 12/16, **undecryptable reported 1/1** |
+| Q12 | Interception-evidence probes | 13 Tier A / 35 A+B | router/AP 11/30, mitm 10/22, DNS 5/12, HDMI 6/9, remote-control 12/16, **undecryptable reported, narrow probe 1, wide probe 3** (see §6b) |
 | Q13 | ''crawlConfig'' fields stated | 6 papers with a ''crawlConfig'' | ''interactionDepth'' 5, ''consentAction'' 0, ''statefulness'' 0, ''browsers'' 0 |
 | Q14 | ''ethics.reviewOutcome'' stated | 34 with an ''ethics'' object | 14 (41.2%) vs corpus-empirical 1,728 of 4,472 (38.6%) |
 | Q15 | ''artifacts.availability'' stated | 34 with an ''artifacts'' object | 27 (79.4%) vs 2,890 of 4,854 (59.5%) |
-| Q16 | ''temporal.spanStart'' stated | 33 empirical | 27 (79.4%) vs 2,882 of 5,118 (56.3%) |
+| Q16 | ''temporal.spanStart'' stated | 34 empirical | 27 (79.4%) vs 2,882 of 5,118 (56.3%) |
 | Q17 | ''detection[].prevalence'' coverage | 70 Tier A tuples | 68 carry a prevalence (97.1%); corpus-wide 26,316 of 27,241 (96.6%) |
 | Q18 | ''population[].listVersion'' stated | 105 population tuples in the 35 | 42 |
 | Q19 | Vantage location stated | 34 with a ''vantage'' tuple | 18 (52.9%) |
-**Denominators that are easy to get wrong here, spelled out.** ''ethics'' and ''artifacts'' are nullable in the schema — one of the 35 papers has neither object — so Q14–Q16 divide by the papers that carry the object, never by 35 and never by the corpus. ''crawlConfig'' is null for 29 of the 35, so Q13 divides by 6, and its zeros are read on the page rather than reported bare. Nothing on either page divides by 5,859.
+**Denominators that are easy to get wrong here, spelled out.** ''ethics'' and ''artifacts'' are nullable in the schema — one of the 35 papers has neither object — so Q14–Q16 divide by the papers that carry the object, never by 35 and never by the corpus. ''crawlConfig'' is null for 29 of the 35, so Q13 divides by 6, and its zeros are read on the page rather than reported bare. No **TV** figure on either page divides by 5,859. The one place that denominator appears is the corpus-share column of the platform table on the content page, which exists precisely to say what share of the whole corpus each platform value has — that column divides by 5,859 by design, and is labelled as doing so. An earlier version of this sentence said "nothing on either page divides by 5,859", which was wrong.
 ===== 4. The candidate pool, and why it has two gates =====
@@ Line 221: / Line 221: @@
   * **Flipping a tier letter** on a paper that stays in the candidate set — ''B'' to ''OUT'' on //Tracking You from a Thousand Miles Away// — **the script exited 0**, silently recomputed the population as 34 instead of 35 and ''OUT'' as 53 instead of 52, and every percentage on the content page would have moved with it. No assertion fired, because the only check was a slug-set difference.
-**Fixed the same day.** ''report_connected_tv.mjs'' now pins the split to what [[design:connected_tv]] publishes and throws on any change:
+**Fixed the same day**, by pinning the split to what [[design:connected_tv]] publishes:
 <code>
@@ Line 227: / Line 227: @@
 </code>
-Re-running the same mutation now fails loudly, with the real message:
+**And that fix was itself insufficient — found by the next review pass, on 2026-09-13.** A count is not a membership. Mutating two entries at once, **swapping** a genuine Tier B paper out and a genuine ''OUT'' paper in, leaves all four counts identical and exits 0, while the population silently acquires a paper that is not about television at all. The reviewer demonstrated it with //BLERP: BLE Re-Pairing Attacks and Defenses// (B to OUT) against a bilingual privacy-policy paper whose only TV content is an ''ACR'' homonym (OUT to B): platform ''web'' went from 1 to 2, PETS 6 to 7, NDSS 6 to 5, and the ''iot'' share from 74.3% to 71.4% — every one of them a published figure, none of them guarded.
+So membership itself is now pinned, per tier, as a digest of the sorted slug list — and the ''topic'' tag, which had no guard at all and drives the ''iot-device-set'' list on the content page, is pinned the same way:
 <code>
-Error: verdict split moved: B is 21, design:connected_tv publishes 22. Full split now
+const PUBLISHED_MEMBERS = {
-{"A":13,"B":21,"ADJ":16,"OUT":53} against published {"A":13,"B":22,"ADJ":16,"OUT":52}.
+  A: 'b7410e5f7a33a91e', B: '4a71c82990115cdf', ADJ: '1eb63ba64295cda8', OUT: '22530ed51f0ceb35',
-If the map change is intended, update PUBLISHED_SPLIT and every figure on the page.
+};
+const PUBLISHED_TOPICS = 'fdb92e2a4c3ab263';
 </code>
-A deliberate change to the map now requires changing that constant, which is the point: it makes moving the population a visible edit rather than an invisible one.
+Re-run on 2026-09-13, all four mutations now fail and the unmutated script exits 0:
+^ Mutation ^ Before 2026-09-13 ^ Now ^
+| Delete a map entry | throws | throws (''audit set has 1 slug(s) with no verdict'') |
+| Flip one tier letter | throws (since 2026-09-12) | throws (''verdict split moved'') |
+| **Swap two papers between tiers** | **exit 0, population silently wrong** | throws (''tier B membership changed'') |
+| **Change a topic tag only** | **exit 0, the iot-device-set list silently wrong** | throws (''topic assignments … changed'') |
+The lesson worth carrying: the round-1 fix was written by reading the failure the reviewer demonstrated, and it closed exactly that failure and nothing adjacent to it. Only a second mutation test, by a second reviewer with the same brief, found the hole next to it.
 ===== 6. Folding, and the residue in full =====
@@ Line 241: / Line 252: @@
 Two folds are used, and only one aggregates anything.
-**Tool-name alias fold.** ''report_connected_tv.mjs'' maps a lower-cased alphanumeric skeleton of each ''tools[].name'' onto a canonical display name, for 25 skeletons: ''mitmproxy / mitmdump / mitmweb'' to //mitmproxy//, ''wireshark / tshark'' to //Wireshark//, ''adb / androiddebugbridge'' to //adb//, ''charles / charlesproxy'' to //Charles Proxy//, plus one-to-one entries for tcpdump, Frida, Pi-hole, VirusTotal, apktool, jadx, FlowDroid, LibScout, EasyList, Scapy, Selenium, OpenWPM, Mercury, PingPong, Appium, Raspberry Pi, Monkey and UIAutomator. Only ''usedOrMentioned == "used"'' tuples are counted, and the unit is the paper.
+**Tool-name alias fold.** ''report_connected_tv.mjs'' maps a lower-cased alphanumeric skeleton of each ''tools[].name'' onto a canonical display name, for **27 skeletons mapping onto 22 canonical names**: ''mitmproxy / mitmdump / mitmweb'' to //mitmproxy//, ''wireshark / tshark'' to //Wireshark//, ''adb / androiddebugbridge'' to //adb//, ''charles / charlesproxy'' to //Charles Proxy//, plus one-to-one entries for tcpdump, Frida, Pi-hole, VirusTotal, apktool, jadx, FlowDroid, LibScout, EasyList, Scapy, Selenium, OpenWPM, Mercury, PingPong, Appium, Raspberry Pi, Monkey and UIAutomator. Only ''usedOrMentioned == "used"'' tuples are counted, and the unit is the paper.
 **The unmapped residue is 191 distinct raw tool names** across the 35 papers. Printed in full, because a residue that lives only in a local file is a residue nobody reads — and because in this case reading it is how the broadcast-side instruments were found:
@@ Line 286: / Line 297: @@
 </code>
-Nothing in the residue was silently merged and nothing was dropped: the page's instrument table is exactly the 25-alias slice, and the residue is everything else. **The residue is the interesting half here.** ''Avalpa OpenCaster'', ''TSDuck'', ''DekTec DTU-215'', ''HiDes UT-100c'', ''Afatech AF9015'' and ''RedOrbit HbbTV Emulator'' are DVB modulation and stream-authoring tools with no counterpart anywhere else on this wiki; ''IRDB'', ''irgen'', ''IrScrutinizer'', ''MakeHex'' and ''VS1838B'' are infrared remote tooling; ''HDMI Video Capture Device'', ''Roku External Control Protocol'', ''tvbus.exe'' and ''NoxPlayer'' are TV-specific automation. A fold that had merged these into "other" would have hidden the page's most useful finding.
+Nothing in the residue was silently merged and nothing was dropped: the page's instrument table is exactly the 27-alias slice, and the residue is everything else. **The residue is the interesting half here.** ''Avalpa OpenCaster'', ''TSDuck'', ''DekTec DTU-215'', ''HiDes UT-100c'', ''Afatech AF9015'' and ''RedOrbit HbbTV Emulator'' are DVB modulation and stream-authoring tools with no counterpart anywhere else on this wiki; ''IRDB'', ''irgen'', ''IrScrutinizer'', ''MakeHex'' and ''VS1838B'' are infrared remote tooling; ''HDMI Video Capture Device'', ''Roku External Control Protocol'', ''tvbus.exe'' and ''NoxPlayer'' are TV-specific automation. A fold that had merged these into "other" would have hidden the page's most useful finding.
 **No fold is applied to ''vantage.locations'' or ''population.sourceList''**, and both are published unfolded on the content page as **rankings only**, never as percentages. This is deliberate: at //n//=35 the folding error that [[literature:corpus]] measures on the vantage field (280 versus 498 for the United States, corpus-wide) is not worth introducing, and the raw strings — "Apartment 1", "lab space", "241 countries and territories", "e-bike route" — are themselves informative about what a TV vantage point is.
@@ Line 294: / Line 305: @@
 ''scripts/ctv_quotecheck.py'' matches every phrase either page quotes, or leans on for a figure, against **both** ''paper.cols.txt'' and the PDF text layer via pypdf. Both are needed, and this run proves why:
-  * **28 of 33** located in both renderings.
+  * **36 of 41** located in both renderings. (It was 28 of 33 until 2026-09-13; round 2 added eight needles, all of which land in both renderings — see below.)
   * **2 in ''.cols'' only** — the FingerprinTV DBF sentence and the Roku ECP URL, which the PDF text layer scrambles.
   * **3 in the PDF only** — de-columning splices. The clearest is {[kumar2019_things]}: the sentence "the most popular vendor, Roku, only accounts for 17.4% of media devices" is spliced in ''.cols'' into "nd the most poputions of IoT device types, except when a device type accounts lar vendor, Roku, only accounts for 17.4% of media devices for fewer than 1% of devices". A ''.cols''-only check would have reported a correct quote as NOTFOUND.
   * **0 in neither.**
+**Eight needles were added on 2026-09-13**, for the claims the round-2 review changed. They are worth listing because each one existed //on the page// before it existed in this checker — which is the drift this file is supposed to prevent:
+^ Needle ^ Why it was added ^
+| ''to bypass certificate pinning'' ({[moghaddam2019_watching]}) | the widened TLS-decryption probe; the narrow probe missed this sentence while the page quoted it |
+| ''we cannot install our own self-signed certificates on the Roku'' ({[varmarken2020_smart]}) | same |
+| ''there is no way to install custom certificates on Android TV'' ({[tileria2022_watch]}) | same |
+| ''We perform a factory reset of the TV for each channel analysis'' ({[tagliaro2023_still]}) | refutes the page's old "no paper states whether the device was factory-reset". Note the needle stops where it does: the full sentence is **spliced in ''.cols''** by the de-columner, which interleaves it with "and the UT-100c HiDes modulator" |
+| ''download the last version of each app (as of August 2020) from AndroZoo'' ({[tileria2022_watch]}) | refutes "every paper built its own set" |
+| ''we randomly selected 1.5K unique package names from Androzoo'' ({[girish2023_room]}) | same |
+| ''published a candidate standard for hybrid TV in America'' ({[oren2014_aether]}) | the single real ATSC mention in all 5,859 papers |
+| ''this time enabling the "Limit Ad Tracking" … "Disable Interest-based Ads" … settings'' ({[moghaddam2019_watching]}) | refutes "exactly one paper measures both sides of the opt-out" |
 **One needle was genuinely wrong, and the check caught it.** The first draft asserted "decryption fails for 1 out of 5 (or fewer) TLS connections for 80% of all apps" against {[varmarken2020_smart]}. The paper says //"decryption fails for 1 out of 10 (or fewer) TLS connections for 55% of all apps; 1 out of 5 (or fewer) TLS connections for 80% of all apps"// — two clauses, and the draft had glued the opening of the first to the end of the second, producing a sentence the paper does not contain. The needle was narrowed to the clause that is actually there. This is the reason bare-number needles are avoided in that script.
@@ Line 337: / Line 360: @@
 cols+pdf  IMC 2011 understanding-couch-potatoes-measurement-and "The average number of set-top boxes provisioned was approximately "
 cols+pdf  USENIX 2019 all-things-considered-an-analysis-of-iot-dev "are the most common type of device in seven of the eleven regions"
+cols+pdf  CCS 2019 watching-you-watch-the-tracking-ecosystem-of "to bypass certificate pinning"
+cols+pdf  PETS 2020 the-tv-is-smart-and-full-of-trackers-measuri "we cannot install our own self-signed certificates on the Roku"
+cols+pdf  PETS 2022 watch-over-your-tv-a-security-and-privacy-an "there is no way to install custom certificates on Android TV"
+cols+pdf  NDSS 2023 i-still-know-what-you-watched-last-sunday-pr "We perform a factory reset of the TV for each channel analysis"
+cols+pdf  PETS 2022 watch-over-your-tv-a-security-and-privacy-an "download the last version of each app (as of August 2020) from And"
+cols+pdf  IMC 2023 in-the-room-where-it-happens-characterizing- "we randomly selected 1.5K unique package names from Androzoo"
+cols+pdf  USENIX 2014 from-the-aether-to-the-ethernet-attacking-th "published a candidate standard for hybrid TV in America"
+cols+pdf  CCS 2019 watching-you-watch-the-tracking-ecosystem-of "this time enabling the "Limit Ad Tracking" (Roku) and the "Disable"
- quotes: 28 in both renderings, 2 in .cols only, 3 in the PDF only, 0 in neither.
+ quotes: 36 in both renderings, 2 in .cols only, 3 in the PDF only, 0 in neither.
 </code>
@@ Line 348: / Line 379: @@
 | HbbTV 2.0.5, published 2026-02-25, incremental over 2.0.4 (March 2023) | HbbTV Association specifications page | fetched; the version table lists 1.0 (2010) through 2.0.5 (2026-02-25) |
 | Roku ECP is "a simple RESTful API accessed using HTTP on port 8060", no authentication documented | Roku developer docs, External Control API | fetched; the port and the ''query/device-info'' endpoint quoted verbatim. Cross-checked against {[moghaddam2019_watching]}, which uses the same URL form |
-| RIDA, ''GetRida()'' / ''IsRidaDisabled()'', 30-day temporary ID under limit-ad-tracking | Roku developer docs, Roku Advertising Framework | fetched |
+| RIDA, ''GetRIDA()'' / ''IsRIDADisabled()'', 30-day temporary ID under limit-ad-tracking | Roku developer docs: ''integrating-roku-advertising-framework'' for ''GetRIDA()'' and the 30-day ID, ''ifDeviceInfo'' for ''IsRIDADisabled()'' (which is //not// on the RAF page) | fetched; re-verified 2026-09-13 |
 | TIFA, ''getTIFA()'' / ''isLATEnabled()'', resettable, "no connection to any PII … or DUID" | Samsung Smart TV developer docs | fetched from the ''unique-identifiers-for-smarttv'' guide |
 | Fire TV Advertising ID, ''advertising_id'' / ''limit_ad_tracking'', Fire OS 5.2.1.1+ on TV | Amazon Developer Policy Center, Advertising ID Policy | fetched |
@@ Line 365: / Line 396: @@
 Two further checks worth recording. The Mon(IoT)r testbed software is live at ''github.com/djdubois/moniotr-core'' (last pushed 2024-08-09) but the lab's tools page does not publish a smart-TV dataset for download, so the page describes testbed captures as a route without promising a TV dataset exists to fetch. And the PETS landing pages were used to recover author lists for five entries the corpus index lacks; {[ahmed2022_analyzing]}'s authors were additionally cross-checked against Crossref because its stored PDF has no usable text layer on the title page.
+==== 8b. Sources added or corrected in round 2 (2026-09-13) ====
+^ Claim ^ Primary source, and how it was checked ^ Result ^
+| ATSC 3.0 broadcaster applications: ''advertisingId'', ''filterCode'', receiver cookies | ATSC **A/344:2026-04**, //ATSC 3.0 Interactive Content//, 14 April 2026. PDF downloaded (HTTP 200, 2,363,014 bytes, 202 pages), text extracted with ''pypdf'', each needle matched in whitespace-collapsed text | All five quotes FOUND verbatim. ''data collection'' returns **0 hits** in the whole standard — the phrase the withdrawn 2016 quote used |
+| A/344 current revision | [[https://www.atsc.org/atsc-documents/3442017-atsc-3-0-interactive-content/|the A/344 document page]] lists 2026-04 (14 April 2026) above 2026-02; the 2026-04 PDF's own title block reads ''A/344:2026-04'' | **2026-04**, not the 2026-02 the author first downloaded. One round-2 reviewer read the standards listing as showing 2025-07 as the latest approved; the PDF's own designation settles it |
+| EU: joint Article 62 GDPR operation on smart TVs | [[https://www.autoriteitpersoonsgegevens.nl/documenten/rapport-verkennend-onderzoek-smart-tvs|Autoriteit Persoonsgegevens]] landing page (HTTP 200) and the report PDF (HTTP 200, 1,005,269 bytes, 14 pages), both needing a browser User-Agent and a Referer header | Real. NL/HU/IT/LI, published 23 September 2025. The off-state percentages (97.52 / 98.84 / 91.10) read out of the PDF's own table |
+| UK: ICO connected-TV programme | [[https://ico.org.uk/about-the-ico/media-centre/news-and-blogs/2026/06/setting-out-our-expectations-for-the-smart-device-industry/|ICO news release]], 11 June 2026 (HTTP 403 to a bare fetcher, HTTP 200 with a browser User-Agent) | Real. Quote and attribution to William Malcolm verified in the fetched HTML |
+| Walmart press release names ''Platform+'' | [[https://corporate.walmart.com/news/2024/12/03/walmart-completes-acquisition-of-vizio|the release]] (HTTP 200, 180,202 bytes), searched after **double** HTML-entity decoding | **It does.** The round-1 footnote said it did not. The plus sign is entity-encoded, so a tag-strip-only search misses it — this is why the check has to decode entities, not just strip tags. ''Inscape'' is genuinely absent |
+| ''ahn2025_watch'' Zenodo access | Zenodo REST API, ''records/15646588'' (HTTP 200): ''access_right: open'', CC-BY-4.0, title ends ''[Public Artifact]'' | The paper has **two** records; the extraction's ''restricted'' is true of ''zenodo.15602938'' only. Round 1 attached the label to the open DOI |
+| FingerprinTV code release | [[https://github.com/UCI-Networking-Group/fingerprintv|README]] fetched raw (HTTP 200) | "The FingerprinTV dataset has already been released." / "Once it is ready for release to the public, the code will be added to this repository. Please stay tuned." Four years on |
+| FCC Fifth FNPRM is still pending | Checked for a Report and Order; none found as of 2026-09-13. The adopted item is **FCC 25-72**, adopted 2025-10-28, released 2025-10-29 | The page's fact-sheet citation stands. One reviewer surfaced a trade-press claim of a May 2026 FCC vote mandating ATSC 3.0 tuners; it appears nowhere on fcc.gov and is **rejected** |
+**Rejected in round 2.** A blog URL for "ADB Wi-Fi 2.0" offered in round 1 returns 404 and stays rejected; the claim rests on ''developer.android.com/tools/adb'' instead. The trade-press "FCC voted 3–2 in May 2026" story is rejected as above. Samsung's briefly-vacated TRO is still judged a procedural detail and stays off the content page.
 ===== 9. Bibliography =====
@@ Line 631: / Line 676: @@
 import fs from 'node:fs';
 import path from 'node:path';
+import { createHash } from 'node:crypto';
 import { loadExtractions, dataRoot, isSentinel, pct, table, wikiTable } from './lib.mjs';
 import { MAP } from './ctv_fold.mjs';
@@ Line 723: / Line 769: @@
 if (splitTotal !== AUDIT.length)
   throw new Error(`verdicts sum to ${splitTotal} but the audit set is ${AUDIT.length}`);
+// A second mutation test, on 2026-09-13, broke the four counts above: SWAPPING
+// two papers between tiers — one genuine Tier B out, one genuine OUT in —
+// leaves A/B/ADJ/OUT unchanged and exits 0, while the population silently gains
+// a paper that is not about television (platform 'web' went 1 -> 2, the venue
+// and year tables both moved). Counts are not membership. So the membership
+// itself is pinned, per tier, as a digest of the sorted slug list.
+const digest = (rows) => createHash('sha256')
+  .update(rows.map((x) => x.p.slug).sort().join('\n')).digest('hex').slice(0, 16);
+const PUBLISHED_MEMBERS = {
+  A: 'b7410e5f7a33a91e', B: '4a71c82990115cdf', ADJ: '1eb63ba64295cda8', OUT: '22530ed51f0ceb35',
+};
+const actualMembers = { A: digest(A), B: digest(B), ADJ: digest(ADJ), OUT: digest(OUT) };
+for (const k of Object.keys(PUBLISHED_MEMBERS)) {
+  if (actualMembers[k] !== PUBLISHED_MEMBERS[k])
+    throw new Error(
+      `tier ${k} membership changed (count is still ${actualSplit[k]}, so the count check above could not see it). ` +
+      `digest is ${actualMembers[k]}, design:connected_tv was published against ${PUBLISHED_MEMBERS[k]}. ` +
+      `Full digests now ${JSON.stringify(actualMembers)}. If the map change is intended, update ` +
+      `PUBLISHED_MEMBERS and re-check every per-venue, per-year, per-platform and per-topic figure on the page.`
+    );
+}
+// The topic tag has no count to pin — it is hand-assigned and published as a
+// ranking. It still drives the "Fifteen of the 35 papers are IoT studies" list,
+// so its membership is pinned the same way.
+const topicDigest = createHash('sha256').update(
+  Object.entries(MAP).filter(([, v]) => v[0] === 'A' || v[0] === 'B')
+    .map(([s, v]) => `${s}\t${v[1]}`).sort().join('\n')).digest('hex').slice(0, 16);
+const PUBLISHED_TOPICS = 'fdb92e2a4c3ab263';
+if (topicDigest !== PUBLISHED_TOPICS)
+  throw new Error(
+    `topic assignments for the 35-paper population changed: digest ${topicDigest}, published against ${PUBLISHED_TOPICS}. ` +
+    `Update PUBLISHED_TOPICS and re-check the "What this literature measures" table and the iot-device-set list on the page.`
+  );
 H('0. CANDIDATE POOL AND AUDIT');
@@ Line 949: / Line 1030: @@
     `${EMP_CORPUS.filter((p) => p.temporal.some((t) => t.spanStart !== null)).length} of ${EMP_CORPUS.length} (${pct(EMP_CORPUS.filter((p) => p.temporal.some((t) => t.spanStart !== null)).length, EMP_CORPUS.length)})`],
 ]);
+function linkOf(a) {
+  if (a.codeUrl) return a.codeUrl;
+  if (a.dataUrl) return a.dataUrl;
+  if (!a.links || a.links.length === 0) return '—';
+  const l = a.links[0];
+  if (typeof l === 'string') throw new Error(`artifacts.links[0] is a string, expected an object: ${l}`);
+  return `${l.url}  (${l.kind}; ${l.what}; authors=${l.belongsToAuthors})`;
+}
 console.log('\nArtifact links released, Tier A:');
 for (const x of A) console.log(x.p.artifacts === null
   ? `  ${x.p.venue} ${x.p.year}  (no artifacts object extracted)`
-  : `  ${x.p.venue} ${x.p.year}  ${x.p.artifacts.availability.padEnd(22)} ${(x.p.artifacts.codeUrl || x.p.artifacts.dataUrl || x.p.artifacts.links[0] || '\u2014')}`);
+  // artifacts.links[] entries are OBJECTS ({url, kind, what, belongsToAuthors}).
+  // Falling back to links[0] itself string-coerced to "[object Object]" and that
+  // literal was published three times on the provenance page \u2014 found by a review
+  // pass on 2026-09-13. Take .url, and print what the link is for.
+  : `  ${x.p.venue} ${x.p.year}  ${x.p.artifacts.availability.padEnd(22)} ${linkOf(x.p.artifacts)}`);
 // ---------------------------------------------------------------------------
+// ---------------------------------------------------------------------------
+// 6b. Two probes the page cites in footnotes. Both were added on 2026-09-13
+//     after review found the page asserting negatives no probe supported.
+// ---------------------------------------------------------------------------
+H('6b. TWO PROBES THE PAGE CITES');
+// (i) The TLS decryption hole. The page originally ran only NARROW, matched 1
+//     of 13 Tier A papers, and published "almost nobody reports the hole" —
+//     while quoting two of the sentences NARROW misses. A narrowing probe must
+//     return a SUBSET of the loose one; both counts are printed so the claim
+//     can be read off the right width.
+const HOLE_NARROW = /could not decrypt|failed to decrypt|decryption fail|unable to (decrypt|intercept)/i;
+const HOLE_WIDE = new RegExp([
+  /(could not|cannot|can ?not|unable to|failed to|no way to)[^.]{0,90}(decrypt|intercept|install (our own |a |custom )?(self-signed )?certificat)/.source,
+  /(bypass|remove|disabl\w*)[^.]{0,40}certificate pinning/.source,
+  /certificate pinning checks/.source,
+].join('|'), 'i');
+const holeN = A.filter((x) => HOLE_NARROW.test(fulltext(x.p)));
+const holeW = A.filter((x) => HOLE_WIDE.test(fulltext(x.p)));
+const holeNs = new Set(holeN.map((x) => x.p.slug));
+const notContained = holeN.filter((x) => !holeW.some((y) => y.p.slug === x.p.slug));
+if (notContained.length)
+  throw new Error(`the narrow decryption probe is not a subset of the wide one: ${notContained.map((x) => x.p.slug).join(', ')}`);
+if (holeN.length > holeW.length)
+  throw new Error(`narrow probe matched ${holeN.length} > wide ${holeW.length}; a narrowing probe must match fewer`);
+console.log(`TLS decryption hole, Tier A (n=${A.length}):`);
+console.log(`  narrow probe (the one the page published until 2026-09-13): ${holeN.length}`);
+console.log(`  wide probe   (the one the page cites now)                 : ${holeW.length}`);
+for (const x of holeW) {
+  const t = fulltext(x.p), m = t.match(HOLE_WIDE), i = t.indexOf(m[0]);
+  console.log(`   ${holeNs.has(x.p.slug) ? 'both  ' : 'wide  '} ${x.p.venue} ${x.p.year} ${x.p.slug}`);
+  console.log(`          ...${t.slice(Math.max(0, i - 100), i + 200)}...`);
+}
+// (ii) ATSC 3.0 / NextGen TV. The page says no paper measures it. That was an
+//      unprobed negative until now; ATSC was never in PROBES.
+const ATSC = /\bATSC\b|\bNextGen ?TV\b|\bNext ?Gen ?TV\b/i;
+const atscHits = ALL.filter((p) => ATSC.test(fulltext(p)));
+console.log(`\nATSC 3.0 / NextGen TV, full text, all ${ALL.length} corpus papers: ${atscHits.length} match`);
+for (const p of atscHits) {
+  const t = fulltext(p), m = t.match(ATSC), i = t.indexOf(m[0]);
+  console.log(`   ${p.venue} ${p.year} ${p.slug}   [${MAP[p.slug] ? MAP[p.slug][0] : 'not in audit set'}]`);
+  console.log(`          ...${t.slice(Math.max(0, i - 130), i + 210)}...`);
+}
+// (iii) The factory-reset gap. crawlConfig.statefulness is empty for all 6
+//       papers that have the object; the page needed the full-text picture too.
+const RESET = /factory[- ]reset/i;
+const resetHits = POP.filter((x) => RESET.test(fulltext(x.p)));
+const withCfg = POP.filter((x) => x.p.crawlConfig !== null);
+console.log(`\nStatefulness: ${withCfg.length} of ${nPOP} papers have a crawlConfig object;`);
+console.log(`  statefulness values among them: ${JSON.stringify(withCfg.map((x) => x.p.crawlConfig.statefulness))}`);
+console.log(`  full-text "factory reset" anywhere in the ${nPOP}: ${resetHits.length}`);
+for (const x of resetHits) {
+  const t = fulltext(x.p), i = t.search(RESET);
+  console.log(`   ${MAP[x.p.slug][0]}  ${x.p.venue} ${x.p.year} ${x.p.slug}`);
+  console.log(`          ...${t.slice(Math.max(0, i - 130), i + 210)}...`);
+}
+// (iv) LLM use inside the population. The page said "not one paper uses an LLM
+//      for anything"; two do, and both tool names are in the printed residue.
+console.log('\nLLM and language-model tools used by the population (tools[].category, used only):');
+for (const x of POP) {
+  for (const t of x.p.tools) {
+    if (t.usedOrMentioned !== 'used') continue;
+    if (!/^llm$/i.test(t.category) && !/gpt|openai|chatgpt|bert|language model/i.test(t.name)) continue;
+    console.log(`   ${x.p.venue} ${x.p.year}  category=${t.category.padEnd(22)} ${t.name}`);
+    console.log(`          quote: ${JSON.stringify((t.evidence?.quote || '').slice(0, 200))}`);
+  }
+}
 // 7. Quote verification lives in scripts/ctv_quotecheck.py, which checks each
 //    quote against BOTH paper.cols.txt and the PDF text layer. Doing it here
@@ Line 1070: / Line 1235: @@
     ('2019', 'USENIX', 'all-things-considered-an-analysis-of-iot-devices-on-home-networks',
      'are the most common type of device in seven of the eleven regions'),
+    # --- added 2026-09-13, for claims the round-2 review changed -------------
+    # The widened TLS-decryption probe: the three Tier A sentences the page now
+    # quotes. The narrow probe matched only the second of these, which is how a
+    # "1 of 13" reached the page while two of the three sat in its own prose.
+    ('2019', 'CCS', 'watching-you-watch-the-tracking-ecosystem-of-over-the-top-tv-streaming-devices',
+     'to bypass certificate pinning'),
+    ('2020', 'PETS', 'the-tv-is-smart-and-full-of-trackers-measuring-smart-tv-advertising-and-tracking',
+     'we cannot install our own self-signed certificates on the Roku'),
+    ('2022', 'PETS', 'watch-over-your-tv-a-security-and-privacy-analysis-of-the-android-tv-ecosystem',
+     'there is no way to install custom certificates on Android TV'),
+    # The factory-reset sentence. In .cols this one is SPLICED by the
+    # de-columner — "and the UT-We perform a factory reset of the TV for each
+    # channel analysis 100c HiDes modulator" — so only the first clause is
+    # contiguous there. The PDF layer carries the whole sentence.
+    ('2023', 'NDSS', 'i-still-know-what-you-watched-last-sunday-privacy-of-the-hbbtv-protocol-in-the-european-smart-tv-landscape',
+     'We perform a factory reset of the TV for each channel analysis'),
+    # AndroZoo: the page said every TV-app paper built its own set.
+    ('2022', 'PETS', 'watch-over-your-tv-a-security-and-privacy-analysis-of-the-android-tv-ecosystem',
+     'download the last version of each app (as of August 2020) from AndroZoo'),
+    ('2023', 'IMC', 'in-the-room-where-it-happens-characterizing-local-communication-and-threats-in-s',
+     'we randomly selected 1.5K unique package names from Androzoo'),
+    # The one real ATSC mention in the whole 5,859-paper corpus.
+    ('2014', 'USENIX', 'from-the-aether-to-the-ethernet-attacking-the-internet-using-broadcast-digital-t',
+     'published a candidate standard for hybrid TV in America'),
+    # Both sides of the opt-out: the paper the page had omitted.
+    ('2019', 'CCS', 'watching-you-watch-the-tracking-ecosystem-of-over-the-top-tv-streaming-devices',
+     'this time enabling the "Limit Ad Tracking" (Roku) and the "Disable Interest-based Ads" (Amazon Fire TV) settings'),
 ]
@@ Line 1975: / Line 2167: @@
   IMC 2011  public                 www.research.att.com/∼kkrama/papers/streamcontrol.pdf
   IMC 2011  (no artifacts object extracted)
-  USENIX 2014  none-mentioned         [object Object]
+  USENIX 2014  none-mentioned         http://www.avalpa.com/the-key-values/15-free-software/33-opencaster  (other; OpenCaster software; authors=false)
   CCS 2019  promised-not-yet-available —
-  PETS 2020  promised-not-yet-available [object Object]
+  PETS 2020  promised-not-yet-available http://athinagroup.eng.uci.edu/projects/smarttv/  (project-page; Project page for tools and testbed datasets; authors=true)
-  USENIX 2021  none-mentioned         [object Object]
+  USENIX 2021  none-mentioned         https://sites.google.com/site/smarttvdemos/  (project-page; Demonstration website for discovered attacks; authors=true)
   PETS 2022  promised-not-yet-available https://github.com/UCI-Networking-Group/fingerprintv
   PETS 2022  public                 https://gitlab.com/s3lab-rhul/watch-over-your-tv-paper
@@ Line 1986: / Line 2178: @@
   NDSS 2024  public                 https://github.com/tejaskannan/smart-tv-keyboard-leakage
   USENIX 2025  restricted             https://doi.org/10.5281/zenodo.15646588
+==============================================================================
+b. TWO PROBES THE PAGE CITES
+==============================================================================
+TLS decryption hole, Tier A (n=13):
+  narrow probe (the one the page published until 2026-09-13): 1
+  wide probe   (the one the page cites now)                 : 3
+   wide   CCS 2019 watching-you-watch-the-tracking-ecosystem-of-over-the-top-tv-streaming-devices
+          ...tificate to the device and use external toolkits (e.g., Frida [29] for the Amazon Fire Stick TV) to bypass certificate pinning. Contributions: We make the following contributions: • We conduct the first large-scale study of privacy practices of OTT streaming channels. Using an automated crawler that...
+   both   PETS 2020 the-tv-is-smart-and-full-of-trackers-measuring-smart-tv-advertising-and-tracking
+          ...tion time with each app is approximately 16 minutes. We do not attempt to decrypt TLS traffic as we cannot install our own self-signed certificates on the Roku. 4.2 Fire TV Data Collection In this section, we describe the Fire TV platform, our app selection methodology, and present an overview of Fi...
+   wide   PETS 2022 watch-over-your-tv-a-security-and-privacy-analysis-of-the-android-tv-ecosystem
+          ...k traffic. We instrumented the APKs using the mitm-proxy script [32] to add Charles certificate and remove certificate pinning checks. This instrumentation is only necessary for TV apps as there is no way to install custom certificates on Android TV. For the mobile apps, we use smartphones with Andr...
+ATSC 3.0 / NextGen TV, full text, all 5859 corpus papers: 4 match
+   USENIX 2012 i-forgot-your-password-randomness-attacks-against-php-applications   [not in audit set]
+          ...mber of bits truncated. Application Attack Application Attack mediawiki 4.2 4.3 5.3 • Joomla 4.3 • Open eClass 4.2 4.3 5.4 • MyBB ATSc 4.1c 5.3c ◦ taskfreak 4.2 4.3 5.3 • IpBoard ATSc 4.1c 4.2c • zen-cart ATS RT • phorum 4.2 4.3 5.3 • osCommerce 2.x ATS RT • HotCRP 4.2 4.3 5.3 • osCommerce 3.x 4.2 4.3 5.4 • gazelle 4.3 5.3 • elgg ATSc 4.2...
+   USENIX 2014 from-the-aether-to-the-ethernet-attacking-the-internet-using-broadcast-digital-t   [A]
+          ...ctive deployment or in advanced stages of testing in most of Europe. In December 2013, the Advanced Television Systems Committee (ATSC), which defines the digital video standards in the US, Canada, South Korea and several USENIX Association other countries, published a candidate standard for hybrid TV in America [6]. This candidate standa...
+   NDSS 2020 automated-cross-platform-reverse-engineering-of-can-bus-commands-from-mobile-apps   [not in audit set]
+          ...E, ATA, ATH... Gauged 17 ATED, ATD, ATP, ATZ... iOBD2 20 ATE, AT ST, AT CA F... LeagendOBD 12 ATE, ATB, ATTR, ATQ... Engie 8 ATE, ATSC, ATI, ATST... TABLE X: AT commands extracted from dongle apps. 16 App # Command AcuraLink 9 Alpine 2 Alpine Tunelt 3 Audi MMI Connect 10 Carbin Control 15 Car-Net 4 Companion 2 Mini Connected Classic 1 Nis...
+   CCS 2025 dont-look-up-there-are-sensitive-internal-links-in-the-clear-on-geo-satellites   [not in audit set]
+          ...ngel Electronics. 2024. STAB HH90 Satellite Dish Motor. https:// angelelectronics.ca/products/stab-hh90-satellite-dish-motor. [8] ATSC. [n. d.]. ATSC. https://www.atsc.org/documents/. [9] Robin Bisping, Johannes Willbold, Martin Strohmeier, and Vincent Lenders. 2024. Wireless Signal Injection Attacks on VSAT Satellite Modems. USENIX Secur...
+Statefulness: 6 of 35 papers have a crawlConfig object;
+  statefulness values among them: ["not-stated","not-stated","not-stated","not-stated","not-stated","not-stated"]
+  full-text "factory reset" anywhere in the 35: 2
+   A  NDSS 2023 i-still-know-what-you-watched-last-sunday-privacy-of-the-hbbtv-protocol-in-the-european-smart-tv-landscape
+          .... For the second test, we start by extracting the HbbTV URLs from the DVB stream using the TSDuck library and the UT-We perform a factory reset of the TV for each channel analysis 100c HiDes modulator. As mentioned in Section II, the DVB to prevent interference in the captured traffic. stream includes the URLs of the HbbTV applications; t...
+   B  NDSS 2026 blerp-ble-re-pairing-attacks-and-defenses
+          ... introduces a usability trade-off: if a device implicit authentication, ensuring that an attacker lacking loses its PK (e.g., via factory reset), it requires manual user the current PK cannot compute the new one. intervention to re-pair. • Transcript Hashing: Devices must maintain a cumulative We implemented this protocol in NimBLE by ext...
+LLM and language-model tools used by the population (tools[].category, used only):
+   CCS 2022  category=ml-model-or-algorithm  BERT
+          quote: "IoTSpotter's BERT-based and BiLSTM classifiers identified 58,859 and 69,270 app descriptions as mobile-IoT, respectively."
+   IMC 2023  category=llm                    ChatGPT (OpenAI's TextCompletion API)
+          quote: "Using OpenAI's TextCompletion API, we develop prompt to infer device vendors and categories based on DHCP hostname, mDNS/SSDP responses, and user labels."
+   PETS 2024  category=llm                    OpenAI Text Completion API
+          quote: "Using OpenAI's Text Completion API [34], we develop prompts to infer device vendors and categories"
+   PETS 2024  category=ml-model-or-algorithm  PrivBERT
+          quote: "We use PrivBERT [69], a pre-trained privacy policy language model to build a binary classifier"
 ==============================================================================
@@ Line 1994: / Line 2227: @@
 ===== 14. Run log =====
-  * **2026-09-12** — page written. Corpus ''data/extract/run1'', 5,859 papers. Candidate pool, audit, report script, quote check, bibliography generation and external verification all run on this date.
+  * **2026-09-12** — page written. Corpus ''data/extract/run1'', 5,859 papers. Candidate pool, audit, report script, quote check, bibliography generation and external verification all run on this date. Three review passes — figures-versus-script, citations-and-quotes and external currency — returned and were applied; they are logged in §15. A fourth, generic pass was still running when the session ended, so it is **not** in §15. An earlier version of this line said all four were logged there; that was wrong.
-  * Sub-agents: four review passes — figures-versus-script, citations-and-quotes, external currency, and a generic pass — logged in §15 with every finding and whether it was accepted.
+  * **2026-09-13** — round 2. All four passes re-run against the published pages, because every focused domain had been changed by round 1's own fixes. 21 findings accepted, 3 rejected, logged in §15b. Two of round 1's accepted fixes turned out to be wrong and one had landed on only one of the two pages. The report script gained membership and topic digests (§5b), a ''linkOf()'' helper, and three probes the page had been citing without having: the wide TLS-decryption probe, the ATSC full-text probe and the factory-reset probe. Section §13 is regenerated from the current script.
-  * No credential or token was printed at any point in this run.
+  * Models: the three focused passes were ''sonnet'', the generic pass was ''fable'', in both rounds. The generic pass found more real defects in round 2 than the three focused passes combined — see the closing note in §15b for why that is structural rather than luck.
+  * No credential or token was printed at any point in either run.
 ===== 15. Review log =====
-Four review passes ran on 2026-09-12 against the published pages, the report script and its output. Each was told explicitly that the author's context might not be exhaustive and to verify rather than assume. **Every finding below was re-checked by hand against the primary source before it was accepted or rejected** — two of the accepted ones needed correcting in the process, and the rejections are recorded because they are the only evidence of whether a reviewer earned its slot.
+Three review passes returned on 2026-09-12 against the published pages, the report script and its output; the fourth was cut off mid-run and was re-run as part of round 2 (§15b). Each was told explicitly that the author's context might not be exhaustive and to verify rather than assume. **Every finding below was re-checked by hand against the primary source before it was accepted or rejected** — two of the accepted ones needed correcting in the process, and the rejections are recorded because they are the only evidence of whether a reviewer earned its slot.
-==== Pass 1 — figures versus script (''sonnet'') ====
+==== Round 1, pass 1 — figures versus script (sonnet) ====
 ^ # ^ Finding ^ Disposition ^
 | 1.1 | The "fifteen IoT-device-set papers" list substitutes {[rye2024_surveilling]} (tagged ''device-population'') for the real fifteenth, //Et Tu Alexa?// (NDSS 2020), which is cited nowhere | **Accepted.** Independently confirmed by extracting the tier/topic tuples from ''ctv_fold.mjs''. Fixed: the list is now exactly the fifteen, {[zhu2020_alexa]} was added to the bibliography, and {[rye2024_surveilling]} is described in its own clause. This was also found independently by the author before the pass returned |
-| 1.2 | Both pages say the alias fold "maps 25 names"; it has **27 skeleton keys onto 22 canonical names** | **Accepted.** Re-parsed the ''ALIAS'' object: 27 and 22, neither of them 25. Corrected on both pages |
+| 1.2 | Both pages say the alias fold "maps 25 names"; it has **27 skeleton keys onto 22 canonical names** | **Accepted.** Re-parsed the ''ALIAS'' object: 27 and 22, neither of them 25. //This row originally read "Corrected on both pages". That was false:// only the content page was corrected in round 1. This provenance page still said 25 until round 2 found it — three separate reviewers, plus the author's own residue sweep, all landed on the same line. See §15b, finding R2.6 |
 | 1.3 | The sampling-size paragraph attributes "3,000,000 IPTV set-top boxes" to {[gopalakrishnan2011_couch]} as one of "the 20 papers stating an ''iot-devices'' population size", but that paper's tuples are unit ''other'' and ''human-participants''. The paper that //is// one of the 20 with large values, {[song2011_score]} (7,000,000 and 140,000), is named nowhere | **Accepted, and it was worse than reported.** The author had already flagged the same sentence for omitting 57 — the PETS 2020 testbed — from a hand-typed list; the reviewer found the framing error underneath it. The whole passage is now generated by the script (§13) rather than typed, and the 3,000,000 figure is kept with an explicit note that it is a subscriber count, not one of the 20 |
 | 1.4 | The map guard only checks the slug set. **Flipping a tier letter exits 0** and silently moves the population from 35 to 34 | **Accepted, and the most valuable finding of the round.** The reviewer mutation-tested rather than read. Fixed with a summing invariant; see §5b |
@@ Line 2012: / Line 2246: @@
 | 1.6 | Venue display labels differ between page ("USENIX Security", "TheWebConf") and script ("USENIX", "WWW") | **Rejected.** The page uses the venues' real names and the numbers underneath are identical. Cosmetic |
-==== Pass 2 — citations and quotes (''sonnet'') ====
+==== Round 1, pass 2 — citations and quotes (sonnet) ====
 ^ # ^ Finding ^ Disposition ^
@@ Line 2023: / Line 2257: @@
 | 2.7 | The Tier-B rule is paraphrased two different ways on the same page ("…for a television" versus "…for them") | **Accepted** as a NIT. Wording made consistent |
-==== Pass 3 — external currency (''sonnet'') ====
+==== Round 1, pass 3 — external currency (sonnet) ====
 ^ # ^ Finding ^ Disposition ^
@@ Line 2037: / Line 2271: @@
 | 3.10 | Samsung's own TRO was granted and vacated the next day, before the February settlement | **Rejected** for the content page. Real, but a procedural detail; the substantive gap was 3.4, which is in |
-==== What the review round cost and returned ====
+==== What round 1 cost and returned ====
-Three passes, eleven accepted findings, four rejections, two accepted-with-correction. The pattern worth recording for the next run: **the defects were all in prose that summarises data, never in the tables the script generates.** Every generated figure survived independent re-derivation; every hand-typed list, paraphrase and quotation that sat next to one had to be fixed. The two most valuable findings — the mutation test that broke the guard, and the author-order check against Crossref — were both things that cannot be done by reading, which is the argument for handing a reviewer the script rather than only the page.
+Three passes returned; a fourth, the generic one, was still running when the session ended and is logged in §15b with round 2 instead. Counting the rows above: **14 accepted** (1.1–1.4, 2.1, 2.2, 2.7, 3.1–3.7), two of them accepted-with-correction, and **3 rejections** (1.6, 3.9, 3.10). An earlier version of this paragraph said "eleven accepted, four rejections", which does not match its own tables. The pattern worth recording for the next run: **the defects were all in prose that summarises data, never in the tables the script generates.** Every generated figure survived independent re-derivation; every hand-typed list, paraphrase and quotation that sat next to one had to be fixed. The two most valuable findings — the mutation test that broke the guard, and the author-order check against Crossref — were both things that cannot be done by reading, which is the argument for handing a reviewer the script rather than only the page.
 ====== References ======
 The same keys and the same shared bibliography as [[design:connected_tv]]; this page adds no entries of its own. No discussion block: comments belong on the content page.
+===== 15b. Review log, round 2 (2026-09-13) =====
+Round 1 ran three focused passes and was cut off with the generic pass still in flight. Round 2 re-ran **all four**, because every one of the three focused domains had been changed by round 1's own fixes — and that turned out to be the right call twice over: two of round 1's accepted fixes were themselves wrong, and a third had landed on only one of the two pages. Every reviewer was told the author's context might not be exhaustive. **Every finding below was re-checked by hand against the primary source or the raw corpus before being accepted or rejected.**
+The headline of this round: **the generic pass, which has no checklist, found more real defects than the three focused passes combined** — and all of them were claims about what the literature does //not// do. A reviewer asked to check figures against a script checks the figures that are in the script. A negative claim has no figure.
+==== Round 2, pass 1 — figures versus script (sonnet) ====
+^ # ^ Finding ^ Disposition ^
+| R2.1 | **The round-1 invariant can be satisfied vacuously.** Swapping two papers between tiers — one genuine Tier B out, one genuine ''OUT'' in — leaves ''A:13, B:22, ADJ:16, OUT:52'' unchanged and exits 0, while platform ''web'' goes 1→2 and the venue, year and platform tables all move | **Accepted, BLOCKER, and the most valuable finding of either round.** Reproduced exactly. §5b now records it, and membership is pinned per tier as a slug-list digest. **The pattern worth keeping: round 1's fix was written from the failure a reviewer demonstrated, and closed exactly that failure and nothing beside it.** It took a second mutation test, from a reviewer with the same brief, to find the hole next door |
+| R2.2 | ''artifacts.links[0]'' is an **object**, so the fallback printed the literal ''[object Object]'' — published three times in §13 | **Accepted.** Confirmed against ''extractions.jsonl''. Fixed with a ''linkOf()'' helper that takes ''.url'' and throws if the shape is ever a string; the three entries now carry real links, one of which (''athinagroup.eng.uci.edu/projects/smarttv/'') is a citable project page that had been hidden behind the bug |
+| R2.3 | Q16 says "33 empirical"; ''isEmpirical'' is true for **34** of the 35, and 27/34 is the 79.4% printed beside it | **Accepted.** Re-derived: exactly one paper (//Lumos//) is not empirical. Corrected |
+| R2.4 | The round-1 rewrite of the 3,000,000 IPTV figure says the paper "records them as subscribers"; the paper says "the average number of **set-top boxes** provisioned was approximately 3 million" | **Accepted — a round-1 fix that introduced a new error.** Verified in ''paper.cols.txt''. The real reason it is outside the 20 is that its extraction ''unit'' is ''other'', not ''iot-devices'' — a taxonomy boundary, not anything the paper did. The page now says that |
+| R2.5 | The ''topic'' tag has no invariant at all; changing one exits 0 and silently moves the "What this literature measures" table and the ''iot-device-set'' list | **Accepted.** Pinned with its own digest. Mutation-tested: it now throws |
+| R2.6 | The provenance page still says the alias fold maps "25 skeletons" while listing 27 of them | **Accepted.** Found independently by three of the four passes and by the author's own residue sweep. Corrected, and round 1's claim to have fixed "both pages" is corrected too |
+| R2.7 | Script reproduces byte-identically; the fifteen ''iot-device-set'' papers are exactly the fifteen named; venue, year, topic, platform, vantage, ''crawlConfig'', ethics and artifacts tables all match cell-for-cell; sentinels never counted as stated; papers never counted as tuples | No action. Recorded because it is the control |
+==== Round 2, pass 2 — citations and quotes (sonnet) ====
+^ # ^ Finding ^ Disposition ^
+| R2.8 | **"Nothing comparable was found from an EU or UK regulator" is false.** The UK ICO published a connected-TV programme on 2026-06-11 | **Accepted, BLOCKER.** Verified at ico.org.uk directly. See R2.11 — the other pass found the EU half independently |
+| R2.9 | Author order for ''zhu2020_alexa'' checked against the NDSS PDF front matter: Yanzi Zhu, Zhujun Xiao, Yuxin Chen, Zhijing Li, Max Liu, Ben Y. Zhao, Haitao Zheng — matches, no swap. All 34 citekeys resolve; 1,025 entries, 1,025 distinct keys; 0 rule-A/B duplicates | No action. This is the check the author cannot self-run, and it is why the pass earns its slot |
+| R2.10 | Every replacement claim from round 1 verified verbatim: the FCC's "extended to July 17, 2027" and its one-word "Privacy" bullet, Kentucky's enacted prohibition, ''GetRIDA()'' / ''IsRIDADisabled()'', the four Texas dates and the "every 500 milliseconds" quote, and the Texas AG's own statement that the cases against Sony, Hisense and TCL "remain ongoing" | No action |
+==== Round 2, pass 3 — external currency (sonnet) ====
+^ # ^ Finding ^ Disposition ^
+| R2.11 | **The EU half of the same blocker**: a joint Article 62 GDPR operation by the Dutch, Hungarian, Italian and Liechtenstein authorities published a final report on smart TVs on 2025-09-23 — a year before this page claimed the file was empty | **Accepted, BLOCKER.** Fetched and read the 14-page report. It is the single most useful external document on this page: **a regulator ran the measurement**, on three televisions, across first install, standby, off and ordinary use. Its off-state result (91–99% of flows to the OS provider) is now on the content page, and the fact that nobody in these seven venues has measured a television in the off state is now an open question. Neither European action uses the phrase "automatic content recognition", which is exactly why an ACR-shaped search could not see them |
+| R2.12 | **The round-1 Walmart fix is itself wrong** — the release does contain "VIZIO's Platform+ segment…"; the plus sign is HTML-entity-encoded, so a tag-strip search misses it | **Accepted — a second round-1 fix that introduced an error.** Reproduced: the string appears only after double entity-decoding. The footnote now quotes what the release says and records why the first check missed it. The reviewer flagged that its own first grep made the same mistake before its ''WebFetch'' caught it — an honest note that is worth more than a clean report |
+| R2.13 | The artifact table labels ''ahn2025_watch'' "restricted" while citing the DOI of the **open** record | **Accepted.** Zenodo's API says ''access_right: open''. The paper has two records; the label belongs to the other one. Both are now named |
+| R2.14 | FCC proceeding still pending, no Report and Order; HbbTV 2.0.5 still current; all five artifact repositories, the Zenodo DOI, mitmproxy v12.2.3 and every vendor doc re-verified | No action |
+| R2.15 | A trade-press claim that the FCC voted in May 2026 to mandate ATSC 3.0 tuners | **Rejected**, by the reviewer that surfaced it and again by the author: it appears nowhere on fcc.gov and the URL 403s. Recorded because this is the kind of claim that gets re-added |
+| R2.16 | One pass read the ATSC standards listing as showing A/344:2025-07 as the latest approved revision | **Rejected on better evidence.** The A/344:2026-04 PDF exists at HTTP 200 and its own title block reads ''A/344:2026-04 … 14 April 2026''. The document's own designation beats a reading of the listing page |
+==== Round 2, pass 4 — generic, no checklist (fable) ====
+This pass found the cluster the other three could not, because **a false negative has no figure to check and no citation to verify**. Five claims about what the literature does not do were wrong, and two were refuted by the provenance page's own printed residue.
+^ # ^ Finding ^ Disposition ^
+| R2.17 | "**LLM-based classification — Absent.** Not one paper in this population uses an LLM for anything." Two papers carry ''tools[]'' entries with ''category: llm'', ''used'' — and **both tool names are printed in this page's own residue block** | **Accepted, and the most embarrassing finding of the round.** {[girish2023_room]} and {[jakaria2024_dots]} both prompt OpenAI's Text Completion API to infer device vendor and category from DHCP hostnames. The claim is now the narrower and more useful one: an LLM is used here as a device-name resolver and has never been pointed at TV app metadata, store descriptions, ACR payloads or policies. **A residue you publish but do not read is not a control** |
+| R2.18 | "No paper states whether the device was factory-reset" was read off ''statefulness'' across the **6** papers that have a ''crawlConfig'' object and asserted over all **35**; {[tagliaro2023_still]} states it in so many words | **Accepted.** Classic denominator slip — the structured field is empty for the 6 that have it and silent for the other 29, which is not the same as those 29 saying nothing. A full-text probe is now in the script, and the page states both the field and the probe |
+| R2.19 | "**Exactly one paper** measures both sides of the opt-out." {[moghaddam2019_watching]} repeated its entire Roku and Fire TV crawl with "Limit Ad Tracking" and "Disable Interest-based Ads" enabled | **Accepted.** Two papers, five years apart, at different layers. Verified in both papers' own text |
+| R2.20 | "Almost nobody reports the hole — a probe matches **1** of 13" while the page **quotes two of the sentences the probe misses, three sections earlier** | **Accepted.** Probe width is a claim. The wide probe matches 3. The script now prints both widths and **asserts the narrow set is a subset of the wide one**, so a narrowing probe that returns more fails instead of publishing |
+| R2.21 | "There is no AndroZoo for TV apps… every paper built its own set." {[tileria2022_watch]}, the page's flagship TV-app paper, pulled its 4,745 APKs **from AndroZoo and APKMirror** | **Accepted.** The true statement is sharper: AndroZoo has no TV facet, so you cannot ask it for TV apps — you arrive with package names found elsewhere. For Roku, Tizen and webOS there is no archive at all |
+| R2.22 | The round-1 log claims four passes ran and that the generic one is "logged in §15 with every finding"; §15 logs three | **Accepted.** The generic pass was still running when the session ended. Corrected, and §15's own arithmetic (it said 11 accepted / 4 rejections against tables holding 14 and 3) corrected with it |
+| R2.23 | "The two vendors with the largest ACR businesses" is an unsourced market-size claim, and §10 says explicitly that installed-base share could not be established | **Accepted.** The two pages contradicted each other. Replaced with "two of the five vendors Texas sued", which is a fact on the record |
+| R2.24 | "Nobody has repeated…", "no TV paper has ever…", "Nobody has published…" — universal phrasing with no population, where the page elsewhere models the right form | **Accepted.** All three now name their population |
+| R2.25 | The lead says "the median TV study is a handful of physical devices"; the page's own medians are 66 and 75.5 | **Accepted.** The lead was describing the TV-specific papers while the median describes IoT testbeds. Both numbers are now given, with the distinction stated |
+| R2.26 | The table says fingerprintv is "promised, not yet available"; the prose two lines later counts "the five public repositories" | **Accepted.** The README still says the code "will be added… stay tuned", four years on, while the dataset is out. That is a better fact than either version, and it is now the row |
+| R2.27 | ''design:mobile_and_app_measurement'', ''privacy:requests'' and ''programming:filter_lists'' contain **no link back** to this page. A reader on the mobile page's pinning section has no signal that step 1 usually fails on a TV | **Accepted.** Reverse links added — see §16 |
+| R2.28 | ''\x27\x27sonnet\x27\x27'' in three §15 headings renders its own apostrophes, because DokuWiki headings ignore monospace markup | **Accepted.** Headings de-monospaced |
+| R2.29 | §5 reprints tables that §13 also carries in full (~150 duplicated lines) | **Rejected.** Deliberate, and stated as such: §5 is the readable verdict map and §13 is the unedited output. Someone checking a verdict should not have to scroll a 900-line block |
+| R2.30 | The page holds its stated boundaries in the outward direction, answers its own question, and the enforcement timeline belongs; rendered DOM matches source on both pages; 0 red links; both anchors resolve | No action. Recorded as the positive control |
+==== What round 2 cost and returned ====
+Four passes, **21 accepted findings**, 3 rejections. Two of round 1's own accepted fixes were wrong (R2.4, R2.12) and one had landed on only one page (R2.6) — which is the argument for re-running a reviewer whose domain you changed, rather than trusting that a fix was a fix.
+Three things are worth carrying to the next page:
+  - **A negative claim is the least-guarded thing on a page.** Every figure here survived independent re-derivation, twice. Five of the six worst defects were sentences saying nobody does something, and none of the three focused briefs could have caught them — the checklist reviewer checks what is there.
+  - **A published residue must be read, not just printed.** The two LLM papers were in this page's own residue block, in plain sight, while the content page said they did not exist.
+  - **A fix closes the failure it was shown, and nothing next to it.** Round 1's invariant stopped a tier flip and let a tier swap through. The only thing that found the difference was mutating the code again, with the same brief, after the fix.
+===== 16. Reverse links added on 2026-09-13 =====
+The round-2 generic pass checked the page's stated boundaries **in both directions** and found the outward direction good and the inward direction missing entirely: three pages this one defers to carried no link back, so a reader arriving at the neighbour had no signal that the TV case exists.
+^ Page ^ Where ^ What it now says ^ Revision ^
+| [[design:mobile_and_app_measurement]] | head of //The Certificate-Pinning Problem// | a ''WRAP tip'' noting that the whole section assumes you can install a CA, which on Roku, Tizen and webOS you cannot, and on Android TV means instrumenting the APK instead | 1789266443 |
+| [[privacy:requests]] | Related pages | where this page's instruments stop: no interception without network-level capture, no page context to attribute a request to, filter-list coverage measured at 22–27% | 1789266444 |
+| [[programming:filter_lists]] | the existing **Smart TVs** coverage row | why the rule syntax itself does not transfer — no URL, no page context, no element hiding | 1789266446 |
+That page had previously pointed its smart-TV row at [[design:website_classification]], which is not where a reader chasing that 22% figure needs to go.
 <bibtex bibliography></bibtex>