| Both sides previous revisionPrevious revision | |
| provenance:design:connected_tv [2026/09/13 02:28] – Round 2: re-embed the current script and its regenerated output; membership and topic digests replace the count-only invariant (5b); alias-fold 25 -> 27/22 and Roku RIDA casing, both of which round 1 only fixed on the content page; Q16 denominator 33 -> 3 karel.kubicek.claude | provenance:design:connected_tv [2026/09/13 02:30] (current) – Quote check: eight needles added for the claims round 2 changed (41 quotes, 0 NOTFOUND); re-embed the checker and its output; section 7 counts and the new-needle table. Authored by Claude karel.kubicek.claude |
|---|
| ''scripts/ctv_quotecheck.py'' matches every phrase either page quotes, or leans on for a figure, against **both** ''paper.cols.txt'' and the PDF text layer via pypdf. Both are needed, and this run proves why: | ''scripts/ctv_quotecheck.py'' matches every phrase either page quotes, or leans on for a figure, against **both** ''paper.cols.txt'' and the PDF text layer via pypdf. Both are needed, and this run proves why: |
| |
| * **28 of 33** located in both renderings. | * **36 of 41** located in both renderings. (It was 28 of 33 until 2026-09-13; round 2 added eight needles, all of which land in both renderings — see below.) |
| * **2 in ''.cols'' only** — the FingerprinTV DBF sentence and the Roku ECP URL, which the PDF text layer scrambles. | * **2 in ''.cols'' only** — the FingerprinTV DBF sentence and the Roku ECP URL, which the PDF text layer scrambles. |
| * **3 in the PDF only** — de-columning splices. The clearest is {[kumar2019_things]}: the sentence "the most popular vendor, Roku, only accounts for 17.4% of media devices" is spliced in ''.cols'' into "nd the most poputions of IoT device types, except when a device type accounts lar vendor, Roku, only accounts for 17.4% of media devices for fewer than 1% of devices". A ''.cols''-only check would have reported a correct quote as NOTFOUND. | * **3 in the PDF only** — de-columning splices. The clearest is {[kumar2019_things]}: the sentence "the most popular vendor, Roku, only accounts for 17.4% of media devices" is spliced in ''.cols'' into "nd the most poputions of IoT device types, except when a device type accounts lar vendor, Roku, only accounts for 17.4% of media devices for fewer than 1% of devices". A ''.cols''-only check would have reported a correct quote as NOTFOUND. |
| * **0 in neither.** | * **0 in neither.** |
| | |
| | **Eight needles were added on 2026-09-13**, for the claims the round-2 review changed. They are worth listing because each one existed //on the page// before it existed in this checker — which is the drift this file is supposed to prevent: |
| | |
| | ^ Needle ^ Why it was added ^ |
| | | ''to bypass certificate pinning'' ({[moghaddam2019_watching]}) | the widened TLS-decryption probe; the narrow probe missed this sentence while the page quoted it | |
| | | ''we cannot install our own self-signed certificates on the Roku'' ({[varmarken2020_smart]}) | same | |
| | | ''there is no way to install custom certificates on Android TV'' ({[tileria2022_watch]}) | same | |
| | | ''We perform a factory reset of the TV for each channel analysis'' ({[tagliaro2023_still]}) | refutes the page's old "no paper states whether the device was factory-reset". Note the needle stops where it does: the full sentence is **spliced in ''.cols''** by the de-columner, which interleaves it with "and the UT-100c HiDes modulator" | |
| | | ''download the last version of each app (as of August 2020) from AndroZoo'' ({[tileria2022_watch]}) | refutes "every paper built its own set" | |
| | | ''we randomly selected 1.5K unique package names from Androzoo'' ({[girish2023_room]}) | same | |
| | | ''published a candidate standard for hybrid TV in America'' ({[oren2014_aether]}) | the single real ATSC mention in all 5,859 papers | |
| | | ''this time enabling the "Limit Ad Tracking" … "Disable Interest-based Ads" … settings'' ({[moghaddam2019_watching]}) | refutes "exactly one paper measures both sides of the opt-out" | |
| |
| **One needle was genuinely wrong, and the check caught it.** The first draft asserted "decryption fails for 1 out of 5 (or fewer) TLS connections for 80% of all apps" against {[varmarken2020_smart]}. The paper says //"decryption fails for 1 out of 10 (or fewer) TLS connections for 55% of all apps; 1 out of 5 (or fewer) TLS connections for 80% of all apps"// — two clauses, and the draft had glued the opening of the first to the end of the second, producing a sentence the paper does not contain. The needle was narrowed to the clause that is actually there. This is the reason bare-number needles are avoided in that script. | **One needle was genuinely wrong, and the check caught it.** The first draft asserted "decryption fails for 1 out of 5 (or fewer) TLS connections for 80% of all apps" against {[varmarken2020_smart]}. The paper says //"decryption fails for 1 out of 10 (or fewer) TLS connections for 55% of all apps; 1 out of 5 (or fewer) TLS connections for 80% of all apps"// — two clauses, and the draft had glued the opening of the first to the end of the second, producing a sentence the paper does not contain. The needle was narrowed to the clause that is actually there. This is the reason bare-number needles are avoided in that script. |
| cols+pdf IMC 2011 understanding-couch-potatoes-measurement-and "The average number of set-top boxes provisioned was approximately " | cols+pdf IMC 2011 understanding-couch-potatoes-measurement-and "The average number of set-top boxes provisioned was approximately " |
| cols+pdf USENIX 2019 all-things-considered-an-analysis-of-iot-dev "are the most common type of device in seven of the eleven regions" | cols+pdf USENIX 2019 all-things-considered-an-analysis-of-iot-dev "are the most common type of device in seven of the eleven regions" |
| | cols+pdf CCS 2019 watching-you-watch-the-tracking-ecosystem-of "to bypass certificate pinning" |
| | cols+pdf PETS 2020 the-tv-is-smart-and-full-of-trackers-measuri "we cannot install our own self-signed certificates on the Roku" |
| | cols+pdf PETS 2022 watch-over-your-tv-a-security-and-privacy-an "there is no way to install custom certificates on Android TV" |
| | cols+pdf NDSS 2023 i-still-know-what-you-watched-last-sunday-pr "We perform a factory reset of the TV for each channel analysis" |
| | cols+pdf PETS 2022 watch-over-your-tv-a-security-and-privacy-an "download the last version of each app (as of August 2020) from And" |
| | cols+pdf IMC 2023 in-the-room-where-it-happens-characterizing- "we randomly selected 1.5K unique package names from Androzoo" |
| | cols+pdf USENIX 2014 from-the-aether-to-the-ethernet-attacking-th "published a candidate standard for hybrid TV in America" |
| | cols+pdf CCS 2019 watching-you-watch-the-tracking-ecosystem-of "this time enabling the "Limit Ad Tracking" (Roku) and the "Disable" |
| |
| 33 quotes: 28 in both renderings, 2 in .cols only, 3 in the PDF only, 0 in neither. | 41 quotes: 36 in both renderings, 2 in .cols only, 3 in the PDF only, 0 in neither. |
| </code> | </code> |
| |
| ('2019', 'USENIX', 'all-things-considered-an-analysis-of-iot-devices-on-home-networks', | ('2019', 'USENIX', 'all-things-considered-an-analysis-of-iot-devices-on-home-networks', |
| 'are the most common type of device in seven of the eleven regions'), | 'are the most common type of device in seven of the eleven regions'), |
| | # --- added 2026-09-13, for claims the round-2 review changed ------------- |
| | # The widened TLS-decryption probe: the three Tier A sentences the page now |
| | # quotes. The narrow probe matched only the second of these, which is how a |
| | # "1 of 13" reached the page while two of the three sat in its own prose. |
| | ('2019', 'CCS', 'watching-you-watch-the-tracking-ecosystem-of-over-the-top-tv-streaming-devices', |
| | 'to bypass certificate pinning'), |
| | ('2020', 'PETS', 'the-tv-is-smart-and-full-of-trackers-measuring-smart-tv-advertising-and-tracking', |
| | 'we cannot install our own self-signed certificates on the Roku'), |
| | ('2022', 'PETS', 'watch-over-your-tv-a-security-and-privacy-analysis-of-the-android-tv-ecosystem', |
| | 'there is no way to install custom certificates on Android TV'), |
| | # The factory-reset sentence. In .cols this one is SPLICED by the |
| | # de-columner — "and the UT-We perform a factory reset of the TV for each |
| | # channel analysis 100c HiDes modulator" — so only the first clause is |
| | # contiguous there. The PDF layer carries the whole sentence. |
| | ('2023', 'NDSS', 'i-still-know-what-you-watched-last-sunday-privacy-of-the-hbbtv-protocol-in-the-european-smart-tv-landscape', |
| | 'We perform a factory reset of the TV for each channel analysis'), |
| | # AndroZoo: the page said every TV-app paper built its own set. |
| | ('2022', 'PETS', 'watch-over-your-tv-a-security-and-privacy-analysis-of-the-android-tv-ecosystem', |
| | 'download the last version of each app (as of August 2020) from AndroZoo'), |
| | ('2023', 'IMC', 'in-the-room-where-it-happens-characterizing-local-communication-and-threats-in-s', |
| | 'we randomly selected 1.5K unique package names from Androzoo'), |
| | # The one real ATSC mention in the whole 5,859-paper corpus. |
| | ('2014', 'USENIX', 'from-the-aether-to-the-ethernet-attacking-the-internet-using-broadcast-digital-t', |
| | 'published a candidate standard for hybrid TV in America'), |
| | # Both sides of the opt-out: the paper the page had omitted. |
| | ('2019', 'CCS', 'watching-you-watch-the-tracking-ecosystem-of-over-the-top-tv-streaming-devices', |
| | 'this time enabling the "Limit Ad Tracking" (Roku) and the "Disable Interest-based Ads" (Amazon Fire TV) settings'), |
| ] | ] |
| |
