Measure The Web

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Provenance: privacy:advertising ======

Working log behind **[[privacy:advertising|Measuring Ad Targeting and Bidding]]**. Corpus-wide caveats (venue coverage, the selection funnel, provisional venue-years) are on [[literature:corpus]] and are not restated here. Citations use the shared [[literature:bibliography]]; the 18 entries this run added are listed under //Bibliography// below and belong to the content page, not to this log. No ''~~DISCUSSION~~'' block: comments belong on the content page.

Voice here is a working log. It is read by somebody checking a number.

===== The run =====

^ Item ^ Value ^
| Date | 2026-09-23 (single sitting, unsupervised drain run) |
| Item | //privacy:advertising (new) — measuring ad targeting and bidding//, chosen by Karel in the 2026-09-22 gap pass; not on the roadmap's Queued table |
| Corpus | ''data/extract/run1/extractions.jsonl'', 5,859 papers, 5,855 with ''paper.cols.txt''; 7 venues, 2010–2026; bibliographic index ''data/corpus2/.meta'', 16,864 records; screening labels ''data/labels/run1'' |
| Page status | **New page.** ''privacy:advertising'' did not exist (''dw.mjs info'': //121 The requested page (revision) does not exist//). ''sitemap.mjs'' on 2026-09-23 listed 189 pages and no red link to it. The neighbours were exported and read before writing — ''privacy:ads_txt'', ''design:algorithm_audits'', ''design:platforms:ad_archives'', ''privacy:cookie_syncing'', ''privacy:privacy_sandbox'', ''design:platforms'', ''privacy:cookies'', ''privacy:tcf_consent_strings'', ''privacy:browser_protection'' — and the page points to them for header bidding, the paired-profile design, ad archives, ID syncing and the Privacy Sandbox. It is a **creation**. It deliberately overlaps with ''privacy:ads_txt'' on **12** of that page's 16 measuring papers (the seven papers tagged ''bids'' here plus ATOM, which ''privacy:ads_txt'' lists under header bidding and this page tags persona/creatives; the two RTB-price papers; the two misinformation-funding papers); the other four — the ''ads.txt'' adoption and pooling studies, the piracy-site study, the MV3 study — are OUT here |
| Models | Page, scripts, verdicts and this log: Claude (Opus 5.5). External-facts pass: one ''sonnet'' sub-agent, instructed to fetch rather than recall and to log rejected sources; every load-bearing fact it returned was re-fetched by ''external_checks_advertising.sh'' before it reached the page. Review layer: see //Review// |
| Scripts added | ''scripts/adv_fold.mjs'' (probes, inclusion rule, 291 hand verdicts), ''scripts/report_advertising.mjs'' (+ ''-output.txt'', run with ''%%--list%%''), ''scripts/verify_advertising_figures.mjs'' (+ ''-output.txt''), ''scripts/external_checks_advertising.sh'' (+ ''-output.txt''), ''scripts/bib_additions_advertising.bib'', ''scripts/build_provenance_advertising.py'', ''scripts/provenance_advertising_prose.txt''; notes in ''notes/advertising_external.md'' |
| Guards run before publication | ''check_wrap.mjs'' (OK), ''check_tables.mjs'' (OK), ''check_page_numbers.mjs'' whole-page against the concatenation of the three outputs (OK after the fixes under //Mistakes caught//), ''bib_dedup_scan.py'' on a fresh export plus the additions (0 definite duplicates; the one candidate pair touching a new key, ''bashir2016_recommended'' / ''bashir2016tracing'', is two different papers), citekey resolution (every marker resolves) |
| Write path | ''node scripts/dw.mjs put'' (JSON-RPC) with ''%%--if-rev%%'' on every save |
| Accidental exposure | None. Credentials stayed in ''.env''; ''GH_TOKEN'' was used by the external-check script and tested for presence only with ''%%${GH_TOKEN:+x}%%'' |

===== Scope and judgement calls =====

^ Decision ^ Why ^ What a reasonable person might have done instead ^
| **An ad's visible behaviour is CONTENT; what its code collects is tracking** (written into the rule after the generic review) | The rule as first written did not separate MadDroid and the mini-games paper (IN: deceptive and aggressive ad behaviour) from Sneaky Piggy (OUT: ads' sensor access and leakage). The boundary keeps the three verdicts as they were and says why | Move Sneaky Piggy IN (71 → 72, creatives 29 → 30). Reasonable; declined because its question is a data flow, the subject of ''privacy:requests'' and ''privacy:javascript'' |
| **A new page, not a section on ''privacy:ads_txt'' or ''design:algorithm_audits''** | ''ads_txt'' is about who may sell inventory and how to read the auction; ''algorithm_audits'' is about arms and nulls. Neither owns the question //was this ad targeted, why, and at what price//, nor the explanation, donation, advertiser-side and crawl-collection observables, which have no home anywhere on the site | Extend ''algorithm_audits'' with an "ads" section. Rejected: 49 of this page's 71 papers are not audits (the 22 that are were checked against ''algorithm_audits_set.mjs''), and the explanation-reliability and dedup material is not a design question |
| **Population = "observes ads or auction outcomes and asks a targeting, price, delivery or content question"**, written before counting | The item asked for observables and their denominators; a rule keyed on the observable is the one that makes the denominator section possible | Restrict to targeting and price only (45 papers). Rejected: the crawl-collection and dedup material, which the item names, would then rest on papers outside the population. The page reports the 45 separately |
| **Ad-content papers (malvertising, political, children's ads) are IN** | They are where the collection, detection and dedup methods are reported; 20 of the 29 crawl-collecting papers ask only a content question | Send them to a malvertising page that does not exist. Rejected; flagged on the page as the content-only subset |
| **Ad-archive papers are IN, observable ''archive'', and pointed to ''design:platforms:ad_archives''** | An archive is one of the ways the literature observes ads; excluding it would make the observable table silently incomplete. None of their figures is restated here | Exclude them because another page owns them. Rejected: ownership is about prose, not about the count |
| **User-perception studies are CONTEXT, not population**, and mixed-method studies that also measure ads are IN | The item asked for a decision. A perception study measures the person; this page's denominators are ads, bids, explanations and campaigns. The four top-30 hits that recruited participants //and// measured ads (a data export, two extension panels, real users' proxied traffic) are IN on the strength of the measurement | Count every paper about ad targeting. Rejected: the page would then publish attitude percentages beside ad counts |
| **Ad fraud, ad blocking and ad injection are OUT** with codes | They observe ads only as the object of a fraud, a blocker or an extension; ''privacy:browser_extensions'' and ''programming:filter_lists'' own two of them | Include ad fraud as a //content// question. Rejected; recorded as 13 ''fraud'' verdicts |
| **Privacy Sandbox measurements are OUT (''sandbox'')** | Owned by ''privacy:privacy_sandbox''; the FLEDGE deployment crawl is cited there | Include the FLEDGE crawl as a targeting observable. Reasonable; declined to avoid a second account of the same paper |
| **Verdicts from title, summary and the extracted ''detection[]'' records**, with the full text read where those did not decide it (a handful: ''When and Why'', the logged-in/out crawl, CanaryTrap, the Meta CAPI study, the Ukraine/Russia and accessibility dedup rules) | 291 candidates in one sitting. The IN papers whose figures the page publishes are then re-read through the needle list, which checks the full text; the others rest on the summary and the detection records | Read every paper. Not done; a second reader would move a few fraud/tracking papers that observe ads incidentally |
| **Andreou et al., NDSS 2018 cited although outside the extraction** | Its venue-year has no screening labels at all, so its absence is a funnel artefact. It is the first bought-campaign test of an explanation interface ({[medjkoune2023_marketing]} is the other); the page would otherwise send a student to self-report studies for the reliability question. Its three quoted figures are checked against the NDSS PDF | Leave it out because the corpus did. Rejected |
| **Two surveys outside the seven venues cited** — {[boerman2017_oba]} (Journal of Advertising) and {[angus2024_donation]} (Computational Communication Research) | The spec asks for the canonical survey outside the venues. Both Crossref-verified. Neither is used for a figure | None; could be dropped without loss to any number |
| **The "What to Report" template sentence carries invented numbers** | It is a template, labelled as illustrative on the page, in the same form as ''design:algorithm_audits''. Its literals are listed in ''verify_advertising_figures.mjs'' so the number guard does not bless them as data | Write it with placeholders. Reasonable |
| **The explanation table mixes controlled checks and self-report checks**, and says which is which | Both are what the literature has; the direction of each check changes what it proves | Publish only the controlled ones. Rejected: the self-report ones are the only evidence on inferred-interest accuracy |
| **The Privacy Sandbox removal status is pointed to, not restated** | As of 2026-09-23 Chrome stable is M154 while the //Deprecate and Remove Protected Audience// entry is still //Proposed// with desktop milestone 153 (external checks). Resolving it belongs on ''privacy:privacy_sandbox'' | State the M153 target. Rejected: it has lapsed |

===== Probes and the candidate set =====

Written into ''adv_fold.mjs'' before the first count. All three run over ''paper.cols.txt'', whitespace-collapsed, case-insensitive, paper-counted.

^ Probe ^ Regex (summary) ^ Threshold ^ Papers ^
| GAP | the 2026-09-22 gap pass's ''ad_targeting_rtb'' family, verbatim | ≥ 5 hits | 127 (118 when run the way the gap pass ran it: ''paper.norm.txt'', no whitespace collapse) |
| FAM | six observable families (targeting, auction, explanation, donation, advertiser side, content), summed | ≥ 5 hits | 236 |
| ADW | ''ads'' / ''advertisement(s)'' / ''advertiser(s)'' | ≥ 40 hits | 157 |
| **union** | | | **291** |

The full regexes are in the ''adv_fold.mjs'' block at the foot of this page. ADW is the widest net and is where the homographs are: Bluetooth Low Energy "advertisements" (9 papers), ADS-B and ACARS, BGP route advertisements. The 118 → 127 difference is rendering plus whitespace: the gap pass read ''paper.norm.txt'' uncollapsed, and a PDF line break inside a phrase undercounts.

**Recall checks.** (1) Against ''design:algorithm_audits'': 22 of its 34 audits are IN here; the other 12 — search, news, ride prices, retail, email filtering, feeds, generative assistants, a data-export audit — are 9 non-candidates and 3 candidates coded OUT (two ''recommendation'', one ''dsr''). Printed by PASS B of the report. This is a consistency check, not an independent recall test: whether an audit is "about ads" was decided by the same reader. (2) Before the candidate rule was fixed, a first, discarded probe set was checked with a plain ads-word sweep (≥ 25 hits) over the papers it missed; that list surfaced Guha et al. 2010 and a dozen ad-content papers, which is why ADW is in the union. (3) PASS F below: a title probe over the //index// for ad measurements outside the extraction.

===== Queries, with their denominators =====

Each row is a figure or table on the content page and the report section that prints it.

^ # ^ Page figure ^ Population / denominator ^ Report section ^
| Q1 | 291 candidates; 71 IN (24.4%), 9 CONTEXT, 211 OUT; OUT codes | the union of the three probes | PASS B |
| Q2 | GAP 127 hits, 42 IN (33.1%), recall 42 of 71 (59.2%); FAM 67 of 71; ADW 64 of 71 | each probe's hits; the 71 | PASS B precision/recall table |
| Q3 | 29 IN papers GAP misses; 22 content-only; the other seven named | IN papers not in GAP | PASS B, PASS C |
| Q4 | top 30 GAP hits: 10 recruited participants, 4 of them IN; 23 of 127 pure perception (8 CONTEXT, 15 OUT:perception) | GAP hits ranked by hit count | PASS B |
| Q5 | bucket table (3 / 10 / 22 / 23 / 13; per year; per 1,000) | IN papers; corpus papers per bucket | PASS C |
| Q6 | venue shares (PoPETs 14 of 510 …) | that venue's papers in the extraction | PASS C |
| Q7 | platform, question tags, 45 core papers | the 71 (multi-valued) | PASS C |
| Q8 | observable table: papers, first–latest, per bucket | the 71 (multi-valued) | PASS C |
| Q9 | crawl-config reporting (statefulness 29 of 41 …) and baselines | IN papers with a ''crawlConfig'' object (41) vs all 1,080 with one | PASS D |
| Q10 | consent action not stated 16, no-interaction 15; accounts 6 of 41 | the 41 | PASS D |
| Q11 | vantage stated 37 of 67; not stated 30 of 67 (44.8%); headless not stated 32 of 41 (78.0%) | IN papers with a vantage tuple; the 41 | PASS D |
| Q12 | artifacts, statistics, ethics rows | the 71 empirical vs 5,118 empirical | PASS D |
| Q13 | 7 papers name an LLM, all 2024–2026 | the 71, ''tools[]'' used/produced | PASS D |
| Q14 | dedup: 9 state a rule, 6 counts only, 1 probe hit something else, 13 no hit | the 29 papers with the ''crawlads'' observable | PASS E / E2 |
| Q15 | EasyList named by 14 of 29 | the 29 | PASS E |
| Q17 | 18 of the 36 IN papers on a platform service have no crawl configuration | IN papers with ''platforms'' including ''other-online-service'' | PASS D |
| Q16 | 18 ad measurements outside the extraction, by reason (7 / 6 / 3 / 2) | title-probe hits in the index that are not extracted, hand-coded | PASS F |

===== Hand classifications, and their residue =====

This page has no name fold. What it has are three hand maps, each checked against the probe that produced its candidates **in both directions** — the report throws if a candidate has no verdict or a verdict matches no candidate:

  * ''VERDICTS'' in ''adv_fold.mjs'' — 291 entries, keyed by ''venue/year/slug-prefix''; each prefix must match exactly one candidate. IN entries carry observable and question tags and a note; OUT entries a code from ''OUT_CODES''. **Residue: 0** (no candidate without a verdict). Every verdict is printed in the report's ''%%--list%%'' section below.
  * ''DEDUP_VERDICT'' in ''report_advertising.mjs'' — one hand verdict per ''crawlads'' paper that matches the dedup vocabulary (16), with the deciding phrase. **Residue: 0**; a stale verdict (a paper that no longer matches) also throws.
  * ''OUTSIDE'' in ''report_advertising.mjs'' — a code for each of the 124 not-extracted title-probe hits, read from the title only, plus one forced paper the title probe cannot see (Olejnik et al., NDSS 2014: no "ad" token in its title). One code is ''U'' (//cannot be decided from the title//) and is not counted as a measurement. **Residue: 0.**

**Observable and question tags are hand-coded, one reader, and the least stable part of the audit.** A second reader would most likely move ''explanation'' vs ''donation'' on papers that use an extension to collect both, and ''targeting'' vs ''content'' on the children's-ads papers.

===== Quotes and figures checked against the papers =====

Every per-paper figure and quoted sentence on the content page is a needle in ''verify_advertising_figures.mjs'': 96 needles, looked up whitespace-collapsed and ligature/dash/tilde/×-folded in ''paper.cols.txt'', then ''paper.norm.txt'', ''paper.txt'' and a ''pypdf'' re-extraction of ''paper.pdf''. **96 located, 0 not located**: 88 in ''.cols'', 5 only in the PDF (Venkatadri 2019's noise sentence, Sunlight's impression count, the second half of a spliced Vadrevu 2019 sentence, Cook 2020's lean towards intentional underbidding, Datta 2015's credit to an earlier study), 3 in the NDSS 2018 PDF of the one non-corpus paper. Three mutation controls — the same needles with one figure changed (1.7× → 1.9×, 98.9% → 98.1%, 169,751 → 196,751) — are correctly **not** located. Spliced sentences are checked as two verbatim halves and labelled //(1/2)//, //(2/2)//. Needles were not taken from ''evidence.quote'': the extraction's ''prevalence'' strings are model summaries. One of them omitted what the figure needs: ''detection[].prevalence'' for {[chen2026_when]} says //gender accuracy 59.13%// with no baseline, and the paper's own table has a **random-control row at 58.80%** on the same metric. The page publishes both.

The full list, with the rendering each needle was found in, is the verify output below.

===== External sources =====

Found by a ''sonnet'' sub-agent told to fetch every claim and log rejected sources (''notes/advertising_external.md''), then **re-fetched by ''external_checks_advertising.sh''** — its output, below, is the verification. Every row passed on 2026-09-23.

^ Claim on the page ^ Primary source ^ Verified how ^
| DSA Art. 26(1)(d), 26(3), 28(2) wording | EUR-Lex, Regulation (EU) 2022/2065, ''eur-lex.europa.eu/eli/reg/2022/2065/oj/eng'' | phrase match; EUR-Lex answers some clients with a 202 bot page, so the script asks for HTML and retries |
| DSA dates (VLOPs August 2023, everyone 17 February 2024) | European Commission, DSA enforcement page | phrase match on the general date; the VLOP date is the page's "four months following notification" after the 25 April 2023 designation |
| TCF v2.2 (2023-05-16), consent the only basis for purposes 3–6; TCF v2.3 (2025-06-19), transition ended 2026-02-28 | IAB Europe's two announcement pages | phrase match. The sub-agent's note called the v2.3 transition "still open as of today"; 2026-02-28 is before 2026-09-23 — **rejected**, the page says it ended |
| Google My Ad Center, 2022-10-20; Ad Settings kept for signed-out users | blog.google | phrase match. **Not found:** any Google wording that My Ad Center //replaces// Ad Settings or "Why this ad" — the page therefore does not say "replaced" |
| Meta removed sensitive detailed-targeting options from 2022-01-19 | Meta Business Help Center article 458835214668072, via a 2023-09-19 Wayback capture (the live page refuses non-browser clients) | phrase match. The widely reported announcement date (November 2021) is **not** on any fetched Meta page and is not on the content page |
| Meta EU subscription (November 2023) and "less personalized ads" (2024-11-12) with its data points | about.fb.com | phrase match |
| Meta ended political, electoral and social-issue ads in the EU, 2025-10-06 | about.fb.com | phrase match; not on the content page (''design:platforms:ad_archives'' owns it) — kept as a check because the ad-archive row points there |
| Meta disabled NYU Ad Observatory accounts, 2021-08-04; Ad Observer still distributed; Australian Ad Observatory "Active"; Who Targets Me extension offered | about.fb.com; adobserver.org; admscentre.org.au; whotargets.me | phrase matches |
| Google Authorized Buyers: the winning price macro returns an encrypted price | developers.google.com | phrase match |
| Chrome keeps third-party cookies (2025-04-22); Privacy Sandbox retirement (2025-10-17) | privacysandbox.google.com | phrase match |
| Safari full third-party cookie blocking (2020-03-24); Firefox Total Cookie Protection by default (2022-06-14) | webkit.org; blog.mozilla.org via a 2022-10-20 Wayback capture | phrase match. **The live Mozilla post has been retitled** "…to //more// users worldwide" with an //Updated Aug. 28, 2024// note; the page says "partitioned them by default since 2022" and does not repeat "all users" |
| TCF Specifications v2.4 published 2026-07-23; CMP web deadline 2026-10-23, apps/CTV 2027-02-23 | IAB Europe notice PDF of 2026-07-16 | text match in a pypdf extraction. Added after the external-currency reviewer found it; the v2.3 page the first pass checked does not mention v2.4 |
| Meta's EU ad choice relaunched in January 2026 after a DMA non-compliance finding | European Commission, DMA site, 2025-12-08 | phrase match. Added after review. The €200m figure the reviewer cited from the April 2025 decision is **not** on the page: it was not fetched from a primary source in this run |
| Tool and dataset liveness | GitHub API (''-L'', so renames resolve), HTTP status after redirects | ''tadatitam/info-flow-experiments'' → ''dattaamit/…''; ''Breezelled/when-ads-become-profiles'' → ''cruiseresearchgroup/…''; ''seemoolab'' is ''seemoo-lab''; ''targeted-andtroublesome'' is ''targeted-and-troublesome''. **Dead:** ''Camq543/YoutubeAds'' (404, no successor found), HBDetector and HARPO (as on ''privacy:ads_txt''). **Down:** ''fdvt.org'' (no connection), ''adanalyst.mpi-sws.org'' (301 to a host that does not answer) |
| Five DOIs for the added NDSS/PoPETs entries; two survey DOIs | api.crossref.org | title and author families printed |

**Rejected sources** (from the sub-agent's log; used, if at all, only to find a primary URL): smartframe.io, searchengineland.com, hookagency.com, foreplay.co (My Ad Center); socialmediatoday.com, jonloomer.com, reshiftmedia.com, medium.com/trapica, adsuploader.com, falcondigital.co.uk, leadsync.me, cadeproject.org, fedma.org, seekingalpha.com, tradingview.com, aljazeera.com, euronews.com (Meta); techpolicy.press, algoodbody.com, twobirds.com, lexology.com, pwc.nl, eu-digital-services-act.com (DSA); secureprivacy.ai, usercentrics.com (TCF); natlawreview.com, kevel.com, cookie-script.com, jentis.com, didomi.io, onetrust.com, segwise.ai, martech.org, adexchanger.com, ghacks.net, computeruser.com, leapbuzz.com, grokipedia.com, en.wikipedia.org (Privacy Sandbox); nyunews.com, Bloomberg, Inside Higher Ed, Engadget, the Cybersecurity for Democracy Medium blog, an NYU press release (Ad Observatory). Reason in every case: secondary, vendor or aggregator.

===== What could not be established =====

  * **Whether any perceptual-hash dedup rule is accurate on display or video ads.** The only measured identity-rule error is Guha et al.'s 2010 text-ad table. Stated as an open question.
  * **The size of the IP-class effect on a targeting outcome.** One paper observed datacentre and Tor addresses being served differently; none measured it.
  * **Whether the 9-of-29 dedup figure is a floor.** A paper that describes its dedup without the probe's words is not seen; the 13 no-hit papers were not read end to end.
  * **The screening loss beyond the title probe.** 18 is from titles only; an ad measurement whose title says nothing about ads is invisible to PASS F, as Olejnik et al. 2014 was until a population paper's references named it.
  * **What the 2026 LLM-based YouTube detector found** (TheWebConf 2026, no abstract in the index, no full text here). Named by title on the provenance side only.
  * **A dedicated Chrome Platform Status removal entry for Topics** — the sub-agent found none; the page points to ''privacy:privacy_sandbox'' for removal status.
  * **DSA Article 40 researcher data access.** The external-currency reviewer reported the delegated act as in force since 2025-10-29, from secondary sources only; not re-fetched from the Commission and not on the page. It is the obvious next route to platform ad-delivery data and belongs in the next revision once verified.
  * **The current wording of Meta's EU ad options** after the January 2026 relaunch: Meta's help pages refuse non-browser clients.
  * **Whether header-bidding auctions other than Google Ad Manager's are first-price.** The page states the Google change (2019, primary source) and that the rendered winning bid is a price //where// the final auction is first-price; it does not generalise.

===== Mistakes caught during the run =====

^ What ^ How it was caught ^ Fix ^
| The report script held every collapsed full text in memory and died with a V8 heap OOM | first run | texts read one at a time; only candidates' texts kept |
| Six first-pass verdicts named papers that were not candidates (they had passed an earlier, discarded probe) | ''resolveVerdicts'' threw: //prefix … matches 0 candidates// | removed; the guard is what the report now runs |
| One outside-extraction hit had no code (//Retargeted-Architecture Binary Code Analysis//, a homograph) | the report threw | coded ''H'' |
| The page's bucket header used two-digit labels (''22–24''), and the number guard matched ''17'', ''18'', ''21'', ''25'', ''26'' to unrelated numbers in the output and flagged only ''24'' | ''check_page_numbers.mjs'' | four-digit labels; a reminder that the guard proves a digit string appears, not that it means the same thing |
| "**45%**" and "**78%**" on the page against ''44.8%''/''78.0%'' in the output; the 45 passed the guard by matching an unrelated 45 | reading the guard output against the report | page now prints the report's own figures, and the report prints the complements |
| The first draft said the GAP probe's 29 misses were //all// content or advertiser-side papers | re-counting in the report | 22 content-only; the other seven include Guha 2010 and AdFisher — now named |
| The first draft said //two YouTube child-ad studies// and //a political-ad detection study// were screened out | PASS F's list | one child-ad study plus one data-cost study; two political-ad studies |
| The first draft said the explanation checks found interfaces //incomplete in every study below//; the self-report checks measure relevance, not completeness | rereading the table | "found wanting in every check below", and the table's two kinds of check are named |
| The first draft said //most// studies read landing pages without clicking — no measurement behind it | rereading | replaced by the one paper that says so |
| The first draft said {[agarwal2020_stop]} priced //personas//; the paper's W<sub>L</sub> / W<sub>R</sub> are left- and right-leaning //websites//, and its prices are for ads its personas were shown there | reading the paper for the needle | sentence rewritten |
| Two needles failed on the first verify run for folding reasons (''×'' vs ''x''; a line-break hyphen inside //impressions//) | ''verify_advertising_figures.mjs'' | fold extended; mutation controls added so a too-loose fold cannot pass |
| The external-check script counted a final HTTP 301 after ''-L'' as OK — it means the redirect target never answered | reading AdAnalyst's row | only a final 2xx is OK now; AdAnalyst is reported down |
| The sub-agent's "TCF v2.3 transition still open" | date arithmetic | rejected, see //External sources// |
| //The Cost of Annoying Ads// was CONTEXT although its object is ad formats, not targeting — the CONTEXT rule as written excludes it | generic review | re-coded OUT ''perception''; CONTEXT 9 → 8, OUT 211 → 212, perception 26 → 27, on the page and here |
| EUR-Lex answered the external-check script with a 202 on one run of three | the script's own FAILED lines | the regulation is fetched once, from two renderings with retries, and all three articles are checked in that copy |

===== Other pages touched in this sitting =====

  * **[[:privacy]]** — row added to the namespace table, count 18 → 19 in the box and in //Methodology//.
  * **[[:roadmap]]** — a row in //Assessed, and not written// for this page, and the //Sock-puppet and personalization audits// row settled (see below).
  * **[[provenance:privacy]]** — a dated note of the count change.
  * **[[literature:bibliography]]** — 18 entries appended before ''</bibtex>'' on a fresh export, saved with ''%%--if-rev%%''.

**The sock-puppet row.** The roadmap still carried //"Sock-puppet and personalization audits — 20 papers … likely a section on Programming:Stateful Stateless"//. That topic was written on 2026-09-11 as [[design:algorithm_audits]] (34 audit papers; re-run of ''report_algorithm_audits.mjs'' on 2026-09-23 reproduces 34), and the ad-specific half is now this page (22 of the 34 audits are in this page's population). The row was rewritten to say it is settled on those two pages. The 20 was not re-derived; it was a title probe from an earlier gap pass and is superseded by both pages' own populations.

===== Post-publication checks, 2026-09-23 =====

^ Check ^ Result ^
| ''literature:bibliography'' | 18 entries appended on a fresh export (rev 1789634327 → 1790126248), 1,057 → 1,075 entries, ''bib_dedup_scan.py'' 0 definite duplicates; cache purged |
| Rendered content page (after ''?purge=true'') | 22 source headings + the table of contents rendered, 11 tables, **56** references in ''bibtex_references'' for 56 distinct keys, 0 "could not be found", 0 red links (''wikilink2'') |
| Rendered provenance page | 18 source headings + table of contents, 5 %%<file>%% blocks, 9 references, 0 red links |
| Published code round trip | ''/_export/code/…?codeblock=3'' (''adv_fold.mjs'') and ''codeblock=0'' (report output) identical to the files apart from DokuWiki's trimmed leading/trailing newline |
| ''report_namespace_overviews.mjs'' | exit 0; ''privacy'': 19 pages, 19 table rows, equal as sets |
| Red links on the four other pages edited | 0 on ''privacy'', ''roadmap'', ''provenance:roadmap'', ''provenance:privacy'' |
| ''sitemap.mjs'' | exits 1 on the 21 pre-existing namespace-relative bare links in ''provenance:*'' namespace pages (none on pages touched here); its page listing is cached and still shows 189 pages |

===== Review =====

Four reviewers, each told that the author's context might not be exhaustive and each handed the page, this log's draft, the report script with its unedited output, the verifier with its output, and the external-check script with its output. The three focused passes ran in parallel on the first complete draft (not yet published); every accepted fix was applied, all three scripts were re-run, and the guards re-passed, before the generic pass. Findings files: ''notes/advertising_review_figures.md'', ''notes/advertising_review_citations.md'', ''notes/advertising_review_external.md'', ''notes/advertising_review_generic.md''.

^ Reviewer ^ Finding ^ Action ^
| figures-vs-script (sonnet) | Re-ran the report (''%%--list%%'') and the verifier: both byte-identical to the committed outputs. Every aggregate table cell on the page matched | — |
| figures-vs-script (sonnet) | //Imprecision.// The longitudinal 76.38% for {[chen2026_when]} is a variant given extra cultural context; the plain model is 74.88% | **Accepted.** Both figures now on the page, with the random control |
| figures-vs-script (sonnet) | //Imprecision.// The identity-rule table showed 8 of the 9 papers the report marks //states//; {[zeng2021_polls]} (MinHash-LSH over ad text) was missing | **Accepted.** Row added; the report's hand-verdict note for it rewritten to name the method |
| figures-vs-script (sonnet) | //Nit ×2.// "150 publishers" and "blank frame in 7.5%" had no needle | **Accepted.** Needles added |
| figures-vs-script (sonnet) | //Nit.// The advertiser-side observable had no worked example in the prose | **Accepted.** New subsection //Buying the ad//, every figure an existing needle |
| figures-vs-script (sonnet) | Latent: ''inferential()'' does not call ''isSentinel''; no sentinel occurs in ''statistics[].kind'' in this corpus | Noted, not changed (the same definition as ''report_algorithm_audits.mjs'', deliberately) |
| citations-and-quotes (sonnet) | All markers resolve; no collisions; no duplicate papers under other keys; the 18 entries' metadata verified against Crossref / landing pages | — |
| citations-and-quotes (sonnet) | //Defect.// "14 match no dedup vocabulary at all" folded in the one paper whose probe hit was a prior tool it declined to use | **Accepted.** Now 9 / 6 / 1 / 13, the report's own four buckets |
| citations-and-quotes (sonnet) | //Imprecision.// González Cabañas's compound surname split wrongly in both new entries, inconsistent with the existing ''caravaca2024_overprofiling'' | **Accepted.** ''González-Cabañas, José'' in both |
| citations-and-quotes (sonnet) | //Imprecision.// Guha 2010's 13–68% / 0–7% came from one of Table 2's two datasets; across both it is 10–69% / 0–12% | **Accepted.** Page gives both datasets' range; the first-row cells are now a needle |
| citations-and-quotes (sonnet) | //Imprecision.// "An address given for login alerts" read as a postal address; the paper tested an email address and a phone number | **Accepted** |
| citations-and-quotes (sonnet) | //Imprecision.// "first measured proof" for {[datta2015_automated]}; the paper credits an earlier manual study | **Accepted.** "first statistically rigorous evidence", with the credit; the paper's sentence is a needle |
| citations-and-quotes (sonnet) | //Imprecision.// Two phrases about {[moti2024_targeted]} were in quotation marks but were paraphrases | **Accepted.** Quotation marks removed, figures kept |
| citations-and-quotes (sonnet) | //Nit.// {[cook2020_headerbidding]} leans towards intentional underbidding after first calling the question undecidable; "slot" is not a calendar effect | **Accepted.** Both rewritten, each confound attributed to its own paper |
| citations-and-quotes (sonnet) | //Nit.// The browser-policy dates in the currency table were the one row without a primary-source footnote | **Accepted.** Footnote added, with the Mozilla retitling |
| citations-and-quotes (sonnet) | //Nit.// {[medjkoune2023_marketing]} reports both 3,321 and 3,221 ads; the page's 3,221 is the analysis-section figure behind its percentages | Noted; no change |
| citations-and-quotes (sonnet, sub-checks it fanned out) | //Defect.// The explanation table said the 98.9% came from "profiles with known histories, plus the repositories"; it is a granularity analysis of real users' notices collected by Who Targets Me | **Accepted.** Row rewritten and reclassified: the lead-in now says three rows are controlled, one measures granularity, three are self-report |
| citations-and-quotes (sonnet, sub-checks) | //Defect.// The Medjkoune row attributed the 76% / 25% to the seven own campaigns; they come from the six profiles' 3,221 organic ads, the campaigns establishing what a placement explanation means | **Accepted.** Row rewritten |
| citations-and-quotes (sonnet, sub-checks) | //Defect.// BlueKai was listed among the platforms behind the 27%; the paper collected BlueKai profiles but did not have them rated | **Accepted.** Row names three platforms; the exclusion sentence is a needle |
| external-currency (sonnet) | //Stale.// TCF Specifications v2.4 were published 2026-07-23 with a CMP web deadline of 2026-10-23; the page presented v2.3 as the latest | **Accepted and re-verified** from IAB Europe's PDF notice by the external-check script |
| external-currency (sonnet) | //Stale.// Meta relaunched the EU ad choice in January 2026 after a DMA non-compliance finding; the page cited only the 2024 option | **Accepted and re-verified** from the Commission's DMA page. The reviewer's €200m fine figure was not added: not fetched from a primary source here |
| external-currency (sonnet) | //Nit, lead only.// DSA Article 40 researcher data access (delegated act reported in force 2025-10-29) is not mentioned | **Deferred.** The reviewer had only secondary sources; recorded under //What could not be established// |
| external-currency (sonnet) | Protected Audience / Topics removal has slipped (M154 stable, removal entry still //Proposed// at M153); the page's "being withdrawn" is accurate | No change here; the slippage belongs on ''privacy:privacy_sandbox'' |
| generic (fable) | **Defect (F1).** "The only field measurement … is self-report" contradicted the page's own 12%-retargeting figure from a donated panel two sections later | **Accepted.** Both measurements named, by kind |
| generic (fable) | F2: the opening box said 98.9% "still" cite only the main form — the under-reporting reading the body rules out — and stated the 9-of-29 dedup count without its probe scope | **Accepted** — "coarse before you ask whether they are true"; "only 9 are found … a probe-scoped floor" |
| generic (fable) | F3: "Seventy-one papers … observe ads" omitted clause (B) of the rule | **Accepted** |
| generic (fable) | F4: "flat … a steady literature, not a growing one" is a trend claim on 10–23 papers a bucket | **Accepted** — now "not a trend in either direction" |
| generic (fable) | F5: "historical elsewhere" for retargeting outside Chromium rests on no measurement and ignores the hashed-email and server-side routes the page itself names | **Accepted** — "current; browser-dependent, and unmeasured outside Chromium" |
| generic (fable) | F6: seven negative universals without "in this corpus" | **Accepted** — qualified in place, and the Open Questions box opens with the seven-venue caveat |
| generic (fable) | F7: the LLM row generalised over seven papers | **Accepted** — two validate against humans, one extracts, one is the finding against a random control |
| generic (fable) | F8: "every study here that checked one against a known truth" headed a table in which four of seven rows are not such checks | **Accepted** |
| generic (fable) | F9: the page said bids cannot show what was paid while citing a paper for the price ads "cleared at"; first-price auctions and the archives' spend ranges were never mentioned | **Accepted.** Bids vs rendered winning bid separated; Google Ad Manager's 2019 move to first-price auctions added from Google's own post (re-fetched by the external-check script); archives' spend ranges pointed to. Not generalised beyond Google — recorded under //What could not be established// |
| generic (fable) | F10: "two orders of magnitude" stated as a property of crawlers; ''privacy:ads_txt'' hedges it as a cross-paper comparison | **Accepted** |
| generic (fable) | F11: this log described the four IN participant papers wrongly | **Accepted** |
| generic (fable) | F12: "disproportionately outside the extraction" had no base rate | **Accepted** — "at least a fifth" |
| generic (fable) | F13: the Sneaky Piggy / MadDroid / mini-games boundary was not in the written rule | **Accepted as a rule clause**, verdicts unchanged; recorded under //Scope and judgement calls// with the alternative |
| generic (fable) | F14: //The Cost of Annoying Ads// fails the CONTEXT rule | **Accepted** — re-coded OUT, three counts updated |
| generic (fable) | F15: the algorithm-audits recall check was prose only, and circular as phrased | **Accepted** — now in PASS B; rephrased as a consistency check |
| generic (fable) | F16: "its eight header-bidding papers" adopted the classification of ATOM used on ''privacy:ads_txt'' | **Accepted** in this log |
| generic (fable) | F17: donation called "least reliable" without a measurement | **Accepted** — "both rest on the platform's own account" |
| generic (fable) | F18: "mostly do not run a crawler" unmeasured | **Accepted** — computed in PASS D: 18 of 36 |
| generic (fable) | F19: "all three live" for donation panels overstated an HTTP-200 check | **Accepted** |

After the generic pass's fixes the three scripts were re-run (outputs below are from that run), and ''check_page_numbers.mjs'', ''check_wrap.mjs'' and ''check_tables.mjs'' re-passed on the content page. The focused reviewers were **not** re-run on the post-generic revision; the changes they did not see are the rows above, each re-checked against the scripts' outputs by the author.

===== Report output, unedited =====

''node scripts/report_advertising.mjs %%--list%% > scripts/report_advertising-output.txt'' — deterministic, ~25 s.

<file text report_advertising-output.txt>

=== PASS A — candidate probes (full text, paper-counted) ===
papers in extraction: 5859; with paper.cols.txt: 5855; missing: 4
probe                                  rendering                             threshold  papers
-------------------------------------  ------------------------------------  ---------  ------
GAP (2026-09-22 gap pass, as run)      paper.norm.txt, raw                   >= 5       118
GAP (same regex)                       paper.cols.txt, whitespace-collapsed  >= 5       127
FAM (six observable families, summed)  paper.cols.txt, collapsed             >= 5       236
ADW (ads|advertisement|advertiser)     paper.cols.txt, collapsed             >= 40      157
UNION = candidate set                                                                   291
(paper.norm.txt present for 5859 papers)
GAP only: 7; FAM only: 67; ADW only: 48; in all three: 60

Per-family reach (papers with >=1 / >=5 hits, of the full-text papers):
family       >=1  >=5
-----------  ---  ---
targeting    536  136
auction      159  43
explanation  130  28
donation     39   12
advside      65   17
content      209  56

=== PASS B — hand verdicts over the candidate set ===
A candidate is IN if both hold:
 (A) the paper's own data includes ads or ad-auction outcomes delivered to a
     measuring identity or to real people: served creatives or impressions,
     landing pages, bids or clearing prices, the platform's ad explanations /
     ad-preference inferences / advertiser lists, or delivery and reach
     statistics for ads the researchers bought (an ad archive counts, and is
     pointed to design:platforms:ad_archives);
 (B) it uses that data to answer a question about TARGETING (whether and why
     an ad reached this identity, or what the platform inferred), PRICE (what
     an advertiser bid or paid), DELIVERY (how the platform distributed an ad
     the researchers bought), or CONTENT (what the served ads were, and where).
     An ad's visible behaviour towards the user (aggressive, deceptive,
     redirecting) is CONTENT; what the ad's code collects and sends is
     tracking, and OUT (boundary written down after review, 2026-09-23).
A candidate is CONTEXT if it is a user study whose object is ad targeting,
explanations or ad settings but whose data answers a perception question only.
Everything else is OUT, with a code saying where it belongs.

candidates: 291; IN 71 (24.4%); CONTEXT 8 (2.7%); OUT 212 (72.9%)

OUT, by code (papers):
code            papers  meaning
--------------  ------  -------------------------------------------------------------------------------------------
tracking        53      tracking, syncing, consent or SDK data flows with no ad observed
homograph       41      "advertisement"/"bid"/"target" in another sense (BLE, ADS-B, auctions, Android intents, ML)
perception      27      user study with no ad observation (and not about ad targeting)
mention         20      advertising mentioned in passing; the paper is about something else
blocking        16      ad blocking, filter lists, anti-adblock: ads counted only as blocking efficacy
fraud           13      ad or click fraud measured from logs, apps or botnets — not targeting, price or content
mechanism       9       auction / allocation / CTR / campaign-optimisation design on platform-internal logs
scam            8       the ad is only the discovery channel for a scam / cloaking / SEO study
dsr             8       data-export (right-of-access) audit — privacy:data_subject_rights
sandbox         6       Privacy Sandbox API analysis or simulation — privacy:privacy_sandbox
defence         3       privacy-preserving ad system, detector or obfuscator evaluated without served-ad outcomes
supplychain     2       ads.txt / sellers.json supply chain only — privacy:ads_txt
recommendation  2       personalisation audit of search/news/recommendations, not ads — design:algorithm_audits
injection       2       ads injected by extensions or malware — privacy:browser_extensions
sponsorship     2       influencer sponsorship inside content, not an ad system

Precision and recall of each probe against the hand audit:
probe            hits  IN  precision  CONTEXT  recall of the IN set
---------------  ----  --  ---------  -------  --------------------
GAP >= 5 (cols)  127   42  33.1%      8        42 of 71 (59.2%)
FAM >= 5         236   67  28.4%      8        67 of 71 (94.4%)
ADW >= 40        157   64  40.8%      6        64 of 71 (90.1%)
UNION            291   71  24.4%      8        71 of 71 (100.0%)
IN papers found by one probe only: GAP 0, FAM 2, ADW 4
IN papers the GAP probe misses (found by FAM or ADW only):
  IMC/2010/challenges-in-measuring-online-advertising-systems
  WWW/2010/analyzing-content-level-properties-of-the-web-adversphere
  CCS/2012/knowing-your-enemy-understanding-and-detecting-malicious-web-advertising
  IMC/2014/the-dark-alleys-of-madison-avenue-understanding-malicious-advertisements
  USENIX/2014/brahmastra-driving-apps-to-test-the-security-of-third-party-components
  PETS/2015/automated-experiments-on-ad-privacy-settings
  IEEE-SP/2018/privacy-risks-with-facebooks-pii-based-targeting-auditing-a-data-brokers-adverti
  USENIX/2018/unveiling-and-quantifying-facebook-exploitation-of-sensitive-personal-data-for-a
  WWW/2018/exposing-search-and-advertisement-abuse-tactics-and-infrastructure-of-technical
  IMC/2019/what-you-see-is-not-what-you-get-discovering-and-tracking-social-engineering-att
  WWW/2019/revisiting-mobile-advertising-threats-with-madlife
  IEEE-SP/2020/a-security-analysis-of-the-facebook-ad-library
  IMC/2020/when-push-comes-to-ads-measuring-the-rise-of-malicious-push-advertising
  WWW/2020/maddroid-characterizing-and-detecting-devious-ad-contents-for-android-apps
  IMC/2021/polls-clickbait-and-commemorative-2-bills-problematic-political-advertising-on-n
  USENIX/2022/an-audit-of-facebooks-political-ad-policy-enforcement
  WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion
  WWW/2023/propaganda-politica-pagada-exploring-u-s-political-facebook-ads-en-espanol
  WWW/2023/the-thin-ideology-of-populist-advertising-on-facebook-during-the-2019-eu-electio
  WWW/2023/are-mobile-advertisements-in-compliance-with-apps-age-group
  WWW/2023/who-funds-misinformation-a-systematic-analysis-of-the-ad-related-profit-routines
  IMC/2024/analyzing-the-in-accessibility-of-online-advertisements
  IMC/2024/beyond-the-guidelines-assessing-metas-political-ad-moderation-in-the-eu
  PETS/2024/client-side-and-server-side-tracking-on-meta-effectiveness-and-accuracy
  NDSS/2025/careful-about-what-app-promotion-ads-recommend-detecting-and-explaining-malware-promotion-via-app-promotion-graph
  WWW/2025/assessing-compliance-in-digital-advertising-a-deep-dive-into-acceptable-ads-stan
  WWW/2025/before-after-the-effect-of-eus-2022-code-of-practice-on-disinformation
  USENIX/2026/when-fun-turns-toxic-a-first-look-at-aggressive-advertising-in-mini-games
  WWW/2026/when-ads-become-profiles-uncovering-the-invisible-risk-of-web-advertising-at-sca
GAP top 30: IN 20, CONTEXT 3, OUT:perception 2, user-perception total 5 (16.7%)
GAP top 60: IN 34, CONTEXT 7, OUT:perception 6, user-perception total 13 (21.7%)
GAP top 127: IN 42, CONTEXT 8, OUT:perception 15, user-perception total 23 (18.1%)
GAP >= 5 hits that recruited human participants (any role): 51 of 127 (40.2%); of those, IN 12, CONTEXT 8, OUT 31
GAP top 30 that recruited participants: 10 of 30; IN 4

CONTEXT papers (reading list, not population):
  USENIX/2017/exploring-user-perceptions-of-discrimination-in-online-targeted-advertising — scenario survey on discriminatory targeting
  CCS/2019/oh-the-places-youve-been-user-reactions-to-longitudinal-transparency-about-third — reactions to tracking inferences; interests inferred by the authors, not by a platform
  IEEE-SP/2023/when-and-why-do-people-want-ad-targeting-explanations-evidence-from-a-four-week — 4,251 ads logged, but the question is when people want an explanation
  PETS/2023/surprised-shocked-worried-user-reactions-to-facebook-data-collection-from-third — reactions to Off-Facebook Activity
  PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing — design probe; interests inferred by the authors
  IEEE-SP/2025/characterizing-the-usability-and-usefulness-of-u-s-ad-transparency-systems — usability of 22 ad-transparency systems
  PETS/2025/misalignments-and-demographic-differences-in-expected-and-actual-privacy-setting — Facebook settings incl. ad profile, as a perception study
  PETS/2025/privacy-perceptions-and-behaviors-towards-targeted-advertising-on-social-media-a — cross-country survey on targeted ads

design:algorithm_audits audit set: 34; IN here 22; the rest:
  not a candidate  IMC/2015/location-location-location-the-impact-of-geolocation-on-web-search-personalizati
  not a candidate  IMC/2015/peeking-beneath-the-hood-of-uber
  not a candidate  WWW/2018/auditing-the-personalization-and-composition-of-politically-related-search-engin
  OUT:recommendation  WWW/2019/measuring-political-personalization-of-google-news-search
  not a candidate  WWW/2022/left-or-right-a-peek-into-the-political-biases-in-email-spam-filtering-algorithm
  OUT:recommendation  PETS/2023/a-utility-preserving-obfuscation-approach-for-youtube-recommendations
  not a candidate  WWW/2023/a-method-to-assess-and-explain-disparate-impact-in-online-retailing
  not a candidate  IMC/2024/poster-identifying-filter-bubble-based-on-feed-level-embedding-similarity-analys
  not a candidate  WWW/2024/tiktok-and-the-art-of-personalization-investigating-exploration-and-exploitation
  not a candidate  USENIX/2025/big-help-or-big-brother-auditing-tracking-profiling-and-personalization-in-gener
  OUT:dsr  IEEE-SP/2026/setting-the-course-but-forgetting-to-steer-analyzing-compliance-with-gdprs-right
  not a candidate  WWW/2026/does-this-button-work-investigating-youtubes-ineffective-user-controls

=== PASS C — the IN population ===
Per year (IN papers / corpus papers that year):
2010: 2/119  2011: 0/116  2012: 1/151  2013: 0/125  2014: 5/166  2015: 2/190  2016: 2/182  2017: 1/231  2018: 5/254  2019: 7/402  2020: 8/404  2021: 2/379  2022: 7/546  2023: 9/719  2024: 7/690  2025*: 9/770  2026*: 4/415

Buckets (both ends bounded; last bucket provisional):
bucket      years  IN  corpus  IN per year  IN per 1,000 corpus papers
----------  -----  --  ------  -----------  --------------------------
2010–2013   4      3   511     0.75         5.9
2014–2017   4      10  769     2.50         13.0
2018–2021   4      22  1439    5.50         15.3
2022–2024   3      23  1955    7.67         11.8
2025–2026*  2      13  1185    6.50         11.0

By venue (share of that venue's own papers in the extraction):
venue    venue papers  IN  share
-------  ------------  --  -----
PETS     510           14  2.7%
WWW      843           20  2.4%
IMC      638           15  2.4%
IEEE-SP  767           5   0.7%
NDSS     701           4   0.6%
USENIX   1410          8   0.6%
CCS      990           5   0.5%

Platform measured (multi-valued; of the IN papers):
web 53 (74.6%); other-online-service 36 (50.7%); mobile 11 (15.5%); iot 3 (4.2%)

Observable families (multi-valued; paper-counted), first / most recent year, and per bucket:
observable   papers  first  latest  2010–2013  2014–2017  2018–2021  2022–2024  2025–2026*
-----------  ------  -----  ------  ---------  ---------  ---------  ---------  ----------
retarget     3       2016   2022    0          1          1          1          0
persona      19      2010   2026    1          6          2          5          5
bids         7       2019   2025    0          0          2          4          1
rtbprice     3       2017   2020    0          1          2          0          0
explanation  14      2015   2026    0          1          6          3          4
donation     11      2018   2026    0          0          5          4          2
advertiser   10      2017   2024    0          1          5          4          0
crawlads     29      2010   2026    3          6          6          7          7
archive      7       2020   2026    0          0          2          4          1
  retarget: 2016 USENIX tracing-information-flows-between-ad-exc | 2018 WWW adbudgetkiller-online-advertising-budget | 2022 CCS cart-ology-intercepting-targeted-adverti
  persona: 2010 IMC challenges-in-measuring-online-advertisi | 2014 CCS your-online-interests-pwned-a-pollution- | 2014 USENIX xray-enhancing-the-web-s-transparency-wi | 2014 WWW adscape-harvesting-and-analyzing-online- | 2015 CCS sunlight-fine-grained-targeting-detectio | 2015 PETS automated-experiments-on-ad-privacy-sett | 2016 USENIX tracing-information-flows-between-ad-exc | 2020 PETS inferring-tracker-advertiser-relationshi | 2020 WWW stop-tracking-me-bro-differential-tracki | 2022 NDSS auto-draft-209 | 2022 PETS atom-ad-network-tomography | 2023 CCS marketing-to-children-through-online-tar | 2023 IMC tracking-profiling-and-ad-targeting-in-t | 2024 PETS opted-out-yet-tracked-are-regulations-en | 2025 IMC from-voice-to-ads-auditing-commercial-sm | 2025 PETS echoes-of-privacy-uncovering-the-profili | 2025 PETS more-and-scammier-ads-the-perils-of-yout | 2025 WWW the-first-early-evidence-of-the-use-of-b | 2026 PETS ad-personalization-and-transparency-in-m
  bids: 2019 IMC no-more-chasing-waterfalls-a-measurement | 2020 PETS inferring-tracker-advertiser-relationshi | 2022 IMC what-factors-affect-targeting-and-bids-i | 2022 NDSS auto-draft-209 | 2023 IMC tracking-profiling-and-ad-targeting-in-t | 2024 PETS opted-out-yet-tracked-are-regulations-en | 2025 WWW the-first-early-evidence-of-the-use-of-b
  rtbprice: 2017 IMC if-you-are-not-paying-for-it-you-are-the | 2018 WWW the-cost-of-digital-advertisement-compar | 2020 WWW stop-tracking-me-bro-differential-tracki
  explanation: 2015 PETS automated-experiments-on-ad-privacy-sett | 2018 USENIX unveiling-and-quantifying-facebook-explo | 2019 NDSS measuring-the-facebook-advertising-ecosy | 2019 NDSS quantity-vs-quality-evaluating-user-inte | 2019 WWW auditing-offline-data-brokers-via-facebo | 2020 PETS canarytrap-detecting-data-misuse-by-thir | 2020 USENIX what-twitter-knows-characterizing-ad-tar | 2023 CCS marketing-to-children-through-online-tar | 2024 PETS how-does-connecting-online-activities-to | 2024 PETS overprofiling-analysis-on-major-internet | 2025 PETS echoes-of-privacy-uncovering-the-profili | 2025 PETS more-and-scammier-ads-the-perils-of-yout | 2026 PETS a-year-under-the-dsa-ad-transparencys-un | 2026 PETS ad-personalization-and-transparency-in-m
  donation: 2018 USENIX unveiling-and-quantifying-facebook-explo | 2019 NDSS measuring-the-facebook-advertising-ecosy | 2020 USENIX what-twitter-knows-characterizing-ad-tar | 2020 WWW facebook-ads-monitor-an-independent-audi | 2021 IMC unique-on-facebook-formulation-and-evide | 2022 IMC what-factors-affect-targeting-and-bids-i | 2023 IEEE-SP collaborative-ad-transparency-promises-a | 2023 USENIX problematic-advertising-and-its-disparat | 2024 PETS overprofiling-analysis-on-major-internet | 2026 PETS a-year-under-the-dsa-ad-transparencys-un | 2026 WWW when-ads-become-profiles-uncovering-the-
  advertiser: 2017 IMC if-you-are-not-paying-for-it-you-are-the | 2018 IEEE-SP privacy-risks-with-facebooks-pii-based-t | 2018 USENIX unveiling-and-quantifying-facebook-explo | 2019 PETS investigating-sources-of-pii-used-in-fac | 2019 WWW auditing-offline-data-brokers-via-facebo | 2021 IMC unique-on-facebook-formulation-and-evide | 2022 IMC measurement-and-analysis-of-implied-iden | 2023 CCS marketing-to-children-through-online-tar | 2023 IEEE-SP collaborative-ad-transparency-promises-a | 2024 PETS client-side-and-server-side-tracking-on-
  crawlads: 2010 IMC challenges-in-measuring-online-advertisi | 2010 WWW analyzing-content-level-properties-of-th | 2012 CCS knowing-your-enemy-understanding-and-det | 2014 IMC the-dark-alleys-of-madison-avenue-unders | 2014 USENIX brahmastra-driving-apps-to-test-the-secu | 2014 WWW adscape-harvesting-and-analyzing-online- | 2015 CCS sunlight-fine-grained-targeting-detectio | 2016 IMC recommended-for-you-a-first-look-at-cont | 2016 USENIX tracing-information-flows-between-ad-exc | 2018 WWW exposing-search-and-advertisement-abuse- | 2019 IMC what-you-see-is-not-what-you-get-discove | 2019 WWW revisiting-mobile-advertising-threats-wi | 2020 IMC when-push-comes-to-ads-measuring-the-ris | 2020 WWW maddroid-characterizing-and-detecting-de | 2021 IMC polls-clickbait-and-commemorative-2-bill | 2022 PETS atom-ad-network-tomography | 2022 WWW conspiracy-brokers-understanding-the-mon | 2023 WWW online-advertising-in-ukraine-and-russia | 2023 WWW are-mobile-advertisements-in-compliance- | 2023 WWW who-funds-misinformation-a-systematic-an | 2024 IEEE-SP targeted-and-troublesome-tracking-and-ad | 2024 IMC analyzing-the-in-accessibility-of-online | 2025 IEEE-SP analyzing-ad-prevalence-characteristics- | 2025 NDSS careful-about-what-app-promotion-ads-rec | 2025 PETS more-and-scammier-ads-the-perils-of-yout | 2025 PETS sheeps-clothing-wolfish-intent-automated | 2025 WWW assessing-compliance-in-digital-advertis | 2025 WWW before-after-the-effect-of-eus-2022-code | 2026 USENIX when-fun-turns-toxic-a-first-look-at-agg
  archive: 2020 IEEE-SP a-security-analysis-of-the-facebook-ad-l | 2020 WWW facebook-ads-monitor-an-independent-audi | 2022 USENIX an-audit-of-facebooks-political-ad-polic | 2023 WWW propaganda-politica-pagada-exploring-u-s | 2023 WWW the-thin-ideology-of-populist-advertisin | 2024 IMC beyond-the-guidelines-assessing-metas-po | 2026 PETS a-year-under-the-dsa-ad-transparencys-un

Question tags (multi-valued; paper-counted):
question   papers  2010–2013  2014–2017  2018–2021  2022–2024  2025–2026*
---------  ------  ---------  ---------  ---------  ---------  ----------
targeting  41      2          7          12         13         7
price      11      0          2          4          4          1
delivery   3       0          0          1          2          0
content    31      1          3          8          11         8
crawlads papers whose only question is content: 20 of 29
IN papers the GAP probe misses: 29; of which content-only question: 22; the rest: IMC 2010 challenges-in-measuring-online [persona,crawlads|targeting]; WWW 2010 analyzing-content-level-proper [crawlads|targeting]; PETS 2015 automated-experiments-on-ad-pr [persona,explanation|targeting]; IEEE-SP 2018 privacy-risks-with-facebooks-p [advertiser|targeting]; USENIX 2018 unveiling-and-quantifying-face [explanation,donation,advertiser|targeting]; PETS 2024 client-side-and-server-side-tr [advertiser|targeting]; WWW 2026 when-ads-become-profiles-uncov [donation|targeting]
observables per IN paper: 1 → 44, 2 → 22, 3+ → 5
IN papers whose only observable is crawlads and only question is content: 20
IN papers asking a targeting, price or delivery question (the page's core): 45
  of which by bucket: 2010–2013 2, 2014–2017 8, 2018–2021 14, 2022–2024 14, 2025–2026* 7

=== PASS D — extraction fields on the IN papers, against the comparable baseline ===
IN papers with a crawlConfig object: 41 of 71; corpus baseline: 1080 papers with one
crawlConfig field  IN (of papers with a config)  baseline (of 1,080 with a config)
-----------------  ----------------------------  ---------------------------------
statefulness       29 of 41 (70.7%)              219 of 1080 (20.3%)
consentAction      21 of 41 (51.2%)              349 of 1080 (32.3%)
headless           9 of 41 (22.0%)               140 of 1080 (13.0%)
interactionDepth   37 of 41 (90.2%)              841 of 1080 (77.9%)
authentication     31 of 41 (75.6%)              779 of 1080 (72.1%)
  IN statefulness: stateful 13, not-stated 12, stateless 10, both 6
  IN consentAction: not-stated 16, no-interaction 15, not-applicable 4, accept-all 3, dismiss-or-remove 1, accept-and-reject 1, cmp-specific-choices 1
  IN headless: not-stated 32, headless 6, headful 3
  IN authentication: none 25, not-stated 10, account-registration 5, manual-login 1
vantage location stated: IN 37 of 67 with a vantage tuple (55.2%); baseline measuredFrom 1228 of 3908 (31.4%)
  complements: vantage NOT stated 30 of 67 (44.8%); headless NOT stated 32 of 41 (78.0%); account-registration or manual-login 6 of 41
ethics review outcome stated: IN 29 of 71 empirical (40.8%); baseline 1728 of 5118 (33.8%)
artifact availability stated: IN 51 of 71 empirical (71.8%); baseline 2890 of 5118 (56.5%)
  IN artifacts.availability: public 41, none-mentioned 18, promised-not-yet-available 6, no-artifacts-object 2, on-request 2, restricted 1, explicitly-withheld 1
runs a non-descriptive statistic: IN 37 of 71 empirical (52.1%); baseline 1637 of 5118 (32.0%)
IN papers measuring an other-online-service platform: 36; of which with no crawlConfig: 18
recruited human participants: IN 16 of 71 (22.5%)
classification.method == llm (used/produced): 5 of 71: IMC/2024/beyond-the-guidelines-assessing-metas-political-ad-moderation-in-the-eu, IEEE-SP/2025/analyzing-ad-prevalence-characteristics-and-compliance-in-alexa-skills, IMC/2025/from-voice-to-ads-auditing-commercial-smart-speakers-for-targeted-advertising-ba, PETS/2025/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic, WWW/2026/when-ads-become-profiles-uncovering-the-invisible-risk-of-web-advertising-at-sca
tools[] names an LLM (used/produced): 7: IMC/2024/beyond-the-guidelines-assessing-metas-political-ad-moderation-in-the-eu [GPT-3.5 Instruct; gpt-4-turbo-2024-04-09] | IEEE-SP/2025/analyzing-ad-prevalence-characteristics-and-compliance-in-alexa-skills [GPT-4o; GPT-3.5; Llama 3.2; Llama 2] | IMC/2025/from-voice-to-ads-auditing-commercial-smart-speakers-for-targeted-advertising-ba [GPT-4o] | PETS/2025/echoes-of-privacy-uncovering-the-profiling-practices-of-voice-assistants [ChatGPT-4o] | PETS/2025/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic [GPT-4o-mini; GPT-4o] | PETS/2026/a-year-under-the-dsa-ad-transparencys-uneven-landscape [OpenAI ChatGPT; Gemini] | WWW/2026/when-ads-become-profiles-uncovering-the-invisible-risk-of-web-advertising-at-sca [Gemini 2.0 Flash; Gemini 2.5 Pro; Gemini 2.5 Flash; GPT-4o; GPT-5; GPT-5 Mini]

=== PASS E — full-text method probes over the IN papers (upper bounds; hand-read verdicts in PASS E2) ===
probe (>=1 hit)  IN papers  crawlads papers
---------------  ---------  ---------------
dedup            33 of 71   16 of 29
screenshot       27 of 71   17 of 29
impressions      49 of 71   20 of 29
landing          30 of 71   19 of 29
encrypted_price  2 of 71    0 of 29
control          19 of 71   7 of 29
clicking         33 of 71   16 of 29
bot              8 of 71    4 of 29
easylist         19 of 71   14 of 29
llm              12 of 71   5 of 29

crawlads papers with a dedup-probe hit: 16 of 29
  states IMC/2010/challenges-in-measuring-online-advertising-systems
         Table 2: false positives/negatives of four ways of uniquely identifying an ad
         ctx: variations of the same ad. False negatives have the effect of over-counting the number of unique ads and in the process, increasing noise. False positives, on the other hand do not
  counts IMC/2014/the-dark-alleys-of-madison-avenue-understanding-malicious-advertisements
         "a corpus of 673,596 unique advertisements"; no identity rule
         ctx: block advertisements. After a period of three months, we have created a corpus of 673,596 unique advertisements. We then analyzed this dataset searching for misbehaving advertiseme
  counts USENIX/2014/brahmastra-driving-apps-to-test-the-security-of-third-party-components
         "566 unique ads, and 3,633 unique landing pages"
         ctx:  collected ads from each of the 220 apps over 5 days, giving us a total collection of 566 unique ads, and 3,633 unique landing pages. Using WoT, we determine that 183 out of the 3,
  counts WWW/2014/adscape-harvesting-and-analyzing-online-display-ads
         "875,209 impressions, and 175,495 distinct ads"
         ctx: creatives and landing pages. Our crawling strategy is focused on maximizing the number of unique ads harvested. Of critical importance to our study is the recognition that a user's
  counts CCS/2015/sunlight-fine-grained-targeting-detection-at-scale-with-statistical-confidence
         "24,961,698 impressions created collectively by 19,543 unique ads"
         ctx:  disabled Gmail ads, we collected 24,961,698 im- pressions created collectively by 19,543 unique ads. As expected, the distribution of impressions per ad is skewed: the median ads 
  states IMC/2016/recommended-for-you-a-first-look-at-content-recommendation-networks
         URL uniqueness with and without URL parameters (94% -> 85%)
         ctx:  and we do not click on advertiser URLs, and thus never we observe that the percentage of unique ads drop to trigger the dynamic redirects. Rather, we extract the 85%. advertiser U
  states IMC/2019/what-you-see-is-not-what-you-get-discovering-and-tracking-social-engineering-att
         perceptual hash (dHash) over screenshots
         ctx: tom JS can be easily detected by existing anti-bot JS li- (see step ④). We then compute a perceptual hash, specifically a braries [5, 17]. To implement a stealthier automated brows
  states WWW/2019/revisiting-mobile-advertising-threats-with-madlife
         visual rather than cryptographic hashing; entity-based dedup 12,036 -> 7,067 landing pages
         ctx: aign (e.g., voting), privacy (e.g., phone number) Table 6 depicts the statistics based on unique ads and total number of ads in each category. 345 out of 7,067 unique ad landing pa
  states IMC/2021/polls-clickbait-and-commemorative-2-bills-problematic-political-advertising-on-n
         MinHash-LSH over ad text, Jaccard > 0.5, grouped by landing-page domain; 1,402,245 ads -> 169,751 unique
         ctx: duplithe text is contained in the HTML markup, so we automatically cated dataset (169,751 unique ads) and it classified 8,836 unique ads extracted the text from these ads using Jav
  counts PETS/2022/atom-ad-network-tomography
         defines unique per persona ("we do not count the same ad shown twice") without the identity rule
         ctx:  which the number of websites in each interest group was varied and the number of ads and unique ads gathered from our ad gathering sites were measured. In this experiment, we alte
  states WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion
         average image hash
         ctx: August 19, 2022.1 We will make our data available upon request. Ad Content Analysis 2.2.1 Deduplication and text extraction from ads. Since ads typically contain text, we created a
  no     IEEE-SP/2024/targeted-and-troublesome-tracking-and-advertising-on-childrens-websites
         dedup of websites; perceptual hashing mentioned for a prior tool
         ctx: blockers, Storey et al. [67] developed an extension that detects the AdChoices icon using perceptual hashing. While we considered applying Storey et al.'s method, we found URL-base
  states IMC/2024/analyzing-the-in-accessibility-of-online-advertisements
         average hash plus the accessibility tree
         ctx:  ad, but prior to scraping its content, a diferent ad was delivered in its place. We also deduplicated the ads, using an average hashing function, as well as the contents of their 
  counts NDSS/2025/careful-about-what-app-promotion-ads-recommend-detecting-and-explaining-malware-promotion-via-app-promotion-graph
         unique ads per app over time; no identity rule
         ctx: p to refresh its ads. We observe that, within a specific time period, the total number of unique ads obtained from an app remains constant [59]. This suggests that ad libraries inf
  states PETS/2025/more-and-scammier-ads-the-perils-of-youtubes-ad-privacy-settings
         deduplicated ad videos by video ID
         ctx: ns. In case of a tie, the ad was discussed with other annotators to reach a consensus. We deduplicated ad videos based on their video ID. Ads were combined across the three privacy
  states PETS/2025/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic
         perceptual hashes grouped with Faiss
         ctx: eenshots or misaligned content. To clean the dataset, we used Meta's Faiss library [5] to deduplicate ads within each category by generating perceptual hashes for images and groupi
dedup verdicts over the 29 crawlads papers: states 9, counts-only 6, probe hit something else 1, no probe hit 13
=> say how a unique ad was identified: 9 of 29 (31.0%) — a probe-scoped count: a paper that describes its dedup in other words is not seen
crawlads papers with no dedup-probe hit: WWW/2010/analyzing-content-level-properties-of-the-web-adversphere, CCS/2012/knowing-your-enemy-understanding-and-detecting-malicious-web-advertising, USENIX/2016/tracing-information-flows-between-ad-exchanges-using-retargeted-ads, WWW/2018/exposing-search-and-advertisement-abuse-tactics-and-infrastructure-of-technical, IMC/2020/when-push-comes-to-ads-measuring-the-rise-of-malicious-push-advertising, WWW/2020/maddroid-characterizing-and-detecting-devious-ad-contents-for-android-apps, WWW/2022/conspiracy-brokers-understanding-the-monetization-of-youtube-conspiracy-theories, WWW/2023/are-mobile-advertisements-in-compliance-with-apps-age-group, WWW/2023/who-funds-misinformation-a-systematic-analysis-of-the-ad-related-profit-routines, IEEE-SP/2025/analyzing-ad-prevalence-characteristics-and-compliance-in-alexa-skills, WWW/2025/assessing-compliance-in-digital-advertising-a-deep-dive-into-acceptable-ads-stan, WWW/2025/before-after-the-effect-of-eus-2022-code-of-practice-on-disinformation, USENIX/2026/when-fun-turns-toxic-a-first-look-at-aggressive-advertising-in-mini-games

=== PASS F — ad-topical papers in the venue index that are NOT in the extraction ===
index records: 16864
title-probe hits (homographs removed): 245; by status: screened-out 107, extracted 121, no-abstract 5, selected-no-text 2, venue-year-unlabelled 10
code  papers  meaning
----  ------  -----------------------------------------------------------------------------------------------
S     18      an ad measurement this page would have counted (targeting, price, delivery or content observed)
M     64      auction / CTR / bidding / allocation / creative-generation method on platform data
D     11      privacy-preserving ad system or protocol
B     7       ad blocking / anti-adblock
T     2       tracking or syncing, no ad observed
C     1       supply chain (privacy:ads_txt)
P     1       perception study
K     1       SoK / survey
H     19      homograph ("ad hoc", "retargetable", "better than advertised")
U     1       cannot be decided from the title

The S rows — ad measurements outside the extraction, with the reason each is missing:
  venue-year-unlabelled  PETS 2012  Betrayed by Your Ads! Reconstructing user profiles from Targeted Ads
  selected-no-text       IMC 2013  Best paper - Follow the money: understanding economics of online aggregation and advertising.
  venue-year-unlabelled  PETS 2013  How Much is too Much? Leveraging Ads Audience Estimation to Evaluate Public Profile Uniqueness
  selected-no-text       WWW 2014  Stranger danger: exploring the ecosystem of ad-based URL shortening services.
  no-abstract            NDSS 2014  Selling Off Privacy at Auction
  screened-out           IMC 2015  Empirical Analysis of Search Advertising Strategies.
  venue-year-unlabelled  NDSS 2016  Are these Ads Safe: Detecting Hidden Attacks through the Mobile App-Web Interfaces
  venue-year-unlabelled  NDSS 2016  The Price of Free: Privacy Leakage in Personalized Mobile In-Apps Ads
  venue-year-unlabelled  NDSS 2016  What Mobile Ads Know About Mobile Users
  venue-year-unlabelled  NDSS 2018  Investigating Ad Transparency Mechanisms in Social Media: A Case Study of Facebooks Explanations
  screened-out           WWW 2021  Understanding the Complexity of Detecting Political Ads.
  screened-out           WWW 2021  Auditing for Discrimination in Algorithms Delivering Job Ads.
  screened-out           WWW 2023  On Detecting Policy-Related Political Ads: An Exploratory Analysis of Meta Ads in 2022 French Election.
  screened-out           WWW 2024  Uncovering the Hidden Data Costs of Mobile YouTube Video Ads.
  screened-out           WWW 2024  Analyzing Ad Exposure and Content in Child-Oriented Videos on YouTube.
  screened-out           WWW 2026  Users Pay Twice: The Hidden Energy Cost of Web Advertising.
  no-abstract            WWW 2026  Is Contextual Advertising Safe? Analyzing Systemic Risks with Ads on YouTube.
  no-abstract            WWW 2026  Safeguarding Children at Scale: Cost-Effective Multimodal LLM Detection of Inappropriate YouTube Advertising.
S rows: 18; by status: selected-no-text 2, screened-out 7, venue-year-unlabelled 6, no-abstract 3
unlabelled venue-years in the index: NDSS/2010, NDSS/2011, NDSS/2016, NDSS/2018, PETS/2010, PETS/2011, PETS/2012, PETS/2013, PETS/2014

=== --list: every verdict ===
OUT     mechanism      gap=5 fam=10 adw=126 CCS/2011/auctions-in-do-not-track-compliant-internet-advertising Bing auction logs, stale-bid simulation
OUT     scam           gap=0 fam=0 adw=42 CCS/2011/fashion-crimes-trending-term-exploitation-on-the-web MFA sites in trending-term search results
IN                     gap=0 fam=99 adw=59 CCS/2012/knowing-your-enemy-understanding-and-detecting-malicious-web-advertising [crawlads|content] malvertising redirection chains, Alexa top 90K
OUT     fraud          gap=1 fam=6 adw=121 CCS/2013/viceroi-catching-click-spam-in-search-ad-networks 
OUT     fraud          gap=0 fam=10 adw=59 CCS/2014/characterizing-large-scale-click-fraud-in-zeroaccess 
OUT     defence        gap=5 fam=24 adw=153 CCS/2014/private-by-design-advertising-meets-the-real-world Privad deployment metrics
OUT     tracking       gap=7 fam=10 adw=4 CCS/2014/the-web-never-forgets-persistent-tracking-mechanisms-in-the-wild 
IN                     gap=23 fam=80 adw=164 CCS/2014/your-online-interests-pwned-a-pollution-attack-against-targeted-advertising [persona|targeting,price] profile pollution vs DoubleClick; attack paper
IN                     gap=27 fam=35 adw=99 CCS/2015/sunlight-fine-grained-targeting-detection-at-scale-with-statistical-confidence [persona,crawlads|targeting] randomised synthetic profiles
OUT     homograph      gap=0 fam=8 adw=1 CCS/2016/predator-proactive-recognition-and-elimination-of-domain-abuse-at-time-of-regist domain-registration "prices"
OUT     mention        gap=5 fam=5 adw=11 CCS/2017/use-privacy-in-data-driven-systems-theory-and-experiments-with-machine-learnt-pr 
OUT     blocking       gap=1 fam=5 adw=151 CCS/2019/adversarial-perceptual-ad-blocking-meets-adversarial-machine-learning 
CONTEXT                gap=20 fam=29 adw=72 CCS/2019/oh-the-places-youve-been-user-reactions-to-longitudinal-transparency-about-third reactions to tracking inferences; interests inferred by the authors, not by a platform
OUT     homograph      gap=26 fam=1 adw=0 CCS/2019/privacy-risks-of-securing-machine-learning-models-against-adversarial-examples targeted adversarial examples
OUT     tracking       gap=3 fam=8 adw=18 CCS/2019/watching-you-watch-the-tracking-ecosystem-of-over-the-top-tv-streaming-devices 
OUT     homograph      gap=5 fam=0 adw=0 CCS/2021/black-box-adversarial-attacks-on-commercial-speech-platforms-with-minimal-inform 
OUT     mention        gap=5 fam=6 adw=31 CCS/2021/consistency-analysis-of-data-usage-purposes-in-mobile-apps 
OUT     fraud          gap=0 fam=0 adw=40 CCS/2021/dissecting-click-fraud-autonomy-in-the-wild 
OUT     homograph      gap=7 fam=0 adw=0 CCS/2021/encodermi-membership-inference-against-pre-trained-encoders-in-contrastive-learn 
OUT     tracking       gap=2 fam=77 adw=207 CCS/2021/this-sneaky-piggy-went-to-the-android-ad-market-misusing-mobile-sensors-for-stea ads' sensor access, not their content
OUT     fraud          gap=38 fam=43 adw=38 CCS/2021/understanding-and-detecting-mobile-ad-fraud-through-the-lens-of-invalid-traffic 
IN                     gap=48 fam=81 adw=173 CCS/2022/cart-ology-intercepting-targeted-advertising-via-ad-network-identity-entanglemen [retarget|targeting] identity entanglement read out through retargeted ads
OUT     dsr            gap=10 fam=12 adw=17 CCS/2022/do-opt-outs-really-opt-me-out traffic after industry opt-outs
OUT     tracking       gap=4 fam=13 adw=16 CCS/2022/freely-given-consent-studying-consent-notice-of-third-party-tracking-and-its-vio 
OUT     sandbox        gap=5 fam=31 adw=22 CCS/2022/privacy-limitations-of-interest-based-advertising-on-the-web-a-post-mortem-empir FLoC recomputed from comScore
OUT     homograph      gap=14 fam=0 adw=0 CCS/2022/when-evil-calls-targeted-adversarial-voice-over-ip-network 
IN                     gap=69 fam=119 adw=276 CCS/2023/marketing-to-children-through-online-targeted-advertising-targeting-mechanisms-a [persona,explanation,advertiser|targeting] six profiles, 3,221 ads + seven own campaigns
OUT     sandbox        gap=0 fam=7 adw=3 CCS/2024/the-privacy-utility-trade-off-in-the-topics-api 
OUT     fraud          gap=12 fam=16 adw=89 CCS/2024/unveiling-collusion-based-ad-attribution-laundering-fraud-detection-analysis-and 
OUT     sandbox        gap=6 fam=7 adw=18 CCS/2025/exploiting-the-shared-storage-api 
OUT     tracking       gap=9 fam=12 adw=16 CCS/2025/piixel-leaks-passive-identification-of-personally-identifiable-information-leaka 
OUT     mention        gap=0 fam=2 adw=43 CCS/2025/whispertest-a-voice-control-based-library-for-ios-ui-automation 
OUT     defence        gap=10 fam=10 adw=10 IEEE-SP/2011/repriv-re-imagining-content-personalization-and-in-browser-privacy in-browser personalisation system
OUT     tracking       gap=17 fam=33 adw=29 IEEE-SP/2012/third-party-web-tracking-policy-and-technology 
OUT     injection      gap=0 fam=3 adw=123 IEEE-SP/2015/ad-injection-at-scale-assessing-deceptive-advertisement-modifications 
OUT     scam           gap=3 fam=6 adw=55 IEEE-SP/2016/cloak-of-visibility-detecting-when-machines-browse-a-different-web advertisement cloaking
OUT     scam           gap=0 fam=23 adw=32 IEEE-SP/2017/how-to-learn-klingon-without-a-dictionary-detection-and-measurement-of-black-key SEO black keywords
OUT     homograph      gap=0 fam=0 adw=53 IEEE-SP/2018/crowd-gps-sec-leveraging-crowdsourcing-to-detect-and-localize-gps-spoofing-attac ADS-B
IN                     gap=1 fam=163 adw=114 IEEE-SP/2018/privacy-risks-with-facebooks-pii-based-targeting-auditing-a-data-brokers-adverti [advertiser|targeting] custom-audience interface as an oracle
OUT     scam           gap=1 fam=5 adw=43 IEEE-SP/2018/surveylance-automatically-detecting-online-survey-scams 
OUT     mention        gap=0 fam=9 adw=26 IEEE-SP/2018/the-spyware-used-in-intimate-partner-violence 
OUT     scam           gap=0 fam=5 adw=7 IEEE-SP/2019/measuring-and-analyzing-search-engine-poisoning-of-linguistic-collisions 
IN                     gap=4 fam=75 adw=364 IEEE-SP/2020/a-security-analysis-of-the-facebook-ad-library [archive|content] Ad Library audit
OUT     tracking       gap=6 fam=6 adw=38 IEEE-SP/2020/do-cookie-banners-respect-my-choice-measuring-legal-compliance-of-banners-from-i 
OUT     homograph      gap=0 fam=6 adw=0 IEEE-SP/2020/ex-vivo-dynamic-analysis-framework-for-android-device-drivers 
OUT     homograph      gap=0 fam=2 adw=170 IEEE-SP/2021/linking-bluetooth-le-classic-and-implications-for-privacy-preserving-bluetooth-b BLE advertisements
OUT     sponsorship    gap=0 fam=1 adw=156 IEEE-SP/2022/investigating-influencer-vpn-ads-on-youtube 
OUT     homograph      gap=0 fam=6 adw=2 IEEE-SP/2023/blue-is-the-new-black-market-privacy-leaks-and-re-victimization-from-police-auct police auctions
IN                     gap=22 fam=86 adw=109 IEEE-SP/2023/collaborative-ad-transparency-promises-and-limitations [donation,advertiser|targeting,delivery] 420 volunteers + 79 own campaigns
CONTEXT                gap=174 fam=275 adw=180 IEEE-SP/2023/when-and-why-do-people-want-ad-targeting-explanations-evidence-from-a-four-week 4,251 ads logged, but the question is when people want an explanation
OUT     homograph      gap=5 fam=0 adw=0 IEEE-SP/2024/multi-instance-adversarial-attack-on-gnn-based-malicious-domain-detection 
OUT     mention        gap=5 fam=6 adw=3 IEEE-SP/2024/sok-technical-implementation-and-human-impact-of-internet-privacy-regulations 
IN                     gap=58 fam=86 adw=285 IEEE-SP/2024/targeted-and-troublesome-tracking-and-advertising-on-childrens-websites [crawlads|content,targeting] ads on 2,004 children's sites from seven vantages
OUT     tracking       gap=5 fam=5 adw=4 IEEE-SP/2024/understanding-the-privacy-practices-of-political-campaigns-a-perspective-from-th 
IN                     gap=9 fam=18 adw=326 IEEE-SP/2025/analyzing-ad-prevalence-characteristics-and-compliance-in-alexa-skills [crawlads|content] audio ads in 45,477 Alexa skills
CONTEXT                gap=56 fam=77 adw=111 IEEE-SP/2025/characterizing-the-usability-and-usefulness-of-u-s-ad-transparency-systems usability of 22 ad-transparency systems
OUT     homograph      gap=0 fam=24 adw=0 IEEE-SP/2025/decentralization-of-ethereums-builder-market MEV-Boost bids
OUT     fraud          gap=2 fam=10 adw=43 IEEE-SP/2025/identifying-incoherent-search-sessions-search-click-fraud-remediation-under-real 
OUT     mention        gap=9 fam=9 adw=8 IEEE-SP/2025/understanding-users-security-and-privacy-concerns-and-attitudes-towards-conversa 
OUT     dsr            gap=8 fam=24 adw=31 IEEE-SP/2026/setting-the-course-but-forgetting-to-steer-analyzing-compliance-with-gdprs-right 
IN                     gap=3 fam=18 adw=108 IMC/2010/challenges-in-measuring-online-advertising-systems [persona,crawlads|targeting] controlled profiles; ad churn and snapshot completeness
OUT     fraud          gap=0 fam=12 adw=122 IMC/2011/understanding-fraudulent-activities-in-online-ad-exchanges 
OUT     mention        gap=5 fam=5 adw=82 IMC/2012/breaking-for-commercials-characterizing-mobile-advertising mobile ad traffic volume and energy; no ad content, targeting or price
OUT     mechanism      gap=0 fam=0 adw=121 IMC/2013/understanding-the-effectiveness-of-video-ads-a-measurement-study CDN logs of ad completion
IN                     gap=0 fam=45 adw=135 IMC/2014/the-dark-alleys-of-madison-avenue-understanding-malicious-advertisements [crawlads|content] 673,596 ads collected by crawl
OUT     blocking       gap=1 fam=5 adw=41 IMC/2016/ad-blockers-global-prevalence-and-impact 
OUT     tracking       gap=2 fam=2 adw=40 IMC/2016/characterizing-website-behaviors-across-logged-in-and-not-logged-in-users ads counted as categorised FQDNs, not collected
IN                     gap=5 fam=7 adw=110 IMC/2016/recommended-for-you-a-first-look-at-content-recommendation-networks [crawlads|content,targeting] CRN widgets and advertiser landing pages
OUT     fraud          gap=1 fam=7 adw=364 IMC/2017/exploring-the-dynamics-of-search-advertiser-fraud 
IN                     gap=100 fam=229 adw=96 IMC/2017/if-you-are-not-paying-for-it-you-are-the-product-how-much-do-advertisers-pay-to [rtbprice,advertiser|price] cleartext and encrypted RTB prices + two own campaigns
OUT     blocking       gap=1 fam=1 adw=45 IMC/2017/the-ad-wars-retrospective-measurement-and-analysis-of-anti-adblock-filter-lists 
OUT     homograph      gap=0 fam=6 adw=12 IMC/2018/following-their-footsteps-characterizing-account-automation-abuse-and-defenses 
OUT     tracking       gap=5 fam=6 adw=41 IMC/2018/how-tracking-companies-circumvented-ad-blockers-using-websockets 
OUT     tracking       gap=20 fam=22 adw=17 IMC/2018/tracing-cross-border-web-tracking 
OUT     supplychain    gap=57 fam=64 adw=325 IMC/2019/a-longitudinal-analysis-of-the-ads-txt-standard RTB inclusion chains serve the ads.txt compliance test
OUT     blocking       gap=0 fam=1 adw=53 IMC/2019/errors-misunderstandings-and-attacks-analyzing-the-crowdsourcing-process-of-ad-b 
IN                     gap=84 fam=147 adw=37 IMC/2019/no-more-chasing-waterfalls-a-measurement-study-of-the-header-bidding-ad-ecosyste [bids|price] HB bids from a clean crawler; shared with privacy:ads_txt
IN                     gap=2 fam=7 adw=147 IMC/2019/what-you-see-is-not-what-you-get-discovering-and-tracking-social-engineering-att [crawlads|content] SE ad campaigns clustered from screenshots
IN                     gap=1 fam=9 adw=132 IMC/2020/when-push-comes-to-ads-measuring-the-rise-of-malicious-push-advertising [crawlads|content] web-push ad notifications collected and clustered
IN                     gap=4 fam=176 adw=566 IMC/2021/polls-clickbait-and-commemorative-2-bills-problematic-political-advertising-on-n [crawlads|content] 1.4M ads from 745 news sites, six locations
IN                     gap=23 fam=243 adw=68 IMC/2021/unique-on-facebook-formulation-and-evidence-of-nano-targeting-individual-users-w [donation,advertiser|targeting,delivery] FDVT interests + 21 nanotargeting campaigns
OUT     homograph      gap=7 fam=0 adw=0 IMC/2022/illuminating-large-scale-ipv6-scanning-in-the-internet scan "targeting"
IN                     gap=13 fam=53 adw=229 IMC/2022/measurement-and-analysis-of-implied-identity-in-ad-delivery-optimization [advertiser|delivery] paired campaigns with synthetic faces
IN                     gap=90 fam=255 adw=318 IMC/2022/what-factors-affect-targeting-and-bids-in-online-advertising-a-field-measurement [donation,bids|targeting,price] 286 participants, extension
IN                     gap=40 fam=66 adw=62 IMC/2023/tracking-profiling-and-ad-targeting-in-the-alexa-echo-smart-speaker-ecosystem [persona,bids|targeting,price] Echo personas, web bids
OUT     tracking       gap=0 fam=8 adw=120 IMC/2023/understanding-the-privacy-risks-of-popular-search-engine-advertising-systems ad-click redirect chains studied for identifier leakage
IN                     gap=2 fam=12 adw=333 IMC/2024/analyzing-the-in-accessibility-of-online-advertisements [crawlads|content] ads on 90 sites for 31 days
IN                     gap=4 fam=70 adw=208 IMC/2024/beyond-the-guidelines-assessing-metas-political-ad-moderation-in-the-eu [archive|content] Meta Ad Library, 16 EU countries
OUT     tracking       gap=3 fam=7 adw=8 IMC/2024/diffaudit-auditing-privacy-practices-of-online-services-for-children-and-adolesc 
OUT     tracking       gap=2 fam=7 adw=18 IMC/2024/watching-tv-with-the-second-party-a-first-look-at-automatic-content-recognition 
OUT     tracking       gap=0 fam=1 adw=42 IMC/2024/whatcha-lookin-at-investigating-third-party-web-content-in-popular-android-apps 
OUT     tracking       gap=8 fam=8 adw=5 IMC/2025/cookieguard-characterizing-and-isolating-the-first-party-cookie-jar 
IN                     gap=46 fam=53 adw=238 IMC/2025/from-voice-to-ads-auditing-commercial-smart-speakers-for-targeted-advertising-ba [persona|targeting] 42 cloned-voice puppets
OUT     homograph      gap=1 fam=7 adw=0 NDSS/2013/analyzing-unique-bid-auction-sites-for-fun-and-profit consumer unique-bid auctions
OUT     homograph      gap=0 fam=1 adw=72 NDSS/2014/scambaiter-understanding-targeted-nigerian-scams-on-craigslist classified ads on Craigslist
OUT     scam           gap=0 fam=0 adw=63 NDSS/2015/parking-sensors-analyzing-and-detecting-parked-domains parked domains
OUT     homograph      gap=0 fam=0 adw=52 NDSS/2017/are-we-there-yet-on-rpki-s-deployment-and-security BGP route advertisements
OUT     scam           gap=0 fam=18 adw=12 NDSS/2017/dial-one-for-scam-a-large-scale-analysis-of-technical-support-scams malvertising only as the route to TSS pages
OUT     mention        gap=2 fam=5 adw=7 NDSS/2019/master-of-web-puppets-abusing-web-browsers-for-persistent-and-stealthy-computation 
IN                     gap=37 fam=137 adw=458 NDSS/2019/measuring-the-facebook-advertising-ecosystem [donation,explanation|targeting] AdAnalyst, 622 volunteers
OUT     homograph      gap=0 fam=0 adw=62 NDSS/2019/nearby-threats-reversing-analyzing-and-attacking-googles-nearby-connections-on-android BLE advertisements
IN                     gap=21 fam=33 adw=106 NDSS/2019/quantity-vs-quality-evaluating-user-interest-profiles-using-ad-preference-managers [explanation|targeting] ad-preference-manager profiles vs participants
OUT     homograph      gap=0 fam=0 adw=48 NDSS/2020/withdrawing-the-bgp-re-routing-curtain-understanding-the-security-impact-of-bgp-poisoning-through-real-world-measurements BGP route advertisements
OUT     blocking       gap=0 fam=0 adw=106 NDSS/2021/cv-inspector-towards-automating-detection-of-adblock-circumvention 
OUT     fraud          gap=3 fam=10 adw=53 NDSS/2021/the-abuser-inside-apps-finding-the-culprit-committing-mobile-ad-fraud 
OUT     homograph      gap=0 fam=0 adw=145 NDSS/2021/trust-the-crowd-wireless-witnessing-to-detect-attacks-on-ads-b-based-air-traffic-surveillance ADS-B
OUT     tracking       gap=8 fam=8 adw=37 NDSS/2021/understanding-worldwide-private-information-collection-on-android 
IN                     gap=86 fam=98 adw=34 NDSS/2022/auto-draft-209 [persona,bids|targeting,price] HARPO: obfuscation scored on real bids and segments
OUT     tracking       gap=6 fam=12 adw=27 NDSS/2023/i-still-know-what-you-watched-last-sunday-privacy-of-the-hbbtv-protocol-in-the-european-smart-tv-landscape 
OUT     tracking       gap=3 fam=14 adw=16 NDSS/2023/navigating-murky-waters-automated-browser-feature-testing-for-uncovering-tracking-vectors 
OUT     homograph      gap=3 fam=6 adw=12 NDSS/2024/anonpsi-an-anonymity-assessment-framework-for-psi 
IN                     gap=3 fam=5 adw=249 NDSS/2025/careful-about-what-app-promotion-ads-recommend-detecting-and-explaining-malware-promotion-via-app-promotion-graph [crawlads|content] app-promotion ads collected by driving apps
OUT     scam           gap=0 fam=1 adw=53 NDSS/2025/ctrlaltdeceive-quantifying-user-exposure-to-online-scams 
OUT     perception     gap=7 fam=10 adw=2 NDSS/2025/transparency-or-information-overload-evaluating-users-comprehension-and-perceptions-of-the-ios-app-privacy-report 
OUT     mention        gap=0 fam=1 adw=43 NDSS/2026/chameleoscan-demystifying-and-detecting-ios-chameleon-apps-via-llm-powered-ui-exploration 
OUT     homograph      gap=6 fam=6 adw=3 NDSS/2026/side-channel-inference-of-user-activities-in-ar-vr-using-gpu-profiling 
OUT     homograph      gap=0 fam=0 adw=57 NDSS/2026/sok-cryptographic-authenticated-dictionaries 
OUT     homograph      gap=1 fam=17 adw=1 NDSS/2026/the-role-of-privacy-guarantees-in-voluntary-donation-of-private-health-data-for-altruistic-goals health data donation
OUT     blocking       gap=1 fam=1 adw=54 PETS/2015/an-automated-approach-for-complementing-ad-blockers-blacklists 
IN                     gap=3 fam=70 adw=230 PETS/2015/automated-experiments-on-ad-privacy-settings [persona,explanation|targeting] AdFisher
OUT     perception     gap=4 fam=19 adw=23 PETS/2016/access-denied-contrasting-data-access-in-the-united-states-and-ireland 
OUT     perception     gap=28 fam=30 adw=60 PETS/2016/do-not-track-me-sometimes-users-contextual-preferences-for-web-tracking 
OUT     perception     gap=0 fam=7 adw=12 PETS/2016/privacy-challenges-in-the-quantified-self-movement-an-eu-perspective 
OUT     tracking       gap=12 fam=14 adw=13 PETS/2017/cross-device-tracking-measurement-and-disclosures 
OUT     blocking       gap=1 fam=5 adw=65 PETS/2017/detecting-anti-ad-blockers-in-the-wild 
OUT     tracking       gap=12 fam=17 adw=37 PETS/2017/on-the-privacy-and-security-of-the-ultrasound-ecosystem 
OUT     blocking       gap=0 fam=6 adw=97 PETS/2018/nomoads-effective-and-efficient-cross-app-mobile-ad-blocking 
OUT     homograph      gap=0 fam=0 adw=48 PETS/2018/undermining-privacy-in-the-aircraft-communications-addressing-and-reporting-syst ADS-B / ACARS
OUT     tracking       gap=1 fam=7 adw=4 PETS/2018/when-the-cookie-meets-the-blockchain-privacy-risks-of-web-payments-via-cryptocur 
OUT     tracking       gap=2 fam=5 adw=12 PETS/2019/4-years-of-eu-cookie-law-results-and-lessons-learned 
IN                     gap=21 fam=136 adw=92 PETS/2019/investigating-sources-of-pii-used-in-facebook-s-targeted-advertising [advertiser|targeting] reach estimates + own campaigns on controlled PII
OUT     perception     gap=21 fam=21 adw=13 PETS/2019/what-if-predicting-individual-users-smart-home-privacy-preferences-and-their-cha 
IN                     gap=9 fam=15 adw=50 PETS/2020/canarytrap-detecting-data-misuse-by-third-party-apps-on-online-social-networks [explanation|targeting] Facebook's advertiser-uploaded-your-info list as ground truth for honeytoken misuse
OUT     tracking       gap=1 fam=7 adw=27 PETS/2020/in-depth-evaluation-of-redirect-tracking-and-link-usage 
IN                     gap=91 fam=148 adw=152 PETS/2020/inferring-tracker-advertiser-relationships-in-the-online-advertising-ecosystem-u [bids,persona|targeting,price] HB bids as the dependent variable
OUT     homograph      gap=5 fam=6 adw=31 PETS/2020/the-price-is-not-right-comparing-privacy-in-free-and-paid-apps 
OUT     tracking       gap=3 fam=4 adw=69 PETS/2020/the-tv-is-smart-and-full-of-trackers-measuring-smart-tv-advertising-and-tracking 
OUT     tracking       gap=12 fam=12 adw=10 PETS/2021/a-calculus-of-tracking-theory-and-practice 
OUT     perception     gap=4 fam=13 adw=41 PETS/2021/awareness-adoption-and-misconceptions-of-web-privacy-tools 
OUT     perception     gap=16 fam=17 adw=19 PETS/2021/did-you-know-this-camera-tracks-your-mood-understanding-privacy-expectations-and 
OUT     perception     gap=36 fam=38 adw=45 PETS/2021/managing-potentially-intrusive-practices-in-the-browser-a-user-centered-perspect 
OUT     homograph      gap=0 fam=0 adw=72 PETS/2021/who-can-find-my-devices-security-and-privacy-of-apple-s-crowd-sourced-bluetooth BLE advertisements
OUT     tracking       gap=3 fam=10 adw=20 PETS/2022/are-iphones-really-better-for-privacy-a-comparative-study-of-ios-and-android-app 
IN                     gap=42 fam=108 adw=211 PETS/2022/atom-ad-network-tomography [persona,crawlads|targeting] isolated personas, ad creatives as features
OUT     perception     gap=6 fam=24 adw=64 PETS/2022/charting-app-developers-journey-through-privacy-regulation-features-in-ad-networ 
OUT     perception     gap=1 fam=7 adw=15 PETS/2022/how-can-and-would-people-protect-from-online-tracking 
OUT     recommendation gap=5 fam=7 adw=16 PETS/2023/a-utility-preserving-obfuscation-approach-for-youtube-recommendations 
OUT     perception     gap=0 fam=5 adw=1 PETS/2023/is-there-a-reverse-privacy-paradox-an-exploratory-analysis-of-gaps-between-priva 
OUT     sandbox        gap=9 fam=10 adw=8 PETS/2023/locality-sensitive-hashing-does-not-guarantee-privacy-attacks-on-googles-floc-an 
OUT     tracking       gap=4 fam=5 adw=15 PETS/2023/privacy-rarely-considered-exploring-considerations-in-the-adoption-of-third-part 
CONTEXT                gap=5 fam=13 adw=15 PETS/2023/surprised-shocked-worried-user-reactions-to-facebook-data-collection-from-third reactions to Off-Facebook Activity
OUT     dsr            gap=3 fam=5 adw=16 PETS/2023/usability-and-enforceability-of-global-privacy-control 
IN                     gap=1 fam=23 adw=65 PETS/2024/client-side-and-server-side-tracking-on-meta-effectiveness-and-accuracy [advertiser|targeting] campaign reach as the oracle for Pixel/CAPI matching
OUT     perception     gap=16 fam=19 adw=28 PETS/2024/cross-contextual-examination-of-older-adults-privacy-concerns-behaviors-and-vuln 
OUT     perception     gap=7 fam=30 adw=17 PETS/2024/crumbling-cookie-categories-deconstructing-common-cookie-categories-to-create-ca 
OUT     perception     gap=7 fam=9 adw=24 PETS/2024/exploring-the-privacy-experiences-of-closeted-users-of-online-dating-services-in 
IN                     gap=16 fam=135 adw=52 PETS/2024/how-does-connecting-online-activities-to-advertising-inferences-impact-privacy-p [explanation|targeting] participants' own labels vs Google Ad Settings: 21% agreement
OUT     sandbox        gap=14 fam=12 adw=110 PETS/2024/interest-disclosing-mechanisms-for-advertising-are-privacy-exposing-not-preservi 
OUT     dsr            gap=5 fam=7 adw=10 PETS/2024/johnny-still-cant-opt-out-assessing-the-iab-ccpa-compliance-framework 
OUT     homograph      gap=0 fam=15 adw=0 PETS/2024/measuring-conditional-anonymity-a-global-study 
IN                     gap=46 fam=78 adw=275 PETS/2024/opted-out-yet-tracked-are-regulations-enough-to-protect-your-privacy [persona,bids|targeting,price] bids after GDPR/CCPA/NAI opt-outs
IN                     gap=12 fam=47 adw=188 PETS/2024/overprofiling-analysis-on-major-internet-players [explanation,donation|targeting] 45,310 interests rated, 1,477 interest-targeted ads
OUT     tracking       gap=4 fam=7 adw=10 PETS/2024/website-data-transparency-in-the-browser 
OUT     perception     gap=0 fam=9 adw=96 PETS/2024/what-do-privacy-advertisements-communicate-to-consumers ads about privacy features
CONTEXT                gap=20 fam=62 adw=101 PETS/2024/what-does-it-mean-to-be-creepy-responses-to-visualizations-of-personal-browsing design probe; interests inferred by the authors
OUT     dsr            gap=2 fam=5 adw=13 PETS/2024/what-to-expect-when-you-re-accessing-an-exploration-of-user-privacy-rights-in-pe 
IN                     gap=16 fam=26 adw=23 PETS/2025/echoes-of-privacy-uncovering-the-profiling-practices-of-voice-assistants [persona,explanation|targeting] persona accounts vs disclosed profiling labels
OUT     perception     gap=8 fam=9 adw=9 PETS/2025/evaluating-a-data-fiduciary-standard-for-privacy-developer-and-end-user-perspect 
OUT     tracking       gap=3 fam=7 adw=9 PETS/2025/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and 
OUT     homograph      gap=0 fam=145 adw=1 PETS/2025/it-s-not-my-data-anymore-exploring-non-users-privacy-perceptions-of-medical-data medical data donation
CONTEXT                gap=21 fam=43 adw=30 PETS/2025/misalignments-and-demographic-differences-in-expected-and-actual-privacy-setting Facebook settings incl. ad profile, as a perception study
IN                     gap=80 fam=133 adw=304 PETS/2025/more-and-scammier-ads-the-perils-of-youtubes-ad-privacy-settings [persona,explanation,crawlads|content,targeting] YouTube pre-roll ads under privacy settings
OUT     perception     gap=7 fam=15 adw=32 PETS/2025/navigating-social-media-privacy-awareness-preferences-and-discoverability 
OUT     homograph      gap=1 fam=1 adw=51 PETS/2025/okay-google-where-s-my-tracker-security-privacy-and-performance-evaluation-of-go BLE advertisements
CONTEXT                gap=117 fam=142 adw=214 PETS/2025/privacy-perceptions-and-behaviors-towards-targeted-advertising-on-social-media-a cross-country survey on targeted ads
OUT     tracking       gap=4 fam=9 adw=7 PETS/2025/privacy-settings-of-third-party-libraries-in-android-apps-a-study-of-facebook-sd 
IN                     gap=7 fam=117 adw=375 PETS/2025/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic [crawlads|content] Acceptable Ads, 4,710 pages, US and Germany
OUT     tracking       gap=3 fam=8 adw=23 PETS/2025/the-effect-of-platform-policies-on-app-privacy-compliance-a-study-of-child-direc 
OUT     tracking       gap=1 fam=5 adw=82 PETS/2025/the-impact-of-default-mobile-sdk-usage-on-privacy-and-data-protection 
OUT     tracking       gap=26 fam=27 adw=42 PETS/2025/tracking-without-borders-studying-the-role-of-webviews-in-bridging-mobile-and-we 
OUT     tracking       gap=5 fam=7 adw=29 PETS/2025/unmasking-the-shadows-a-cross-country-study-of-online-tracking-in-illegal-movie 
OUT     perception     gap=3 fam=6 adw=21 PETS/2025/user-centric-textual-descriptions-of-privacy-enhancing-technologies-for-ad-track 
OUT     tracking       gap=3 fam=5 adw=6 PETS/2025/your-signal-their-data-an-empirical-privacy-analysis-of-wireless-scanning-sdks-i 
IN                     gap=65 fam=195 adw=216 PETS/2026/a-year-under-the-dsa-ad-transparencys-uneven-landscape [explanation,donation,archive|targeting,content] 48,511 explanations vs repositories
IN                     gap=34 fam=71 adw=283 PETS/2026/ad-personalization-and-transparency-in-mobile-ecosystems-a-comparative-analysis [persona,explanation|targeting] controlled app-store accounts
OUT     tracking       gap=4 fam=10 adw=298 PETS/2026/clicking-into-exposure-uncovering-privacy-risks-of-google-click-identifier-in-yo gclid on YouTube ad clicks
OUT     perception     gap=2 fam=5 adw=3 PETS/2026/designing-reflective-thinking-based-contextual-privacy-policy-for-mobile-applica 
OUT     dsr            gap=4 fam=13 adw=15 PETS/2026/exercising-the-ccpa-opt-out-right-on-android-legally-mandated-but-practically-ch 
OUT     perception     gap=4 fam=5 adw=9 PETS/2026/i-don-t-think-it-needs-to-be-political-privacy-experiences-and-concerns-of-femhe 
OUT     tracking       gap=3 fam=5 adw=8 PETS/2026/no-privacy-for-privates-how-military-communities-experience-and-perceive-the-pri 
OUT     blocking       gap=2 fam=10 adw=110 PETS/2026/privacy-vs-profit-the-impact-of-googles-manifest-version-3-mv3-update-on-ad-bloc 
OUT     tracking       gap=7 fam=7 adw=2 PETS/2026/the-empire-strikes-back-at-your-privacy-an-archaeology-of-tracking-on-government 
OUT     tracking       gap=6 fam=11 adw=5 PETS/2026/the-tcf-doesn-t-really-a-a-id-automatic-privacy-analysis-and-legal-compliance-of 
OUT     homograph      gap=0 fam=18 adw=17 USENIX/2011/a-study-of-android-application-security Android intent "retargeting"
OUT     homograph      gap=0 fam=6 adw=1 USENIX/2013/effective-inter-component-communication-mapping-in-android-an-essential-step-tow Android intent "retargeting"
IN                     gap=0 fam=5 adw=45 USENIX/2014/brahmastra-driving-apps-to-test-the-security-of-third-party-components [crawlads|content] in-app ad landing pages and child-inappropriate ads
OUT     fraud          gap=1 fam=10 adw=68 USENIX/2014/understanding-the-dark-side-of-domain-parking parking monetisation chains
IN                     gap=20 fam=36 adw=89 USENIX/2014/xray-enhancing-the-web-s-transparency-with-differential-correlation [persona|targeting] differential correlation over shadow accounts
OUT     tracking       gap=8 fam=8 adw=9 USENIX/2016/internet-jones-and-the-raiders-of-the-lost-trackers-an-archaeological-study-of-w 
OUT     mention        gap=0 fam=1 adw=68 USENIX/2016/investigating-commercial-pay-per-install-and-the-distribution-of-unwanted-softwa 
OUT     mention        gap=0 fam=1 adw=49 USENIX/2016/measuring-pup-prevalence-and-pup-distribution-through-pay-per-install-services 
OUT     homograph      gap=0 fam=0 adw=134 USENIX/2016/protecting-privacy-of-ble-device-users BLE advertisements
IN                     gap=104 fam=120 adw=140 USENIX/2016/tracing-information-flows-between-ad-exchanges-using-retargeted-ads [retarget,persona,crawlads|targeting] retargeted ads as evidence of exchange-to-exchange flows
OUT     tracking       gap=3 fam=5 adw=24 USENIX/2017/a-privacy-analysis-of-cross-device-tracking 
CONTEXT                gap=22 fam=27 adw=88 USENIX/2017/exploring-user-perceptions-of-discrimination-in-online-targeted-advertising scenario survey on discriminatory targeting
IN                     gap=3 fam=314 adw=57 USENIX/2018/unveiling-and-quantifying-facebook-exploitation-of-sensitive-personal-data-for-a [explanation,donation,advertiser|targeting] FDVT ad preferences + own campaigns
OUT     homograph      gap=0 fam=12 adw=0 USENIX/2019/on-the-lack-of-location-privacy-in-crowdsourcing-applications 
OUT     mention        gap=0 fam=0 adw=43 USENIX/2019/policylint-investigating-internal-privacy-policy-contradictions-on-google-play 
OUT     mention        gap=3 fam=3 adw=51 USENIX/2020/actions-speak-louder-than-words-entity-sensitive-privacy-policy-and-data-flow-an 
OUT     perception     gap=16 fam=28 adw=115 USENIX/2020/the-impact-of-ad-blockers-on-product-search-and-purchase-behavior-a-lab-experime lab experiment with contextual ads blocked
IN                     gap=96 fam=141 adw=138 USENIX/2020/what-twitter-knows-characterizing-ad-targeting-practices-user-perceptions-and-ad [donation,explanation|targeting] 231 participants' own Twitter ad files
OUT     fraud          gap=0 fam=0 adw=96 USENIX/2021/adcube-webvr-ad-fraud-and-practical-confinement-of-third-party-ads WebVR ad fraud attacks
OUT     perception     gap=7 fam=14 adw=10 USENIX/2021/are-privacy-dashboards-good-for-end-users-evaluating-user-perceptions-and-reacti 
OUT     homograph      gap=0 fam=1 adw=53 USENIX/2021/disrupting-continuity-of-apples-wireless-ecosystem-security-new-tracking-dos-and BLE advertisements
OUT     tracking       gap=4 fam=7 adw=29 USENIX/2021/share-first-ask-later-or-never-studying-violations-of-gdprs-explicit-consent-in 
IN                     gap=1 fam=185 adw=626 USENIX/2022/an-audit-of-facebooks-political-ad-policy-enforcement [archive|content] Ad Library enforcement audit
OUT     blocking       gap=5 fam=5 adw=32 USENIX/2022/khaleesi-breaker-of-advertising-and-tracking-request-chains 
OUT     mention        gap=0 fam=6 adw=2 USENIX/2022/mistrust-plugins-you-must-a-large-scale-study-of-malicious-plugins-in-wordpress 
OUT     blocking       gap=5 fam=5 adw=28 USENIX/2022/webgraph-capturing-advertising-and-tracking-information-flows-for-robust-blockin 
OUT     blocking       gap=1 fam=1 adw=131 USENIX/2023/autofr-automated-filter-rule-generation-for-adblocking 
OUT     blocking       gap=4 fam=5 adw=17 USENIX/2023/defining-broken-user-experiences-and-remediation-tactics-when-ad-blocking-or-tra 
IN                     gap=16 fam=102 adw=344 USENIX/2023/problematic-advertising-and-its-disparate-exposure-on-facebook [donation|content,targeting] 32,587 panel ads coded
OUT     defence        gap=0 fam=5 adw=156 USENIX/2023/trident-towards-detecting-and-mitigating-web-based-social-engineering-attacks SE-ad navigation detector
OUT     tracking       gap=11 fam=28 adw=45 USENIX/2024/attention-please-an-investigation-of-the-app-tracking-transparency-permission 
OUT     dsr            gap=2 fam=8 adw=4 USENIX/2024/data-subjects-reactions-to-exercising-their-right-of-access 
OUT     homograph      gap=13 fam=0 adw=0 USENIX/2024/false-claims-against-model-ownership-resolution 
OUT     sandbox        gap=44 fam=80 adw=143 USENIX/2024/fledging-will-continue-until-privacy-improves-empirical-analysis-of-googles-priv Protected Audience adoption and attacks
OUT     perception     gap=1 fam=5 adw=11 USENIX/2024/im-not-convinced-that-they-dont-collect-more-than-is-necessary-user-controlled-d 
OUT     tracking       gap=0 fam=13 adw=62 USENIX/2024/navigating-the-privacy-compliance-maze-understanding-risks-with-privacy-configur 
OUT     homograph      gap=0 fam=0 adw=48 USENIX/2024/security-and-privacy-analysis-of-samsungs-crowd-sourced-bluetooth-location-track BLE advertisements
OUT     perception     gap=2 fam=7 adw=4 USENIX/2025/addressing-the-address-books-interdependent-privacy-issues 
OUT     sponsorship    gap=0 fam=1 adw=160 USENIX/2025/as-advertised-understanding-the-impact-of-influencer-vpn-ads 
OUT     homograph      gap=0 fam=12 adw=0 USENIX/2025/lemix-enabling-testing-of-embedded-applications-as-linux-applications 
OUT     mention        gap=0 fam=11 adw=44 USENIX/2025/shadowed-realities-an-investigation-of-ui-attacks-in-webxr 
OUT     homograph      gap=0 fam=0 adw=63 USENIX/2026/privacyshield-relaying-ble-beacons-to-counter-unsolicited-tracking BLE advertisements
OUT     homograph      gap=0 fam=0 adw=42 USENIX/2026/security-and-privacy-analysis-of-tiles-location-tracking-protocol BLE advertisements
IN                     gap=0 fam=2 adw=98 USENIX/2026/when-fun-turns-toxic-a-first-look-at-aggressive-advertising-in-mini-games [crawlads|content] aggressive ad behaviour in mini-games
IN                     gap=3 fam=15 adw=24 WWW/2010/analyzing-content-level-properties-of-the-web-adversphere [crawlads|targeting] ads from 282 PlanetLab vantages; location and behavioural targeting
OUT     perception     gap=0 fam=9 adw=171 WWW/2013/the-cost-of-annoying-ads annoyance of ad formats in a lab task; not about targeting (was CONTEXT until the generic review, 2026-09-23)
OUT     mechanism      gap=0 fam=26 adw=35 WWW/2013/towards-a-robust-modeling-of-temporal-interest-change-patterns-for-behavioral-ta behavioural-targeting model on server logs
OUT     perception     gap=8 fam=15 adw=11 WWW/2013/your-browsing-behavior-for-a-big-mac-economics-of-personal-information-online PII valuation via reverse auctions
IN                     gap=17 fam=28 adw=227 WWW/2014/adscape-harvesting-and-analyzing-online-display-ads [persona,crawlads|targeting] 340 personas, 875,209 impressions
OUT     mechanism      gap=1 fam=5 adw=1 WWW/2014/from-devices-to-people-attribution-of-search-activity-in-multi-user-settings search-log attribution
OUT     fraud          gap=0 fam=0 adw=89 WWW/2015/crowd-fraud-detection-in-internet-advertising 
OUT     injection      gap=0 fam=65 adw=106 WWW/2015/understanding-malvertising-through-ad-injecting-browser-extensions 
OUT     tracking       gap=5 fam=5 adw=10 WWW/2016/an-empirical-study-of-web-cookies 
OUT     mention        gap=0 fam=7 adw=15 WWW/2017/j-force-forced-execution-on-javascript 
OUT     mention        gap=0 fam=6 adw=1 WWW/2017/security-implications-of-redirection-trail-in-popular-websites-worldwide 
IN                     gap=27 fam=79 adw=201 WWW/2018/adbudgetkiller-online-advertising-budget-draining-attack [retarget|targeting] inferred retargeting strategies; attack paper
IN                     gap=0 fam=3 adw=59 WWW/2018/exposing-search-and-advertisement-abuse-tactics-and-infrastructure-of-technical [crawlads|content] search ads collected daily across five engines
IN                     gap=58 fam=84 adw=139 WWW/2018/the-cost-of-digital-advertisement-comparing-user-and-advertiser-views [rtbprice|price] passive RTB prices, 1,270 users
OUT     perception     gap=10 fam=10 adw=22 WWW/2018/your-secrets-are-safe-how-browsers-explanations-impact-misconceptions-about-priv 
IN                     gap=6 fam=30 adw=36 WWW/2019/auditing-offline-data-brokers-via-facebooks-advertising-platform [advertiser,explanation|targeting] broker attributes via the ad platform
OUT     tracking       gap=6 fam=7 adw=14 WWW/2019/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private 
OUT     tracking       gap=6 fam=7 adw=37 WWW/2019/cookie-synchronization-everything-you-always-wanted-to-know-but-were-afraid-to-a 
OUT     mechanism      gap=4 fam=11 adw=33 WWW/2019/dealing-with-interdependencies-and-uncertainty-in-multi-channel-advertising-camp 
OUT     recommendation gap=2 fam=5 adw=4 WWW/2019/measuring-political-personalization-of-google-news-search 
OUT     mechanism      gap=1 fam=8 adw=7 WWW/2019/multiple-treatment-effect-estimation-using-deep-generative-model-with-task-embed 
OUT     fraud          gap=74 fam=81 adw=38 WWW/2019/nameles-an-intelligent-system-for-real-time-filtering-of-invalid-ad-traffic 
OUT     homograph      gap=0 fam=20 adw=0 WWW/2019/quality-effects-on-user-preferences-and-behaviorsin-mobile-news-streaming 
OUT     mechanism      gap=0 fam=0 adw=43 WWW/2019/quality-sensitive-training-social-advertisement-generation-by-leveraging-user-cl 
IN                     gap=1 fam=62 adw=175 WWW/2019/revisiting-mobile-advertising-threats-with-madlife [crawlads|content] 84K in-app ads and landing pages
OUT     tracking       gap=2 fam=5 adw=16 WWW/2019/the-chain-of-implicit-trust-an-analysis-of-the-web-third-party-resources-loading 
OUT     mention        gap=5 fam=5 adw=2 WWW/2020/apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web 
OUT     tracking       gap=8 fam=12 adw=14 WWW/2020/beyond-the-front-page-measuring-third-party-dynamics-in-the-field 
OUT     mention        gap=0 fam=0 adw=61 WWW/2020/deconstructing-googles-web-light-service 
IN                     gap=7 fam=123 adw=305 WWW/2020/facebook-ads-monitor-an-independent-auditing-system-for-political-ads-on-faceboo [donation,archive|content] volunteer-collected Facebook ads vs Ad Library
OUT     blocking       gap=0 fam=0 adw=64 WWW/2020/filter-list-generation-for-underserved-regions 
OUT     mention        gap=8 fam=8 adw=5 WWW/2020/finding-a-choice-in-a-haystack-automatic-extraction-of-opt-out-statements-from-p 
IN                     gap=1 fam=8 adw=79 WWW/2020/maddroid-characterizing-and-detecting-devious-ad-contents-for-android-apps [crawlads|content] devious in-app ad contents
IN                     gap=13 fam=23 adw=38 WWW/2020/stop-tracking-me-bro-differential-tracking-of-user-demographics-on-hyper-partisa [persona,rtbprice|price,targeting] RTB prices per demographic persona
OUT     tracking       gap=4 fam=5 adw=12 WWW/2021/user-tracking-in-the-post-cookie-era-how-websites-bypass-gdpr-consent-to-track-u 
OUT     tracking       gap=4 fam=5 adw=4 WWW/2021/websocket-adoption-and-the-landscape-of-the-real-time-web 
IN                     gap=7 fam=20 adw=185 WWW/2022/conspiracy-brokers-understanding-the-monetization-of-youtube-conspiracy-theories [crawlads|content] YouTube ads on conspiracy vs mainstream videos
OUT     blocking       gap=1 fam=1 adw=64 WWW/2022/investigating-advertisers-domain-changing-behaviors-and-their-impacts-on-ad-bloc 
OUT     mechanism      gap=8 fam=8 adw=24 WWW/2023/ad-auction-design-with-coupon-dependent-conversion-rate-in-the-auto-bidding-worl 
IN                     gap=0 fam=36 adw=140 WWW/2023/are-mobile-advertisements-in-compliance-with-apps-age-group [crawlads|content] 11,270 in-app ad views
OUT     mechanism      gap=29 fam=41 adw=62 WWW/2023/fairness-aware-guaranteed-display-advertising-allocation-under-traffic-cost-cons 
IN                     gap=0 fam=11 adw=197 WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion [crawlads|content] ads on 500 news sites from three proxy locations
IN                     gap=4 fam=50 adw=281 WWW/2023/propaganda-politica-pagada-exploring-u-s-political-facebook-ads-en-espanol [archive|content] Ad Library API
IN                     gap=4 fam=19 adw=98 WWW/2023/the-thin-ideology-of-populist-advertising-on-facebook-during-the-2019-eu-electio [archive|content] Meta Ad Library
IN                     gap=3 fam=5 adw=127 WWW/2023/who-funds-misinformation-a-systematic-analysis-of-the-ad-related-profit-routines [crawlads|content] served ads on fake-news sites; shared with privacy:ads_txt
OUT     tracking       gap=6 fam=11 adw=14 WWW/2024/a-study-of-gdpr-compliance-under-the-transparency-and-consent-framework 
OUT     blocking       gap=1 fam=5 adw=53 WWW/2024/adflush-a-real-world-deployable-machine-learning-solution-for-effective-advertis 
IN                     gap=3 fam=3 adw=202 WWW/2025/assessing-compliance-in-digital-advertising-a-deep-dive-into-acceptable-ads-stan [crawlads|content] Acceptable Ads compliance on 10,000 domains
IN                     gap=1 fam=2 adw=126 WWW/2025/before-after-the-effect-of-eus-2022-code-of-practice-on-disinformation [crawlads|content] served ads and brand adjacency; shared with privacy:ads_txt
IN                     gap=21 fam=39 adw=43 WWW/2025/the-first-early-evidence-of-the-use-of-browser-fingerprinting-for-online-trackin [persona,bids|targeting,price] FPTrace: bids under fingerprint changes
OUT     supplychain    gap=8 fam=14 adw=178 WWW/2025/welcome-to-the-dark-side-analyzing-the-revenue-flows-of-fraud-in-the-online-ad-e 
OUT     homograph      gap=2 fam=10 adw=0 WWW/2026/bowling-with-chatgpt-on-the-evolving-user-interactions-with-conversational-ai-sy donated chat logs
OUT     dsr            gap=12 fam=20 adw=42 WWW/2026/does-ad-free-mean-less-data-collection-an-empirical-study-of-platform-data-pract 
IN                     gap=3 fam=19 adw=39 WWW/2026/when-ads-become-profiles-uncovering-the-invisible-risk-of-web-advertising-at-sca [donation|targeting] Australian Ad Observatory exposures re-analysed with Gemini
</file>

===== Figure and quote verification, unedited =====

''node scripts/verify_advertising_figures.mjs > scripts/verify_advertising_figures-output.txt''. The font warnings pypdf prints go to stderr and are not included.

<file text verify_advertising_figures-output.txt>
control ok (mutated needle not located)  2017/IMC/if-you-are-not  "we found encrypted prices to be ~1.9x higher"
control ok (mutated needle not located)  2026/PETS/a-year-under-the-dsa  "98.1% of all explanation texts cite only the main targeting form"
control ok (mutated needle not located)  2021/IMC/polls-clickbait  "we collected 1,402,245 ads (196,751 unique ads) from 6 locations"
OK   paper.cols.txt 2016/USENIX/tracing-information-flows  | Bashir 2016: 5,102 retargets of 31,850 labelled images
       "labeled 31,850 images, of which 7,563 are behavioral targeted ads and 5,102 are retargeted ads"
OK   paper.cols.txt 2016/USENIX/tracing-information-flows  | Bashir 2016: 35,448 chains
       "including 35,448 chains associated with 5,102 unique retargeted ads"
OK   paper.cols.txt 2016/USENIX/tracing-information-flows  | Bashir 2016: cookie-matching heuristics miss 31%
       "are unable to identify 31% of ad exchange pairs that share data"
OK   paper.cols.txt 2022/CCS/cart-ology  | Cart-ology: ~15 h latency
       "entanglement becomes visible roughly 15 hours later"
OK   paper.cols.txt 2018/WWW/adbudgetkiller  | AdBudgetKiller: 254 of 291
       "we successfully revealed targeting strategies from 254 out of 291 websites (about 87%)"
OK   paper.cols.txt 2022/IMC/what-factors  | Zeng 2022: 18.3% self-reported prior visit
       "Participants responded "Yes" for 18.3% of ads"
OK   paper.cols.txt 2022/IMC/what-factors  | Zeng 2022: +$1.07 CPM
       "with an estimated increase of $1.07 for ads with"
OK   paper.cols.txt 2014/WWW/adscape  | Adscape: profile contamination
       "60% of profiles gained more than 9 new interests after 50 visits"
OK   paper.cols.txt 2014/WWW/adscape  | Adscape: impressions vs distinct
       "This data collection produced 875,209 impressions, and 175,495 distinct ads"
OK   paper.cols.txt 2010/IMC/challenges-in  | Guha 2010: ~10 reloads to saturate
       "The knee of the graph (around 10 reloads)"
OK   paper.cols.txt 2010/IMC/challenges-in  | Guha 2010: churn
       "1-4% per minute depending on the search keyword"
OK   paper.cols.txt 2010/IMC/challenges-in  | Guha 2010: dedup error direction
       "False negatives have the effect of over-counting the number of unique ads"
OK   paper.cols.txt 2010/IMC/challenges-in  | Guha 2010: Table 2 cells
       "DisplayURL 7 13 12 10 Title + DisplayURL 0 45 0 50 Title + Summary 0 68 0 69"
OK   paper.cols.txt 2022/PETS/atom  | ATOM: plateau at 40 sites per interest group
       "until the size of our interest groups reached 40 websites"
OK   paper.cols.txt 2025/PETS/echoes  | Echoes: Google 18.0 days
       "it took 18.0 ± 4.1 days for all profiling labels to appear"
OK   paper.cols.txt 2025/PETS/echoes  | Echoes: Alexa 7.7 days
       "it took Alexa 7.7 ± 1.3 days to generate profiling labels"
OK   paper.cols.txt 2019/PETS/investigating-sources  | Venkatadri 2019: 2FA number, 22 days
       "the phone number became targetable after 22 days"
OK   paper.cols.txt 2019/PETS/investigating-sources  | Venkatadri 2019: login alerts, 17 days
       "became targetable after 17 days"
OK   pypdf          2019/PETS/investigating-sources  | Venkatadri 2019: reach noise
       "the noise is uniformly distributed between 0 and 20"
OK   paper.cols.txt 2017/IMC/if-you-are-not  | Papadopoulos 2017: 1.7x
       "we found encrypted prices to be ~1.7x higher"
OK   paper.cols.txt 2017/IMC/if-you-are-not  | Papadopoulos 2017: median user ~25 CPM
       "the median user costs ~25 CPM, and up to 73% of the users cost < 100 CPM"
OK   paper.cols.txt 2018/WWW/the-cost-of-digital  | Papadopoulos 2018
       "The median advertiser paid 0.00071 Euro per delivered ad, but the median user paid 0.0022 Euro per ad in downloaded bytes"
OK   paper.cols.txt 2020/WWW/stop-tracking  | Agarwal 2020
       "The median price for W L is $0.561 CPM, and the median price for W R is $0.667 CPM"
OK   paper.cols.txt 2020/PETS/inferring-tracker  | Cook 2020: zero bids
       "zero bids making up 22% of all bids"
OK   paper.cols.txt 2023/IMC/tracking-profiling  | Iqbal 2023: 30x
       "go as much as 30x and 27x higher than the mean of vanilla persona"
OK   paper.cols.txt 2015/PETS/automated-experiments  | AdFisher: 1852 vs 318
       "Google showed the ads 1852 times to the male group but just 318 times to the female group"
OK   paper.cols.txt 2015/PETS/automated-experiments  | AdFisher: rehab ads
       "The experimental group saw these ads a total of 3309 times (16% of the ads); the control group never saw any of them"
OK   paper.cols.txt 2023/CCS/marketing-to-children  | Medjkoune 2023: 25% profiling
       "25% of ads collected on children-focused videos had ad explanations suggesting they have been targeted based on profiling"
OK   paper.cols.txt 2023/CCS/marketing-to-children  | Medjkoune 2023: 3,221 ads
       "Across the six profiles, we collected 3,221 ads"
OK   paper.cols.txt 2023/CCS/marketing-to-children  | Medjkoune 2023: 76% non-personalised explanation
       "76% of ads in the Children YTB Kids List"
OK   paper.cols.txt 2026/PETS/a-year-under-the-dsa  | Ben Zaamia 2026: 98.9% (YouTube)
       "98.9% of all explanation texts cite only the main targeting form"
OK   paper.cols.txt 2026/PETS/a-year-under-the-dsa  | Ben Zaamia 2026: 48,511
       "Using 48,511 user-facing "Why am I seeing this ad?" (WAIST) notices"
OK   paper.cols.txt 2024/PETS/overprofiling  | Overprofiling: <50% relevant
       "less than 50% of the assigned interests are relevant"
OK   paper.cols.txt 2019/NDSS/quantity-vs-quality  | Bashir 2019: 27%
       "Participants were strongly interested in only 27% of interests in their profiles"
OK   paper.cols.txt 2024/PETS/how-does-connecting  | Ad Settings agreement 21%
       "only 21% of assigned interests occurred in Ad Settings"
OK   paper.cols.txt 2025/PETS/more-and-scammier  | Mai 2025: 1.30x
       "disabling ad personalization can lead to the user being shown as much as 1.30 times more pre-roll ads"
OK   paper.cols.txt 2025/PETS/more-and-scammier  | Mai 2025: impressions vs unique
       "we manually annotated 10,628 unique pre-roll ads (corresponding to 38,851 impressions)"
OK   paper.cols.txt 2020/USENIX/what-twitter-knows  | Wei 2020: 240,651 ads
       "Our 231 participants' data files collectively contained 240,651 ads that had been targeted with at least one targeting type"
OK   paper.cols.txt 2020/USENIX/what-twitter-knows  | Wei 2020: 30 types
       "we observed 30 different targeting types, with 45,209 unique instances of those targeting types"
OK   paper.cols.txt 2019/NDSS/measuring-the-facebook  | Andreou 2019: 17% lookalike (sic "substancial")
       "A substancial fraction (17%) of ads are targeted using Lookalike audiences"
OK   paper.cols.txt 2019/NDSS/measuring-the-facebook  | Andreou 2019: 12% retargeting
       "A non-trivial fraction (12%) of ads are part of Retargeting"
OK   paper.cols.txt 2019/NDSS/measuring-the-facebook  | Andreou 2019: 622 users
       "we analyze data from 622 real-world Facebook users"
OK   paper.cols.txt 2023/IEEE-SP/collaborative-ad  | Gkiouzepi 2023: 65%
       "if we only look at experiments where our ad has been received by ten or more monitored users, the accuracy increases to 65%"
OK   paper.cols.txt 2023/IEEE-SP/when-and-why  | CONTEXT: when people want explanations
       "participants wanted ad targeting explanations for approximately 30% (1,281 out of 4,251) of the ads"
OK   paper.cols.txt 2022/IMC/measurement-and-analysis-of-implied  | Kaplan 2022
       "images of Black people are delivered more to Black Facebook users (73.8%) when compared to images of white people (56.3%)"
OK   paper.cols.txt 2021/IMC/unique-on-facebook  | Gonzalez-Cabanas 2021: 9 of 21
       "9 out of the 21 campaigns successfully"
OK   paper.cols.txt 2024/PETS/client-side-and-server  | CAPI 34-51%
       "The Conversions API matched between 34% to 51% of website visitors to user profiles"
OK   pypdf          2015/CCS/sunlight  | Sunlight: impressions vs unique (line-break hyphen folded)
       "we collected 24,961,698 impressions created collectively by 19,543 unique ads"
OK   paper.cols.txt 2021/IMC/polls-clickbait  | Zeng 2021: 1.4M vs 169,751
       "we collected 1,402,245 ads (169,751 unique ads) from 6 locations"
OK   paper.cols.txt 2023/WWW/online-advertising-in-ukraine  | Ukraine: 1,197 unique, 4,319 times (sic "confict")
       "35 confict-related clusters with 1,197 unique ads, appearing 4,319 times"
OK   paper.cols.txt 2023/WWW/online-advertising-in-ukraine  | Ukraine: average hash (sic "frst")
       "We frst deduplicated ad images based on the average image hash"
OK   paper.cols.txt 2024/IEEE-SP/targeted-and-troublesome  | Moti 2024: ad detection 7.5% non-ads
       "misidentified non-ads in 7.5%"
OK   paper.cols.txt 2024/IEEE-SP/targeted-and-troublesome  | Moti 2024: 36% / 27% of pages
       "An average of 36% of the pages contained one or more ads, and we detected targeted ads on 27% of the pages we crawled"
OK   paper.cols.txt 2024/IEEE-SP/targeted-and-troublesome  | Moti 2024: 40,281 disclosure pages
       "Our crawler captured a total of 40,281 ad disclosure pages"
OK   paper.cols.txt 2014/IMC/the-dark-alleys  | Zarras 2014: ~1%
       "about 1% of all the collected advertisements show a malicious behavior"
OK   paper.cols.txt 2022/WWW/conspiracy-brokers  | Conspiracy brokers
       "We collect 184,218 ad impressions from 6,347 unique advertisers"
OK   paper.cols.txt 2019/IMC/what-you-see-is-not  | SEACMA: perceptual hash
       "We then compute a perceptual hash"
OK   paper.cols.txt 2019/IMC/what-you-see-is-not  | Vadrevu 2019: ads tailored to OS/browser
       "many ads, including SEACMA ads, were tailored to the kind of operating system or browser being used"
OK   paper.cols.txt 2019/IMC/what-you-see-is-not  | Vadrevu 2019: ad networks treat datacentre/Tor IPs differently (1/2)
       "Tor exit nodes, and Amazon AWS IP ranges"
OK   pypdf          2019/IMC/what-you-see-is-not  | Vadrevu 2019: ... never serve SEACMA ads to them (2/2)
       "seemed to never serve SEACMA ads"
OK   paper.cols.txt 2025/PETS/more-and-scammier  | Mai 2025: 12 h between sequences
       "We waited at least 12 hours to avoid bot detection"
OK   paper.cols.txt 2015/PETS/automated-experiments  | AdFisher: settings page unchanged
       "visiting webpages associated with substance abuse changed the ads shown but not the settings page"
OK   paper.cols.txt 2024/PETS/opted-out  | Liu 2024: higher bids from advertisers not leaked to
       "did not explicitly leak data. It can be seen that all personas, with the exception of Shopping for CCPA, receive higher bids on average than the control persona"
OK   paper.cols.txt 2025/WWW/the-first-early  | Liu 2025: fingerprint changes move bids
       "The bid value dataset exhibits notable differences in trends, mean values, median values, and maximum values after changing browser fingerprints"
OK   paper.cols.txt 2025/IMC/from-voice-to-ads  | Le 2025: 42 puppets
       "using 42 puppets, Alexa's suggested first-party features for training"
OK   paper.cols.txt 2024/IEEE-SP/targeted-and-troublesome  | Moti 2024: 85% correct
       "The crawler correctly detected ads in 85%"
OK   paper.cols.txt 2024/IEEE-SP/targeted-and-troublesome  | Moti 2024: 73% of disclosed ads targeted (table row)
       "Avg./Sum 70,303 36% 27% 73%"
OK   paper.cols.txt 2016/IMC/recommended-for-you  | Bashir 2016b: no clicks
       "and we do not click on advertiser URLs"
OK   paper.cols.txt 2016/IMC/recommended-for-you  | Bashir 2016b: 94%
       "94% only appear on a single publisher"
OK   paper.cols.txt 2016/IMC/recommended-for-you  | Bashir 2016b: ... drop to 85% (1/2, spliced)
       "we observe that the percentage of unique ads drop to"
OK   paper.cols.txt 2016/IMC/recommended-for-you  | Bashir 2016b: 85% (2/2, spliced)
       "Rather, we extract the 85%."
OK   paper.cols.txt 2022/IMC/what-factors  | Zeng 2022: 286 participants
       "In total, 286 participants successfully completed data collection for our study"
OK   paper.cols.txt 2016/USENIX/tracing-information-flows  | Bashir 2016: 738 e-commerce sites
       "In total, the personas cover 738 unique websites"
OK   paper.cols.txt 2014/WWW/adscape  | Adscape: 180 sites, 340 profiles
       "These pairs contain 180 distinct websites and 340 distinct profiles"
OK   paper.cols.txt 2024/IEEE-SP/targeted-and-troublesome  | Moti 2024: 2,004 sites
       "Our final list contained 2,004 websites in 48 distinct languages"
OK   paper.cols.txt 2026/WWW/when-ads-become  | Chen 2026: data source
       "longitudinal dataset sourced from the Australian Ad Observatory"
OK   paper.cols.txt 2026/PETS/a-year-under-the-dsa  | Ben Zaamia 2026: data source
       "ads collected through Who Targets Me (WTM)"
OK   paper.cols.txt 2016/USENIX/tracing-information-flows  | Bashir 2016: 150 publishers
       "We manually select 150 publishers"
OK   paper.cols.txt 2024/IEEE-SP/targeted-and-troublesome  | Moti 2024: blank or empty 7.5%
       "or empty ads in 7.5%"
OK   paper.cols.txt 2026/WWW/when-ads-become  | Chen 2026: plain longitudinal model
       "Gemini 2.0 Flash 74.88"
OK   paper.cols.txt 2021/IMC/polls-clickbait  | Zeng 2021: MinHash-LSH dedup
       "to identify ads with a Jaccard similarity > 0.5"
OK   paper.cols.txt 2010/IMC/challenges-in  | Guha 2010: Table 2 first row (both datasets)
       "Approach % FP % FN % FP % FN RedirURL 0 38 1 52"
OK   paper.cols.txt 2023/CCS/marketing-to-children  | Medjkoune 2023: seven campaigns
       "we created seven real-world ad campaigns"
OK   paper.cols.txt 2019/NDSS/quantity-vs-quality  | Bashir 2019: BlueKai not rated
       "we omit analysis of BlueKai from this section"
OK   pypdf          2020/PETS/inferring-tracker  | Cook 2020: leans to intent
       "zero bids are unlikely to be caused as a result of configuration errors"
OK   pypdf          2015/PETS/automated-experiments  | Datta 2015: credits earlier study
       "We are not the first to study how Google uses information"
OK   paper.cols.txt 2022/IMC/what-factors  | Zeng 2022: 7,117 rendered winners
       "Only in 7,117 ads of these ads was the winner actually rendered on the page"
OK   paper.cols.txt 2019/NDSS/measuring-the-facebook  | Andreou 2019: retargeting definition
       "ads are part of Retargeting, meaning an advertiser is trying to reach a user who had previously interacted with them"
OK   paper.cols.txt 2025/PETS/sheeps-clothing  | Roongta 2025: GPT-4o-mini kappa 0.74
       "OpenAI's GPT-4o-mini attains an inter-annotator agreement of 0.74 across all categories"
OK   paper.cols.txt 2025/IMC/from-voice-to-ads  | Le 2025: GPT-4o ad extraction
       "We then employed the GPT-4o model [79] to extract ads from the transcripts"
OK   paper.cols.txt 2026/WWW/when-ads-become  | When ads become profiles: random-control row
       "Random Control 58.80 58.50"
OK   paper.cols.txt 2026/WWW/when-ads-become  | When ads become profiles: 59.13%
       "the model achieves 59.13% accuracy for Gender"
OK   paper.cols.txt 2026/WWW/when-ads-become  | When ads become profiles: 76.38%
       "Gemini† achieves 76.38% accuracy"
OK   paper.cols.txt 2026/WWW/when-ads-become  | When ads become profiles: longitudinal random control
       "Random Control 70.39 70.38"
OK   paper.cols.txt 2024/IMC/beyond-the-guidelines  | Bouchaud 2024: GPT-3.5 as a coder
       "once through the GPT-3.5 Instruct model"
OK   pypdf(ext)     out/adv/ext/andreou2018.pdf  | Andreou 2018 (NDSS, outside the extraction)
       "In all explanations provided by Facebook across all monitored users, only one attribute was ever shown, while all users had all attributes"
OK   pypdf(ext)     out/adv/ext/andreou2018.pdf  | Andreou 2018: 28 three-attribute campaigns
       "we conducted 28 controlled experiments that target three attributes"
OK   pypdf(ext)     out/adv/ext/andreou2018.pdf  | Andreou 2018: 51 two-attribute campaigns
       "and 51 that target two attributes"

98 needles; located by: paper.cols.txt 90, pypdf 5, pypdf(ext) 3; weak (<20 chars): 0; not located: 0

=== EXTERNAL FIGURES (non-corpus; primary source on the provenance page) ===
Regulation (EU) 2022/2065 (Digital Services Act), Article 26(1)(d)
=== NUMBERS WRITTEN WITH A THOUSANDS SEPARATOR THAT THE GUARD SPLITS ===
24,961,698 (Sunlight impressions; needle above)
=== TEMPLATE LITERALS (the illustrative "What to Report" sentence; not data) ===
Chromium 154; ten personas (five treatment, five control); 40 sites; seven days; 50 publishers; 93% precision on 200 hand-checked frames; Hamming <= 4; 12,408 impressions; 2,917 unique creatives
</file>

===== External checks, unedited =====

''bash scripts/external_checks_advertising.sh > scripts/external_checks_advertising-output.txt 2>&1''

<file text external_checks_advertising-output.txt>
== 1. Non-corpus paper quoted on the page ==
OK   Andreou et al. NDSS 2018 PDF fetched (1869513 bytes; sha256 45b8f8e906b9174b); needles checked by verify_advertising_figures.mjs
OK   Crossref 10.14722/ndss.2018.23191: Investigating Ad Transparency Mechanisms in Social Media: A Case Study of Facebook's Explanations | Andreou, Venkatadri, Goga, Gummadi, Loiseau, Mislove
OK   Crossref 10.14722/ndss.2019.23280: Measuring the Facebook Advertising Ecosystem | Andreou, Silva, Benevenuto, Goga, Loiseau, Mislove
OK   Crossref 10.14722/ndss.2014.23270: Selling off Privacy at Auction | Olejnik, Tran, Castelluccia
OK   Crossref 10.56553/popets-2024-0055: How Does Connecting Online Activities to Advertising Inferences Impact Privacy Perceptions? | Farke, Balash, Golla, Aviv
OK   Crossref 10.56553/popets-2025-0057: Privacy Perceptions and Behaviors Towards Targeted Advertising on Social Media: A Cross-Country Study on the Effect of Culture and Religion | Kaushik, Sharma, Yu, Ali, Knijnenburg, Wang, Zou

== 2. Platform explanation and control surfaces ==
OK   [200] Google My Ad Center launch (2022-10-20) — "If you're not signed into Google, you can still control your preferences in Ad Settings"
OK   [200] Google Ads Transparency Center launch (2023-03-29) — "Ads Transparency Center"
OK   [200] Meta: sensitive detailed-targeting removal (Wayback of Business Help 458835214668072) — "Starting Jan 19, 2022, we're removing some detailed targeting options"
OK   [200] Meta: EU less personalized ads (2024-11-12) — "we'll show ads based only on context"
OK   [200] Meta: EU political/electoral/social-issue ads ended (2025-10-06) — "political, electoral and social issue adverts are no longer able to be delivered in the EU"

== 3. Law and framework text ==
OK   [200] DSA Art. 26(1)(d) — "meaningful information directly and easily accessible from the advertisement about the main parameters used to determine the recipient to whom the advertisement is presented"
OK   [200] DSA Art. 26(3) — "shall not present advertisements to recipients of the service based on profiling as defined in Article 4, point (4), of Regulation (EU) 2016/679 using special categories of personal data"
OK   [200] DSA Art. 28(2) — "when they are aware with reasonable certainty that the recipient of the service is a minor"
OK   [200] DSA application dates (Commission) — "The DSA's general date of applicability is 17 February 2024"
OK   [200] TCF v2.2 (2023-05-16), LI removed for purposes 3-6 — "Vendors will only be able to select consent as an acceptable legal basis for purposes 3, 4, 5 and 6"
OK   [200] TCF v2.3 (2025-06-19) — "was released on 19th June 2025"
OK   [200] TCF v2.3 transition end — "28th February 2026"
OK   TCF v2.4: published 2026-07-23, web deadline 2026-10-23 (IAB Europe notice of 2026-07-16)
OK   [200] Meta EU ad choice relaunched under the DMA (Commission, 2025-12-08) — "Meta will present these new options to users in the EU in January 2026"

== 4. Prices and browsers ==
OK   [200] Google Ad Manager unified first-price auctions (2019-09-05) — "we're transitioning to unified first price auctions for Google Ad Manager"
OK   [200] Google Authorized Buyers: winning price is encrypted — "it returns the winning price in an encrypted form"
OK   [200] Chrome keeps third-party cookies (2025-04-22) — "will not be rolling out a new standalone prompt for third-party cookies"
OK   [200] Privacy Sandbox retirement (2025-10-17) — "Protected Audience (Chrome and Android)"
OK   [200] Safari full third-party cookie blocking (2020-03-24) — "Cookies for cross-site resources are now blocked by default across the board"
OK   [200] Firefox Total Cookie Protection default (2022-06-14, archived original) — "Firefox is rolling out Total Cookie Protection by default to all Firefox users worldwide"
OK   chromestatus 6552486106234880: Deprecate and Remove Protected Audience | Proposed | desktop milestone 153
OK   Chrome stable (Linux): milestone 154 154.0.8037.57
OK   [200] Meta disables NYU Ad Observatory accounts (2021-08-04) — "we disabled the accounts, apps, Pages and platform access associated with NYU's Ad Observatory Project"
OK   [200] Ad Observer still distributed — "It copies the ads you see on Facebook and YouTube"
OK   [200] Australian Ad Observatory status — "Active"
OK   [200] Who Targets Me extension — "Browser Extension"
OK   Crossref 10.1080/00913367.2017.1339368: Online Behavioral Advertising: A Literature Review and Research Agenda | Journal of Advertising
OK   Crossref 10.5117/ccr2024.2.6.angu: Enabling Online Advertising Transparency through Data Donation Methods | Computational Communication Research

== 5. Tools and datasets named on the page (HTTP status after redirects; GitHub archived / pushed_at) ==
OK   github.com/tadatitam/info-flow-experiments -> dattaamit/info-flow-experiments archived=False pushed_at=2021-02-18 license=GPL-3.0
OK   github.com/UWCSESecurityLab/adscraper -> UWCSESecurityLab/adscraper archived=False pushed_at=2025-06-05 license=MIT
OK   github.com/CybersecurityForDemocracy/youtube-ad-settings -> CybersecurityForDemocracy/youtube-ad-settings archived=False pushed_at=2025-06-26 license=CC-BY-4.0
OK   github.com/Racro/AcceptableAds_PETS -> Racro/AcceptableAds_PETS archived=False pushed_at=2025-07-01 license=GPL-3.0
OK   github.com/seemoo-lab/appstore-ad-tools -> seemoo-lab/appstore-ad-tools archived=False pushed_at=2026-06-11 license=GPL-3.0
OK   github.com/phani-vadrevu/seacma -> phani-vadrevu/seacma archived=False pushed_at=2019-10-21 license=None
OK   github.com/NEU-SNS/voice-assistants-pets25 -> NEU-SNS/voice-assistants-pets25 archived=False pushed_at=2025-02-25 license=Apache-2.0
OK   github.com/Breezelled/when-ads-become-profiles -> cruiseresearchgroup/when-ads-become-profiles archived=False pushed_at=2025-12-04 license=MIT
DEAD github.com/Camq543/YoutubeAds — expected: 404 on 2026-09-23; Conspiracy Brokers' code link
OK   github.com/fcaravaca/OverprofilingPETS -> fcaravaca/OverprofilingPETS archived=False pushed_at=2024-06-14 license=None
OK   github.com/MadDroid-2020/MadDroid-WWW -> MadDroid-2020/MadDroid-WWW archived=False pushed_at=2020-04-17 license=None
OK   github.com/AppPromotionAdsResearch/AdGPE -> AppPromotionAdsResearch/AdGPE archived=False pushed_at=2025-01-22 license=None
OK   github.com/ahsan238/Ad-Compliance -> ahsan238/Ad-Compliance archived=False pushed_at=2025-04-29 license=MIT
OK   github.com/targeted-and-troublesome/targeted-and-troublesome-crawler -> targeted-and-troublesome/targeted-and-troublesome-crawler archived=False pushed_at=2025-02-04 license=NOASSERTION
DEAD github.com/mipach/HBDetector — expected: 404 since at least 2026-09-02 (privacy:ads_txt)
DEAD github.com/bitzj2015/Harpo-NDSS22 — expected: 404 since at least 2026-09-02 (privacy:ads_txt)
OK   [200 https://systems.cs.columbia.edu/sunlight/] https://columbia.github.io/sunlight/
OK   [200 https://xray.cs.columbia.edu/] https://xray.cs.columbia.edu/
OK   [200 https://privsec-research.github.io/alexaechos/] https://privsec-research.github.io/alexaechos
OK   [200 https://personalization.ccs.neu.edu/] http://personalization.ccs.neu.edu/
DOWN [000 https://www.fdvt.org/] https://www.fdvt.org/ — expected: no connection on https or http, 2026-09-23
DOWN [301 https://adanalystplus.imag.fr/] https://adanalyst.mpi-sws.org/ — expected: redirects to adanalystplus.imag.fr, which does not answer, 2026-09-23
OK   [200 https://badads.cs.washington.edu/political.html] https://badads.cs.washington.edu/political.html
OK   [200 https://ads.cs.washington.edu/] https://ads.cs.washington.edu/
OK   [200 https://facebook-targeting.ccs.neu.edu/] https://facebook-targeting.ccs.neu.edu/
OK   [200 https://whotargets.me/en/] https://whotargets.me/
OK   [200 https://gitlab.inria.fr/oagoga/collaborative-transparency-IEEE-SP2023] https://gitlab.inria.fr/oagoga/collaborative-transparency-IEEE-SP2023

== summary ==
FAILED checks: 0
</file>

===== Probes, inclusion rule and verdicts =====

''scripts/adv_fold.mjs'' as run.

<file javascript adv_fold.mjs>
// Population rule, candidate probes and hand verdicts for privacy:advertising
// ("Measuring Ad Targeting and Bidding"). Written 2026-09-23.
//
// There is no extraction field for "observed an ad". The population is a hand
// audit of a full-text CANDIDATE set, against the inclusion rule below, which
// was written before the first count was taken. The candidate set is recall-
// oriented and is NOT a population: most of it is off topic.
//
// Candidate set = union of three probes over paper.cols.txt (whitespace
// collapsed, case-insensitive, paper-counted):
//   GAP   the 2026-09-22 gap pass's own ad_targeting_rtb family, >= 5 hits
//         (kept verbatim so this page's population can be compared with the
//         118 that queued it)
//   FAM   the sum of six observable families below, >= 5 hits
//   ADW   /\b(ads|advertisements?|advertisers?)\b/ >= 40 hits — the widest net,
//         there to catch ad-content papers that never say "targeting"; it is
//         also where the homographs live (BLE "advertisements", ADS-B)
//
// Every candidate gets exactly one verdict. report_advertising.mjs throws if a
// candidate has no verdict or a verdict names a paper that is not a candidate.

export const GAP_RE = /real-?time bidding|\bRTB\b|bid request|bidstream|ad auction|header bidding|ad targeting|targeted ad|ad personali[sz]|ad explanation|why am I seeing/gi;

export const FAMILIES = {
  targeting: /ad targeting|targeted (ads?|advertis\w*)\b|behaviou?ral(ly)? target\w*|interest-based (ads?|advertis\w*)|ad personali[sz]\w*|personali[sz]ed (ads?|advertis\w*)\b|re-?target(ing|ed)\b|remarketing|micro-?target\w*|nano-?target\w*/gi,
  auction: /real-?time bidding|\bRTB\b|bid requests?|bid responses?|bidstream|ad auctions?|header bidding|\bprebid|winning bids?|clearing prices?|\bCPMs?\b|price notifications?|encrypted prices?/gi,
  explanation: /why am I seeing|ad explanations?|targeting explanations?|about this ad|ad preferences?|ads? settings|ad (preference )?managers?|my ad center|ad interests?|advertiser interests?/gi,
  donation: /data donation|donated (data|ads?)\b|ad observer|who targets me|ad observatory|adanalyst|facebook data valuation|\bFDVT\b/gi,
  advside: /custom audiences?|lookalike audiences?|PII-based targeting|potential reach|reach estimates?|ads manager|targeting (options?|attributes?|parameters?|criteria)|we (ran|launched|created|placed|bought|purchased) (\w+ ){0,3}(ad|advertising) campaigns?/gi,
  content: /malvertis\w*|scam ads?\b|problematic ads?\b|political ads?\b|in-app ads?\b|search ads?\b|ad creatives?|ad landing pages?|landing pages? of (the )?ads|ad screenshots?|screenshots? of (the )?ads|collected [\d,.]+[kKM]? (unique |distinct )?ads\b|[\d,]{3,} (unique |distinct )?(ads|advertisements|ad impressions)\b/gi,
};

export const ADW_RE = /\b(ads|advertisements?|advertisers?)\b/gi;
export const THRESH = { GAP: 5, FAM: 5, ADW: 40 };

export const INCLUSION_RULE = `A candidate is IN if both hold:
 (A) the paper's own data includes ads or ad-auction outcomes delivered to a
     measuring identity or to real people: served creatives or impressions,
     landing pages, bids or clearing prices, the platform's ad explanations /
     ad-preference inferences / advertiser lists, or delivery and reach
     statistics for ads the researchers bought (an ad archive counts, and is
     pointed to design:platforms:ad_archives);
 (B) it uses that data to answer a question about TARGETING (whether and why
     an ad reached this identity, or what the platform inferred), PRICE (what
     an advertiser bid or paid), DELIVERY (how the platform distributed an ad
     the researchers bought), or CONTENT (what the served ads were, and where).
     An ad's visible behaviour towards the user (aggressive, deceptive,
     redirecting) is CONTENT; what the ad's code collects and sends is
     tracking, and OUT (boundary written down after review, 2026-09-23).
A candidate is CONTEXT if it is a user study whose object is ad targeting,
explanations or ad settings but whose data answers a perception question only.
Everything else is OUT, with a code saying where it belongs.`;

// Observable families (multi-valued per IN paper).
export const OBSERVABLES = {
  retarget: 'retargeting as a controlled stimulus',
  persona: 'trained personas / profiles / sock puppets vs control',
  bids: 'header-bidding bids read from the page',
  rtbprice: 'RTB win-notification prices in passive traffic',
  explanation: "the platform's own explanations, ad-preference pages or advertiser lists",
  donation: "real people's ads or ad data (extension panel, data export)",
  advertiser: 'researcher-bought ads, targeting interfaces, reach estimates',
  crawlads: 'ad creatives / screenshots / landing pages collected by a crawler or app driver',
  archive: 'an ad-transparency archive (owned by design:platforms:ad_archives)',
};
export const QUESTIONS = ['targeting', 'price', 'delivery', 'content'];

// OUT codes: where the candidate belongs instead.
export const OUT_CODES = {
  perception: 'user study with no ad observation (and not about ad targeting)',
  mechanism: 'auction / allocation / CTR / campaign-optimisation design on platform-internal logs',
  fraud: 'ad or click fraud measured from logs, apps or botnets — not targeting, price or content',
  blocking: 'ad blocking, filter lists, anti-adblock: ads counted only as blocking efficacy',
  supplychain: 'ads.txt / sellers.json supply chain only — privacy:ads_txt',
  tracking: 'tracking, syncing, consent or SDK data flows with no ad observed',
  sandbox: 'Privacy Sandbox API analysis or simulation — privacy:privacy_sandbox',
  recommendation: 'personalisation audit of search/news/recommendations, not ads — design:algorithm_audits',
  injection: 'ads injected by extensions or malware — privacy:browser_extensions',
  scam: 'the ad is only the discovery channel for a scam / cloaking / SEO study',
  defence: 'privacy-preserving ad system, detector or obfuscator evaluated without served-ad outcomes',
  dsr: 'data-export (right-of-access) audit — privacy:data_subject_rights',
  sponsorship: 'influencer sponsorship inside content, not an ad system',
  homograph: '"advertisement"/"bid"/"target" in another sense (BLE, ADS-B, auctions, Android intents, ML)',
  mention: 'advertising mentioned in passing; the paper is about something else',
};

// Verdicts, keyed by `${venue}/${year}/${slug-prefix}`. obs = observable
// families, q = question tags, unit = the unit the paper's headline ad figure
// is counted in (filled for IN papers only; see UNITS below).
const V = (v, rest = {}) => ({ v, ...rest });
const IN = (obs, q, note = '') => V('IN', { obs, q, note });
const CTX = (note) => V('CONTEXT', { note });
const OUT = (code, note = '') => V('OUT', { code, note });

export const VERDICTS = {
  // ---- 2010–2015
  'IMC/2010/challenges-in-measuring-online-advertising-systems': IN(['persona', 'crawlads'], ['targeting'], 'controlled profiles; ad churn and snapshot completeness'),
  'WWW/2010/analyzing-content-level-properties-of-the-web-adversphere': IN(['crawlads'], ['targeting'], 'ads from 282 PlanetLab vantages; location and behavioural targeting'),
  'CCS/2011/auctions-in-do-not-track-compliant-internet-advertising': OUT('mechanism', 'Bing auction logs, stale-bid simulation'),
  'IEEE-SP/2011/repriv-re-imagining-content-personalization': OUT('defence', 'in-browser personalisation system'),
  'IMC/2011/understanding-fraudulent-activities-in-online-ad-exchanges': OUT('fraud'),
  'USENIX/2011/a-study-of-android-application-security': OUT('homograph', 'Android intent "retargeting"'),
  'CCS/2011/fashion-crimes-trending-term-exploitation': OUT('scam', 'MFA sites in trending-term search results'),
  'CCS/2012/knowing-your-enemy-understanding-and-detecting-malicious-web-advertising': IN(['crawlads'], ['content'], 'malvertising redirection chains, Alexa top 90K'),
  'IEEE-SP/2012/third-party-web-tracking-policy-and-technology': OUT('tracking'),
  'IMC/2012/breaking-for-commercials-characterizing-mobile-advertising': OUT('mention', 'mobile ad traffic volume and energy; no ad content, targeting or price'),
  'IMC/2013/understanding-the-effectiveness-of-video-ads': OUT('mechanism', 'CDN logs of ad completion'),
  'NDSS/2013/analyzing-unique-bid-auction-sites': OUT('homograph', 'consumer unique-bid auctions'),
  'USENIX/2013/effective-inter-component-communication-mapping': OUT('homograph', 'Android intent "retargeting"'),
  'WWW/2013/the-cost-of-annoying-ads': OUT('perception', 'annoyance of ad formats in a lab task; not about targeting (was CONTEXT until the generic review, 2026-09-23)'),
  'WWW/2013/your-browsing-behavior-for-a-big-mac': OUT('perception', 'PII valuation via reverse auctions'),
  'WWW/2013/towards-a-robust-modeling-of-temporal-interest-change': OUT('mechanism', 'behavioural-targeting model on server logs'),
  'CCS/2013/viceroi-catching-click-spam': OUT('fraud'),
  'CCS/2014/private-by-design-advertising-meets-the-real-world': OUT('defence', 'Privad deployment metrics'),
  'CCS/2014/the-web-never-forgets-persistent-tracking': OUT('tracking'),
  'CCS/2014/your-online-interests-pwned-a-pollution-attack': IN(['persona'], ['targeting', 'price'], 'profile pollution vs DoubleClick; attack paper'),
  'CCS/2014/characterizing-large-scale-click-fraud-in-zeroaccess': OUT('fraud'),
  'IMC/2014/the-dark-alleys-of-madison-avenue': IN(['crawlads'], ['content'], '673,596 ads collected by crawl'),
  'USENIX/2014/understanding-the-dark-side-of-domain-parking': OUT('fraud', 'parking monetisation chains'),
  'USENIX/2014/xray-enhancing-the-web-s-transparency': IN(['persona'], ['targeting'], 'differential correlation over shadow accounts'),
  'USENIX/2014/brahmastra-driving-apps-to-test': IN(['crawlads'], ['content'], 'in-app ad landing pages and child-inappropriate ads'),
  'WWW/2014/adscape-harvesting-and-analyzing-online-display-ads': IN(['persona', 'crawlads'], ['targeting'], '340 personas, 875,209 impressions'),
  'WWW/2014/from-devices-to-people-attribution': OUT('mechanism', 'search-log attribution'),
  'NDSS/2014/scambaiter-understanding-targeted-nigerian-scams': OUT('homograph', 'classified ads on Craigslist'),
  'CCS/2015/sunlight-fine-grained-targeting-detection': IN(['persona', 'crawlads'], ['targeting'], 'randomised synthetic profiles'),
  'IEEE-SP/2015/ad-injection-at-scale': OUT('injection'),
  'PETS/2015/automated-experiments-on-ad-privacy-settings': IN(['persona', 'explanation'], ['targeting'], 'AdFisher'),
  'WWW/2015/understanding-malvertising-through-ad-injecting': OUT('injection'),
  'NDSS/2015/parking-sensors-analyzing-and-detecting': OUT('scam', 'parked domains'),
  'PETS/2015/an-automated-approach-for-complementing-ad-blockers': OUT('blocking'),
  'WWW/2015/crowd-fraud-detection-in-internet-advertising': OUT('fraud'),
  // ---- 2016–2019
  'CCS/2016/predator-proactive-recognition': OUT('homograph', 'domain-registration "prices"'),
  'IMC/2016/recommended-for-you-a-first-look-at-content-recommendation': IN(['crawlads'], ['content', 'targeting'], 'CRN widgets and advertiser landing pages'),
  'IMC/2016/ad-blockers-global-prevalence-and-impact': OUT('blocking'),
  'IMC/2016/characterizing-website-behaviors-across-logged-in': OUT('tracking', 'ads counted as categorised FQDNs, not collected'),
  'IEEE-SP/2016/cloak-of-visibility': OUT('scam', 'advertisement cloaking'),
  'PETS/2016/access-denied-contrasting-data-access': OUT('perception'),
  'PETS/2016/do-not-track-me-sometimes': OUT('perception'),
  'PETS/2016/privacy-challenges-in-the-quantified-self': OUT('perception'),
  'USENIX/2016/internet-jones-and-the-raiders': OUT('tracking'),
  'USENIX/2016/tracing-information-flows-between-ad-exchanges': IN(['retarget', 'persona', 'crawlads'], ['targeting'], 'retargeted ads as evidence of exchange-to-exchange flows'),
  'USENIX/2016/investigating-commercial-pay-per-install': OUT('mention'),
  'USENIX/2016/measuring-pup-prevalence': OUT('mention'),
  'USENIX/2016/protecting-privacy-of-ble-device-users': OUT('homograph', 'BLE advertisements'),
  'WWW/2016/an-empirical-study-of-web-cookies': OUT('tracking'),
  'CCS/2017/use-privacy-in-data-driven-systems': OUT('mention'),
  'IMC/2017/if-you-are-not-paying-for-it': IN(['rtbprice', 'advertiser'], ['price'], 'cleartext and encrypted RTB prices + two own campaigns'),
  'IMC/2017/exploring-the-dynamics-of-search-advertiser-fraud': OUT('fraud'),
  'IMC/2017/the-ad-wars-retrospective-measurement': OUT('blocking'),
  'IEEE-SP/2017/how-to-learn-klingon-without-a-dictionary': OUT('scam', 'SEO black keywords'),
  'NDSS/2017/dial-one-for-scam': OUT('scam', 'malvertising only as the route to TSS pages'),
  'NDSS/2017/are-we-there-yet-on-rpki': OUT('homograph', 'BGP route advertisements'),
  'PETS/2017/cross-device-tracking-measurement': OUT('tracking'),
  'PETS/2017/on-the-privacy-and-security-of-the-ultrasound': OUT('tracking'),
  'PETS/2017/detecting-anti-ad-blockers-in-the-wild': OUT('blocking'),
  'USENIX/2017/exploring-user-perceptions-of-discrimination': CTX('scenario survey on discriminatory targeting'),
  'USENIX/2017/a-privacy-analysis-of-cross-device-tracking': OUT('tracking'),
  'WWW/2017/j-force-forced-execution': OUT('mention'),
  'WWW/2017/security-implications-of-redirection-trail': OUT('mention'),
  'IEEE-SP/2018/privacy-risks-with-facebooks-pii-based-targeting': IN(['advertiser'], ['targeting'], 'custom-audience interface as an oracle'),
  'IEEE-SP/2018/the-spyware-used-in-intimate-partner-violence': OUT('mention'),
  'IEEE-SP/2018/crowd-gps-sec': OUT('homograph', 'ADS-B'),
  'IEEE-SP/2018/surveylance': OUT('scam'),
  'IMC/2018/following-their-footsteps': OUT('homograph'),
  'IMC/2018/how-tracking-companies-circumvented-ad-blockers': OUT('tracking'),
  'IMC/2018/tracing-cross-border-web-tracking': OUT('tracking'),
  'PETS/2018/nomoads-effective-and-efficient': OUT('blocking'),
  'PETS/2018/undermining-privacy-in-the-aircraft': OUT('homograph', 'ADS-B / ACARS'),
  'PETS/2018/when-the-cookie-meets-the-blockchain': OUT('tracking'),
  'USENIX/2018/unveiling-and-quantifying-facebook-exploitation': IN(['explanation', 'donation', 'advertiser'], ['targeting'], 'FDVT ad preferences + own campaigns'),
  'WWW/2018/adbudgetkiller': IN(['retarget'], ['targeting'], 'inferred retargeting strategies; attack paper'),
  'WWW/2018/the-cost-of-digital-advertisement': IN(['rtbprice'], ['price'], 'passive RTB prices, 1,270 users'),
  'WWW/2018/your-secrets-are-safe': OUT('perception'),
  'WWW/2018/exposing-search-and-advertisement-abuse-tactics': IN(['crawlads'], ['content'], 'search ads collected daily across five engines'),
  'CCS/2019/oh-the-places-youve-been': CTX('reactions to tracking inferences; interests inferred by the authors, not by a platform'),
  'CCS/2019/privacy-risks-of-securing-machine-learning': OUT('homograph', 'targeted adversarial examples'),
  'CCS/2019/adversarial-perceptual-ad-blocking': OUT('blocking'),
  'CCS/2019/watching-you-watch': OUT('tracking'),
  'IEEE-SP/2019/measuring-and-analyzing-search-engine-poisoning': OUT('scam'),
  'IMC/2019/a-longitudinal-analysis-of-the-ads-txt-standard': OUT('supplychain', 'RTB inclusion chains serve the ads.txt compliance test'),
  'IMC/2019/no-more-chasing-waterfalls': IN(['bids'], ['price'], 'HB bids from a clean crawler; shared with privacy:ads_txt'),
  'IMC/2019/what-you-see-is-not-what-you-get': IN(['crawlads'], ['content'], 'SE ad campaigns clustered from screenshots'),
  'IMC/2019/errors-misunderstandings-and-attacks': OUT('blocking'),
  'NDSS/2019/measuring-the-facebook-advertising-ecosystem': IN(['donation', 'explanation'], ['targeting'], 'AdAnalyst, 622 volunteers'),
  'NDSS/2019/quantity-vs-quality-evaluating-user-interest-profiles': IN(['explanation'], ['targeting'], 'ad-preference-manager profiles vs participants'),
  'NDSS/2019/master-of-web-puppets': OUT('mention'),
  'NDSS/2019/nearby-threats': OUT('homograph', 'BLE advertisements'),
  'PETS/2019/investigating-sources-of-pii-used-in-facebook': IN(['advertiser'], ['targeting'], 'reach estimates + own campaigns on controlled PII'),
  'PETS/2019/what-if-predicting-individual-users': OUT('perception'),
  'PETS/2019/4-years-of-eu-cookie-law': OUT('tracking'),
  'USENIX/2019/on-the-lack-of-location-privacy': OUT('homograph'),
  'USENIX/2019/policylint': OUT('mention'),
  'WWW/2019/auditing-offline-data-brokers-via-facebooks': IN(['advertiser', 'explanation'], ['targeting'], 'broker attributes via the ad platform'),
  'WWW/2019/before-and-after-gdpr': OUT('tracking'),
  'WWW/2019/cookie-synchronization-everything': OUT('tracking'),
  'WWW/2019/nameles': OUT('fraud'),
  'WWW/2019/quality-effects-on-user-preferences': OUT('homograph'),
  'WWW/2019/revisiting-mobile-advertising-threats-with-madlife': IN(['crawlads'], ['content'], '84K in-app ads and landing pages'),
  'WWW/2019/dealing-with-interdependencies-and-uncertainty': OUT('mechanism'),
  'WWW/2019/measuring-political-personalization-of-google-news': OUT('recommendation'),
  'WWW/2019/multiple-treatment-effect-estimation': OUT('mechanism'),
  'WWW/2019/quality-sensitive-training': OUT('mechanism'),
  'WWW/2019/the-chain-of-implicit-trust': OUT('tracking'),
  // ---- 2020–2023
  'IEEE-SP/2020/a-security-analysis-of-the-facebook-ad-library': IN(['archive'], ['content'], 'Ad Library audit'),
  'IEEE-SP/2020/do-cookie-banners-respect-my-choice': OUT('tracking'),
  'IEEE-SP/2020/ex-vivo-dynamic-analysis': OUT('homograph'),
  'IMC/2020/when-push-comes-to-ads': IN(['crawlads'], ['content'], 'web-push ad notifications collected and clustered'),
  'NDSS/2020/withdrawing-the-bgp-re-routing-curtain': OUT('homograph', 'BGP route advertisements'),
  'PETS/2020/canarytrap': IN(['explanation'], ['targeting'], "Facebook's advertiser-uploaded-your-info list as ground truth for honeytoken misuse"),
  'PETS/2020/inferring-tracker-advertiser-relationships': IN(['bids', 'persona'], ['targeting', 'price'], 'HB bids as the dependent variable'),
  'PETS/2020/the-price-is-not-right': OUT('homograph'),
  'PETS/2020/in-depth-evaluation-of-redirect-tracking': OUT('tracking'),
  'PETS/2020/the-tv-is-smart-and-full-of-trackers': OUT('tracking'),
  'USENIX/2020/the-impact-of-ad-blockers-on-product-search': OUT('perception', 'lab experiment with contextual ads blocked'),
  'USENIX/2020/what-twitter-knows': IN(['donation', 'explanation'], ['targeting'], "231 participants' own Twitter ad files"),
  'USENIX/2020/actions-speak-louder-than-words': OUT('mention'),
  'WWW/2020/apophanies-or-epiphanies': OUT('mention'),
  'WWW/2020/beyond-the-front-page': OUT('tracking'),
  'WWW/2020/facebook-ads-monitor': IN(['donation', 'archive'], ['content'], 'volunteer-collected Facebook ads vs Ad Library'),
  'WWW/2020/finding-a-choice-in-a-haystack': OUT('mention'),
  'WWW/2020/stop-tracking-me-bro': IN(['persona', 'rtbprice'], ['price', 'targeting'], 'RTB prices per demographic persona'),
  'WWW/2020/deconstructing-googles-web-light': OUT('mention'),
  'WWW/2020/filter-list-generation-for-underserved': OUT('blocking'),
  'WWW/2020/maddroid': IN(['crawlads'], ['content'], 'devious in-app ad contents'),
  'CCS/2021/black-box-adversarial-attacks': OUT('homograph'),
  'CCS/2021/consistency-analysis-of-data-usage': OUT('mention'),
  'CCS/2021/encodermi': OUT('homograph'),
  'CCS/2021/this-sneaky-piggy': OUT('tracking', "ads' sensor access, not their content"),
  'CCS/2021/understanding-and-detecting-mobile-ad-fraud': OUT('fraud'),
  'CCS/2021/dissecting-click-fraud-autonomy': OUT('fraud'),
  'IEEE-SP/2021/linking-bluetooth-le-classic': OUT('homograph', 'BLE advertisements'),
  'IMC/2021/polls-clickbait-and-commemorative': IN(['crawlads'], ['content'], '1.4M ads from 745 news sites, six locations'),
  'IMC/2021/unique-on-facebook': IN(['donation', 'advertiser'], ['targeting', 'delivery'], 'FDVT interests + 21 nanotargeting campaigns'),
  'NDSS/2021/the-abuser-inside-apps': OUT('fraud'),
  'NDSS/2021/understanding-worldwide-private-information': OUT('tracking'),
  'NDSS/2021/cv-inspector': OUT('blocking'),
  'NDSS/2021/trust-the-crowd': OUT('homograph', 'ADS-B'),
  'PETS/2021/a-calculus-of-tracking': OUT('tracking'),
  'PETS/2021/awareness-adoption-and-misconceptions': OUT('perception'),
  'PETS/2021/did-you-know-this-camera': OUT('perception'),
  'PETS/2021/managing-potentially-intrusive-practices': OUT('perception'),
  'PETS/2021/who-can-find-my-devices': OUT('homograph', 'BLE advertisements'),
  'USENIX/2021/adcube': OUT('fraud', 'WebVR ad fraud attacks'),
  'USENIX/2021/are-privacy-dashboards-good': OUT('perception'),
  'USENIX/2021/disrupting-continuity': OUT('homograph', 'BLE advertisements'),
  'USENIX/2021/share-first-ask-later': OUT('tracking'),
  'WWW/2021/user-tracking-in-the-post-cookie-era': OUT('tracking'),
  'WWW/2021/websocket-adoption': OUT('tracking'),
  'CCS/2022/cart-ology': IN(['retarget'], ['targeting'], 'identity entanglement read out through retargeted ads'),
  'CCS/2022/do-opt-outs-really-opt-me-out': OUT('dsr', 'traffic after industry opt-outs'),
  'CCS/2022/privacy-limitations-of-interest-based-advertising': OUT('sandbox', 'FLoC recomputed from comScore'),
  'CCS/2022/when-evil-calls': OUT('homograph'),
  'CCS/2022/freely-given-consent': OUT('tracking'),
  'IEEE-SP/2022/investigating-influencer-vpn-ads': OUT('sponsorship'),
  'IMC/2022/illuminating-large-scale-ipv6-scanning': OUT('homograph', 'scan "targeting"'),
  'IMC/2022/measurement-and-analysis-of-implied-identity': IN(['advertiser'], ['delivery'], 'paired campaigns with synthetic faces'),
  'IMC/2022/what-factors-affect-targeting-and-bids': IN(['donation', 'bids'], ['targeting', 'price'], '286 participants, extension'),
  'NDSS/2022/auto-draft-209': IN(['persona', 'bids'], ['targeting', 'price'], 'HARPO: obfuscation scored on real bids and segments'),
  'PETS/2022/atom-ad-network-tomography': IN(['persona', 'crawlads'], ['targeting'], 'isolated personas, ad creatives as features'),
  'PETS/2022/charting-app-developers-journey': OUT('perception'),
  'PETS/2022/are-iphones-really-better': OUT('tracking'),
  'PETS/2022/how-can-and-would-people-protect': OUT('perception'),
  'USENIX/2022/an-audit-of-facebooks-political-ad-policy': IN(['archive'], ['content'], 'Ad Library enforcement audit'),
  'USENIX/2022/khaleesi': OUT('blocking'),
  'USENIX/2022/mistrust-plugins-you-must': OUT('mention'),
  'USENIX/2022/webgraph': OUT('blocking'),
  'WWW/2022/conspiracy-brokers': IN(['crawlads'], ['content'], 'YouTube ads on conspiracy vs mainstream videos'),
  'WWW/2022/investigating-advertisers-domain-changing': OUT('blocking'),
  'CCS/2023/marketing-to-children-through-online-targeted': IN(['persona', 'explanation', 'advertiser'], ['targeting'], 'six profiles, 3,221 ads + seven own campaigns'),
  'IEEE-SP/2023/blue-is-the-new-black-market': OUT('homograph', 'police auctions'),
  'IEEE-SP/2023/collaborative-ad-transparency': IN(['donation', 'advertiser'], ['targeting', 'delivery'], '420 volunteers + 79 own campaigns'),
  'IEEE-SP/2023/when-and-why-do-people-want-ad-targeting': CTX('4,251 ads logged, but the question is when people want an explanation'),
  'IMC/2023/tracking-profiling-and-ad-targeting-in-the-alexa': IN(['persona', 'bids'], ['targeting', 'price'], 'Echo personas, web bids'),
  'IMC/2023/understanding-the-privacy-risks-of-popular-search': OUT('tracking', 'ad-click redirect chains studied for identifier leakage'),
  'NDSS/2023/i-still-know-what-you-watched': OUT('tracking'),
  'NDSS/2023/navigating-murky-waters': OUT('tracking'),
  'PETS/2023/a-utility-preserving-obfuscation-approach': OUT('recommendation'),
  'PETS/2023/is-there-a-reverse-privacy-paradox': OUT('perception'),
  'PETS/2023/locality-sensitive-hashing-does-not-guarantee': OUT('sandbox'),
  'PETS/2023/surprised-shocked-worried': CTX('reactions to Off-Facebook Activity'),
  'PETS/2023/privacy-rarely-considered': OUT('tracking'),
  'PETS/2023/usability-and-enforceability-of-global': OUT('dsr'),
  'USENIX/2023/problematic-advertising-and-its-disparate': IN(['donation'], ['content', 'targeting'], '32,587 panel ads coded'),
  'USENIX/2023/trident': OUT('defence', 'SE-ad navigation detector'),
  'USENIX/2023/autofr': OUT('blocking'),
  'USENIX/2023/defining-broken': OUT('blocking'),
  'WWW/2023/ad-auction-design-with-coupon': OUT('mechanism'),
  'WWW/2023/fairness-aware-guaranteed-display': OUT('mechanism'),
  'WWW/2023/online-advertising-in-ukraine-and-russia': IN(['crawlads'], ['content'], 'ads on 500 news sites from three proxy locations'),
  'WWW/2023/propaganda-politica-pagada': IN(['archive'], ['content'], 'Ad Library API'),
  'WWW/2023/the-thin-ideology-of-populist': IN(['archive'], ['content'], 'Meta Ad Library'),
  'WWW/2023/are-mobile-advertisements-in-compliance': IN(['crawlads'], ['content'], '11,270 in-app ad views'),
  'WWW/2023/who-funds-misinformation': IN(['crawlads'], ['content'], 'served ads on fake-news sites; shared with privacy:ads_txt'),
  // ---- 2024–2026
  'CCS/2024/unveiling-collusion-based-ad-attribution': OUT('fraud'),
  'CCS/2024/the-privacy-utility-trade-off-in-the-topics': OUT('sandbox'),
  'IEEE-SP/2024/multi-instance-adversarial-attack': OUT('homograph'),
  'IEEE-SP/2024/sok-technical-implementation-and-human': OUT('mention'),
  'IEEE-SP/2024/targeted-and-troublesome': IN(['crawlads'], ['content', 'targeting'], "ads on 2,004 children's sites from seven vantages"),
  'IEEE-SP/2024/understanding-the-privacy-practices-of-political': OUT('tracking'),
  'IMC/2024/analyzing-the-in-accessibility-of-online': IN(['crawlads'], ['content'], 'ads on 90 sites for 31 days'),
  'IMC/2024/beyond-the-guidelines': IN(['archive'], ['content'], 'Meta Ad Library, 16 EU countries'),
  'IMC/2024/diffaudit': OUT('tracking'),
  'IMC/2024/watching-tv-with-the-second-party': OUT('tracking'),
  'IMC/2024/whatcha-lookin-at': OUT('tracking'),
  'NDSS/2024/anonpsi': OUT('homograph'),
  'PETS/2024/client-side-and-server-side-tracking-on-meta': IN(['advertiser'], ['targeting'], "campaign reach as the oracle for Pixel/CAPI matching"),
  'PETS/2024/cross-contextual-examination-of-older': OUT('perception'),
  'PETS/2024/crumbling-cookie-categories': OUT('perception'),
  'PETS/2024/exploring-the-privacy-experiences-of-closeted': OUT('perception'),
  'PETS/2024/how-does-connecting-online-activities': IN(['explanation'], ['targeting'], "participants' own labels vs Google Ad Settings: 21% agreement"),
  'PETS/2024/interest-disclosing-mechanisms': OUT('sandbox'),
  'PETS/2024/johnny-still-cant-opt-out': OUT('dsr'),
  'PETS/2024/measuring-conditional-anonymity': OUT('homograph'),
  'PETS/2024/opted-out-yet-tracked': IN(['persona', 'bids'], ['targeting', 'price'], 'bids after GDPR/CCPA/NAI opt-outs'),
  'PETS/2024/overprofiling-analysis-on-major': IN(['explanation', 'donation'], ['targeting'], '45,310 interests rated, 1,477 interest-targeted ads'),
  'PETS/2024/what-do-privacy-advertisements': OUT('perception', 'ads about privacy features'),
  'PETS/2024/what-does-it-mean-to-be-creepy': CTX('design probe; interests inferred by the authors'),
  'PETS/2024/website-data-transparency-in-the-browser': OUT('tracking'),
  'PETS/2024/what-to-expect-when-you-re-accessing': OUT('dsr'),
  'USENIX/2024/attention-please': OUT('tracking'),
  'USENIX/2024/false-claims-against-model-ownership': OUT('homograph'),
  'USENIX/2024/fledging-will-continue': OUT('sandbox', 'Protected Audience adoption and attacks'),
  'USENIX/2024/data-subjects-reactions': OUT('dsr'),
  'USENIX/2024/im-not-convinced': OUT('perception'),
  'USENIX/2024/navigating-the-privacy-compliance-maze': OUT('tracking'),
  'USENIX/2024/security-and-privacy-analysis-of-samsungs': OUT('homograph', 'BLE advertisements'),
  'WWW/2024/a-study-of-gdpr-compliance-under-the-transparency': OUT('tracking'),
  'WWW/2024/adflush': OUT('blocking'),
  'CCS/2025/exploiting-the-shared-storage-api': OUT('sandbox'),
  'CCS/2025/piixel-leaks': OUT('tracking'),
  'CCS/2025/whispertest': OUT('mention'),
  'IEEE-SP/2025/analyzing-ad-prevalence-characteristics': IN(['crawlads'], ['content'], 'audio ads in 45,477 Alexa skills'),
  'IEEE-SP/2025/characterizing-the-usability-and-usefulness': CTX('usability of 22 ad-transparency systems'),
  'IEEE-SP/2025/decentralization-of-ethereums-builder': OUT('homograph', 'MEV-Boost bids'),
  'IEEE-SP/2025/understanding-users-security-and-privacy': OUT('mention'),
  'IEEE-SP/2025/identifying-incoherent-search-sessions': OUT('fraud'),
  'IMC/2025/cookieguard': OUT('tracking'),
  'IMC/2025/from-voice-to-ads': IN(['persona'], ['targeting'], '42 cloned-voice puppets'),
  'NDSS/2025/transparency-or-information-overload': OUT('perception'),
  'NDSS/2025/careful-about-what-app-promotion': IN(['crawlads'], ['content'], 'app-promotion ads collected by driving apps'),
  'NDSS/2025/ctrlaltdeceive': OUT('scam'),
  'PETS/2025/echoes-of-privacy': IN(['persona', 'explanation'], ['targeting'], 'persona accounts vs disclosed profiling labels'),
  'PETS/2025/evaluating-a-data-fiduciary': OUT('perception'),
  'PETS/2025/it-s-not-my-data-anymore': OUT('homograph', 'medical data donation'),
  'PETS/2025/misalignments-and-demographic-differences': CTX('Facebook settings incl. ad profile, as a perception study'),
  'PETS/2025/more-and-scammier-ads': IN(['persona', 'explanation', 'crawlads'], ['content', 'targeting'], 'YouTube pre-roll ads under privacy settings'),
  'PETS/2025/navigating-social-media-privacy': OUT('perception'),
  'PETS/2025/privacy-perceptions-and-behaviors-towards-targeted': CTX('cross-country survey on targeted ads'),
  'PETS/2025/sheeps-clothing-wolfish-intent': IN(['crawlads'], ['content'], "Acceptable Ads, 4,710 pages, US and Germany"),
  'PETS/2025/tracking-without-borders': OUT('tracking'),
  'PETS/2025/unmasking-the-shadows': OUT('tracking'),
  'PETS/2025/intractable-cookie-crumbs': OUT('tracking'),
  'PETS/2025/okay-google-where-s-my-tracker': OUT('homograph', 'BLE advertisements'),
  'PETS/2025/privacy-settings-of-third-party-libraries': OUT('tracking'),
  'PETS/2025/the-effect-of-platform-policies-on-app': OUT('tracking'),
  'PETS/2025/the-impact-of-default-mobile-sdk': OUT('tracking'),
  'PETS/2025/user-centric-textual-descriptions': OUT('perception'),
  'PETS/2025/your-signal-their-data': OUT('tracking'),
  'USENIX/2025/lemix': OUT('homograph'),
  'USENIX/2025/shadowed-realities': OUT('mention'),
  'USENIX/2025/addressing-the-address-books': OUT('perception'),
  'USENIX/2025/as-advertised': OUT('sponsorship'),
  'WWW/2025/the-first-early-evidence-of-the-use-of-browser': IN(['persona', 'bids'], ['targeting', 'price'], 'FPTrace: bids under fingerprint changes'),
  'WWW/2025/welcome-to-the-dark-side': OUT('supplychain'),
  'WWW/2025/assessing-compliance-in-digital-advertising': IN(['crawlads'], ['content'], 'Acceptable Ads compliance on 10,000 domains'),
  'WWW/2025/before-after-the-effect-of-eus-2022': IN(['crawlads'], ['content'], 'served ads and brand adjacency; shared with privacy:ads_txt'),
  'IEEE-SP/2026/setting-the-course-but-forgetting': OUT('dsr'),
  'NDSS/2026/side-channel-inference-of-user-activities': OUT('homograph'),
  'NDSS/2026/the-role-of-privacy-guarantees': OUT('homograph', 'health data donation'),
  'NDSS/2026/chameleoscan': OUT('mention'),
  'NDSS/2026/sok-cryptographic-authenticated': OUT('homograph'),
  'PETS/2026/a-year-under-the-dsa': IN(['explanation', 'donation', 'archive'], ['targeting', 'content'], '48,511 explanations vs repositories'),
  'PETS/2026/ad-personalization-and-transparency-in-mobile': IN(['persona', 'explanation'], ['targeting'], 'controlled app-store accounts'),
  'PETS/2026/clicking-into-exposure': OUT('tracking', 'gclid on YouTube ad clicks'),
  'PETS/2026/the-empire-strikes-back': OUT('tracking'),
  'PETS/2026/the-tcf-doesn-t-really-a-a-id': OUT('tracking'),
  'PETS/2026/designing-reflective-thinking-based': OUT('perception'),
  'PETS/2026/exercising-the-ccpa-opt-out-right': OUT('dsr'),
  'PETS/2026/i-don-t-think-it-needs-to-be-political': OUT('perception'),
  'PETS/2026/no-privacy-for-privates': OUT('tracking'),
  'PETS/2026/privacy-vs-profit': OUT('blocking'),
  'USENIX/2026/privacyshield': OUT('homograph', 'BLE advertisements'),
  'USENIX/2026/security-and-privacy-analysis-of-tiles': OUT('homograph', 'BLE advertisements'),
  'USENIX/2026/when-fun-turns-toxic': IN(['crawlads'], ['content'], 'aggressive ad behaviour in mini-games'),
  'WWW/2026/bowling-with-chatgpt': OUT('homograph', 'donated chat logs'),
  'WWW/2026/does-ad-free-mean-less-data-collection': OUT('dsr'),
  'WWW/2026/when-ads-become-profiles': IN(['donation'], ['targeting'], 'Australian Ad Observatory exposures re-analysed with Gemini'),
};

// Resolve a verdict prefix against the candidate list. Throws on zero or
// several matches, so a slug that changes upstream fails loudly.
export function resolveVerdicts(candidateKeys) {
  const byPrefix = new Map();
  for (const pref of Object.keys(VERDICTS)) {
    const hits = candidateKeys.filter((k) => k.startsWith(pref));
    if (hits.length !== 1) throw new Error(`verdict prefix ${pref} matches ${hits.length} candidates`);
    byPrefix.set(hits[0], { key: hits[0], prefix: pref, ...VERDICTS[pref] });
  }
  const missing = candidateKeys.filter((k) => !byPrefix.has(k));
  if (missing.length) throw new Error(`FAILURE: ${missing.length} candidates have no verdict:\n  ${missing.join('\n  ')}`);
  for (const r of byPrefix.values()) {
    if (r.v === 'OUT' && !OUT_CODES[r.code]) throw new Error(`unknown OUT code ${r.code} on ${r.key}`);
    if (r.v === 'IN') {
      for (const o of r.obs) if (!OBSERVABLES[o]) throw new Error(`unknown observable ${o} on ${r.key}`);
      for (const q of r.q) if (!QUESTIONS.includes(q)) throw new Error(`unknown question ${q} on ${r.key}`);
    }
  }
  return byPrefix;
}
</file>

===== Bibliography =====

Entries appended to [[literature:bibliography]] in this sitting. PoPETs and USENIX authors from the landing pages' own metadata, cross-checked against the paper's title block; NDSS and PoPETs DOIs from Crossref. //Recommended For You// is entered without its title's quotation marks.

<file bibtex bib_additions_advertising.bib>
@inproceedings{zarras2014_dark,
  author        = {Zarras, Apostolis and Kapravelos, Alexandros and Stringhini, Gianluca and Holz, Thorsten and Kruegel, Christopher and Vigna, Giovanni},
  title         = {The Dark Alleys of Madison Avenue: Understanding Malicious Advertisements},
  booktitle     = {Proceedings of the ACM Internet Measurement Conference},
  year          = {2014},
  series        = {IMC 2014},
  doi           = {10.1145/2663716.2663719},
}

@inproceedings{barford2014_adscape,
  author        = {Barford, Paul and Canadi, Igor and Krushevskaja, Darja and Ma, Qiang and Muthukrishnan, S.},
  title         = {Adscape: Harvesting and Analyzing Online Display Ads},
  booktitle     = {Proceedings of the ACM Web Conference},
  year          = {2014},
  series        = {TheWebConf 2014},
  doi           = {10.1145/2566486.2567992},
}

@inproceedings{bashir2016_recommended,
  author        = {Bashir, Muhammad Ahmad and Arshad, Sajjad and Wilson, Christo},
  title         = {Recommended For You: A First Look at Content Recommendation Networks},
  booktitle     = {Proceedings of the ACM Internet Measurement Conference},
  year          = {2016},
  series        = {IMC 2016},
  doi           = {10.1145/2987443.2987469},
}

@inproceedings{plane2017_exploring,
  author        = {Plane, Angelisa C. and Redmiles, Elissa M. and Mazurek, Michelle L. and Tschantz, Michael Carl},
  title         = {Exploring User Perceptions of Discrimination in Online Targeted Advertising},
  booktitle     = {Proceedings of the USENIX Security Symposium},
  year          = {2017},
  series        = {USENIX Security 2017},
  url           = {https://www.usenix.org/conference/usenixsecurity17/technical-sessions/presentation/plane},
}

@inproceedings{cabanas2018_unveiling,
  author        = {González-Cabañas, José and Cuevas, Ángel and Cuevas, Rubén},
  title         = {Unveiling and Quantifying Facebook Exploitation of Sensitive Personal Data for Advertising Purposes},
  booktitle     = {Proceedings of the USENIX Security Symposium},
  year          = {2018},
  series        = {USENIX Security 2018},
  url           = {https://www.usenix.org/conference/usenixsecurity18/presentation/cabanas},
}

@inproceedings{andreou2018_investigating,
  author        = {Andreou, Athanasios and Venkatadri, Giridhari and Goga, Oana and Gummadi, Krishna P. and Loiseau, Patrick and Mislove, Alan},
  title         = {Investigating Ad Transparency Mechanisms in Social Media: A Case Study of Facebook's Explanations},
  booktitle     = {Proceedings of the Network and Distributed System Security Symposium},
  year          = {2018},
  series        = {NDSS 2018},
  doi           = {10.14722/ndss.2018.23191},
}

@inproceedings{olejnik2014_selling,
  author        = {Olejnik, Lukasz and Tran, Minh-Dung and Castelluccia, Claude},
  title         = {Selling Off Privacy at Auction},
  booktitle     = {Proceedings of the Network and Distributed System Security Symposium},
  year          = {2014},
  series        = {NDSS 2014},
  doi           = {10.14722/ndss.2014.23270},
}

@inproceedings{vadrevu2019_discovering,
  author        = {Vadrevu, Phani and Perdisci, Roberto},
  title         = {What You See is NOT What You Get: Discovering and Tracking Social Engineering Attack Campaigns},
  booktitle     = {Proceedings of the ACM Internet Measurement Conference},
  year          = {2019},
  series        = {IMC 2019},
  doi           = {10.1145/3355369.3355600},
}

@inproceedings{andreou2019_facebook,
  author        = {Andreou, Athanasios and Silva, Márcio and Benevenuto, Fabrício and Goga, Oana and Loiseau, Patrick and Mislove, Alan},
  title         = {Measuring the Facebook Advertising Ecosystem},
  booktitle     = {Proceedings of the Network and Distributed System Security Symposium},
  year          = {2019},
  series        = {NDSS 2019},
  doi           = {10.14722/ndss.2019.23280},
}

@inproceedings{chen2019_madlife,
  author        = {Chen, Gong and Meng, Wei and Copeland, John A.},
  title         = {Revisiting Mobile Advertising Threats with MAdLife},
  booktitle     = {Proceedings of the ACM Web Conference},
  year          = {2019},
  series        = {TheWebConf 2019},
  doi           = {10.1145/3308558.3313549},
}

@inproceedings{cabanas2021_unique,
  author        = {González-Cabañas, José and Cuevas, Ángel and Cuevas, Rubén and López-Fernández, Juan and García, David},
  title         = {Unique on Facebook: Formulation and Evidence of (Nano)targeting Individual Users with non-PII Data},
  booktitle     = {Proceedings of the ACM Internet Measurement Conference},
  year          = {2021},
  series        = {IMC 2021},
  doi           = {10.1145/3487552.3487861},
}

@inproceedings{ballard2022_conspiracy,
  author        = {Ballard, Cameron and Goldstein, Ian and Mehta, Pulak and Smothers, Genesis and Take, Kejsi and Zhong, Victoria and Greenstadt, Rachel and Lauinger, Tobias and McCoy, Damon},
  title         = {Conspiracy Brokers: Understanding the Monetization of YouTube Conspiracy Theories},
  booktitle     = {Proceedings of the ACM Web Conference},
  year          = {2022},
  series        = {TheWebConf 2022},
  doi           = {10.1145/3485447.3512142},
}

@inproceedings{yeung2024_accessibility,
  author        = {Yeung, Christina and Kohno, Tadayoshi and Roesner, Franziska},
  title         = {Analyzing the (In)Accessibility of Online Advertisements},
  booktitle     = {Proceedings of the ACM Internet Measurement Conference},
  year          = {2024},
  series        = {IMC 2024},
  doi           = {10.1145/3646547.3688427},
}

@inproceedings{farke2024_connecting,
  author        = {Farke, Florian M. and Balash, David G. and Golla, Maximilian and Aviv, Adam J.},
  title         = {How Does Connecting Online Activities to Advertising Inferences Impact Privacy Perceptions?},
  booktitle     = {Proceedings on Privacy Enhancing Technologies},
  year          = {2024},
  series        = {PoPETs 2024},
  doi           = {10.56553/popets-2024-0055},
}

@inproceedings{sabir2025_alexa,
  author        = {Sabir, Aafaq and B., Abhinaya S. and Ahmed, Dilawer and Das, Anupam},
  title         = {Analyzing Ad Prevalence, Characteristics, and Compliance in Alexa Skills},
  booktitle     = {Proceedings of the IEEE Symposium on Security and Privacy},
  year          = {2025},
  series        = {IEEE S&P 2025},
  doi           = {10.1109/sp61157.2025.00257},
}

@inproceedings{kaushik2025_perceptions,
  author        = {Kaushik, Smirity and Sharma, Tanusree and Yu, Yaman and Ali, Amna and Knijnenburg, Bart Piet and Wang, Yang and Zou, Yixin},
  title         = {Privacy Perceptions and Behaviors Towards Targeted Advertising on Social Media: A Cross-Country Study on the Effect of Culture and Religion},
  booktitle     = {Proceedings on Privacy Enhancing Technologies},
  year          = {2025},
  series        = {PoPETs 2025},
  doi           = {10.56553/popets-2025-0057},
}

@article{boerman2017_oba,
  author        = {Boerman, Sophie C. and Kruikemeier, Sanne and Zuiderveen Borgesius, Frederik J.},
  title         = {Online Behavioral Advertising: A Literature Review and Research Agenda},
  journal       = {Journal of Advertising},
  volume        = {46},
  number        = {3},
  pages         = {363--376},
  year          = {2017},
  doi           = {10.1080/00913367.2017.1339368},
}

@article{angus2024_donation,
  author        = {Angus, Daniel and Obeid, Abdul Karim and Burgess, Jean and Parker, Christine and Andrejevic, Mark and Carah, Nicholas and Tan, Xue Ying},
  title         = {Enabling Online Advertising Transparency through Data Donation Methods},
  journal       = {Computational Communication Research},
  volume        = {6},
  number        = {2},
  year          = {2024},
  doi           = {10.5117/ccr2024.2.6.angu},
}
</file>

====== References ======

<bibtex bibliography></bibtex>