Measure The Web

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Public Relations ======

Your paper says "of the 10,000 sites we could load from Frankfurt in March 2024, 74% set a third-party cookie before any interaction with the consent banner". The headline says "Three quarters of the web tracks you illegally". Both sentences describe the same crawl. Only one of them is yours, and it is the other one that your funding agency, your future co-authors and the company you measured will read.

This page is about the mechanics of that trip: **which sentence in your paper is the one that travels, what it loses on the way, and what you can prepare in advance so that what arrives is still true.** It is not media training, and it is not about whether to talk to the press at all — by the time this matters, the decision has usually been taken by someone else, because the paper is public.

<WRAP important>
**The single most consequential thing on this page is that the number and its denominator have to travel in the same sentence.** Across the 27,241 measured findings extracted from the 5,859-paper corpus behind this site, 11,523 are stated in a sentence containing a percentage — and **47.9% of those sentences do not say what the percentage is a percentage //of//**. Read that as what it is: a regex over one model-selected sentence per finding, not an audit of how papers are written. It says half of the sentences //most likely to be lifted out of a paper// are not self-contained; it does not say the papers omit their denominators, and mostly they do not. A sentence that does not carry its own denominator cannot survive being quoted, because quoting is exactly the operation that removes the surrounding paragraph. Section //[[#the_denominator_has_to_travel_with_the_number|The denominator has to travel with the number]]// gives the measurement and the residue.
</WRAP>

===== How little of this the field documents =====

Before taking any advice from this page, know how thin the evidence under it is. The corpus behind this site is 5,859 papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026, with a structured record per paper. **It has no field for press engagement**, so the question has to be asked of the free text and of the papers themselves. Doing that turns up three papers.

^ Question ^ Population ^ N ^ Answer ^
| Papers whose ethics text mentions press, media, journalists, reporters or an embargo | empirical ∧ has an ethics record | 4,472 | 34 (0.8%) |
| … of those 34, the mention is a **vendor disclosure embargo**, not the press | — | 34 | 16 (47.1%) |
| … the word "media" in "social media", "media payload", "media documents" | — | 34 | 14 (41.2%) + 2 in residue |
| … the word "press" in a genuine press sense — and **neither is about the paper's own press engagement** (see below) | — | 34 | 2 (5.9%) |
| Papers whose **full text** mentions any of 20 press/media terms | all ''paper.cols.txt'' files((5,869 files against 5,859 extraction records: a few papers have retrievable full text but no extraction record. Both denominators are named where they are used.)) | 5,869 | 674 (11.5%) |
| Papers matching a broad **self-reference** regex ("inquiries from reporters", "media outreach", "press office", "pursued public disclosure", …) | all ''paper.cols.txt'' files | 5,869 | 80 candidates |
| … of those 80, actually describing press engagement with **their own results**, after reading every hit | — | 80 | **3** |

The two "press contact" papers in the ethics text are not what you would hope. One used companies' public press contacts as a channel for reaching the companies it had measured — a notification channel, not a press strategy — and reports that "none responded". The other anonymises case-study details "when those details appear in the public press", which is a redaction rule.

So: **the extraction's ethics fields, over 4,472 papers, contain no description of how any of them handled their own results going public.** That is a statement about a 20-word structured field, and the right response to it is to go and read the papers. A separate sweep of the full text of all 5,869 files, with a deliberately broad self-reference regex, produced 80 candidates; **every one was read, and three survived** — one from 2023, one from 2024, one from 2026. The other 77 are the regex matching ''our median'' as ''our media'', //Mediatek//, "reporters" meaning users who file abuse reports, "dedicated website" meaning somebody else's, press releases in a reference list, and journalists as a study population rather than as an audience.

<WRAP todo>**Three, not two — and the third was found by a reviewer, not by the sweep.** The first version of the regex demanded ''reporters (who|seeking|contacted|reached out)'' and so missed {[vlummens2026_bridges]}, which says "reporters asked" and thanks a colleague "for helping with media outreach" — and which has an entire //Public Disclosure// subsection, making it the best-documented case of the three. It was caught because a sibling page, [[Practices:Legal enforcement]], already cited the same paper for its disclosure timeline. The regex has been widened and the sweep re-run; the count went 59 → 80 candidates and 2 → 3 real hits. **Take the "three" as a floor, not a census**: a keyword sweep under-recalls by construction, and this one demonstrably did.</WRAP>

<WRAP info>
Compare [[Practices:Notifying websites]], where 48.3% of the same 4,472 papers say they notified somebody and the field has a decade of controlled experiments on how to do it. The two activities are the same activity — telling somebody outside your paper what you found — and one of them has a literature. **The corpus cannot tell you whether press engagement is rare or merely unreported** — only that it is unreported. Both readings are live, and they imply the same thing for you: there is no accumulated practice to copy. Treat everything below as reasoning from three cases and from adjacent evidence, not as a summary of established practice.((This page was drafted by Claude and no named researcher has yet vouched for its judgement calls. Per the [[:Contributing#subjective_statements|subjective-statement convention]] such statements should carry a name — add yours here, and to the two other footnotes on this page that flag a judgement rather than a measurement, if you agree with them.))
</WRAP>

===== The denominator has to travel with the number =====

The extraction behind this site records, for every measured finding in every paper, the verbatim sentence the paper stated it in. That gives an unusual measurement: **how well does this field's own writing survive being quoted?** A sentence in an evidence quote is being read in isolation — which is precisely the condition a sentence is in when it appears in a news article.

Of 27,241 measured findings across 5,655 papers, 11,523 are stated in a sentence containing a percentage, contributed by 4,316 papers:

^ Of the 11,523 percentage-bearing sentences ^ Sentences ^ Share ^
| name what the percentage is a percentage **of**, in the same sentence | 6,003 | 52.1% |
| **do not** | **5,520** | **47.9%** |
| … of those, the number is a **classifier or attack performance figure** (accuracy, precision, recall, success rate, F-score, overhead) | 1,690 | 30.6% of the undenominated |
| carry a **date or month** in the same sentence | 270 | **2.3%** |

And it is getting worse, not better:

^ Bucket ^ Percentage-bearing sentences ^ Denominated ^ Dated ^ Are performance figures ^ Denominated, **excluding** performance figures ^
| 2010–2013 | 994 | 63.0% | 2.0% | 12.9% | 67.8% (n=866) |
| 2014–2017 | 1,674 | 62.1% | 1.9% | 16.5% | 68.4% (n=1,398) |
| 2018–2021 | 3,023 | 54.6% | 2.8% | 17.7% | 61.7% (n=2,489) |
| 2022–2024 | 3,615 | 47.2% | 2.4% | 21.6% | 53.8% (n=2,835) |
| 2025–2026* | 2,217 | 44.2% | 2.1% | 21.9% | 51.3% (n=1,732) |

<WRAP todo>* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and IEEE S&P 2026 and TheWebConf 2026 abstracts are not in OpenAlex, so those venue-years are under-represented by construction. See [[Literature:Corpus]].</WRAP>

The obvious explanation for the fall is that the field publishes more model-performance figures than it used to, and a precision score has no population to be a percentage of. The last two columns test that, and it is only part of the story: performance figures do rise from 12.9% to 21.9% of the sentences, **but denomination falls just as steeply within the sentences that are not performance figures** — 67.8% to 51.3%. Whatever is happening is happening to prevalence sentences too.

Two cautions before you use these numbers. The rule is a regular expression — a percentage counts as denominated if the same sentence attaches it to "of //something//", or names a population count before it — so it is a proxy, generously drawn, and it says nothing about whether the paper states its denominator //elsewhere//. Most do. And the claim is narrower than "the field omits denominators": it is that **half of this field's quotable sentences are not self-contained**.

That last point is the one to act on. The most frequently misquoted number in a measurement paper is not the prevalence figure — it is the classifier's precision:

  * You wrote: //"our detector achieves 91% precision"//. That is a property of your detector on your labelled set.
  * It arrives as: //"91% of sites were found to be vulnerable"//. That is a property of the web.

There is nothing in the first sentence, read alone, that stops the second. The fix is not to hedge; it is to make the sentence carry its own subject: //"of the 4,120 sites our detector flagged, manual review confirmed 91%"//. Same number, one extra clause, and it cannot be turned into a prevalence claim.

==== What a self-contained sentence contains ====

The four things a measurement sentence needs in order to be quotable are the four things this field routinely leaves to the surrounding paragraph:

  - **The unit.** Sites? Pages? Domains? Requests? Cookies? Companies? "1,000 trackers" and "the 20 companies that own them" are the same measurement.
  - **The denominator, as a count and as a description.** Not "of websites" but "of the 10,000 sites that loaded successfully" — the successful-load population is almost never the population you sampled, and the gap is your unstated attrition. See [[Design:Website selection]] and [[Design:Sampling]].
  - **The date.** 2.3% of this field's quotable sentences have one. A measurement of the ad-tech ecosystem is a measurement of a specific fortnight; the reader will assume it is a measurement of now, and by publication it is eighteen months old.
  - **The vantage point.** "As seen from a data-centre IP in Frankfurt" is a different web from a residential connection in São Paulo, and consent banners in particular are served on the basis of geolocation. See [[Design:Crawling location]].

===== The ladder: present → executed → collecting → identifying → harmful → illegal =====

Every rung on this ladder is a separate claim requiring separate evidence, and coverage climbs it for free unless you nail each rung down. The clearest demonstration in the corpus is a paper written specifically to measure the gap between two of the rungs.

{[perez2021_smart]} defines the three terms it needs and keeps them apart:

  * **vulnerable** — "flagged by a static analysis tool as such. As we will see later, this means that some contracts may be vulnerable because of a false-positive."
  * **exploitable** — "vulnerable and the vulnerability could be exploited by an external attacker."
  * **exploited** — "received a transaction on Ethereum's main network which triggered one of its vulnerabilities. Therefore, a contract can be vulnerable or even exploitable without having been exploited."

Then it measures the distance: surveying "the 23,327 vulnerable contracts reported by six recent academic projects", it finds "that, despite the amounts at stake, only 1.98% of them have been exploited since deployment. This corresponds to at most 8,487 ETH (~1.7 million USD), or only 0.27% of" the total at stake. **The exploited count is a fiftieth of the vulnerable count, and the money actually taken is about a three-hundred-and-seventieth of the money "at stake".** None of those six papers was wrong. They measured "vulnerable" and said so.

The same ladder, in the vocabulary of web measurement:

^ Rung ^ What it takes to claim it ^ What the previous rung licenses ^
| a resource is **present** on the page | a crawl, and a detector with a stated precision | nothing about behaviour |
| the resource is **executed / the API is called** | instrumentation, not a filter-list match | presence ≠ execution; a blocked script is still "present" in a HAR |
| it **collects** something | observed values leaving the browser, tied to an identifier | execution ≠ collection |
| the collected data **identifies a person** | linkage evidence, or an entropy argument | collection ≠ identification |
| a person was **harmed** | an outcome, not an inference | identification ≠ harm |
| a **law was broken** | a legal analysis, and someone with standing to make it | none of the above |

That last rung is the one this field climbs most casually, and it is the one where the corpus can say something. Of the 402 papers that assessed compliance with a law:

^ ''legal.foundViolations'', strongest finding per paper ^ Papers ^ Share of 402 ^
| yes | 94 | 23.4% |
| partial | 83 | 20.6% |
| no | 74 | 18.4% |
| **not-assessed** | **151** | **37.6%** |
| not-stated | 0 | 0.0% |

(One row per paper, taking the strongest value it records, so the column sums to 402. This is the same tabulation [[Practices:Legal enforcement]] publishes, deliberately — the alternative, counting a paper once per distinct value it uses, gives 177 papers with a ''not-assessed'' tuple somewhere and is not a count of papers that declined to assess.)

**Fewer than a quarter of the papers that engaged with a law recorded an unqualified violation**; another 20.6% recorded a qualified one, and **37.6% stopped short of assessing compliance at all** — they measured a technical fact and named the legal provision it bears on, without asserting anything about whether it was breached. Among the 131 papers that both crawled and assessed a law — the population closest to a cookie-compliance study — 32.8% have at least one unqualified violation finding.

The enum records what a paper wrote, not why, so read that 37.6% as a description of the literature rather than as evidence of a shared norm. But the reason to write that way is independent of the count: a violation is a determination made by a regulator or a court about a specific controller's specific processing, using facts you do not have — the legal basis claimed, the contracts with processors, the consent record. What you measured is a signal that bears on it. **The sentence "we observed X, which is inconsistent with Article 5(3) of the ePrivacy Directive absent consent" is defensible and quotable; "X% of sites break the law" is neither.**((No citation for the phrasing recommendation; it is a judgement, drafted by Claude, consistent with how the 402 legal papers in the corpus write. Per the [[:Contributing#subjective_statements|subjective-statement convention]], add your name here if you agree.)) [[Practices:Legal enforcement]] covers what to do when you want the determination actually made.

===== The press release is where the exaggeration enters =====

Nothing in this corpus measures the accuracy of press coverage. The field that does measure it is health and biomedical research, and its results are consistent enough across an observational study, a replication and a randomised trial that they are worth importing — with the obvious caveat that **these are studies of health-science press releases, not of security or measurement ones**, and the numbers below should be read as direction, not as calibration.

{[sumner2014_exaggeration]} coded 462 press releases on biomedical and health-related science, issued by 20 leading UK universities in 2011, against the peer-reviewed papers behind them and against the 668 resulting news stories:((Sample sizes verified against the BMJ abstract via the PubMed record (PMID 25498121) on 2026-08-18. An erratum exists — //BMJ// 2014;349:g7666 — and its content could not be retrieved: bmj.com is Cloudflare-gated to both ''curl'' and ''WebFetch''. It is recorded in PubMed as an erratum, not a retraction.))

  * **40%** (95% CI 33–46%) of press releases contained exaggerated advice, **33%** (26–40%) exaggerated causal claims from correlational data, and **36%** (28–46%) exaggerated inference to humans from animal research.
  * When the press release exaggerated, **58% / 81% / 86%** of the news stories did too, against **17% / 18% / 10%** when it did not. Odds ratios 6.5, 20 and 56.
  * And the finding that matters most for how you behave: //"there was little evidence that exaggeration in press releases increased the uptake of news."//

{[bratton2019_replication]} repeated the study on an independent sample and replicated two of the three: causal-claim exaggeration (82% vs 16%, OR 23.7) and animal-to-human inference (72% vs 9%, OR 26.5). **Advice exaggeration did not replicate** — 49% of news exaggerated when the release did, against 60% when it did not, which is no effect. The uptake finding held: //"There was no evidence for higher news uptake for exaggerated press releases, consistent with previous results."//

Then the experiment that closes the argument. {[adams2019_causality]} randomised 312 press releases across nine press offices into four arms — headline-and-claim alignment, an explicit causality statement, both, or neither — and tracked 2,257 resulting news items:

> There was no evidence of lost news uptake for press releases with aligned headlines and claims (ITT 55% vs 55%, OR = 0.7 to 1.3, AT 58% vs 60%, OR = 0.7 to 1.7), or causality statements/caveats (ITT 53% vs 56%, OR = 0.8 to 1.0, AT 66% vs 52%, OR = 1.3 to 2.7).

Its conclusion: //"Cautious claims and explicit caveats about correlational findings may penetrate into news without harming news interest."//

Three things follow directly.

  - **The overclaim is usually inserted upstream of the journalist.** The strongest predictor of an exaggerated story is an exaggerated press release, and the press release is the document you can still edit. Ask to see it; the office will normally send it.
  - **The fear that hedging kills coverage is measured, and false.** It is the reason researchers give for accepting a press release they know is too strong, and a randomised trial found no cost in uptake. You are trading nothing for accuracy.
  - **The exaggeration that replicates is the causal one.** Of the three types coded in that literature, the causal one had both the largest effect and the one that survived replication most strongly. Its web-measurement analogue is "sites that use CMP //X// show more tracking" becoming "CMP //X// causes more tracking" — the same move, on data with the same limitation. If you have an association, put the word "associated" in the sentence you want quoted.

For a checklist written for the other side of the exchange, the UK Science Media Centre still publishes its //10 best practice guidelines for reporting science and health stories//, which asks journalists for the sample size, the population, whether the finding is causal, and the funding source.((Fetched from [[https://www.sciencemediacentre.org/publications/publications-for-journalists/|sciencemediacentre.org]] on 2026-08-18; the guidelines PDF returns HTTP 200 at the linked path. Note that the SMC's //Before the Headlines// statistical-appraisal service, which is often recommended in older advice, **no longer exists** — the SMC's own page states "Please note that we no longer offer this service", and the most recent appraisal listed is from December 2019.)) Reading it as an author is the cheapest way to see which four facts a good reporter will go looking for, and therefore which four your quotable sentence should already contain.

===== You probably have no embargo to offer =====

Advice written for the life sciences assumes an embargo: the paper is secret until a coordinated release, so a press office can brief reporters in advance under an agreement. **In security and measurement venues that assumption is mostly false**, and it changes what you can promise.((Each row fetched from the venue's own 2026 site on 2026-08-18: usenix.org instructions for presenters and authors; sp2026.ieee-security.org call for papers; conferences.sigcomm.org/imc/2026 submission instructions; sigsac.org/ccs/CCS2026 call for papers; ndss-symposium.org; petsymposium.org; www2026.thewebconf.org. "None found" means the site was checked and no author-facing press policy was present — not that none exists. One caveat on the last row: TheWebConf 2026 has already been held and its call-for-papers pages now return 404, so that row rests on the conference's current homepage, which contains no press or media language, rather than on the original call. The ACM Digital Library became fully open access on 2026-01-01, which removes the paywall that used to function as a soft embargo for CCS, IMC and TheWebConf papers.))

^ Venue ^ Author-facing press or embargo policy, as of 2026-08-18 ^ When the paper becomes public ^
| **USENIX Security** | **The only venue with a real embargo mechanism.** "Authors may request an embargo for their papers by the deadlines listed below… All embargoed papers will be released on the first day of the symposium." Authors specify whether the PDF, abstract, title or author list is embargoed | to registered attendees before the symposium; embargoed papers on day 1 |
| **IEEE S&P** | No press or embargo policy. Pre-decision only: "Authors should refrain from widely advertising their results" | **immediately on acceptance** — "Papers will immediately be published, open access, in the Computer Society's Digital Library", months before the conference |
| **ACM IMC** | Pre-decision only, and explicit: "do not do general press releases" while under review | at publication, ACM DL |
| **ACM CCS** | None found on the official call for papers | at publication, ACM DL |
| **NDSS** | None found on the official site | around the symposium |
| **PETS / PoPETs** | None found | quarterly issues, ahead of the annual symposium |
| **TheWebConf** | None found | at publication, ACM DL |

The practical consequences:

  * **At IEEE S&P your paper is public and citable long before you present it.** Anyone can find it, including a reporter and including the company you measured. If you were planning to notify affected parties "before publication", acceptance is your deadline, not the conference.
  * **USENIX Security is where a coordinated release is actually possible**, and it is the venue to pick if your finding needs a vendor fix to land first. The embargo has to be requested, by a deadline, and it is per-artefact.
  * **"Do not do general press releases" at IMC applies during review, not after.** It is a double-blind rule, not a publicity rule, but it is worth reading before you let a press office get ahead of your acceptance.
  * The ACM Digital Library's move to full open access on 1 January 2026 removed the paywall that used to slow re-reporting of CCS, IMC and TheWebConf papers. Expect faster and less mediated pickup than the older advice assumes.

===== Three papers that documented their own handling =====

These are the three papers found in seven venues, 2010–2026, that describe handling the public-facing side of their own results. All three are worth reading in the original; all three are from 2023 or later, which is the most encouraging thing on this page.

==== A coordinated release, with the vendors given a day to answer ====

{[vlummens2026_bridges]} found Meta and Yandex silently bridging web identity to native apps through localhost, and ran the most complete disclosure process documented in this corpus. Its //Public Disclosure// paragraph is the closest thing the field has to a worked example:

> Because the tracking technique was actively abused, we pursued public disclosure in parallel with affected vendor coordination before submission of this paper. Public disclosures increase societal awareness, and motivate and expedite mitigations. The public disclosure occurred through **a dedicated website** … with independent journalistic reporting. As part of the same release process, **reporters asked Meta and Yandex's statements the day before the publication, giving them a chance to respond.**

Five decisions in that paragraph, each of which you will have to make:

  - **Publish to the public in parallel with vendor coordination, not after it** — and give the reason. Theirs is that the technique was //actively abused//, so waiting had a cost to users. That is an argument a reviewer can evaluate; "we disclosed responsibly" is not.
  - **A dedicated website, not a press release.** The site is the artefact reporters, standards bodies and lawyers all cite; the paper came later. It is also what lets you control the "what this does not show" text.
  - **Right of reply, arranged by you, one day ahead.** The reporters solicited Meta's and Yandex's statements before publication. This is the item most worth copying wholesale: it converts the vendor's response from a rebuttal published against you into a part of the story published with you. It is precisely the mechanism the printer case below lacked.
  - **Anonymity is manageable.** They "omitted the mention of our public disclosure in our initial paper submission to preserve anonymity and unbiased reviews", and told the USENIX'26 program chairs about the public status. If you think a public release rules out a double-blind venue, it does not — it rules out //not telling the chairs//.
  - **Name the framework.** They justify the decision against the Menlo Report principles of Beneficence and Public Interest, which is what turns a judgement call into a reviewable one.

The outcomes they report are the argument for doing any of this: both companies terminated the observed behaviour on 3 June 2025, the day of the coordinated public disclosure — after Meta's variant had been running since September 2024 and Yandex's for over eight years — and the findings went on to inform class actions in several jurisdictions, expert testimony to the Spanish Congress on the DSA, and Local Network Access standardisation documents at the IETF and W3C. Compare the response rates in [[Practices:Notifying websites]] before concluding that quiet notification would have done the same.

==== Reporters pull toward the sub-finding you did not make ====

{[bellini2024_safer]}, a systematisation of safety practices for research with at-risk users, is the only paper in the corpus with a passage of advice on this:

> Our experiences have also demonstrated a role for caution in handling media attention after publication. After publishing our work understanding abuser tactics in IPV, we received inquiries from reporters seeking our expertise. Some reporters may be incentivized to seek sensationalist headlines. For example, several have contacted us wanting to explicitly write stories on stalkerware, despite the fact that our research suggested that it is a **less prevalent** attack vector in IPV than everyday privacy violations like account compromise. Our statements to reporters explicitly state this, and we always tell them to contact the communications staff of our partner organizations.

Three practices are in that passage, and all three generalise past at-risk research:

  - **The pull is toward the most dramatic thing your paper mentions, not the thing your paper found.** A paper whose finding is "mundane account compromise dominates" contains the word "stalkerware", and that is the word that gets the call. In web measurement the equivalents are the named brand, the sensitive category, and the single most extreme site in your sample. If your paper contains one, expect it to become the story.
  - **Put the correction in every statement, not once.** Not a caveat at the end of the interview — the sentence that contradicts the expected story, in the answer to every question. What survives editing is what was repeated.
  - **Route to the people who will still be there next year.** Bellini et al. send reporters to partner organisations' communications staff. You are a temporary participant in a topic that the advocacy organisation, the CERT or the regulator works on permanently, and they are better placed to be quoted about the field than you are about anything beyond your own measurement.

The same paper's neighbouring advice — "we discourage researchers from promising or implying assurances that they cannot guarantee … researchers do not assure participants that the research will result in systemic change, or that such change will be swift" — is written about participants, and reads exactly as well about press.

==== Withholding per-target findings, and who you ask before you do ====

{[zimmeck2023_gpc]} measured whether websites honour Global Privacy Control signals under the CCPA, notified the operators, and reported to the California Attorney General's office. Its disclosure section is the most complete account in the corpus of the decision //not// to publish:

> We have not disclosed any individual finding of non-compliance publicly and will not do so in the future. One reason is that such publication could interfere with potential GPC enforcement actions by the OAG, to which we are not privy. To ensure that all applicable legal and ethical considerations of our report to the OAG were considered we conferred internally with our institution's IRB staff, legal counsel, communications staff, university administrators, and faculty colleagues about the procedure and potential ramifications.

Two things to take from it. First, **the aggregate finding and the per-site list are separable, and separating them is a normal choice, not a retreat.** The paper publishes its rates; it does not publish which named site failed. If your finding is a compliance failure, decide explicitly which of those two you are publishing, before anyone asks you for the list.

Second, that list of five internal parties — IRB, legal counsel, **communications staff**, administrators, colleagues — is the only place in 5,859 papers where a university communications office appears as part of a research decision. It is also the cheapest item on this page: the office exists, it costs you nothing, and it has done this many times to your once.

===== Three cases where the coverage outran the paper =====

None of these is in the corpus as a //press// finding — two of the three papers are in it as ordinary measurement papers, and the record of what happened to them is outside the paper. That is the shape of the evidence available on this topic, and it is why the cases have to be assembled by hand.

==== "Are these devices spying on us?" — a project page written to answer the question the paper was not asked ====

{[dubois2020_speakers]} played 134 hours of television at popular smart speakers and measured how often they woke without a wake word: "0.95 misactivations per hour, or 1.43 times for every 10,000 words spoken, with some devices having 10% of their misactivation durations lasting at least 10 seconds". The paper's own conclusion is careful about what that is: "we did not find evidence of malicious or intentional misactivations, meaning that the misactivations we observed may just be caused by a suboptimal wake word recognition engine."

The coverage the authors list on their own project page includes //The New York Times//, "Are Alexa and Google Assistant spying on us?", and //Which?// under the same headline; //The Independent// ran "Smart Speakers Could Accidentally Record Users up to 19 Times Per Day, Study Reveals".((Press list and quotes fetched from the authors' project page, [[https://moniotrlab.khoury.northeastern.edu/publications/smart-speakers-study-pets20/|moniotrlab.khoury.northeastern.edu]], on 2026-08-18.)) Note that the //Independent// headline is arithmetically fair — roughly 0.95 per hour over a waking day — and that is the point: **the number survived and the framing did not.** The authors' response was a public Q&A on the project page, in the reader's words rather than the paper's:

> **Are these devices constantly recording our conversations?** In short, we found no evidence to support this. The devices do wake up frequently, but often for short intervals (with some exceptions).

and, one paragraph earlier, the scope caveat that no news article will print for you:

> Of course, all our findings pertain only to the source material (audio from selected TV shows) and we cannot make claims about more general trends.

**Write the answer to the question you will be asked, not to the question you studied.** Nobody asked these authors about wake-word recognition engines. Everybody asked whether the speaker is spying. The Q&A answers that in the first line and then gives the real finding, which is the only ordering that works.

==== Panoptispy: the finding was the absence, and the absence did not travel ====

{[pan2018_panoptispy]} tested Android apps for audio and video exfiltration. Its result on audio is a negative one, stated plainly: "We did not find any true positive audio files in our extracted dataset, i.e., no apps appeared to exfiltrate audio in our tests." What it //did// find was a different and less familiar risk — third-party SDKs silently capturing screen recordings.

The study is routinely invoked in the "your phone is listening to you" genre. The authors' own institutional coverage frames it correctly and is worth copying as a model: it names the urban legend, says the researchers found no evidence of recorded conversations, and moves the reader to the thing that was found.((Northeastern Global News, //Is your smartphone spying on you?//, [[https://news.northeastern.edu/2018/07/06/is-your-smartphone-spying-on-you/|news.northeastern.edu]], fetched 2026-08-18.))

**A null result is easy to invert on the way out.** If your headline finding is "we looked for X and did not find it", assume that the story will be about X, and put the thing you //did// find in the same sentence as the negative.

==== The printers that were not going to catch fire ====

The most complete documented case in this literature, because all three sides are on the record. Columbia researchers disclosed an unauthenticated remote firmware-update vulnerability in HP LaserJet printers (CVE-2011-4161). NBC News ran the work as an exclusive on 29 November 2011, opening by asking whether a hacker could give a printer instructions "so frantic that it could eventually catch fire".((Bob Sullivan, //Exclusive: Millions of printers open to devastating hack attack, researchers say//, NBC News, 29 November 2011, [[https://www.nbcnews.com/technolog/exclusive-millions-printers-open-devastating-hack-attack-researchers-say-118851|nbcnews.com]], fetched 2026-08-18.))

HP responded publicly the following day: "Today there has been sensational and inaccurate reporting regarding a potential security vulnerability with some HP LaserJet printers. No customer has reported unauthorized access. Speculation regarding potential for devices to catch fire due to a firmware change is false."((HP's statement, quoted verbatim at [[https://www.foxnews.com/tech/hp-refutes-reports-that-printers-can-be-remotely-set-on-fire|foxnews.com]] — HTTP 200, quote confirmed word-for-word on 2026-08-18 by two independent checks. The same statement was reported by phys.org, //HP slams 'sensational' reports about LaserJet printer hack vulnerability//, 30 November 2011, [[https://phys.org/news/2011-11-hp-slams-sensational-laserjet-printers.html|phys.org]]; that URL is **Cloudflare-gated and returns HTTP 403** to ''curl'' and ''WebFetch'', so it is cited as corroboration that could not be independently re-fetched, not as the source of record.)) A month later //MIT Technology Review// wrote the retrospective: "It was an image so sensational it was destined to make headlines … It probably was unlikely that HP printers were ever going to burst into flames."((David Zax, //Exploding HP Printers Turn Out to Be Not So Explosive//, MIT Technology Review, 28 December 2011, [[https://www.technologyreview.com/2011/12/28/188623/exploding-hp-printers-turn-out-to-be-not-so-explosive/|technologyreview.com]], fetched 2026-08-18.))

And the researchers' own peer-reviewed account, published at NDSS fifteen months later, opens its impact discussion by distancing itself from the coverage: "Contrary to the sensationalized media coverage regarding the HP-RFU vulnerability, it would be unwise for the attacker to destroy a compromised printer physically" {[cui2013_when]}.

Three lessons, and the last one is the expensive one:

  - **The demonstration becomes the finding.** The paper is about persistent firmware implants — malware whose "presence on this device will most likely go undetected" — on over 90,000 internet-reachable printers. That is a serious result. What survived was the fire.
  - **The vendor's rebuttal is now the story's second half, permanently.** HP's statement is factually narrow ("no customer has reported unauthorized access") and rhetorically total. Once the vendor has called your coverage sensational, every subsequent article carries that framing, and correcting it is your job, not the reporter's.
  - **You will end up writing the correction into your own paper.** {[cui2013_when]} spends a sentence of its NDSS submission dissociating itself from the story it helped create. That sentence is the cost of not having decided in advance which demonstration would lead.

<WRAP info>
**Two things this page deliberately does not contain, and why.** No peer-reviewed study was found of the effect of //branding// a vulnerability — giving it a name and a logo — on attention or on patching; the only sources are trade-press and vendor blog opinion pieces, and they are not cited on this page. And no professional body publishes guidance for researchers on talking to media: ACM, IEEE, USENIX, IAPP and FIRST were all checked, and FIRST's multi-party coordination guidelines cover vendor embargoes, not press.((Checked on 2026-08-18: acm.org code of ethics and media centre, usenix.org, iapp.org, and FIRST's //Guidelines and Practices for Multi-Party Vulnerability Coordination and Disclosure// v1.1. "None found" means the site was searched and nothing author-facing on media was present.)) If you find either — a real study of branding, or institutional guidance — it belongs on this page, and both rejections are logged with their sources on [[provenance:practices:public_relations]].
</WRAP>

===== The interview is where the story is decided =====

The one peer-reviewed study of this problem from a computing venue is outside the seven this site indexes: {[smith2020_disseminating]} interviewed 12 HCI researchers about how their work reached the public, and found miscommunication entering at four points along the "Media Production Pipeline": **press releases, interviews, media incentives and formats, and Web 2.0 affordances**. It is a small qualitative study and should be read as one — but **all 12 participants "retained concerns about at least one instance of perceived miscommunication"**, and its account of the interview is the part with no substitute:

> Quotes from a press release may be undesirable for journalists to include in their stories since all newsrooms have access to those same quotes. To stand out, journalists gather unique, catchy quotes through interviews that can range from around 5 to 60+ minutes; **these interviews frequently determine how the science is portrayed moreso than the original paper or press release.** In most cases, journalists also condense interviews into a dramatically shorter format. "You kinda spend a half-hour, 45 minutes, talking with someone, and then there's a one sentence summary of all that."

Two failure modes from that paper generalise exactly:

  * **The pre-constructed narrative.** "They've got essentially a hypothesis … They're not trying to find out what the evidence shows, but particular scientific anecdotes consistent with that hypothesis." The paper's worked example is a researcher who knew what was happening and answered anyway: //"he was like, 'Tell me about the worst things that can happen,' and I was like, 'Ok,' and I knew exactly what was going to happen, which it did. The article was like, '5 Horrible Things that Can Happen if You Don't Read the Terms' … with nothing about the hedging."// The answerable version of that question is the one you supply: //"the worst case we actually observed was X, in N of 10,000 sites"//.
  * **Cherry picking.** A minor result in the press release becomes the headline. This is the same mechanism as the stalkerware pull in {[bellini2024_safer]}, reported independently by a different community — which is about as much corroboration as this topic offers. **Assume the most quotable line in your press release is the one that will run, and audit the press release on that basis rather than on whether it is accurate overall.**

The concrete preparation this implies is small: decide the one sentence, expect a 45-minute call to be reduced to it, and repeat it. If the interview lasts an hour and the story quotes one sentence, the only question that matters is which sentence you said more than once.((No citation for this last formulation; it is a restatement of {[smith2020_disseminating]}'s interview findings, not a measured result. Per the [[:Contributing#subjective_statements|subjective-statement convention]], add your name here if you agree.))

===== Publicity is an instrument, and also a confounder =====

Two corpus findings that pull in opposite directions, and you need both.

**Attention moves ecosystems, sometimes.** {[kotzias2018_coming]}, measuring nine years of TLS deployment, concludes that "the impact of security research on the TLS ecosystem is sometimes spectacular, but sometimes also quite slow. In a few cases — especially those that attracted a lot of media attention or had significant impact — the ecosystem changed very quickly. The best example is Heartbleed." Against that, RC4: "while RC4 use started to drop soon after attacks were announced, it still took several years for RC4 usage to reduce significantly." Publicity is one of the few levers a researcher has that reaches parties they cannot email, and [[Practices:Notifying websites]] documents how poorly the direct channels perform — but the same paper is the evidence that it works unevenly and that "the ecosystem changed quickly" is a claim about two named cases, not a rate.

**Regulators act on what is publicly visible.** {[saemann2022_fines]} coded 856 GDPR fines from the CMS Enforcement Tracker and could identify the cause of the investigation for 295 of them; in 111 of those (reported as 45%) the authority opened the case on its own initiative after personal data was //"unlawfully published on a website or social media"//, and the paper concludes that //"respective DPA actions are primarily based on customer complaints and unwanted disclosure of data"//.((Two caveats. The 111 cases are data //exposures// — a controller's personal data visible in public — not researchers publishing findings, so reading them as "publication triggers enforcement" is an inference, not the paper's claim. And 111 of 295 is 37.6%, not the 45% the paper states; [[Practices:Legal enforcement]] notes the same internal inconsistency and that the percentages appear to rest on a base near 245. The shape is safe to use; the exact figure is not.)) The transferable part is narrow but real: a self-initiated case needs something an authority can see without a complainant, and a published measurement with a reproducible method is exactly that. [[Practices:Legal enforcement]] reads the same paper from the regulator's side.

That is an argument for publicity, and simultaneously the reason {[zimmeck2023_gpc]} withheld its per-site list: publication that pre-empts an authority's own case can cut the other way.

**Attention contaminates measurements, including yours.** {[deuber2023_assessing]}, de-anonymising German court decisions, excluded an entire tier of its population for this reason: "We did not consider decisions from the federal courts, as these often receive a great deal of media attention, which might have distorted the results." If you are measuring anything that has been in the news — a specific breach, a named vendor after an enforcement action, a site after a viral post — the coverage is part of your treatment. Say so, or exclude it as they did.

The two combine into an awkward corollary for longitudinal work: **if your first paper gets coverage, your second measurement of the same population is measuring a population that read about you.** Nothing in the corpus measures this effect. It is worth one sentence in your limitations section.((No citation: no paper in this corpus measures the effect of its own publicity on a follow-up measurement. Flagged as an open question rather than a finding.))

===== Naming a company =====

Naming is where the legal and the editorial risks meet, and it is the decision most likely to be made for you if you have not made it first.

  * **The exposure is measured, not folkloric.** {[gamerogarrido2017_legalrisks]} put the "chilling effects" question on a quantitative footing by surveying 110 self-identified vulnerability researchers: **49.1% had feared legal action** over their vulnerability work, and of those, **52.7% said that fear caused them to modify a project**; **22% had actually been threatened with legal action**. That is the base rate to have in mind before you name a company in a press release rather than only in a paper.
  * **Aggregate and per-target findings are separable** — {[zimmeck2023_gpc]} above. This is the mechanism that lets you publish the finding without publishing the target list.
  * **Give the named party the finding before the journalist has it.** That is [[Practices:Notifying websites|notification]], and the disclosure timeline you followed is also your press timeline: a reporter who calls a company that has never heard of your work gets a "we dispute these findings" quote, and that quote is now in the story permanently. A company that has had your report for 90 days gets asked about its remediation instead.
  * **Expect to be asked for the list.** Decide in advance what you will say to "which sites?" — including whether the artefact you published already answers it. If your dataset is public and contains the per-site labels, you have already published the list, whatever your paper says. See [[:Artifacts]].
  * **Your detector's false positives become named accusations.** A detector with 91% precision that flags 4,000 sites names about 360 of them wrongly. That is tolerable in a table and not tolerable in a headline. If you publish per-site results, the precision figure belongs next to them, in the same sentence.

===== What to prepare, and what to check =====

The single-sentence version: **write the sentence you want quoted, put it in the abstract, and make it survive being cut out of the paper.** Everything else follows from that. In practice, five artefacts, none of which takes more than an hour:

  - **The quotable sentence.** Unit, denominator, date, vantage. Test it by deleting the rest of the abstract and reading what is left. If it becomes a claim about "the web", rewrite it. If it is a model-performance figure, say what set it was measured on — 30.6% of this corpus's undenominated percentage sentences are performance figures, and they are the ones that read as prevalence to everyone outside the field.
  - **An answer to the question you will actually be asked**, which is the alarming one your paper touches but did not study. Answer it in the first line, then give your finding, as {[dubois2020_speakers]} does. If your headline result is a null one, put the thing you //did// find in the same sentence as the negative.
  - **A "what this does not show" list, written down.** Three to five lines, in the paper if the venue allows it and on the project page regardless. Every rung of the ladder you did not climb; whether you are asserting a violation or an inconsistency with a named provision; whether the per-target list is published, and whether your artefact already publishes it whatever the paper says. This is the document that gets copied into a story, because it is the only part nobody else can write.
  - **A named contact and a routing rule.** Who takes the call; who gets sent onward (co-authors on parts you did not do, partner organisations, the CERT, the regulator); what you decline to comment on. {[bellini2024_safer]} routes to partner organisations by default.
  - **The release plan and its dates**: when your paper actually becomes public at your venue (at IEEE S&P, that is acceptance, not the conference), when the affected parties were notified, and whether anyone has arranged for them to be asked for a statement before publication rather than after. {[vlummens2026_bridges]} gave Meta and Yandex one day; the alternative is the printer case.

Then four things to check that no artefact covers, because they are decisions rather than documents:

  - Has your institution's **communications office** been told, and have you read the press release? Its headline and its first sentence should make the same claim your abstract does. That is the document the exaggeration enters through {[sumner2014_exaggeration,bratton2019_replication]}, and caveating it costs no coverage {[adams2019_causality]}.
  - If your finding is an **association**, does the word "associated" appear in the sentence you want quoted?
  - Does your **limitations section** mention publicity as a confounder, if you plan to measure the same population again?
  - If you are publishing per-site results, is the **precision figure in the same sentence** as them?

===== Papers to read first =====

If you read three: {[vlummens2026_bridges]} for the only fully worked disclosure-and-press process in the corpus, with its timeline and its right-of-reply arrangement; {[bellini2024_safer]} §6.6 for what reporters ask for and how the authors answer it; and {[adams2019_causality]} for the randomised evidence that caveating a press release costs you nothing. Then {[sumner2014_exaggeration]} and {[bratton2019_replication]} for where the exaggeration enters and which kind of it replicates, {[smith2020_disseminating]} for the interview, {[zimmeck2023_gpc]} for the decision not to publish a per-site list, {[perez2021_smart]} for the vulnerable/exploitable/exploited distinction that every one of these arguments rests on, and {[cui2013_when]} with the 2011 coverage around it for what it looks like when this goes wrong.

===== Related pages =====

  * [[Practices:Notifying websites]] — the other post-publication channel, and the one with a literature. Your disclosure timeline is also your press timeline.
  * [[Practices:Legal enforcement]] — when you want a violation actually determined rather than asserted, and the regulator's side of {[saemann2022_fines]}.
  * [[Practices:Ethics]] — harm, review, and what you owe the parties you measured.
  * [[Design:Website selection]] and [[Design:Sampling]] — where your denominator comes from and what it is not.
  * [[Design:Crawling location]] — why "the web" in your sentence needs a vantage point attached.
  * [[:Artifacts]] — the per-site list you may have published without deciding to.
  * [[Statistics:Biases]] — the gap between what you sampled and what you will be quoted as describing.
  * [[Writing:Conferences]] — venue choice, which also decides when your paper becomes public and whether an embargo is available at all.

===== Methodology and limitations of these figures =====

Every corpus figure above comes from ''scripts/report_public_relations.mjs'' and ''scripts/pr_fulltext_grep.sh'' over ''data/extract/run1'' (5,859 papers) and ''data/fulltext'' (5,869 ''paper.cols.txt'' files), 2010–2026. Four limits carry directly:

  * **Seven venues only** — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent, and CHI and SOUPS are where at-risk and usable-security work most often appears — the community that produced the one paper here with explicit press advice. Every claim about "the field" on this page is a claim about these seven venues.
  * **The count of three documented cases is a floor.** It came from a keyword sweep over full text, and the first version of that sweep missed one of the three; the regex was widened and re-run, but a keyword sweep under-recalls by construction and this one demonstrably did. If you know of a fourth, it belongs here.
  * **The denominator figure is a regex over a model-selected sentence.** It asks whether a percentage in a sentence is attached to "of //something//" in that same sentence. It cannot tell a well-written sentence from a lucky one; it says nothing about whether the paper gives its denominator elsewhere, and mostly papers do; and ''evidence.quote'' is one sentence chosen by an extraction model as the best evidence for a finding, which may be systematically better or worse written than the paper's average. **This is the weakest assumption on the page**, and the reason the figure is framed as being about quotable sentences rather than about papers. The residue in both directions is printed by ''%%--examples%%'' and reproduced in full on the provenance page.
  * **''legal.foundViolations'' is an enum and can carry a percentage**, but "assessed a law" is itself an extraction judgement, 402 papers is a small population for a five-way split, and the enum records what a paper wrote rather than why it wrote it.

Two limits on the non-corpus evidence, which carries at least as much of this page:

  * **The press-release evidence is from health and biomedical science, not from computing.** {[sumner2014_exaggeration]}, {[bratton2019_replication]} and {[adams2019_causality]} study UK university press releases on health topics. The mechanism they identify — exaggeration entering upstream, causal inflation replicating most strongly, caveats costing no uptake — is plausibly general, and nothing has tested it on security or measurement research. Read the direction, not the decimals. The one computing-venue study, {[smith2020_disseminating]}, is 12 interviews at CHI: qualitative, small, and outside this corpus.
  * **The three miscoverage cases were assembled by hand and are not a sample.** Every quote was fetched from a primary source on 2026-08-18 — the authors' own project page, the paper PDF, the news article, the vendor statement — but cases where coverage was accurate leave no trace of this kind, so the set is selected on the outcome. It shows what the failure looks like; it says nothing about how often it happens.

The complete query log, both report scripts with their unedited output, the fold rules with their unmapped residue, the quotes that were spot-checked against ''paper.cols.txt'', the reviewer findings and what was rejected, and every external source that was checked or discarded are on **[[provenance:practices:public_relations]]**. Corpus-wide caveats are on [[Literature:Corpus]].

====== References ======
<bibtex bibliography></bibtex>


~~DISCUSSION~~