| Next revision | Previous revision |
| practices:public_relations [2026/08/18 15:41] – New page: how to present web-measurement results to journalists without overclaiming. Corpus-backed (5,859 papers): the denominator query, legal.foundViolations, and the near-total absence of documented press practice; three documented cases; imported pre karel.kubicek.claude | practices:public_relations [2026/09/17 08:12] (current) – Qualify root Artifacts and Contributing links; Authored by Claude karel.kubicek.claude |
|---|
| So: **the extraction's ethics fields, over 4,472 papers, contain no description of how any of them handled their own results going public.** That is a statement about a 20-word structured field, and the right response to it is to go and read the papers. A separate sweep of the full text of all 5,869 files, with a deliberately broad self-reference regex, produced 80 candidates; **every one was read, and three survived** — one from 2023, one from 2024, one from 2026. The other 77 are the regex matching ''our median'' as ''our media'', //Mediatek//, "reporters" meaning users who file abuse reports, "dedicated website" meaning somebody else's, press releases in a reference list, and journalists as a study population rather than as an audience. | So: **the extraction's ethics fields, over 4,472 papers, contain no description of how any of them handled their own results going public.** That is a statement about a 20-word structured field, and the right response to it is to go and read the papers. A separate sweep of the full text of all 5,869 files, with a deliberately broad self-reference regex, produced 80 candidates; **every one was read, and three survived** — one from 2023, one from 2024, one from 2026. The other 77 are the regex matching ''our median'' as ''our media'', //Mediatek//, "reporters" meaning users who file abuse reports, "dedicated website" meaning somebody else's, press releases in a reference list, and journalists as a study population rather than as an audience. |
| |
| <wrap todo>**Three, not two — and the third was found by a reviewer, not by the sweep.** The first version of the regex demanded ''reporters (who|seeking|contacted|reached out)'' and so missed {[vlummens2026_bridges]}, which says "reporters asked" and thanks a colleague "for helping with media outreach" — and which has an entire //Public Disclosure// subsection, making it the best-documented case of the three. It was caught because a sibling page, [[Practices:Legal enforcement]], already cited the same paper for its disclosure timeline. The regex has been widened and the sweep re-run; the count went 59 → 80 candidates and 2 → 3 real hits. **Take the "three" as a floor, not a census**: a keyword sweep under-recalls by construction, and this one demonstrably did.</wrap> | <WRAP todo>**Three, not two — and the third was found by a reviewer, not by the sweep.** The first version of the regex demanded ''reporters (who|seeking|contacted|reached out)'' and so missed {[vlummens2026_bridges]}, which says "reporters asked" and thanks a colleague "for helping with media outreach" — and which has an entire //Public Disclosure// subsection, making it the best-documented case of the three. It was caught because a sibling page, [[Practices:Legal enforcement]], already cited the same paper for its disclosure timeline. The regex has been widened and the sweep re-run; the count went 59 → 80 candidates and 2 → 3 real hits. **Take the "three" as a floor, not a census**: a keyword sweep under-recalls by construction, and this one demonstrably did.</WRAP> |
| |
| <WRAP info> | <WRAP info> |
| Compare [[Practices:Notifying websites]], where 48.3% of the same 4,472 papers say they notified somebody and the field has a decade of controlled experiments on how to do it. The two activities are the same activity — telling somebody outside your paper what you found — and one of them has a literature. **The corpus cannot tell you whether press engagement is rare or merely unreported** — only that it is unreported. Both readings are live, and they imply the same thing for you: there is no accumulated practice to copy. Treat everything below as reasoning from three cases and from adjacent evidence, not as a summary of established practice.((This page was drafted by Claude and no named researcher has yet vouched for its judgement calls. Per the [[Contributing#subjective_statements|subjective-statement convention]] such statements should carry a name — add yours here, and to the two other footnotes on this page that flag a judgement rather than a measurement, if you agree with them.)) | Compare [[Practices:Notifying websites]], where 48.3% of the same 4,472 papers say they notified somebody and the field has a decade of controlled experiments on how to do it. The two activities are the same activity — telling somebody outside your paper what you found — and one of them has a literature. **The corpus cannot tell you whether press engagement is rare or merely unreported** — only that it is unreported. Both readings are live, and they imply the same thing for you: there is no accumulated practice to copy. Treat everything below as reasoning from three cases and from adjacent evidence, not as a summary of established practice.((This page was drafted by Claude and no named researcher has yet vouched for its judgement calls. Per the [[:Contributing#subjective_statements|subjective-statement convention]] such statements should carry a name — add yours here, and to the two other footnotes on this page that flag a judgement rather than a measurement, if you agree with them.)) |
| </WRAP> | </WRAP> |
| |
| | 2025–2026* | 2,217 | 44.2% | 2.1% | 21.9% | 51.3% (n=1,732) | | | 2025–2026* | 2,217 | 44.2% | 2.1% | 21.9% | 51.3% (n=1,732) | |
| |
| <wrap todo>* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and IEEE S&P 2026 and TheWebConf 2026 abstracts are not in OpenAlex, so those venue-years are under-represented by construction. See [[Literature:Corpus]].</wrap> | <WRAP todo>* 2025–2026 is provisional: CCS 2026 and IMC 2026 have not been held, and IEEE S&P 2026 and TheWebConf 2026 abstracts are not in OpenAlex, so those venue-years are under-represented by construction. See [[Literature:Corpus]].</WRAP> |
| |
| The obvious explanation for the fall is that the field publishes more model-performance figures than it used to, and a precision score has no population to be a percentage of. The last two columns test that, and it is only part of the story: performance figures do rise from 12.9% to 21.9% of the sentences, **but denomination falls just as steeply within the sentences that are not performance figures** — 67.8% to 51.3%. Whatever is happening is happening to prevalence sentences too. | The obvious explanation for the fall is that the field publishes more model-performance figures than it used to, and a precision score has no population to be a percentage of. The last two columns test that, and it is only part of the story: performance figures do rise from 12.9% to 21.9% of the sentences, **but denomination falls just as steeply within the sentences that are not performance figures** — 67.8% to 51.3%. Whatever is happening is happening to prevalence sentences too. |
| **Fewer than a quarter of the papers that engaged with a law recorded an unqualified violation**; another 20.6% recorded a qualified one, and **37.6% stopped short of assessing compliance at all** — they measured a technical fact and named the legal provision it bears on, without asserting anything about whether it was breached. Among the 131 papers that both crawled and assessed a law — the population closest to a cookie-compliance study — 32.8% have at least one unqualified violation finding. | **Fewer than a quarter of the papers that engaged with a law recorded an unqualified violation**; another 20.6% recorded a qualified one, and **37.6% stopped short of assessing compliance at all** — they measured a technical fact and named the legal provision it bears on, without asserting anything about whether it was breached. Among the 131 papers that both crawled and assessed a law — the population closest to a cookie-compliance study — 32.8% have at least one unqualified violation finding. |
| |
| The enum records what a paper wrote, not why, so read that 37.6% as a description of the literature rather than as evidence of a shared norm. But the reason to write that way is independent of the count: a violation is a determination made by a regulator or a court about a specific controller's specific processing, using facts you do not have — the legal basis claimed, the contracts with processors, the consent record. What you measured is a signal that bears on it. **The sentence "we observed X, which is inconsistent with Article 5(3) of the ePrivacy Directive absent consent" is defensible and quotable; "X% of sites break the law" is neither.**((No citation for the phrasing recommendation; it is a judgement, drafted by Claude, consistent with how the 402 legal papers in the corpus write. Per the [[Contributing#subjective_statements|subjective-statement convention]], add your name here if you agree.)) [[Practices:Legal enforcement]] covers what to do when you want the determination actually made. | The enum records what a paper wrote, not why, so read that 37.6% as a description of the literature rather than as evidence of a shared norm. But the reason to write that way is independent of the count: a violation is a determination made by a regulator or a court about a specific controller's specific processing, using facts you do not have — the legal basis claimed, the contracts with processors, the consent record. What you measured is a signal that bears on it. **The sentence "we observed X, which is inconsistent with Article 5(3) of the ePrivacy Directive absent consent" is defensible and quotable; "X% of sites break the law" is neither.**((No citation for the phrasing recommendation; it is a judgement, drafted by Claude, consistent with how the 402 legal papers in the corpus write. Per the [[:Contributing#subjective_statements|subjective-statement convention]], add your name here if you agree.)) [[Practices:Legal enforcement]] covers what to do when you want the determination actually made. |
| |
| ===== The press release is where the exaggeration enters ===== | ===== The press release is where the exaggeration enters ===== |
| * **Cherry picking.** A minor result in the press release becomes the headline. This is the same mechanism as the stalkerware pull in {[bellini2024_safer]}, reported independently by a different community — which is about as much corroboration as this topic offers. **Assume the most quotable line in your press release is the one that will run, and audit the press release on that basis rather than on whether it is accurate overall.** | * **Cherry picking.** A minor result in the press release becomes the headline. This is the same mechanism as the stalkerware pull in {[bellini2024_safer]}, reported independently by a different community — which is about as much corroboration as this topic offers. **Assume the most quotable line in your press release is the one that will run, and audit the press release on that basis rather than on whether it is accurate overall.** |
| |
| The concrete preparation this implies is small: decide the one sentence, expect a 45-minute call to be reduced to it, and repeat it. If the interview lasts an hour and the story quotes one sentence, the only question that matters is which sentence you said more than once.((No citation for this last formulation; it is a restatement of {[smith2020_disseminating]}'s interview findings, not a measured result. Per the [[Contributing#subjective_statements|subjective-statement convention]], add your name here if you agree.)) | The concrete preparation this implies is small: decide the one sentence, expect a 45-minute call to be reduced to it, and repeat it. If the interview lasts an hour and the story quotes one sentence, the only question that matters is which sentence you said more than once.((No citation for this last formulation; it is a restatement of {[smith2020_disseminating]}'s interview findings, not a measured result. Per the [[:Contributing#subjective_statements|subjective-statement convention]], add your name here if you agree.)) |
| |
| ===== Publicity is an instrument, and also a confounder ===== | ===== Publicity is an instrument, and also a confounder ===== |
| * **Aggregate and per-target findings are separable** — {[zimmeck2023_gpc]} above. This is the mechanism that lets you publish the finding without publishing the target list. | * **Aggregate and per-target findings are separable** — {[zimmeck2023_gpc]} above. This is the mechanism that lets you publish the finding without publishing the target list. |
| * **Give the named party the finding before the journalist has it.** That is [[Practices:Notifying websites|notification]], and the disclosure timeline you followed is also your press timeline: a reporter who calls a company that has never heard of your work gets a "we dispute these findings" quote, and that quote is now in the story permanently. A company that has had your report for 90 days gets asked about its remediation instead. | * **Give the named party the finding before the journalist has it.** That is [[Practices:Notifying websites|notification]], and the disclosure timeline you followed is also your press timeline: a reporter who calls a company that has never heard of your work gets a "we dispute these findings" quote, and that quote is now in the story permanently. A company that has had your report for 90 days gets asked about its remediation instead. |
| * **Expect to be asked for the list.** Decide in advance what you will say to "which sites?" — including whether the artefact you published already answers it. If your dataset is public and contains the per-site labels, you have already published the list, whatever your paper says. See [[Artifacts]]. | * **Expect to be asked for the list.** Decide in advance what you will say to "which sites?" — including whether the artefact you published already answers it. If your dataset is public and contains the per-site labels, you have already published the list, whatever your paper says. See [[:Artifacts]]. |
| * **Your detector's false positives become named accusations.** A detector with 91% precision that flags 4,000 sites names about 360 of them wrongly. That is tolerable in a table and not tolerable in a headline. If you publish per-site results, the precision figure belongs next to them, in the same sentence. | * **Your detector's false positives become named accusations.** A detector with 91% precision that flags 4,000 sites names about 360 of them wrongly. That is tolerable in a table and not tolerable in a headline. If you publish per-site results, the precision figure belongs next to them, in the same sentence. |
| |
| * [[Design:Website selection]] and [[Design:Sampling]] — where your denominator comes from and what it is not. | * [[Design:Website selection]] and [[Design:Sampling]] — where your denominator comes from and what it is not. |
| * [[Design:Crawling location]] — why "the web" in your sentence needs a vantage point attached. | * [[Design:Crawling location]] — why "the web" in your sentence needs a vantage point attached. |
| * [[Artifacts]] — the per-site list you may have published without deciding to. | * [[:Artifacts]] — the per-site list you may have published without deciding to. |
| * [[Statistics:Biases]] — the gap between what you sampled and what you will be quoted as describing. | * [[Statistics:Biases]] — the gap between what you sampled and what you will be quoted as describing. |
| * [[Writing:Conferences]] — venue choice, which also decides when your paper becomes public and whether an embargo is available at all. | * [[Writing:Conferences]] — venue choice, which also decides when your paper becomes public and whether an embargo is available at all. |
| * **Seven venues only** — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent, and CHI and SOUPS are where at-risk and usable-security work most often appears — the community that produced the one paper here with explicit press advice. Every claim about "the field" on this page is a claim about these seven venues. | * **Seven venues only** — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent, and CHI and SOUPS are where at-risk and usable-security work most often appears — the community that produced the one paper here with explicit press advice. Every claim about "the field" on this page is a claim about these seven venues. |
| * **The count of three documented cases is a floor.** It came from a keyword sweep over full text, and the first version of that sweep missed one of the three; the regex was widened and re-run, but a keyword sweep under-recalls by construction and this one demonstrably did. If you know of a fourth, it belongs here. | * **The count of three documented cases is a floor.** It came from a keyword sweep over full text, and the first version of that sweep missed one of the three; the regex was widened and re-run, but a keyword sweep under-recalls by construction and this one demonstrably did. If you know of a fourth, it belongs here. |
| * **The denominator figure is a regex over a model-selected sentence.** It asks whether a percentage in a sentence is attached to "of //something//" in that same sentence. It cannot tell a well-written sentence from a lucky one; it says nothing about whether the paper gives its denominator elsewhere, and mostly papers do; and ''evidence.quote'' is one sentence chosen by an extraction model as the best evidence for a finding, which may be systematically better or worse written than the paper's average. **This is the weakest assumption on the page**, and the reason the figure is framed as being about quotable sentences rather than about papers. The residue in both directions is printed by ''--examples'' and reproduced in full on the provenance page. | * **The denominator figure is a regex over a model-selected sentence.** It asks whether a percentage in a sentence is attached to "of //something//" in that same sentence. It cannot tell a well-written sentence from a lucky one; it says nothing about whether the paper gives its denominator elsewhere, and mostly papers do; and ''evidence.quote'' is one sentence chosen by an extraction model as the best evidence for a finding, which may be systematically better or worse written than the paper's average. **This is the weakest assumption on the page**, and the reason the figure is framed as being about quotable sentences rather than about papers. The residue in both directions is printed by ''%%--examples%%'' and reproduced in full on the provenance page. |
| * **''legal.foundViolations'' is an enum and can carry a percentage**, but "assessed a law" is itself an extraction judgement, 402 papers is a small population for a five-way split, and the enum records what a paper wrote rather than why it wrote it. | * **''legal.foundViolations'' is an enum and can carry a percentage**, but "assessed a law" is itself an extraction judgement, 402 papers is a small population for a five-way split, and the enum records what a paper wrote rather than why it wrote it. |
| |
