Working log behind Notifying websites. Every query with its denominator, the report script and its unedited output, the folding rules with their residue, the quotes that were spot-checked, the external sources that were verified and the ones that were rejected, and what could not be established. Corpus-wide caveats are on Corpus and are not restated here.

Not prose. Read it if you are checking a number.

House convention on provenance: pages is mixed and this page picked the more traceable end of it. provenance:privacy:fingerprinting and provenance:design:ip_classification use no inline citation markers at all and name papers by slug; provenance:privacy:requests uses citekeys and a bibliography block, as this page does. Two of the four carry a footer back-link, so this one does too. No ~~DISCUSSION~~ block, matching all four: comments belong on the content page.

The item as specified pointed at ethics.notifiedAffectedParties and ethics.harmMitigation and asked for “how to reach an operator at scale, response rates reported in the literature, and the disclosure-timeline conventions”. Two judgement calls:

• Scope excludes one-off coordinated disclosure to a named vendor. That is a queue with an SLA and nothing on this page would help. The page says so in its second paragraph. The consequence is that a large share of the 1,636 notifiedAffectedParties == yes papers are out of scope for the how-to part while remaining in scope for the how often part; the page keeps those two questions in separate sections rather than pooling them.
• Scope includes network-scanning campaigns, not only web ones. The page is titled “Notifying Websites” but 10 of the 22 hand-classified campaigns notified network operators rather than site owners, and the methodological literature (control groups, channel comparison, decay curves, the one RCT) is disproportionately on that side. Excluding it would have produced a page with three campaigns on it. The page states which population each figure comes from. A reasonable person would have narrowed this to web only and produced a thinner, more on-title page.
• Not broadening a neighbour. Ethics exists but is 2.6 kB of bullet-point notes covering crawling harm, robots.txt and ethics approval. Notification is one line in its outline. Broadening it would have buried a 30 kB page inside a stub on a different topic. The two pages cross-link instead.

All figures from scripts/report_notifying_websites.mjs. Denominators, in the order the page uses them:

The old task hint (notifiedAffectedParties stated by 40.9%, harmMitigation by 57.4%, of 3,814 empirical papers) was computed on the 4,322-paper corpus and is superseded in both numerator and denominator. No figure was carried over.

Run as node scripts/report_notifying_websites.mjs. Flags: –wiki for DokuWiki tables, –list for the hand-classified campaigns, –quotes for ethics evidence quotes, –residue for every unmapped fold entry and every unreviewed candidate.

out/notifying_websites-output.txt in the workdir is two concatenated runs: the plain run (lines 1–143) followed by the –list run (lines 144–448), which repeats the plain output before adding the campaign list. A diff against a single fresh invocation therefore shows ~305 spurious extra lines. This misled one reviewer; head -143 and tail -n +144 each match a fresh run byte-for-byte. The script itself is deterministic — verified.

Sections of the script, so a figure on the content page can be traced to the block that produced it:

• A — ethics.notifiedAffectedParties by value and by population
• B — four-year buckets: notification, silence, disclosure detail, harm mitigation, regulator contact
• C — ethics.regulatorContact; ethics.harmMitigation folded into families with residue
• D — ethics.disclosureDetail folded into channel and outcome families with residue, plus every entry carrying a countable ratio
• E — full-text candidate scan, hand classification, and the unreviewed residue
• F — (–quotes) ethics evidence quotes for web papers with notifiedAffectedParties == yes

corpus: 5859 extraction records, 7 venues, 2010–2026
populations: empirical 5118 | empirical with an ethics record 4472 (87.4%) | crawled 1120 (with ethics record 992) | network-scan 930 (with ethics record 858)

==========================================================================
A. HOW OFTEN THE FIELD SAYS IT NOTIFIED ANYONE
==========================================================================

-- ethics.notifiedAffectedParties, of 4472 empirical papers with an ethics record --
Value           Papers  Share
--------------  ------  -----
yes             1636    36.6%
partial         524     11.7%
no              194     4.3%
not-applicable  1001    22.4%
not-stated      1117    25.0%
stated something (yes|partial|no): 2354 = 52.6%; notified (yes|partial): 2160 = 48.3%

-- the same field, per population --
Population (all ∧ has an ethics record)  N     yes    partial  no     not-applicable  not-stated
---------------------------------------  ----  -----  -------  -----  --------------  ----------
empirical                                4472  36.6%  11.7%    4.3%   22.4%           25.0%
empirical ∧ measured the web platform    1362  33.2%  11.7%    7.0%   19.5%           28.6%
ran an automated web crawl               992   29.8%  13.7%    9.4%   15.9%           31.1%
ran a network scan or probe              858   37.6%  23.4%    4.2%   10.7%           24.0%
assessed compliance with a law           376   42.0%  14.6%    9.8%   21.5%           12.0%
crawled ∧ assessed a law                 123   35.0%  14.6%    14.6%  15.4%           20.3%
recruited human participants             1283  39.7%  7.6%     1.9%   35.3%           15.4%

==========================================================================
B. IS NOTIFICATION BECOMING A NORM? (four-year buckets)
==========================================================================
Indicator                              Denominator         2010–2013  2014–2017  2018–2021  2022–2024  2025–2026*
-------------------------------------  ------------------  ---------  ---------  ---------  ---------  ----------
Papers with an ethics record           empirical           338/460    610/718    1089/1272  1482/1649  953/1019
Notified (yes|partial)                 empirical ∧ ethics  21.3%      37.7%      45.9%      52.8%      60.3%
Said nothing (not-stated)              empirical ∧ ethics  61.5%      40.2%      26.5%      16.9%      13.1%
Notified (yes|partial), crawlers only  crawled ∧ ethics    14.6%      34.7%      41.4%      50.6%      54.3%
Gave any disclosureDetail text         empirical ∧ ethics  31.7%      48.4%      58.9%      70.8%      81.6%
Named a harm-mitigation step           empirical ∧ ethics  46.4%      61.1%      65.0%      76.8%      85.0%
Contacted a regulator or CERT (yes)    empirical ∧ ethics  0.3%       3.0%       4.7%       3.4%       2.8%

==========================================================================
C. REGULATOR / CERT CONTACT, AND HARM MITIGATION
==========================================================================

-- ethics.regulatorContact, of 4472 --
Value       Papers  Share
----------  ------  -----
yes         147     3.3%
no          1982    44.3%
not-stated  2343    52.4%

of the 376 that assessed a law: yes 27 (7.2%)

of the 2160 that notified an affected party, 135 also contacted a regulator or CERT (6.3%)
of the 147 that contacted a regulator or CERT, 135 ALSO notified the affected party directly (91.8%) — the intermediary is an addition, not a substitute

-- ethics.harmMitigation: 3186 of 4472 (71.2%) name at least one step --
Family (a paper can be in several)                                              Papers  Share of 3186
------------------------------------------------------------------------------  ------  -------------
anonymisation, aggregation, no PII collected                                    732     23.0%
NOTIFICATION — told the affected party                                          594     18.6%
rate limiting, bandwidth caps, low request volume                               287     9.0%
consent, permission, or cooperation obtained                                    259     8.1%
withheld, redacted, or delayed publication                                      241     7.6%
isolated testbed, own accounts, own devices                                     178     5.6%
no exploitation, non-invasive probing, restricted scope                         163     5.1%
access-controlled or encrypted storage                                          156     4.9%
NOTIFICATION — opt-out, exclusion list, or an explanatory page for the scanner  114     3.6%
debriefing, compensation, or support for participants                           65      2.0%
ethics board or legal review                                                    25      0.8%
unmapped residue: 1069 of 3186 (33.6%)
  (run with --residue to print all of it)

==========================================================================
D. WHAT ethics.disclosureDetail SAYS — CHANNELS AND OUTCOMES
==========================================================================
2870 of 4472 (64.2%) give free-text disclosure detail. The field is capped at 20 words, so it names a channel only when the channel was
the notable thing. These are rankings of an under-reported field, not adoption rates.

-- channel named in disclosureDetail, of 2870 papers that gave any disclosure detail --
Channel family                                                                 Papers  Share of 2870
-----------------------------------------------------------------------------  ------  -------------
a named large platform or app store (Google, Apple, Meta, Microsoft, Amazon…)  335     11.7%
direct email to the operator or developer                                      113     3.9%
CERT / CSIRT / national coordinator                                            39      1.4%
hosting provider / ISP / upstream                                              38      1.3%
bug bounty / vulnerability reward programme                                    36      1.3%
WHOIS / registry contact                                                       18      0.6%
data protection authority / regulator                                          17      0.6%
standards body / working group                                                 13      0.5%
generic contact form or phone                                                  10      0.3%
no channel named at all: 2303 of 2870 (80.2%)

-- channel named in disclosureDetail, of 2160 papers that notified (yes|partial) AND gave a detail — the population where "which channel?" is a fair question --
Channel family                                                                 Papers  Share of 2160
-----------------------------------------------------------------------------  ------  -------------
a named large platform or app store (Google, Apple, Meta, Microsoft, Amazon…)  318     14.7%
direct email to the operator or developer                                      99      4.6%
CERT / CSIRT / national coordinator                                            39      1.8%
bug bounty / vulnerability reward programme                                    35      1.6%
hosting provider / ISP / upstream                                              35      1.6%
WHOIS / registry contact                                                       15      0.7%
data protection authority / regulator                                          14      0.6%
standards body / working group                                                 13      0.6%
generic contact form or phone                                                  9       0.4%
no channel named at all: 1633 of 2160 (75.6%)

-- outcome named in disclosureDetail, of 2870 --
Outcome family                    Papers  Share of 2870
--------------------------------  ------  -------------
fixed / patched / removed         373     13.0%
acknowledged / confirmed          359     12.5%
CVE assigned or bounty paid       96      3.3%
no response / ignored / declined  25      0.9%
still pending / awaiting          18      0.6%
no outcome named: 2155 of 2870 (75.1%)

-- disclosureDetail entries carrying a number: 7 of 2870 (0.2%) --
  IMC/2016/ad-blockers-global-prevalence-and-impact
    Panel was 100% opt-in with thorough privacy disclosure pages.
  USENIX/2021/lzr-identifying-unexpected-internet-services
    No complaints or operator questions from the 1% scans; seven organizations complained during 100% scans.
  CCS/2022/an-extensive-study-of-residential-proxies-in-china
    Contacted 318 of 559 sensitive organizations; received 11 responses by submission.
  CCS/2022/understanding-iot-security-from-a-market-scale-perspective
    Reported confirmed case-study vulnerabilities to 12 of 18 vendors; reporting to six remained in progress.
  CCS/2022/do-opt-outs-really-opt-me-out
    Informed 10 of 11 inconsistent trackers by email; one corrected its opt-out implementation.
  USENIX/2024/fv8-a-forced-execution-javascript-engine-for-detecting-evasive-techniques
    Reported 110 malicious extensions to Google; 62.7% were removed.
  IMC/2025/poster-how-do-you-know-my-name-investigating-the-role-of-domain-names-for-target
    Cooperated with toplist maintainers and limited honey domains to 0.02% of top-10ⁿ domains.

==========================================================================
E. NOTIFICATION CAMPAIGNS IN THE FULL TEXT (hand-classified)
==========================================================================
Stage                                                   Papers
------------------------------------------------------  ------
paper.cols.txt files read                               5869
candidates (notify ∧ outcome ∧ number in one sentence)  179
hand-classified as a campaign                           22
hand-classified and rejected, with a reason             16
UNREVIEWED residue — matched, nobody read the paper     144

sanity: campaigns not caught by the regex (added from a wider first-pass scan): 0
         rejected slugs not caught by the regex: 3 — how-website-owners-face-privacy-issues-thematic-analysis-of-responses-from-a-cov, from-whois-to-whowas-a-large-scale-measurement-study-of-domain-registration-privacy-under-the-gdpr, the-cve-wayback-machine-measuring-coordinated-disclosure-from-exploits-against-t

-- campaigns by decade-ish bucket, of 22 hand-classified --
Bucket      Campaigns
----------  ---------
2010–2013   1
2014–2017   4
2018–2021   4
2022–2024   8
2025–2026*  5
What was notified about  Campaigns
-----------------------  ---------
network                  10
web                      10
mobile                   2

Two of the four figures in ethics.disclosureDetail ratios above are quoted on the content page and were re-verified against the JSONL by exact substring match: an-extensive-study-of-residential-proxies-in-china (CCS 2022) and fv8-a-forced-execution-javascript-engine-for-detecting-evasive-techniques (USENIX Sec 2024), one match each.

Free-text fields agree run-to-run on roughly 20% of exact strings, so nothing here is aggregated by exact string. Three folds were written, in scripts/report_notifying_websites.mjs and scripts/notify_fold.mjs.

Nine regex families, not mutually exclusive (a paper that told a CERT and the vendor belongs in both; papers counted once per family). Published on the content page as a ranking, never as adoption rates.

• Scoped to the right population, after a post-review check. The first version reported the channel fold over all 2,870 papers with a disclosureDetail, giving 80.2% with no channel named. Reading a sample of that residue showed the problem: disclosureDetail serves two purposes in the schema, notifying an operator and debriefing a participant, so the residue included entries like “Participants provided written or verbal consent; screen recordings were deleted after data collection” — papers that were never notifying an operator, for which “which channel?” is not a question. The script now prints both populations, and the page quotes the scoped one: 1,633 of the 2,160 papers that said they notified (75.6%) name no channel. All 2,160 yes/partial papers turn out to carry a detail, so that subset needs no separate silence figure.
• The residue is still the headline: three quarters of the papers that say they notified do not say through what. That is not a fold failure, it is the field's content — a 20-word summary of “we disclosed responsibly” names no channel.
• The largest matched family, “a named large platform or app store”, is a deliberately loose rule matching the strings google play, app store, play store, apple, google, amazon, facebook, meta, microsoft, marketplace, mozilla, chrome, android. It over-matches: a paper that used Google Play as a population source and disclosed elsewhere lands here. The label on the content page says “a named large platform”, not “notified a platform”, for that reason. Do not read 11.7% as “11.7% notified a platform”.
• hosting provider / ISP / upstream also matches the bare token registrar, which overlaps the WHOIS family. Both are under 1.5% and reported as a ranking, so the overlap is stated rather than resolved.

Five families, non-exclusive. Residue 2,155 of 2,870 (75.1%), published above. Not used on the content page except as the 75.1% figure.

Eleven families, non-exclusive. Residue 1,069 of 3,186 (33.6%). This fold is diagnostic only — no folded harm-mitigation figure appears on the content page; the page uses only the non-null count (71.2%) and its per-bucket trend, neither of which depends on the fold. The residue is therefore recorded here by shape rather than in full, and prints with node scripts/report_notifying_websites.mjs –residue.

Shape of the 1,069, read from the first ~60 entries and a shuffled sample of 45: almost entirely study-specific harm limitation with no generalisable family — “Used a wired testbed and 900-MHz transmitter to minimize GPS interference”, “Purchased unlimited daily plans so test traffic was not counted”, “Suppressed uploads from BitTorrent clients to avoid distributing illegal software”, “Captured only four bytes of application payload to minimise privacy impact”. A first attempt with mutually-exclusive first-match-wins rules and narrower regexes gave 46.5% residue and a nonsensical zero in one family; broadening the regexes and making families non-exclusive took it to 33.6%. It was not driven lower, because the remainder is genuinely one-off. Anyone wanting a publishable harm-mitigation taxonomy should treat that residue as the work, not as noise.

The one fold that is a hand list rather than a regex family list, and the reason is in the file: the corpus signal for “the authors ran a campaign” is a sentence, not a name. ethics.notifiedAffectedParties == yes does not distinguish a bulk campaign to thousands of strangers from one email to Google's security team, and those are different methods with different rates.

Candidate rule: over normaliseText(paper.cols.txt) (de-hyphenate line breaks, collapse whitespace), a sentence under 400 characters matching all three of

• NOTIFY_RE = /notif|disclos|reported (?:it |them |the )?to|informed|contacted|reach(?:ed)? out/i
• OUTCOME_RE = /respon|repl(?:y|ied|ies)|acknowledg|fix|patch|remediat|resolv|ignor|no answer|bounce|took? (?:it )?down|removed/i
• NUM_RE = a percentage, a comma-grouped integer, or an N of M ratio

Result over 5,869 files: 179 candidates → 22 campaigns, 16 rejected with a reason, 144 unreviewed.

Inclusion threshold, applied by hand: the authors notified parties they did not already know, at a scale where the contact had to be looked up, and reported a delivery, response or remediation outcome. Rough scale floor ≥100 parties, which is why PETS 2022's 40-organisation SAR re-test is in the rejected list and quoted only in prose.

An earlier, wider first-pass scan (a looser regex, 197 candidates) was used to seed the hand list; the script reports that 0 campaigns and 3 rejections fall outside the committed narrow regex, so the committed rule reproduces the campaign set. The three rejections it misses are how-website-owners-face-privacy-issues…, from-whois-to-whowas… and the-cve-wayback-machine… — kept in REJECTED anyway because they are papers a future run will want the reason for.

Rejections, with the deciding reason (this is the part that stops the next run re-adding them):

The six campaigns not on the content page's rates table, because they report a volume sent but no response or remediation figure: freely-given-consent… (CCS 2022, 1,127 developers), extended-hell-o… (USENIX Sec 2023), dancer-in-the-dark… (USENIX Sec 2024, 19% response — this one is quoted in prose), beyond-exploit-scanning… (USENIX Sec 2025, 4,983 emails), hey-your-secrets-leaked… (IEEE S&P 2025, 3,906 emails), and beyond-the-horizon… (IEEE S&P 2025, 15,837 ASes).

So the rates table has 17 rows: 16 of the 22 hand-classified campaigns, plus [1Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)], which is not in CAMPAIGNS because its full text is not in the corpus (see below) and the fold only classifies papers the scan could read. catch-22… is on the table for its channel, not for a rate.

Matched the candidate rule; nobody read the paper. Any of these may be a campaign the content page should have listed. The rule has known false positives — a candidate sentence can be a paper describing someone else's campaign in related work, or a survey response rate, or a sentence about “disclosure” in an unrelated sense (self-disclosure in social media, information disclosure as a vulnerability class), all of which are visible in the list below.

IMC 2010 internet-background-radiation-revisited
IMC 2010 selecting-representative-ip-addresses-for-internet-topology-studies
CCS 2012 before-we-knew-it-an-empirical-study-of-zero-day-attacks-in-the-real-world
NDSS 2013 when-firmware-modifications-attack-a-case-study-of-embedded-exploitation
USENIX 2013 an-empirical-study-of-vulnerability-rewards-programs
WWW 2013 two-years-of-short-urls-internet-measurement-security-threats-and-countermeasure
USENIX 2014 a-look-at-targeted-attacks-through-the-lense-of-an-ngo
USENIX 2014 brahmastra-driving-apps-to-test-the-security-of-third-party-components
IEEE-SP 2015 the-attack-of-the-clones-a-study-of-the-impact-of-shared-code-on-vulnerability-p
NDSS 2015 checking-more-and-alerting-less-detecting-privacy-leakages-via-enhanced-data-flo
PETS 2015 portrait-of-a-privacy-invasion
IEEE-SP 2016 seeking-nonsense-looking-for-trouble-efficient-promotional-infection-detection-t
IEEE-SP 2016 talos-neutralizing-vulnerabilities-with-security-workarounds-for-rapid-response
IMC 2016 weak-keys-remain-widespread-in-network-devices
PETS 2016 towards-mining-latent-client-identifiers-from-network-traffic
IMC 2017 if-you-are-not-paying-for-it-you-are-the-product-how-much-do-advertisers-pay-to
NDSS 2017 are-we-there-yet-on-rpki-s-deployment-and-security
PETS 2017 wiretapping-end-to-end-encrypted-voip-calls-real-world-attacks-on-zrtp
USENIX 2018 from-patching-delays-to-infection-symptoms-using-risk-profiles-for-an-early-disc
USENIX 2018 wpse-fortifying-web-protocols-via-browser-side-security-monitoring
CCS 2019 a-usability-evaluation-of-lets-encrypt-and-certbot-usable-security-done-right
NDSS 2019 automating-patching-of-vulnerable-open-source-software-versions-in-application-binaries
NDSS 2019 digital-healthcare-associated-infection-a-case-study-on-the-security-of-a-major-multi-campus-hospital-system
NDSS 2019 we-value-your-privacy-now-take-some-cookies-measuring-the-gdprs-impact-on-web-privacy
PETS 2019 accessileaks-investigating-privacy-leaks-exposed-by-the-android-accessibility-se
USENIX 2019 50-ways-to-leak-your-data-an-exploration-of-apps-circumvention-of-the-android-pe
USENIX 2019 evaluating-the-contextual-integrity-of-privacy-regulation-parents-iot-toy-privac
USENIX 2019 the-betrayal-at-cloud-city-an-empirical-analysis-of-cloud-based-mobile-backends
WWW 2019 studying-preferences-and-concerns-about-information-disclosure-in-email-notifica
CCS 2020 dangerous-skills-got-certified-measuring-the-trustworthiness-of-skill-certificat
IEEE-SP 2020 a-security-analysis-of-the-facebook-ad-library
IEEE-SP 2020 the-many-kinds-of-creepware-used-for-interpersonal-attacks
IMC 2020 behind-closed-doors-a-network-tale-of-spoofing-intrusion-and-false-dns-security
NDSS 2020 broken-metre-attacking-resource-metering-in-evm
NDSS 2020 complex-security-policy-a-longitudinal-analysis-of-deployed-content-security-policies
PETS 2020 the-price-is-not-right-comparing-privacy-in-free-and-paid-apps
USENIX 2020 actions-speak-louder-than-words-entity-sensitive-privacy-policy-and-data-flow-an
USENIX 2020 cached-and-confused-web-cache-deception-in-the-wild
USENIX 2020 plug-n-pwned-comprehensive-vulnerability-analysis-of-obd-ii-dongles-as-a-new-ove
WWW 2020 an-empirical-study-of-the-use-of-integrity-verification-mechanisms-for-web-subre
CCS 2021 i-need-a-better-description-an-investigation-into-user-expectations-for-differen
IEEE-SP 2021 sok-quantifying-cyber-risk
NDSS 2021 all-the-numbers-are-us-large-scale-abuse-of-contact-discovery-in-mobile-messengers
NDSS 2021 shadow-attacks-hiding-and-replacing-content-in-signed-pdfs
PETS 2021 defining-privacy-how-users-interpret-technical-terms-in-privacy-policies
PETS 2021 digital-inequality-through-the-lens-of-self-disclosure
USENIX 2021 a-large-scale-interview-study-on-information-security-in-and-attacks-against-sma
USENIX 2021 injection-attacks-reloaded-tunnelling-malicious-payloads-over-dns
USENIX 2021 now-im-a-bit-angry-individuals-awareness-perception-and-responses-to-data-breach
CCS 2022 clues-in-tweets-twitter-guided-discovery-and-analysis-of-sms-spam
IEEE-SP 2022 timing-based-browsing-privacy-vulnerabilities-via-site-isolation
IMC 2022 characterizing-permanently-dead-links-on-wikipedia
IMC 2022 mind-your-manrs-measuring-the-manrs-ecosystem
NDSS 2022 auto-draft-219
PETS 2022 developers-say-the-darnedest-things-privacy-compliance-processes-followed-by-dev
PETS 2022 exploring-the-privacy-concerns-of-bystanders-in-smart-homes-from-the-perspective
PETS 2022 we-may-share-the-number-of-diaper-changes-a-privacy-and-security-analysis-of-mob
USENIX 2022 movery-a-precise-approach-for-modified-vulnerable-code-clone-discovery-from-modi
WWW 2022 measuring-alexa-skill-privacy-practices-across-three-years
WWW 2022 understanding-the-practice-of-security-patch-management-across-multiple-branches
CCS 2023 dont-leak-your-keys-understanding-measuring-and-exploiting-the-appsecret-leaks-i
CCS 2023 passive-ssh-key-compromise-via-lattices
IEEE-SP 2023 no-one-drinks-from-the-firehose-how-organizations-filter-and-prioritize-vulnerab
IEEE-SP 2023 no-privacy-in-the-electronics-repair-industry
IEEE-SP 2023 three-birds-with-one-stone-efficient-partitioning-attacks-on-interdependent-cryp
IEEE-SP 2023 utopia-automatic-generation-of-fuzz-driver-using-unit-tests
IMC 2023 a-longitudinal-study-of-vulnerable-client-side-resources-and-web-developers-upda
IMC 2023 rovista-measuring-and-analyzing-the-route-origin-validation-rov-in-rpki
IMC 2023 wolf-in-sheeps-clothing-evaluating-security-risks-of-the-undelegated-record-on-d
PETS 2023 investigating-how-users-imagine-their-personal-privacy-assistant
PETS 2023 lessons-in-vcr-repair-compliance-of-android-app-developers-with-the-california-c
PETS 2023 on-the-role-and-form-of-personal-information-disclosure-in-cyberbullying-inciden
PETS 2023 privacy-rarely-considered-exploring-considerations-in-the-adoption-of-third-part
PETS 2023 revoked-just-now-users-behaviors-toward-fitness-data-sharing-with-third-party-ap
USENIX 2023 a-bugs-life-analyzing-the-lifecycle-and-mitigation-process-of-content-security-p
USENIX 2023 all-of-them-claim-to-be-the-best-multi-perspective-study-of-vpn-users-and-vpn-pr
USENIX 2023 arana-discovering-and-characterizing-password-guessing-attacks-in-practice
USENIX 2023 are-you-spying-on-me-large-scale-analysis-on-iot-data-exposure-through-companion
USENIX 2023 downgrading-dnssec-how-to-exploit-crypto-agility-for-hijacking-signed-zones
USENIX 2023 internet-service-providers-and-individuals-attitudes-barriers-and-incentives-to
USENIX 2023 my-privacy-for-their-security-employees-privacy-perspectives-and-expectations-wh
USENIX 2023 panda-security-analysis-of-algorand-smart-contracts
USENIX 2023 pushed-by-accident-a-mixed-methods-study-on-strategies-of-handling-secret-inform
USENIX 2023 whip-improving-static-vulnerability-detection-in-web-application-by-forcing-tool
CCS 2024 airgapagent-protecting-privacy-conscious-conversational-agents
CCS 2024 are-we-getting-well-informed-an-in-depth-study-of-runtime-privacy-notice-practic
CCS 2024 defying-the-odds-solanas-unexpected-resilience-in-spite-of-the-security-challeng
IEEE-SP 2024 breach-extraction-attacks-exposing-and-addressing-the-leakage-in-second-generati
IEEE-SP 2024 from-virtual-touch-to-tesla-command-unlocking-unauthenticated-control-chains-fro
IEEE-SP 2024 shedding-light-on-cvss-scoring-inconsistencies-a-user-centric-study-on-evaluatin
IEEE-SP 2024 sok-sgx-fail-how-stuff-gets-exposed
IEEE-SP 2024 tabbed-out-subverting-the-android-custom-tab-security-model
NDSS 2024 dont-interrupt-me-a-large-scale-study-of-on-device-permission-prompt-quieting-in-chrome
NDSS 2024 understanding-the-implementation-and-security-implications-of-protective-dns-services
PETS 2024 exploring-the-privacy-experiences-of-closeted-users-of-online-dating-services-in
PETS 2024 security-and-privacy-with-second-hand-storage-devices-a-user-centric-perspective
PETS 2024 the-medium-is-the-message-how-secure-messaging-apps-leak-sensitive-data-to-push
USENIX 2024 diffie-hellman-picture-show-key-exchange-stories-from-commercial-vowifi-deployme
USENIX 2024 digital-discrimination-of-users-in-sanctioned-states-the-case-of-the-cuba-embarg
USENIX 2024 globalconfusion-trustzone-trusted-application-0-days-by-design
USENIX 2024 malla-demystifying-real-world-large-language-model-integrated-malicious-services
USENIX 2024 rethinking-the-security-threats-of-stale-dns-glue-records
USENIX 2024 spf-beyond-the-standard-management-and-operational-challenges-in-practice-and-pr
USENIX 2024 towards-more-practical-threat-models-in-artificial-intelligence-security
USENIX 2024 unpacking-privacy-labels-a-measurement-and-developer-perspective-on-googles-data
WWW 2024 unfiltered-measuring-cloud-based-email-filtering-bypasses
CCS 2025 a-decade-long-landscape-of-advanced-persistent-threats-longitudinal-analysis-and
CCS 2025 is-this-a-scam-the-nature-and-quality-of-reddit-discussion-about-scams
IEEE-SP 2025 gptracker-a-large-scale-measurement-of-misused-gpts
IEEE-SP 2025 sniffing-location-privacy-of-video-conference-users-using-free-audio-channels
NDSS 2025 hitchhiking-vaccine-enhancing-botnet-remediation-with-remote-code-deployment-reuse
NDSS 2025 was-this-you-investigating-the-design-considerations-for-suspicious-login-notifications
NDSS 2025 who-is-trying-to-access-my-account-exploring-user-perceptions-and-reactions-to-risk-based-authentication-notifications
PETS 2025 ai-is-from-the-devil-behaviors-and-concerns-toward-personal-data-sharing-with-ll
PETS 2025 how-unique-is-whose-web-browser-the-role-of-demographics-in-browser-fingerprinti
PETS 2025 privacy-perceptions-and-behaviors-towards-targeted-advertising-on-social-media-a
PETS 2025 the-effect-of-platform-policies-on-app-privacy-compliance-a-study-of-child-direc
USENIX 2025 a-mixed-methods-study-of-open-source-software-maintainers-on-vulnerability-manag
USENIX 2025 an-industry-interview-study-of-software-signing-for-supply-chain-security
USENIX 2025 collisionrepair-first-aid-and-automated-patching-for-storage-collision-vulnerabi
USENIX 2025 exploring-user-security-and-privacy-attitudes-and-concerns-toward-the-use-of-gen
USENIX 2025 gpuhammer-rowhammer-attacks-on-gpu-memories-are-practical
USENIX 2025 malicious-llm-based-conversational-ai-makes-users-reveal-personal-information
USENIX 2025 misty-registry-an-empirical-study-of-flawed-domain-registry-operation
USENIX 2025 privacy-law-enforcement-under-centralized-governance-a-qualitative-analysis-of-f
USENIX 2025 regulating-smart-device-support-periods-user-expectations-and-the-european-cyber
USENIX 2025 your-shield-is-my-sword-a-persistent-denial-of-service-attack-via-the-reuse-of-u
IEEE-SP 2026 consumer-beware-exploring-data-brokers-ccpa-compliance
NDSS 2026 chameleoscan-demystifying-and-detecting-ios-chameleon-apps-via-llm-powered-ui-exploration
NDSS 2026 cross-boundary-mobile-tracking-exploring-java-to-javascript-information-diffusion-in-webviews
NDSS 2026 demystifying-the-access-control-mechanism-of-esxi-vmkernel
NDSS 2026 tickets-to-hide-an-inside-look-into-the-anti-abuse-ecosystem-through-internal-abuse-data
NDSS 2026 time-and-time-again-leveraging-tcp-timestamps-to-improve-remote-timing-attacks
NDSS 2026 unveiling-byovd-threats-malwares-use-and-abuse-of-kernel-drivers
PETS 2026 more-space-less-privacy-measuring-the-effectiveness-of-ip-based-website-fingerpr
PETS 2026 personal-data-flows-and-privacy-policy-traceability-in-third-party-llm-apps-in-t
PETS 2026 precision-leads-recalling-you-improved-location-privacy-for-shared-mobility-serv
PETS 2026 the-role-of-online-forums-in-developer-understanding-of-privacy-law-a-reddit-cas
USENIX 2026 abuse-risks-are-often-inherent-to-product-features-exploring-ai-vendors-bug-boun
USENIX 2026 inconsistent-incomplete-and-insecure-a-survey-of-account-security-interfaces
USENIX 2026 the-art-of-hide-and-seek-making-pickle-based-model-supply-chain-poisoning-stealt
WWW 2026 bowling-with-chatgpt-on-the-evolving-user-interactions-with-conversational-ai-sy
WWW 2026 falconscope-effective-and-efficient-detection-of-hidden-web-interfaces-in-iot-de
WWW 2026 unveiling-the-resilience-of-llm-enhanced-search-engines-against-black-hat-seo-ma

Filed as deferred work, not left as a TODO on a wiki page: DRAIN_DB=/workspace/drain/work.db python3 /workspace/drain/sync.py add-item –task wiki-measuretheweb –key 'notifying_websites: read the 144 unreviewed campaign candidates', with the shortlist above and the two open security.txt questions attached. The first published version of that list had 135 of the 144 entries, because it was built through a shell pipeline that dropped nine rows rather than copied from the script. It is now generated directly from report_notifying_websites.mjs –residue and asserted to be exactly 144 long. A residue list that is itself truncated is worse than no residue list, and this one was, for three revisions. The list above is the honest coverage boundary of this page. The highest-value follow-up is reading the ~15 of these that look like real campaigns — NDSS 2017 are-we-there-yet-on-rpki-s-deployment-and-security (a “19% success rate” for ROAlert notifications), IMC 2016 weak-keys-remain-widespread-in-network-devices, IMC 2020 behind-closed-doors…, NDSS 2020 complex-security-policy…, WWW 2024 unfiltered… (“over 80% of the domains contacted were unresponsive”), IEEE-SP 2025 gptracker… — and adding them to CAMPAIGNS.

Seven papers with notifiedAffectedParties == yes ∧ platforms ∋ web, picked deterministically one per year across 2013–2026, checked after de-hyphenating and whitespace-collapsing both the quote and paper.cols.txt:

4 exact, 1 partial, 2 spliced-but-locatable, 0 unlocatable, 0 fabricated. Consistent with the corpus README's 57.9% exact / 37% via an 8-word run / 0.9% unlocatable. No ethics quote is quoted on the content page, so this check was for confidence in the enum, not to validate a published string.

Every load-bearing quote on the content page was re-checked by the orchestrator against paper.cols.txt after the sub-agents reported it, because sub-agent line numbers are not by themselves evidence.

Two errors in the first published version, caught by the orchestrator's own re-check after publication and fixed in revision 2:

• The [13Sasaki, Takayuki; Fujita, Akira; Gañán, Carlos Hernandez; van Eeten, Michel; Yoshioka, Katsunari; Matsumoto, Tsutomu (2022): "Exposed Infrastructures: Discovery, Attacks and Remediation of Insecure ICS Remote Management Devices", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] row said “93 (58%) responded — 'higher than most previous notification experiments'”. Wrong on three counts. The channel is manual telephone calls, not email — the paper explicitly contrasts itself with prior studies that used IP-WHOIS abuse contacts and claims to be “the first study to directly contact the organization operating the device”. The 93 (58%) is a spliced Table VI cell, not a response rate. And “higher than most previous notification experiments” is about the paper's remediation rate, not its response rate. The corrected row uses the paper's own funnel: 160 operators notified about 317 devices, the person in charge reached for 212, “50% of the persons in charge … stated that they mitigated or will mitigate”, devices “reduced by 58% when we were able to contact the persons in charge” against 13% for un-notified devices, χ² p<0.0001. This came from a sub-agent report that the orchestrator did not re-verify before first publication — the one place on the page where the verification discipline recorded above was not actually followed.
• The [14Cetin, Orcun; Gañán, Carlos; Altena, Lisette; Kasama, Takahiro; Inoue, Daisuke; Tamiya, Kazuki; Tie, Ying; Yoshioka, Katsunari; van Eeten, Michel (2019): "Cleaning Up the Internet of Evil Things: Real-World Evidence on ISP and Consumer Efforts to Remove Mirai", in: Proceedings of the Network and Distributed System Security Symposium. (Link)] control column said “74% natural remediation in the control”. The paper says “surprisingly high natural remediation rates of 58[–]74% for this control group and for two reference networks” — a range across three groups, not a single control figure. Also added: “Email-only notifications have no observable impact compared to a control group”, which is the more important half of that paper for this page.

Discovering the first of these prompted a new subsection on the content page: phone calls and hand-read imprints are the channels with the highest measured rates in this literature, and the page's earlier flat statement that telephone “does not scale” was true only at [12Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)]'s N of 44,790. That claim is now qualified rather than repeated.

One internal inconsistency, not resolved: [12Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] states a 2.4% control fix rate for client-side XSS in prose and 2.2% (4/185) in Table 1. The page quotes the table.

One inconsistency reported by a sub-agent and excluded: [15Stöver, Alina; Gerber, Nina; Pridöhl, Henning; Maass, Max; Bretthauer, Sebastian; Spiecker gen. Döhmann, Indra; Hollick, Matthias; Herrmann, Dominik (2023): "How Website Owners Face Privacy Issues: Thematic Analysis of Responses from a Covert Notification Study Reveals Diverse Circumstances and Challenges", in: Proceedings on Privacy Enhancing Technologies. (DOI)] Figure 1 says 1,495 owner responses while the body text says 1,043 (1,043 + 452 bounces/auto-replies = 1,495). Neither number is on the content page.

[1Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)], NDSS 2018, “Didn't You Hear Me? – Towards More Successful Web Vulnerability Notifications”. data/fulltext/2018/NDSS/ does not exist — NDSS 2016 and NDSS 2018 full text was not retrieved at all, while both years are present in data/corpus2/.meta. This is the canonical follow-up to the two 2016 campaigns, so it was fetched from the publisher (ndss-symposium.org PDF from the index's pdfUrl) and extracted with pdfminer.six (pdftotext is not installed in this workdir). Figures taken from it, with their locations in that extraction: population >24,000 domains in seven arms of ~4,000; fix rates 24% (Git) / 17% (WordPress) notified vs 13% / 14% control; six email variants with Fisher's exact + Holm–Bonferroni (m=6, α=0.05), all Git arms significant, only the Mailbot arm significant for WordPress, and “no group performed significantly better than all others”; bounce-all rates 1,085/8,160 (13.3%) Git and 1,906/12,442 (15.3%) WordPress; 74.42% / 33.33% fix rate among those who viewed the report; 24.7% among Git operators who opened the mail but never viewed the report; first reminder worth ~7 pp at two weeks and ~11 pp at four weeks, second reminder worth nothing. This is the one page figure not reproducible from the corpus mount.

A Sonnet sub-agent ran the currency pass; every load-bearing item was then re-fetched by the orchestrator, because the sub-agent hit HTTP 403 on several primary sources and fell back to search-engine caches. What follows separates the two.

• CERT/CC: 45-day default embargo — certcc.github.io, quoted verbatim, HTTP 200 on the sub-agent's fetch.
• Google Project Zero: 90+30 days, 14-day grace, 7 days for actively-exploited bugs, plus the Reporting Transparency trial from 29 July 2025 — projectzero.google. Also caught that googleprojectzero.blogspot.com now 301-redirects to projectzero.google, so the blogspot URL is stale; the page says so.
• Shadowserver: still free, 201 national CSIRTs, 175 countries — shadowserver.org.
• RFC 2142 still current, unrevised — rfc-editor.org / datatracker.
• RIPE validates abuse-mailbox at least annually; ARIN validates Abuse POCs annually — docs.db.ripe.net, arin.net NRPM.
• Abusix Abuse Contact DB still free and not rate-limited — docs.abusix.com. Relevant because [12Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] used it in 2016.
• Venue calls: IMC 2026 (direct), IEEE S&P 2026 (direct), CCS 2026 (direct), PoPETs 2026 (direct — no vulnerability-notification clause found, which is what the page says).

• USENIX Security 2026 ethics wording. usenix.org returns 403 to automated fetches. The sub-agent read it from a search-engine cache. The page carries a footnote saying exactly this and telling the reader to check the live call. This is the only external claim on the page resting on a cache.
• ISO/IEC 29147:2018 and ISO/IEC 30111:2019 review status. iso.org returns 403 to both the sub-agent and the orchestrator. The page names the two standards and what they cover but makes no claim about edition, review status or a pending revision — that was cut rather than published on a search snippet.
• CISA's Coordinated Vulnerability Disclosure process. cisa.gov 403s. The sub-agent reported new joint CISA/NSA CVD guidance dated 15 July 2026 from secondary sources. Cut from the page entirely rather than published unverified. Worth adding once someone can fetch it.
• CERT/CC Vultron. Reported as explicitly “not yet ready for production use”. Cut, because a non-production protocol is not a channel a PhD student can use this year.
• Bugcrowd / Intigriti / YesWeHack third-party report acceptance. No primary policy page found either way. The page therefore claims the 2016 “reward programmes only serve their customers” premise no longer holds for HackerOne specifically, and says nothing about the others.

The sub-agent's first pass surfaced these; all were rejected as SEO, marketing or reseller content rather than primary, and none reached the page: whois.is blog, ipgeolocation.io, viewdns.info, myiphelp.com, decryptiondigest.com, secureframe.com, plaxidityx.com, standards.iteh.ai ISO reseller mirrors, dokumen.pub, secportal.io, craevidence.com, 0xlanks.me, uriports.com and iotdef.com security.txt adoption-statistics blogs, claimdepot.com, and the WHOIS-marketing explainers at abion.com, ionos.com, dynadot.com and safenames.net. Two — dreyfus.fr and cadeproject.org — were used only to corroborate the ICANN Registration Data Policy date while the ICANN page was 403ing, and became unnecessary once the orchestrator fetched ICANN directly.

notes/find_contacts.py on the page is the real file, and the code block under it is its real, unedited output from a run on 2026-08-13 against google.com ethz.ch cispa.de bbc.co.uk wikipedia.org.

Three things about that output are worth recording rather than tidying away:

• The same script on the same five domains gave 80% RIR-abuse coverage in one run and 100% in another, purely because stat.ripe.net timed out once for google.com. The published block is the 100% run (it matches the current code); the 80% run is recorded here, and the content page states the discrepancy explicitly rather than presenting either number as a coverage measurement. Five domains and a transient — this is why the page says the sample is hand-picked and not representative.
• .ch and .de having no RDAP service in the IANA bootstrap is a real finding, not a script bug — confirmed by reading dns.json directly.
• A reviewer found a real bug and it was fixed. _rdap_base() fetched the IANA bootstrap registry with no error handling, so a single transient failure there raised an uncaught URLError and killed the whole batch — contradicting the script's own promise that failures are printed rather than swallowed. It now raises a RuntimeError naming the URL and the reason, and refuses to continue, on the grounds that the alternative (a column of dashes) would silently understate RDAP coverage for every domain at once. Verified both paths: the happy path prints the table above, and pointing BOOTSTRAP at an unreachable host raises with full context. This is the second defect on this page that only surfaced because someone re-ran the code rather than reading it.

Design decisions in that script, all of them arguable:

• Domain WHOIS omitted on the currency grounds above. A 2016-vintage version of this script would have made WHOIS the primary source.
• RIPEstat rather than an Abusix DNS TXT lookup for abuse contacts, because this workdir has neither dig nor dnspython and code that cannot be executed cannot be published as tested. Abusix is the cheaper choice at scale and is what [12Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)] used; RIPEstat is the choice that runs with nothing but the Python standard library. The script's docstring does not say this; the page does not either. It should.
• mailto: scraping omitted, though it is the highest-yield source. Documented in the docstring with the reason.
• The five test domains are hand-picked, not sampled, and the page says so and gives the contrasting corpus figure, because 80% security.txt coverage against [6Poteat, Tara; Li, Frank (2021): "Who You Gonna Call? An Empirical Evaluation of Website security.txt Deployment", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]'s 3–4% at the top 10K is the kind of number that gets quoted out of context.

21 new entries added to Bibliography before its closing </bibtex>. Generated with node scripts/bibgen.mjs, i.e. from data/corpus2/.meta publisher metadata rather than from recall. Checked before saving: 23 distinct citation keys used on the content page, 0 unresolved, 0 colliding with the 239 keys already in the file.

Two entries already existed and were reused rather than duplicated: maass2021_effective and hantke2024_redlines.

Author lists that the index does not carry — 100% of PETS and USENIX records have no authors and no DOI, they are scraped from venue pages — were fetched from the venue pages with curl and a browser User-Agent (WebFetch gets 403 from usenix.org): stock2016_hey, li2016_youve, nguyen2021_sharefirst, squarcina2021_subdomain, munteanu2025_catch22, utz2023_comparing, stoever2023_owners. Keys generated by bibgen with a guessed first author were renamed to the real one after fetching: findlay2021_securitytxt → poteat2021_securitytxt, curtain2026_behind → stivala2026_behind, keys2025_file → elyadmani2025_keys, tajalizadehkhoob2021_whowas → lu2021_whowas, hlavacek2024_rov → qin2024_rov, elazari2017_legalrisks → gamerogarrido2017_legalrisks, mirian2022_ics → sasaki2022_ics, tatang2022_spfail → bennett2022_spfail. bibgen also emitted the slug in place of the title for cetin2019_cleaning; fixed by hand.

stock2018_didnt was hand-written from the NDSS index record plus the DOI printed in the PDF (10.14722/ndss.2018.23171), since bibgen cannot reach a venue-year with no full text.

• Whether the 22 hand-classified campaigns are the whole set. They are not; 144 candidates were never read. See the TODO above.
• Any response rate from the corpus schema. ethics.disclosureDetail carries a countable ratio in 7 of 2,870 cases. Every rate on the content page came from a paper's results section, read by hand. A schema field for “notification outcome, with denominator” would make this page regenerable; there isn't one.
• Whether security.txt contacts actually work. [6Poteat, Tara; Li, Frank (2021): "Who You Gonna Call? An Empirical Evaluation of Website security.txt Deployment", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] measured adoption and field validity but never sent anything, and explicitly leaves that to future work. No paper in this corpus tests delivery through security.txt. This is the most obvious open question on the page and it is five years old.
• Current security.txt adoption. The only corpus figure is from a 2021 paper on a list (Alexa) that was discontinued in 2022. Nothing in the 2022–2026 slice re-measures it. The page therefore gives a 2021 figure and dates it; a reader in 2026 should assume it has risen and has no published number to cite.
• Whether the framing effect is real. [2Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)] found a large, significant fine-framing effect; [3Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)] found none. Different countries, issues, media and populations. The page presents the contradiction rather than picking a side. Resolving it needs a study that varies framing while holding medium and jurisdiction fixed.
• Whether the RCT result generalises. [9Lone, Qasim; Frik, Alisa; Luckie, Matthew; Korczyński, Maciej; van Eeten, Michel; Gañán, Carlos (2022): "Deployment of Source Address Validation by Network Operators: A Randomized Control Trial", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] found no effect on network operators deploying SAV — an expensive, low-visibility configuration change. It does not follow that notification fails for a web vulnerability an operator can fix in ten minutes, and [2Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)]'s 56.6%-vs-9.2% with a proper control says it does not. The page says the RCT undermines control-free studies, not that notification does not work. A reasonable person could read the page as more pessimistic than the evidence supports; this was the hardest calibration call on it.
• Non-EU/US practice. Every campaign in the table notified operators in Europe, North America or globally, and framing effects rest on German Impressum law. There is nothing here about notifying operators in jurisdictions with no equivalent legal hook.
• Venues. EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent from the corpus. CHI/SOUPS is where much of the operator-facing usable-security work appears, so the page's account of what website owners think and need is thinner than the literature.

Four reviewers, all told the context might not be exhaustive, all given the page text, the report script, its output, and these notes. Three focused passes ran in parallel first; the generic pass ran after their findings were applied.

• A table rendered with every figure in the wrong column, and no reviewer caught it. The trend table's first row labelled its indicator with a backslash-escaped pipe — | Notified (''yes''\|''partial'') | — but a backslash is not a DokuWiki escape for a literal pipe. The renderer treated it as a cell separator, splitting the label in two and shifting all six percentages one column left, so the 2010–2013 figure appeared under Denominator and the last bucket rendered empty. Caught by parsing the live page's HTML, where that row came back as ['Notified (yes\', 'partial)', 'empirical ∧ ethics', '21.3%', …] against a seven-column header. Fixed by writing the label as “yes or partial” and avoiding the pipe entirely. The figures reviewer checked every number against the script and passed the page; every number was right and the table was still wrong. Checking a wiki page means parsing the rendered HTML, not only the source — the table-cell counts of both pages are now asserted from the live HTML.
• The downloadable code was verified end to end. curl on /_export/code/practices/notifying_websites?codeblock=0 → python3 -m py_compile → an actual run against ethz.ch. What a reader downloads from the page compiles and works. Worth doing because the page's <file> block is a hand-transcribed copy with em-dashes substituted, and a substitution inside a string literal would have broken it silently.
• All eight internal links (Artifacts, Design:Website selection, Literature:Corpus, Practices:Ethics, Practices:Legal enforcement, Practices:Public relations, Statistics:Hypothesis testing, Statistics:Pvalue corrections) fetched 200 with real content — no red links. ?do=sitemap is cached and still listed the new page as missing, as expected.

The crawled ∧ assessed a law cross-tab (N=123) was not a reviewer finding — it was added afterwards because none of the existing rows described the reader the page is written for, someone who crawled the web and found a legal-compliance violation. It is the one cell where no (14.6%) is as common as partial, more than three times the corpus-wide 4.3%. The script row and the page row were added together and the pasted output above was regenerated. This means the reviewers did not see this figure; it is the newest and least-checked number on the page, though it comes from the same enum and the same script as the rows above it.

Every substantive finding across the three passes was accepted. Two are worth separating from the rest because they say something about the process rather than the page:

• The figures pass found no figure error at all — every corpus number checked out — but found the one real code defect. Re-running beats re-reading.
• The quotes pass independently reproduced the [13Sasaki, Takayuki; Fujita, Akira; Gañán, Carlos Hernandez; van Eeten, Michel; Yoshioka, Katsunari; Matsumoto, Tsutomu (2022): "Exposed Infrastructures: Discovery, Attacks and Remediation of Insecure ICS Remote Management Devices", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] diagnosis the orchestrator had already made, and caught an over-correction on [14Cetin, Orcun; Gañán, Carlos; Altena, Lisette; Kasama, Takahiro; Inoue, Daisuke; Tamiya, Kazuki; Tie, Ying; Yoshioka, Katsunari; van Eeten, Michel (2019): "Cleaning Up the Internet of Evil Things: Real-World Evidence on ISP and Consumer Efforts to Remove Mirai", in: Proceedings of the Network and Distributed System Security Symposium. (Link)] that the orchestrator's own fix had introduced. A correction is as capable of being wrong as the thing it corrects.

[1]

Stock, Ben; Pellegrino, Giancarlo; Li, Frank; Backes, Michael; Rossow, Christian (2018): "Didn't You Hear Me? - Towards More Successful Web Vulnerability Notifications", in: Proceedings of the Network and Distributed System Security Symposium. (DOI)

[2]

Maass, Max; Stöver, Alina; Pridöhl, Henning; Bretthauer, Sebastian; Herrmann, Dominik; Hollick, Matthias; Spiecker, Indra (2021): "Effective Notification Campaigns on the Web: A Matter of Trust, Framing, and Support", in: Proceedings of the USENIX Security Symposium. (Link)

[3]

Utz, Christine; Michels, Matthias; Degeling, Martin; Marnau, Ninja; Stock, Ben (2023): "Comparing Large-Scale Privacy and Security Notifications", in: Proceedings on Privacy Enhancing Technologies. (DOI)

[4]

El Yadmani, Soufian; Gadyatskaya, Olga; Zhauniarovich, Yury (2025): "The File That Contained the Keys Has Been Removed: An Empirical Analysis of Secret Leaks in Cloud Buckets and Responsible Disclosure Outcomes", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[5]

Bennett, Nathaniel; Sowards, Rebekah; Deccio, Casey T. (2022): "SPFail: Discovering, Measuring, and Remediating Vulnerabilities in Email Sender Validation", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[6]

Poteat, Tara; Li, Frank (2021): "Who You Gonna Call? An Empirical Evaluation of Website security.txt Deployment", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[7]

Stivala, Giada; Mrowczynski, Rafael; Hellenthal, Maria; Pellegrino, Giancarlo (2026): "Behind the Curtain: How Shared Hosting Providers Respond to Vulnerability Notifications", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[8]

Li, Frank; Durumeric, Zakir; Czyz, Jakub; Karami, Mohammad; Bailey, Michael; McCoy, Damon; Savage, Stefan; Paxson, Vern (2016): "You've Got Vulnerability: Exploring Effective Vulnerability Notifications", in: Proceedings of the USENIX Security Symposium. (Link)

[9]

Lone, Qasim; Frik, Alisa; Luckie, Matthew; Korczyński, Maciej; van Eeten, Michel; Gañán, Carlos (2022): "Deployment of Source Address Validation by Network Operators: A Randomized Control Trial", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[10]

Lu, Chaoyi; Liu, Baojun; Zhang, Yiming; Li, Zhou; Zhang, Fenglu; Duan, Haixin; Liu, Ying; Chen, Joann Qiongna; Liang, Jinjin; Zhang, Zaifeng; Hao, Shuang; Yang, Min (2021): "From WHOIS to WHOWAS: A Large-Scale Measurement Study of Domain Registration Privacy under the GDPR", in: Proceedings of the Network and Distributed System Security Symposium. (Link)

[11]

Canali, Davide; Balzarotti, Davide; Francillon, Aurélien (2013): "The Role of Web Hosting Providers in Detecting Compromised Websites", in: Proceedings of the ACM Web Conference. (DOI)

[12]

Stock, Ben; Pellegrino, Giancarlo; Rossow, Christian; Johns, Martin; Backes, Michael (2016): "Hey, You Have a Problem: On the Feasibility of Large-Scale Web Vulnerability Notification", in: Proceedings of the USENIX Security Symposium. (Link)

[13]

Sasaki, Takayuki; Fujita, Akira; Gañán, Carlos Hernandez; van Eeten, Michel; Yoshioka, Katsunari; Matsumoto, Tsutomu (2022): "Exposed Infrastructures: Discovery, Attacks and Remediation of Insecure ICS Remote Management Devices", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[14]

Cetin, Orcun; Gañán, Carlos; Altena, Lisette; Kasama, Takahiro; Inoue, Daisuke; Tamiya, Kazuki; Tie, Ying; Yoshioka, Katsunari; van Eeten, Michel (2019): "Cleaning Up the Internet of Evil Things: Real-World Evidence on ISP and Consumer Efforts to Remove Mirai", in: Proceedings of the Network and Distributed System Security Symposium. (Link)

[15]

Stöver, Alina; Gerber, Nina; Pridöhl, Henning; Maass, Max; Bretthauer, Sebastian; Spiecker gen. Döhmann, Indra; Hollick, Matthias; Herrmann, Dominik (2023): "How Website Owners Face Privacy Issues: Thematic Analysis of Responses from a Covert Notification Study Reveals Diverse Circumstances and Challenges", in: Proceedings on Privacy Enhancing Technologies. (DOI)

← back to the content page · corpus-level provenance