The working log behind email_authentication: every query with its denominator, the hand maps, the folds and their residue, the quotes that were checked and the one that turned out not to be a quote, the external sources that were accepted and the ones that were rejected, and what the four reviewers found. Corpus-level caveats — how the 5,859 papers were selected, why 2025–2026 are provisional — are on corpus and are not restated here.

This is a log, not prose. It is for someone checking a number.

The 31 papers were not derived by a query written for this page. They are the INFRA verdict of scripts/msg_fold.mjs, the hand map published on 2026-09-02 with email_tracking and printed by scripts/report_email_tracking.mjs section H. That map screened a 354-paper candidate pool, built by unioning a title-slug regex with a full-text density threshold over message-channel vocabulary, and gave every paper in the pool one of eleven verdicts. INFRA is the one whose object is the machinery that carries the message rather than the message.

Reusing it rather than writing a new query is a deliberate choice with a cost and a benefit:

• Benefit. The population is already published, already hand-read, already carries a one-line reason per paper, and the count already appears in two places on privacy:email_tracking. Re-deriving it would have produced a second, slightly different 31 and left the two pages disagreeing.
• Cost. The candidate pool that produced it was built for a page about message tracking. Its recall for email-infrastructure work is therefore a property of somebody else's regex. Section 3 is the recall check that closes this.

scripts/report_email_authentication.mjs imports the map and throws if the INFRA slice is no longer 31 papers, and again if any of its own hand maps stops covering exactly those 31 slugs:

if (INFRA.length !== 31)
  throw new Error(`msg_fold INFRA slice is ${INFRA.length}, page was written against 31`);
if (PAPERS.length !== 31)
  throw new Error(`only ${PAPERS.length} of 31 INFRA slugs found in extractions.jsonl`);
...
assertComplete('KEYS', KEYS);
assertComplete('MECH', MECH);
assertComplete('METHOD', METHOD);

assertComplete reports both directions — a slug in the population with no map entry, and a map entry for a slug not in the population. Both were exercised during development by deleting entries; both fire.

31 of 5,859 is not a rate, and the page says so. The only honest denominator for it is the 354-paper candidate pool, which belongs to the other page.

A full-text probe over all 5,859 papers (paper.cols.txt, whitespace collapsed; 4 papers have no .cols file and count as negatives), counting occurrences of each mechanism family:

const PATS = {
  spf: /\bSPF\b|Sender Policy Framework/g,   dkim: /\bDKIM\b|DomainKeys/g,
  dmarc: /\bDMARC\b/g,                       starttls: /STARTTLS|\bSTART ?TLS\b/g,
  dane: /\bTLSA\b|\bDANE\b/g,                mtasts: /MTA-STS|TLS-RPT|TLSRPT/g,
  smime: /S\/MIME|SMIMEA|OPENPGPKEY/g,       smtp: /\bSMTP\b|\bMTA\b|\bMX record/g,
  bimi: /\bBIMI\b|Brand Indicators/g,        arc: /Authenticated Received Chain|\bARC (chain|set|seal)/g,
};
// mech = spf+dkim+dmarc+dane+mtasts+bimi+arc ; tls = starttls+smime

Widening the probe added 32 unmapped papers and none of them belongs on the page. Read by hand on 2026-09-09, they are DANE-for-HTTPS (When HTTPS Meets CDN, IEEE S&P 2014, 60 DANE/TLSA hits and not one about mail), DNSSEC registrar and resolver work, ALPACA, DROWN, Let's Encrypt, a DNS toolkit paper, PDF encryption, and two software-engineering interview studies where S/MIME appears in a quotation. The full list is in section H of the report output.

Two things this establishes and one it does not:

1. The hand map is not missing a cluster. A regex twice as permissive as the one that would sensibly be written for this topic finds nothing new.
2. The probe cannot replace the hand map. The TIGHT probe misses one of the 31 — IMC/2017/email-typosquatting, whose subject is misdirected mail and which names no authentication mechanism at all. A page defined by the regex would have dropped it.
3. It does not establish that the seven venues are the field. EuroS&P, ACSAC, RAID, AsiaCCS, TMA, PAM, ANRW and ACM CCR are outside the corpus entirely, and this literature publishes in several of them.

All counts are of papers, never of extraction tuples. Sentinels (not-stated, none-mentioned) are never counted as an answer. Reproduce with node scripts/report_email_authentication.mjs.

The full per-paper lists behind Q3, Q4 and Q16 are printed in the report output; they are not reproduced here because they are the same 31 slugs eleven times over.

53 figures, each hand-keyed in the FIGURES array of the report script with its population and its evidence quote, and each printed by report section E. They are not repeated here — section E of the output embedded in section 8 below is the authoritative list, and it is generated, so it cannot drift from the script. Read it rather than this page if you are checking a number.

Three figures on the page are derived rather than quoted, and are marked here because a derived percentage is where an inferred denominator hides:

• “4,167,633 domains (41.7%)” — the paper states the count and states the population as the top ten million; the percentage is mine, 4,167,633 / 10,000,000.
• “882,183 domains (8.8%)” — same paper, same arithmetic.
• MTA-STS “0.07%–0.13% by TLD” — the paper's own Table 1 gives the per-TLD percentages and the header of the column they are shares of reads “Domains with MX Records”. The four zone totals in that table sum to 86.8 million; the page does not publish 68,030 / 86.8M as a combined rate, because the paper does not, and because the prose and the table disagree with each other on .org (0.12% in the prose, 0.13% in Table 1). The page quotes the range, not a combined figure.

Two of the page's tables are hand maps, not extraction fields, because the schema has no field for them. Both live in the report script, both are asserted complete against the 31, and both are my judgement.

• MECH — which deployed mechanism the paper measures, read off its own results section. Not which it mentions: every one of the 31 mentions SPF. A paper whose object is where mail goes rather than any single mechanism gets delivery-path.
• METHOD — the instrument families, defined in the METHODS dictionary in the script so the definition travels with the count. A paper gets every family it actually ran, which is why the mean is 2.7.

Judgement calls inside them worth naming:

• email-typosquatting gets delivery-path and honey-domain. It measures neither an authentication mechanism nor a transport-security mechanism; it measures what happens to mail that is addressed wrongly. It is in the population because msg_fold put it there and because misdirection is a delivery-path property.
• heads-in-the-clouds gets delivery-path and dns-scan only. Its subject is university cloud migration generally; MX-record concentration is one section of it. It is the weakest fit of the 31 and I would not have admitted it from scratch.
• s-mine gets smtp-probe for its internet-wide LDAP scan. LDAP is not SMTP; the family is “actively connect to servers and drive a protocol” and there is no better bucket. Flagged rather than renamed.
• DNSSEC is listed as a measured mechanism for three papers because DANE cannot be measured without it, not because those papers are DNSSEC studies.

Two free-text fields were folded. Both residues are printed by the script and are reproduced here, because a residue that exists only in a local script output is a residue nobody will ever look at.

Rule (report section D):

[/alexa/i, 'Alexa (retired Nov 2022)'],
[/tranco/i, 'Tranco'],
[/majestic|umbrella|domcop|quantcast/i, 'another top list (Majestic / Umbrella / DomCop)'],
[/zone file|verisign|internetstiftelsen|public interest registry|czds/i, 'TLD zone files'],
[/passive dns|farsight|sie|dnsdb|openintel/i, 'passive DNS / OpenINTEL'],
[/adobe/i, 'Adobe 2013 breach address list'],
[/enron/i, 'Enron corpus'],
[/gmail|coremail|smtp connection log|reception log|delivery log/i, 'operator logs'],
[/university|times higher|the top100|institution/i, 'university lists'],
[/custom (seed|client|list)|selected public|public software|specialized mailing|mailing list|nanog|mailop/i, 'hand-built list'],
[/ipv4|address space|scans\.io|censys|zmap/i, 'IPv4 address space'],

43 distinct strings did not fold. They are almost all one-off study-specific populations rather than reusable frames, which is the finding: outside the top-list/zone-file/passive-DNS trio, everybody builds their own. The full residue is in report section D and is reproduced verbatim in the embedded output below. A sample of what is in it: “48 universities”, “DomCop Top 10M list”, “dmarcian's DMARC Data Reporters”, “emailclientmarketshare.com”, “mailsac.com”, “public LDAP servers”, “reachable mail servers identified in setup guides”, “shared infrastructure settings: cloud servers, proxy services, serverless functions, CI/CD platforms, and CDN services”, “Tor exit list”, “VPNGate”.

One residue entry is a sentinel and should be read as one, not as a frame: “not-stated” for USENIX/2021/why-tls-is-better-without-starttls.

One is a pointer rather than a source and would be wrong to fold into anything: “based on [56]”, “previous work [44]” and “prior research [40]” (all USENIX/2025/email-spoofing-with-smtp-smuggling). The extractor recorded the citation instead of the list. That is an extraction limitation, not a study one.

Denominator: the 26 of the 31 papers that name at least one tool as used. The fold merges implementations of the same thing (libspf2/pyspf/Mail::SPF/policyd-spf/spfmilter/python-postfix-policyd-spf → “an SPF validator library”; GnuPG/gpg/GPGME/Enigmail/GPG Suite/OpenKeychain/OpenPGP.js → “a PGP implementation”). The unmapped residue is large — roughly a hundred distinct strings across the 31 papers — and is printed in full in report section I. It is dominated by single-use infrastructure (QEMU, GNU Parallel, mitmproxy, Qualtrics, jc, hashcat, Google Translate) and by named commercial services (Netstar URL Categorization, Coremail spam filter, GoDaddy Domains API, HaveIBeenPwned).

The fold is not cosmetic. Without it, OpenSSL and openssl are two tools, ZMap, Zmap and XMap are three, and every one of them falls below any reporting threshold.

Every figure the page publishes carries an evidence.quote in the FIGURES array, and the script locates each one in paper.cols.txt in three modes — exact after whitespace collapse, whitespace-and-hyphen folded, punctuation folded. Four figures needed two fragments, giving 57 fragments in total; three of those second fragments exist because the source sentence is not contiguous in any available rendering, and the fourth is a corroborating quotation rather than a splice.

Final state: 55 exact, 2 punctuation-folded, 0 unlocatable. The script prints *** N QUOTE(S) COULD NOT BE LOCATED *** and the page's quotes were rewritten until that line disappeared.

That is a selection effect and it should be stated as one. Reaching zero meant changing the quote, never the figure. Ten quotes were rewritten in the first pass and one in the last; in every case the number and the claim stayed and only the string moved, usually by shortening to the contiguous part of the same sentence. The full before/after list:

No figure was dropped for being unquotable, and no figure's value changed to make a quote fit.

paper.cols.txt repairs two-column reading order, but it cannot repair a sentence that the PDF itself interrupts with a table row, a figure caption or a running head. Three of the page's figures sit on such a sentence and carry a spliced annotation in the script. A fourth figure carries a second quote fragment that is an independent corroborating quotation rather than a split sentence, and a fifth spliced sentence – the DANE rollover figure – is stated in the page's prose rather than in a figure row, so three is a floor rather than a total. I checked each against paper.txt and paper.norm.txt as well; none of the three renderings contains the sentence contiguously, so this is the PDF's layout, not the extraction pipeline's fault.

So: three figures carry a spliced annotation, one carries a second fragment for a different reason, two were shortened to a contiguous fragment rather than split, and at least one prose figure outside the tracked array is spliced too.

Consequence for the page: it does not present any of these five as one continuous quotation. It states the figure and leaves the split-quote evidence here.

As a check on the extraction rather than on the page: all 206 detection[].evidence.quote values across the 31 papers were located.

36.9% is far above the 0.9% unlocatable rate data/extract/README.md reports corpus-wide, and that difference needs explaining rather than reporting. Of the 76: 11 contain an explicit ellipsis inserted by the extractor and were never expected to match. The remaining 65 were sampled by hand on 2026-09-09 — eight of them read in context in paper.cols.txt:

So: 7 of 8 sampled are real sentences broken by page furniture; 1 of 8 is a genuine extractor paraphrase that preserves the figure. A .cols-only quote check on a two-column mail-measurement paper produces false failures at a rate that would be alarming if you did not look at them, and one true failure that a passing check would have hidden. Both directions are worth knowing.

Committed as scripts/report_email_authentication.mjs. Re-runnable against the current data/extract/run1; every figure on the content page comes from its output, and it throws rather than silently continuing if the population or any hand map moves.

report_email_authentication.mjs

// Report script for `security:email_authentication`.
//
// Every figure on that page is printed here with its own denominator, and
// every quote the page puts in quotation marks is verified against
// data/fulltext/<year>/<venue>/<slug>/paper.cols.txt before it is printed.
//
// Population rule: the page's 31 papers are NOT re-derived here. They are the
// `INFRA` verdict of scripts/msg_fold.mjs — the hand map published with
// `privacy:email_tracking` (2026-09-02), which screened a 354-paper candidate
// pool and separated "the object of measurement is a message" from "the object
// of measurement is the machinery that carries the message". This script
// imports that map and asserts that the INFRA slice is still 31 papers, so a
// change to msg_fold.mjs breaks this report instead of silently moving the
// page's denominator.
//
// Everything this script adds on top of that population is hand-keyed below
// (MECH, METHOD, FIGURES) and checked for completeness against the 31: a slug
// missing from any map throws.
//
// Usage: node scripts/report_email_authentication.mjs
//        node scripts/report_email_authentication.mjs --format wiki   (tables only)
 
import fs from 'node:fs';
import path from 'node:path';
import { loadExtractions, dataRoot } from './lib.mjs';
import { MAP as MSG_MAP } from './msg_fold.mjs';
 
const ROOT = dataRoot();
const H = (s) => console.log('\n' + '='.repeat(78) + '\n' + s + '\n' + '='.repeat(78));
 
// ---------------------------------------------------------------------------
// A. Population
// ---------------------------------------------------------------------------
const INFRA = Object.entries(MSG_MAP)
  .filter(([, v]) => v[0] === 'INFRA')
  .map(([slug, v]) => ({ slug, reason: v[1] }));
const INFRA_SLUGS = new Set(INFRA.map((x) => x.slug));
 
const ALL = loadExtractions();
const PAPERS = ALL.filter((p) => INFRA_SLUGS.has(p.slug)).sort(
  (a, b) => a.year - b.year || a.venue.localeCompare(b.venue)
);
const bySlug = new Map(PAPERS.map((p) => [p.slug, p]));
 
if (INFRA.length !== 31)
  throw new Error(`msg_fold INFRA slice is ${INFRA.length}, page was written against 31`);
if (PAPERS.length !== 31)
  throw new Error(`only ${PAPERS.length} of 31 INFRA slugs found in extractions.jsonl`);
 
const key = (p) => `${p.venue}/${p.year}/${p.slug}`;
 
// ---------------------------------------------------------------------------
// B. Hand maps. A slug missing from either map throws (see assertComplete).
// ---------------------------------------------------------------------------
 
// ---------------------------------------------------------------------------
// B0. Citekey per paper. Hand-keyed so the page's reference table is generated
//     rather than retyped; assertComplete below throws if a paper has no key.
// ---------------------------------------------------------------------------
const KEYS={
 'security-by-any-other-name-on-the-effectiveness-of-provider-based-email-security':'foster2015_security',
 'neither-snow-nor-rain-nor-mitm-an-empirical-analysis-of-email-delivery-security':'durumeric2015_neither',
 'email-typosquatting':'szurdi2017_email',
 'efail-breaking-s-mime-and-openpgp-email-encryption-using-exfiltration-channels':'poddebniak2018_efail',
 'end-to-end-measurements-of-email-spoofing-attacks':'hu2018_measurements',
 'johnny-you-are-fired-spoofing-openpgp-and-s-mime-signatures-in-emails':'mueller2019_johnny',
 'a-longitudinal-and-comprehensive-study-of-the-dane-ecosystem-in-email':'lee2020_longitudinal',
 'composition-kills-a-case-study-of-email-sender-authentication':'chen2020_composition',
 'weak-links-in-authentication-chains-a-large-scale-analysis-of-email-sender-spoof':'shen2021_weak',
 'why-tls-is-better-without-starttls-a-security-analysis-of-starttls-in-the-email':'poddebniak2021_better',
 'spfail-discovering-measuring-and-remediating-vulnerabilities-in-email-sender-val':'bennett2022_spfail',
 'a-large-scale-and-longitudinal-measurement-study-of-dkim-deployment':'wang2022_longitudinal',
 'under-the-hood-of-dane-mismanagement-in-smtp':'lee2022_under',
 'revisiting-email-forwarding-security-under-the-authenticated-received-chain-prot':'wang2022_revisiting',
 '27-years-and-81-million-opportunities-later-investigating-the-use-of-email-encry':'stransky2022_years',
 'heads-in-the-clouds-measuring-universities-migration-to-public-clouds-implicatio':'fiebig2023_heads',
 'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild':'czybik2023_lazy',
 'youve-got-report-measurement-and-security-implications-of-dmarc-reporting':'ashiq2023_report',
 'content-type-multipart-oracle-tapping-into-format-oracles-in-email-end-to-end-en':'ising2023_content',
 'extended-hell-o-a-comprehensive-large-scale-study-on-email-confidentiality-and-i':'blechschmidt2023_hello',
 'bounce-in-the-wild-a-deep-dive-into-email-delivery-failures-from-a-large-email-s':'li2024_bounce',
 'breakspf-how-shared-infrastructures-magnify-spf-vulnerabilities-across-the-internet':'wang2024_breakspf',
 'fakebehalf-imperceptible-email-spoofing-attacks-against-the-delegation-mechanism':'ma2024_fakebehalf',
 'spf-beyond-the-standard-management-and-operational-challenges-in-practice-and-pr':'ashiq2024_beyond',
 'unraveling-the-complexities-of-mta-sts-deployment-and-management-in-securing-ema':'ashiq2025_unraveling',
 'email-spoofing-with-smtp-smuggling-how-the-shared-email-infrastructures-magnify':'wang2025_email',
 'a-multifaceted-study-on-the-use-of-tls-and-auto-detect-in-email-ecosystems':'tang2025_multifaceted',
 's-mine-collecting-and-analyzing-s-mime-certificates-at-scale':'oendaroe2025_mine',
 'understanding-and-characterizing-intermediate-paths-of-email-delivery-the-hidden':'li2025_characterizing',
 'automatic-insecurity-exploring-email-auto-configuration-in-the-wild':'wen2025_automatic',
 'coordmail-exploiting-smtp-timeout-and-command-interaction-to-coordinate-email-middleware-for-convergence-amplification-attack':'li2026_coordmail',
};
 
// Which deployed mechanism(s) the paper MEASURES — not which it mentions.
// Read off the paper's own results section. `delivery-path` means the object is
// where mail goes and who carries it, with no single mechanism as the subject.
const MECH = {
  'security-by-any-other-name-on-the-effectiveness-of-provider-based-email-security': ['STARTTLS', 'SPF', 'DKIM', 'DMARC', 'DNSSEC'],
  'neither-snow-nor-rain-nor-mitm-an-empirical-analysis-of-email-delivery-security': ['STARTTLS', 'SPF', 'DKIM', 'DMARC'],
  'email-typosquatting': ['delivery-path'],
  'efail-breaking-s-mime-and-openpgp-email-encryption-using-exfiltration-channels': ['S/MIME+OpenPGP'],
  'end-to-end-measurements-of-email-spoofing-attacks': ['SPF', 'DKIM', 'DMARC'],
  'johnny-you-are-fired-spoofing-openpgp-and-s-mime-signatures-in-emails': ['S/MIME+OpenPGP'],
  'a-longitudinal-and-comprehensive-study-of-the-dane-ecosystem-in-email': ['DANE', 'STARTTLS', 'DNSSEC'],
  'composition-kills-a-case-study-of-email-sender-authentication': ['SPF', 'DKIM', 'DMARC'],
  'weak-links-in-authentication-chains-a-large-scale-analysis-of-email-sender-spoof': ['SPF', 'DKIM', 'DMARC'],
  'why-tls-is-better-without-starttls-a-security-analysis-of-starttls-in-the-email': ['STARTTLS'],
  'spfail-discovering-measuring-and-remediating-vulnerabilities-in-email-sender-val': ['SPF'],
  'a-large-scale-and-longitudinal-measurement-study-of-dkim-deployment': ['DKIM'],
  'under-the-hood-of-dane-mismanagement-in-smtp': ['DANE', 'STARTTLS', 'DNSSEC'],
  'revisiting-email-forwarding-security-under-the-authenticated-received-chain-prot': ['ARC', 'SPF', 'DKIM', 'DMARC'],
  '27-years-and-81-million-opportunities-later-investigating-the-use-of-email-encry': ['S/MIME+OpenPGP'],
  'heads-in-the-clouds-measuring-universities-migration-to-public-clouds-implicatio': ['delivery-path'],
  'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild': ['SPF', 'DMARC'],
  'youve-got-report-measurement-and-security-implications-of-dmarc-reporting': ['DMARC'],
  'content-type-multipart-oracle-tapping-into-format-oracles-in-email-end-to-end-en': ['S/MIME+OpenPGP'],
  'extended-hell-o-a-comprehensive-large-scale-study-on-email-confidentiality-and-i': ['SPF', 'DKIM', 'DMARC', 'MTA-STS', 'DANE', 'STARTTLS', 'S/MIME+OpenPGP'],
  'bounce-in-the-wild-a-deep-dive-into-email-delivery-failures-from-a-large-email-s': ['delivery-path', 'SPF', 'DKIM', 'DMARC'],
  'breakspf-how-shared-infrastructures-magnify-spf-vulnerabilities-across-the-internet': ['SPF', 'DMARC'],
  'fakebehalf-imperceptible-email-spoofing-attacks-against-the-delegation-mechanism': ['SPF', 'DKIM', 'DMARC'],
  'spf-beyond-the-standard-management-and-operational-challenges-in-practice-and-pr': ['SPF'],
  'unraveling-the-complexities-of-mta-sts-deployment-and-management-in-securing-ema': ['MTA-STS', 'STARTTLS'],
  'email-spoofing-with-smtp-smuggling-how-the-shared-email-infrastructures-magnify': ['SPF', 'DKIM', 'delivery-path'],
  'a-multifaceted-study-on-the-use-of-tls-and-auto-detect-in-email-ecosystems': ['STARTTLS', 'DANE', 'autoconfig'],
  's-mine-collecting-and-analyzing-s-mime-certificates-at-scale': ['S/MIME+OpenPGP'],
  'understanding-and-characterizing-intermediate-paths-of-email-delivery-the-hidden': ['delivery-path', 'SPF'],
  'automatic-insecurity-exploring-email-auto-configuration-in-the-wild': ['autoconfig', 'STARTTLS'],
  'coordmail-exploiting-smtp-timeout-and-command-interaction-to-coordinate-email-middleware-for-convergence-amplification-attack': ['delivery-path', 'SPF', 'DKIM'],
};
 
// The instrument. A paper gets every family it actually ran.
const METHODS = {
  'dns-scan': 'resolve DNS records (TXT/MX/TLSA/_mta-sts) over a domain list',
  'passive-dns': 'query a passive-DNS archive instead of resolving live',
  'smtp-probe': 'open SMTP sessions against real MTAs and drive the protocol',
  'account-delivery-test': 'register accounts at real providers, send crafted mail, observe the inbox',
  'client-matrix': 'test N mail clients (MUAs) by hand or semi-automatically',
  'provider-logs': 'analyse logs, headers or mailboxes from a real mail operator',
  'operator-survey': 'survey mail administrators',
  'notification-experiment': 'notify misconfigured operators, then rescan',
  'software-testbed': 'run MTA/library implementations in a controlled lab',
  'code-analysis': 'read the implementation source',
  'honey-domain': 'register domains or addresses and receive live mail',
  'user-study': 'recruit end users',
};
 
const METHOD = {
  'security-by-any-other-name-on-the-effectiveness-of-provider-based-email-security': ['dns-scan', 'smtp-probe', 'account-delivery-test'],
  'neither-snow-nor-rain-nor-mitm-an-empirical-analysis-of-email-delivery-security': ['dns-scan', 'smtp-probe', 'provider-logs'],
  'email-typosquatting': ['honey-domain', 'dns-scan'],
  'efail-breaking-s-mime-and-openpgp-email-encryption-using-exfiltration-channels': ['client-matrix'],
  'end-to-end-measurements-of-email-spoofing-attacks': ['dns-scan', 'account-delivery-test', 'client-matrix', 'user-study'],
  'johnny-you-are-fired-spoofing-openpgp-and-s-mime-signatures-in-emails': ['client-matrix', 'code-analysis'],
  'a-longitudinal-and-comprehensive-study-of-the-dane-ecosystem-in-email': ['dns-scan', 'smtp-probe', 'account-delivery-test', 'software-testbed'],
  'composition-kills-a-case-study-of-email-sender-authentication': ['account-delivery-test', 'client-matrix'],
  'weak-links-in-authentication-chains-a-large-scale-analysis-of-email-sender-spoof': ['account-delivery-test', 'client-matrix'],
  'why-tls-is-better-without-starttls-a-security-analysis-of-starttls-in-the-email': ['client-matrix', 'smtp-probe', 'software-testbed'],
  'spfail-discovering-measuring-and-remediating-vulnerabilities-in-email-sender-val': ['smtp-probe', 'code-analysis', 'notification-experiment'],
  'a-large-scale-and-longitudinal-measurement-study-of-dkim-deployment': ['dns-scan', 'passive-dns', 'provider-logs'],
  'under-the-hood-of-dane-mismanagement-in-smtp': ['dns-scan', 'smtp-probe', 'operator-survey'],
  'revisiting-email-forwarding-security-under-the-authenticated-received-chain-prot': ['provider-logs', 'account-delivery-test', 'code-analysis'],
  '27-years-and-81-million-opportunities-later-investigating-the-use-of-email-encry': ['provider-logs'],
  'heads-in-the-clouds-measuring-universities-migration-to-public-clouds-implicatio': ['dns-scan'],
  'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild': ['dns-scan', 'notification-experiment', 'account-delivery-test'],
  'youve-got-report-measurement-and-security-implications-of-dmarc-reporting': ['dns-scan', 'passive-dns', 'account-delivery-test', 'operator-survey'],
  'content-type-multipart-oracle-tapping-into-format-oracles-in-email-end-to-end-en': ['client-matrix'],
  'extended-hell-o-a-comprehensive-large-scale-study-on-email-confidentiality-and-i': ['dns-scan', 'smtp-probe', 'account-delivery-test', 'notification-experiment', 'operator-survey'],
  'bounce-in-the-wild-a-deep-dive-into-email-delivery-failures-from-a-large-email-s': ['provider-logs', 'dns-scan'],
  'breakspf-how-shared-infrastructures-magnify-spf-vulnerabilities-across-the-internet': ['dns-scan', 'account-delivery-test', 'software-testbed'],
  'fakebehalf-imperceptible-email-spoofing-attacks-against-the-delegation-mechanism': ['account-delivery-test', 'client-matrix', 'user-study'],
  'spf-beyond-the-standard-management-and-operational-challenges-in-practice-and-pr': ['dns-scan', 'smtp-probe', 'operator-survey', 'software-testbed'],
  'unraveling-the-complexities-of-mta-sts-deployment-and-management-in-securing-ema': ['dns-scan', 'smtp-probe', 'account-delivery-test', 'operator-survey'],
  'email-spoofing-with-smtp-smuggling-how-the-shared-email-infrastructures-magnify': ['account-delivery-test', 'smtp-probe', 'software-testbed'],
  'a-multifaceted-study-on-the-use-of-tls-and-auto-detect-in-email-ecosystems': ['client-matrix', 'smtp-probe', 'dns-scan'],
  's-mine-collecting-and-analyzing-s-mime-certificates-at-scale': ['smtp-probe', 'client-matrix'],
  'understanding-and-characterizing-intermediate-paths-of-email-delivery-the-hidden': ['provider-logs', 'dns-scan'],
  'automatic-insecurity-exploring-email-auto-configuration-in-the-wild': ['dns-scan', 'client-matrix', 'software-testbed'],
  'coordmail-exploiting-smtp-timeout-and-command-interaction-to-coordinate-email-middleware-for-convergence-amplification-attack': ['smtp-probe', 'account-delivery-test', 'software-testbed'],
};
 
function assertComplete(name, map) {
  const missing = [...INFRA_SLUGS].filter((s) => !(s in map));
  const extra = Object.keys(map).filter((s) => !INFRA_SLUGS.has(s));
  if (missing.length || extra.length)
    throw new Error(`${name}: missing ${missing.length} ${JSON.stringify(missing)}; extra ${extra.length} ${JSON.stringify(extra)}`);
}
assertComplete('KEYS', KEYS);
assertComplete('MECH', MECH);
assertComplete('METHOD', METHOD);
for (const [slug, ms] of Object.entries(METHOD))
  for (const m of ms) if (!(m in METHODS)) throw new Error(`${slug}: unknown method family ${m}`);
 
// ---------------------------------------------------------------------------
// C. Hand-keyed figures. Every one of these appears on the page. `quote` is
//    checked verbatim against paper.cols.txt below; `pdfOnly: true` marks a
//    quote the .cols rendering splices and that was verified against paper.pdf
//    text instead (see the provenance page).
// ---------------------------------------------------------------------------
const FIGURES = [
  // --- adoption, in publication order, each with the population it measured ---
  { g: 'adoption', slug: 'neither-snow-nor-rain-nor-mitm-an-empirical-analysis-of-email-delivery-security',
    what: 'STARTTLS', value: '81.8% (648,030)', pop: 'mail-enabled domains among the Alexa Top Million, 26 Apr 2015',
    quote: 'In total, 648,030 (81.8%) of mail-enabled domains supported STARTTLS' },
  { g: 'adoption', slug: 'neither-snow-nor-rain-nor-mitm-an-empirical-analysis-of-email-delivery-security',
    what: 'DMARC', value: '1.1%', pop: 'Alexa Top Million domains with an MX record, Apr 2015',
    quote: 'only 35% successfully configure encryption, and 1.1% specify a DMARC authentication policy' },
  { g: 'adoption', slug: 'end-to-end-measurements-of-email-spoofing-attacks',
    what: 'SPF', value: '44.9%', pop: 'Alexa top 1 million domains, Jan 2018 snapshot',
    quote: 'About 44.9% of the domains have published a valid SPF record in 2018' },
  { g: 'adoption', slug: 'a-large-scale-and-longitudinal-measurement-study-of-dkim-deployment',
    what: 'DKIM', value: '28.1% (lower bound)', pop: 'Alexa Top 1 million domains, probed with 40 selectors harvested from passive DNS',
    quote: '28.1% of Alexa Top 1 million domains have enabled DKIM' },
  { g: 'adoption', slug: 'a-longitudinal-and-comprehensive-study-of-the-dane-ecosystem-in-email',
    what: 'DANE (TLSA)', value: '0.60%–0.73%', pop: 'second-level domains WITH an MX record in .com / .net',
    quote: '0.60% (.com) ∼ 0.73% (.net) of all',
    quote2: 'domains with MX records have corresponding TLSA records in the latest snapshot',
    spliced: 'a column break drops the figure caption "4.1 Datasets" into the middle of the sentence' },
  { g: 'adoption', slug: 'extended-hell-o-a-comprehensive-large-scale-study-on-email-confidentiality-and-i',
    what: 'SPF', value: '4,167,633 domains (41.7%)', pop: 'Tranco top 10 million domains, 2023',
    quote: 'Our analysis shows that only 4,167,633 do',
    quote2: 'mains in the top ten million implement SPF',
    spliced: 'the sentence is broken mid-word by a column break' },
  { g: 'adoption', slug: 'extended-hell-o-a-comprehensive-large-scale-study-on-email-confidentiality-and-i',
    what: 'DMARC', value: '882,183 domains (8.8%)', pop: 'Tranco top 10 million domains, final scan June 2023',
    quote: 'Of the top 10 million domains, 882,183 deployed a DMARC policy at the time of our final scan in June 2023' },
  { g: 'adoption', slug: 'extended-hell-o-a-comprehensive-large-scale-study-on-email-confidentiality-and-i',
    what: 'MTA-STS', value: '6,948 records, 569 with a policy file', pop: 'Tranco top 10 million domains WITH an MX record, 2023',
    quote: 'However, only 569 of these domains actually had a policy file hosted in that location' },
  { g: 'adoption', slug: 'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild',
    what: 'SPF', value: '56.5% / 60.2%', pop: '12 million domains (all) vs the top 1 million of them, 2023',
    quote: 'a clear increase in SPF usage to 56.5 % can be observed from our scan' },
  { g: 'adoption', slug: 'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild',
    what: 'DMARC', value: '13.6% / 22.6%', pop: '12 million domains (all) vs the top 1 million of them, 2023',
    quote: 'is now at 22.6 % for the top 1 million domains and 13.6 % for all domains' },
  { g: 'adoption', slug: 'breakspf-how-shared-infrastructures-magnify-spf-vulnerabilities-across-the-internet',
    what: 'SPF', value: '60.9% published, 55.9% valid', pop: 'Tranco top million domains, 2023–24',
    quote: '60.9% of the top million domains have deployed SPF records, and 55.9% have deployed valid SPF records' },
  { g: 'adoption', slug: 'unraveling-the-complexities-of-mta-sts-deployment-and-management-in-securing-ema',
    what: 'MTA-STS', value: '68,030 domains (0.07%–0.13%)', pop: 'domains WITH AN MX RECORD in the .com/.net/.org/.se zone files (86.8M; Table 1 column header reads "Domains with MX Records"), 29 Sep 2024',
    quote: 'have risen 3-4 times, with adoption ranging between 53,800 (0.07%)',
    quote2: '.com 73,939,004 53,800 (0.07%)',
    spliced: 'Table 1, whose header column reads "Domains with MX Records", is interleaved with a figure legend; quote2 is the .com row' },
 
  // --- what the receiver actually does, as opposed to what the sender publishes ---
  { g: 'enforcement', slug: 'security-by-any-other-name-on-the-effectiveness-of-provider-based-email-security',
    what: 'SPF enforcement', value: '10 providers acted', pop: '22 large providers where the authors held an account, 2014–15',
    quote: 'Almost all providers performed the SPF DNS query, but only 10 took action, half at the SMTP layer' },
  { g: 'enforcement', slug: 'spf-beyond-the-standard-management-and-operational-challenges-in-practice-and-pr',
    what: 'inbound SPF validation before DATA', value: '6.8% (81,843)', pop: '1.2 million reachable SMTP servers derived from four zone files',
    quote: 'initiate SPF queries prior to issuing the DATA command' },
  { g: 'enforcement', slug: 'unraveling-the-complexities-of-mta-sts-deployment-and-management-in-securing-ema',
    what: 'sender-side MTA-STS validation', value: '19.6% (469)', pop: '2,394 sender domains observed in the authors’ deliverability tests',
    quote: 'Our sender-side dataset reveals that 19.6% of domains perform MTA-STS validation when sending email to an MTA-STS enabled domain' },
  { g: 'enforcement', slug: 'a-longitudinal-and-comprehensive-study-of-the-dane-ecosystem-in-email',
    what: 'DANE-validating senders', value: '4 providers', pop: '29 email providers tested end to end',
    quote: 'we find that only four email service providers (mail.com, comcast.net, gmx.com, tutanota.com) actually fetch TLSA records' },
  { g: 'enforcement', slug: 'end-to-end-measurements-of-email-spoofing-attacks',
    what: 'forged mail reaching the inbox', value: '34 of 35 providers', pop: '35 public email providers, Dec 2017 – Jan 2018',
    quote: 'out of the 35 email services allowed at least one forged email to arrive the inbox' },
  { g: 'enforcement', slug: 'weak-links-in-authentication-chains-a-large-scale-analysis-of-email-sender-spoof',
    what: 'sender-inconsistency checks', value: '12 services', pop: '30 email services and 23 clients tested, 2021',
    quote: 'only 12 services perform the sender inconsistency checks' },
 
  // --- misconfiguration: the second thing every one of these papers measures ---
  { g: 'misconfig', slug: 'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild',
    what: 'SPF authorising >100,000 IPv4 addresses', value: '34.7%', pop: 'the SPF-publishing domains of the 12 million scanned, 2023',
    quote: '34.7 % of the domains allow emails to be sent from over 100 000 IP addresses' },
  { g: 'misconfig', slug: 'spf-beyond-the-standard-management-and-operational-challenges-in-practice-and-pr',
    what: 'SPF needing >10 DNS lookups', value: '6.5%', pop: '55 million domains that have SPF records, latest of 17 monthly zone-file scans',
    quote: 'We use our latest scan and focus on 55 M domains that have SPF records' },
  { g: 'misconfig', slug: 'unraveling-the-complexities-of-mta-sts-deployment-and-management-in-securing-ema',
    what: 'MTA-STS incorrectly configured', value: '29.6% (20,144)', pop: 'the 68,030 domains that published an MTA-STS record, Sep 2024',
    quote: '29.6% of domains with MTA-STS records in our latest scan',
    quote2: 'We find that out of the 20,144 misconfigured domains, 640 (3.2%) domains will encounter email delivery failures' },
  { g: 'misconfig', slug: 'under-the-hood-of-dane-mismanagement-in-smtp',
    what: 'invalid TLSA records', value: 'over 22%', pop: 'SMTP servers serving .com domains, hourly snapshots',
    quote: 'the percentage of SMTP servers with invalid TLSA records is over 22% when they serve domains in .com' },
  { g: 'misconfig', slug: 'a-large-scale-and-longitudinal-measurement-study-of-dkim-deployment',
    what: 'DKIM keys ≤ 1024 bits', value: '84%', pop: '3,627,871 domains with DKIM keys in passive DNS',
    quote: 'of 3,627,871 domains still use DKIM keys that are less or equal to 1024 bits' },
  { g: 'misconfig', slug: 'a-large-scale-and-longitudinal-measurement-study-of-dkim-deployment',
    what: 'shared DKIM keys', value: '66.9% (2,427,682)', pop: 'the DKIM-publishing domains in the passive-DNS set',
    quote: 'we find 61,062 DKIM keys shared by more than one domain' },
  { g: 'misconfig', slug: 'youve-got-report-measurement-and-security-implications-of-dmarc-reporting',
    what: 'external rua without an authorisation record', value: '26% (520K)', pop: '2 million DMARC records naming an external reporting domain',
    quote: 'we find that 520K (26%) of them do not have the authorization DMARC records' },
  { g: 'misconfig', slug: 'extended-hell-o-a-comprehensive-large-scale-study-on-email-confidentiality-and-i',
    what: 'STARTTLS certificate validation failure', value: '30.0%', pop: '2,112,682 connectable MXs, 2023',
    quote: 'Of the remaining 2,112,682 MXs, only 1,478,060 (70.0%) passed proper certificate validation' },
  { g: 'misconfig', slug: 'automatic-insecurity-exploring-email-auto-configuration-in-the-wild',
    what: 'auto-configuration support', value: '7.52% (79,212)', pop: '1,053,469 domains scanned for Autodiscover / Autoconfig / SRV',
    quote: 'domains support at least one auto-configuration mechanism' },
 
  { g: 'misconfig', slug: 'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild',
    what: 'SPF with no restrictive all qualifier', value: '5.9% (427,767)', pop: 'the SPF-publishing domains of the 12 million scanned, 2023 (427,767 / 0.059 = 7.25M, i.e. the SPF-publishing subset, not all 12M)',
    quote: 'the SPF configuration is missing a restrictive all policy' },
  { g: 'clients', slug: 'end-to-end-measurements-of-email-spoofing-attacks',
    what: 'providers showing any security indicator on a forged message', value: '9 of 35', pop: '35 popular email providers, web and mobile interfaces inspected by hand',
    quote: 'only 9 of 35 providers have implemented some security indicators' },
  { g: 'clients', slug: 'end-to-end-measurements-of-email-spoofing-attacks',
    what: 'providers showing a MISLEADING element on a forged message', value: '25 of 35', pop: 'the same 35 providers',
    quote: '25 out of 35 providers will automatically load the spoofed sender\'s photo' },
  { g: 'logs', slug: '27-years-and-81-million-opportunities-later-investigating-the-use-of-email-encry',
    what: 'emails encrypted', value: '0.06% (46,973)', pop: 'the same 81.6 million emails at one university',
    quote: '46,973 (0.06%) emails were encrypted' },
 
  // --- the client side ---
  { g: 'clients', slug: 'efail-breaking-s-mime-and-openpgp-email-encryption-using-exfiltration-channels',
    what: 'exfiltration channels', value: '23 of 35 S/MIME, 10 of 28 OpenPGP', pop: '48 mail clients tested',
    quote: '23 of the 35 tested S/MIME email clients' },
  { g: 'clients', slug: 'johnny-you-are-fired-spoofing-openpgp-and-s-mime-signatures-in-emails',
    what: 'signature spoofing', value: '14 of 20 OpenPGP, 15 of 22 S/MIME', pop: '25 mail clients tested',
    quote: 'in 14 out of 20 tested OpenPGP-capable email clients' },
  { g: 'clients', slug: 'why-tls-is-better-without-starttls-a-security-analysis-of-starttls-in-the-email',
    what: 'plaintext downgrade', value: '15 of 28', pop: '28 mail clients tested, 2021',
    quote: 'In total, 15 of 28 clients could be downgraded to plaintext and leaked sensitive data' },
  { g: 'clients', slug: 'a-multifaceted-study-on-the-use-of-tls-and-auto-detect-in-email-ecosystems',
    what: 'silent TLS downgrade', value: '19 of 49', pop: '49 mail clients tested, 2025',
    quote: 'we found that 19 clients may inadvertently downgrade the security of the email protocols to no-TLS without notifying the user' },
  { g: 'clients', slug: 'automatic-insecurity-exploring-email-auto-configuration-in-the-wild',
    what: 'auto-configuration attack', value: '22 of 29', pop: '29 mail clients exercised against controlled servers',
    quote: '22/29 clients were vulnerable to at least one of the attack scenarios' },
 
  // --- provider-side logs ---
  { g: 'logs', slug: '27-years-and-81-million-opportunities-later-investigating-the-use-of-email-encry',
    what: 'users who ever used S/MIME or PGP', value: '5.46%', pop: '81.6 million emails from 37,089 accounts at one university, 27 years',
    quote: 'We found that only 5.46% of all users ever used S/MIME or PGP' },
  { g: 'logs', slug: 'bounce-in-the-wild-a-deep-dive-into-email-delivery-failures-from-a-large-email-s',
    what: 'hard bounces', value: '8.11% (24M)', pop: '298 million emails at one large Chinese ESP',
    quote: '259M (87.07%) are non-bounced, 14M (4.82%) are soft-bounced, and 24M (8.11%) are hard-bounced' },
  { g: 'logs', slug: 'bounce-in-the-wild-a-deep-dive-into-email-delivery-failures-from-a-large-email-s',
    what: 'bounces caused by sender authentication failure', value: '2.19% (701K)', pop: 'the same 298 million emails; share of all, not of bounces',
    quote: 'We find that 701K (2.19%) emails are hard-bounced due to sender authentication failure' },
  { g: 'logs', slug: 'understanding-and-characterizing-intermediate-paths-of-email-delivery-the-hidden',
    what: 'paths relying entirely on third-party relaying', value: '82.7% (86.9M)', pop: '105 million emails with a reconstructable intermediate path',
    quote: 'We find that 82.7% of email intermediate paths fall into the Third-party hosting category' },
  { g: 'logs', slug: 'understanding-and-characterizing-intermediate-paths-of-email-delivery-the-hidden',
    what: 'concentration of the relay market (HHI)', value: '40%', pop: 'middle nodes across all reconstructed intermediate paths',
    quote: 'we obtain an HHI of 40% for the middle node market, which indicates a highly concentrated market' },
  { g: 'logs', slug: 'neither-snow-nor-rain-nor-mitm-an-empirical-analysis-of-email-delivery-security',
    what: 'inbound Gmail messages authenticated by SPF, DKIM or both', value: '94.40%', pop: 'Gmail SMTP handshake logs, April 2015',
    quote: 'During April 2015, 94.40% of incoming Gmail messages were authenticated with DKIM, SPF, or both' },
 
  // --- notification experiments ---
  { g: 'notify', slug: 'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild',
    what: 'SPF errors fixed two weeks after notification', value: '6,931 errors', pop: 'notified domains, rescanned two weeks later; total errors fell 3.28%',
    quote: 'We observe that 6 931 errors have been fixed by that time' },
  { g: 'notify', slug: 'extended-hell-o-a-comprehensive-large-scale-study-on-email-confidentiality-and-i',
    what: 'SPF misconfiguration remediated one month after notification', value: '39.7% vs 20.7% control',
    pop: 'notified vs undeliverable-notification domains; Fisher exact p = 7.8e-43',
    quote: 'only 1,076 were still misconfigured in November, which is a decrease of 39.7%' },
  { g: 'notify', slug: 'spfail-discovering-measuring-and-remediating-vulnerabilities-in-email-sender-val',
    what: 'still vulnerable after four months', value: 'just over 80%', pop: 'domains inferred vulnerable to the libSPF2 flaw, remeasured over four months',
    quote: 'just over 80% of domains that we were able to infer results for were still vulnerable' },
 
  // --- operator surveys ---
  { g: 'survey', slug: 'spf-beyond-the-standard-management-and-operational-challenges-in-practice-and-pr',
    what: 'operators who validate SPF inbound', value: '87.1% (54)', pop: '62 survey respondents who answered that question (95 started)',
    quote: 'answered yes, while only 8 explicitly indicated they do not' },
  { g: 'survey', slug: 'unraveling-the-complexities-of-mta-sts-deployment-and-management-in-securing-ema',
    what: 'operators who had heard of MTA-STS', value: '94.7% (89)', pop: '94 respondents who answered that question (117 engaged with the survey)',
    quote: 'While awareness of MTA-STS was high (94.7%)' },
  { g: 'survey', slug: 'youve-got-report-measurement-and-security-implications-of-dmarc-reporting',
    what: 'DMARC publishers who also send reports', value: '66% (26)', pop: '39 operators who said they publish DMARC records; 74 answered any question',
    quote: 'Out of 39 operators who published DMARC records, we also find that 26 (66%) also send DMARC reports' },
 
  // --- shared infrastructure: the 2024-2026 turn ---
  { g: 'shared', slug: 'breakspf-how-shared-infrastructures-magnify-spf-vulnerabilities-across-the-internet',
    what: 'domains exploitable through a shared-infrastructure IP inside their own SPF', value: '23,916',
    pop: 'Tranco top million; 23 of them in the top 1,000',
    quote: 'We uncover that 23,916 domains are vulnerable to BreakSPF attacks' },
  { g: 'shared', slug: 'breakspf-how-shared-infrastructures-magnify-spf-vulnerabilities-across-the-internet',
    what: 'SPF records covering more than 65,536 addresses', value: '51.7%', pop: 'the SPF-publishing domains of the Tranco top million',
    quote: 'findings revealed that 51.7% of domains have SPF records' },
  { g: 'shared', slug: 'email-spoofing-with-smtp-smuggling-how-the-shared-email-infrastructures-magnify',
    what: 'exploitable sender/receiver pairs', value: '36 of 440', pop: '20 sending x 22 receiving public email services',
    quote: 'we identified 36 exploitable pairs out of the 440' },
  { g: 'shared', slug: 'coordmail-exploiting-smtp-timeout-and-command-interaction-to-coordinate-email-middleware-for-convergence-amplification-attack',
    what: 'reflected mail passing SPF or DKIM', value: '77.89% (7,851)', pop: '10,079 exploitable bounce servers found by probing',
    quote: 'We discover that emails from 7,851 (77.89%) bounce servers could be verified by SPF or DKIM' },
  { g: 'shared', slug: 'lazy-gatekeepers-a-large-scale-study-on-spf-configuration-in-the-wild',
    what: 'domains spoofable by renting shared web hosting', value: '26,095', pop: 'domains whose SPF authorised the rented hosts; 4 of 5 providers allowed it',
    quote: 'we are able to send emails with valid SPF entries from 26 095 domains' },
];
 
// ---------------------------------------------------------------------------
// D. Quote verification against paper.cols.txt (three whitespace/hyphen modes)
// ---------------------------------------------------------------------------
const colsCache = new Map();
function colsText(p) {
  if (colsCache.has(p.slug)) return colsCache.get(p.slug);
  const f = path.join(ROOT, 'fulltext', String(p.year), p.venue, p.slug, 'paper.cols.txt');
  const v = fs.existsSync(f) ? fs.readFileSync(f, 'utf8') : null;
  colsCache.set(p.slug, v);
  return v;
}
const norm = (s) => s.replace(/[‘’]/g, "'").replace(/[“”]/g, '"').replace(/[‐-―]/g, '-').replace(/\s+/g, ' ').trim();
const nospace = (s) => norm(s).replace(/[\s-]/g, '');
const nopunct = (s) => nospace(s).replace(/[^a-z0-9]/gi, '').toLowerCase();
 
function locate(p, quote) {
  const t = colsText(p);
  if (t === null) return 'NO-COLS-FILE';
  if (norm(t).includes(norm(quote))) return 'exact';
  if (nospace(t).includes(nospace(quote))) return 'whitespace-folded';
  if (nopunct(t).includes(nopunct(quote))) return 'punctuation-folded';
  // The .cols rendering repairs two-column reading order but still interposes
  // page furniture (running heads, table captions, figure labels) inside a
  // sentence that spans a column break. A quote that is real in the PDF but
  // interrupted in .cols shows up here as 'spliced': both ends are present,
  // the middle is not contiguous. Anything on the page must be 'exact' or a
  // fragment short enough to be contiguous -- see FIGURES.
  const h = nopunct(t);
  let pre = 0, suf = 0;
  for (let k = 1; k <= quote.length; k++) { if (h.includes(nopunct(quote.slice(0, k)))) pre = k; else break; }
  for (let k = 1; k <= quote.length; k++) { if (h.includes(nopunct(quote.slice(quote.length - k)))) suf = k; else break; }
  if (pre >= 25 && suf >= 25) return `spliced(head ${pre}ch / tail ${suf}ch)`;
  return `NOT FOUND (longest head ${pre}ch: "${quote.slice(0, pre)}")`;
}
 
// ---------------------------------------------------------------------------
// E. Output
// ---------------------------------------------------------------------------
const wikiOnly = process.argv.includes('--format') && process.argv[process.argv.indexOf('--format') + 1] === 'wiki';
 
H('A. POPULATION');
console.log(`corpus            : ${ALL.length} extracted papers, 7 venues, 2010-2026`);
console.log(`page population   : ${PAPERS.length} papers, verdict INFRA in scripts/msg_fold.mjs`);
console.log(`  denominator note: this is 0.53% of ${ALL.length}. It is NOT a share of anything`);
console.log(`                    meaningful; it is a hand-mapped topical slice of a 354-paper`);
console.log(`                    candidate pool published with privacy:email_tracking.`);
const yrs = {}, vns = {};
for (const p of PAPERS) { yrs[p.year] = (yrs[p.year] || 0) + 1; vns[p.venue] = (vns[p.venue] || 0) + 1; }
console.log('\nby year  :', Object.entries(yrs).map(([k, v]) => `${k}:${v}`).join('  '));
console.log('by venue :', Object.entries(vns).sort((a, b) => b[1] - a[1]).map(([k, v]) => `${k}:${v}`).join('  '));
console.log(`\n2018-2026: ${PAPERS.filter(p => p.year >= 2018).length} of 31; 2022-2026: ${PAPERS.filter(p => p.year >= 2022).length} of 31`);
console.log('\nThe list, oldest first:');
for (const p of PAPERS) console.log(`  ${key(p)}\n      ${MSG_MAP[p.slug][1]}`);
 
H('A2. THE 31, AS THE PAGE PRINTS THEM (DokuWiki table, generated -- do not retype)');
{
  const VEN = { USENIX: 'USENIX Sec', IMC: 'IMC', NDSS: 'NDSS', CCS: 'CCS', 'IEEE-SP': 'IEEE S&P', WWW: 'TheWebConf', PETS: 'PETS' };
  console.log('^ Year ^ Venue ^ Paper ^ What it measures ^');
  for (const p of PAPERS) console.log(`| ${p.year} | ${VEN[p.venue]} | {[${KEYS[p.slug]}]} | ${MSG_MAP[p.slug][1]} |`);
  const dupes = Object.values(KEYS).filter((k, i, a) => a.indexOf(k) !== i);
  if (dupes.length) throw new Error(`duplicate citekeys: ${dupes}`);
  console.log(`\n${new Set(Object.values(KEYS)).size} distinct citekeys for ${PAPERS.length} papers.`);
}
 
H('A3. WHO WROTE THEM. Author concentration across the 31.');
{
  // Authors come from data/corpus2/.meta plus out/authors.json, the same two
  // sources bibgen.mjs uses -- PETS and USENIX .meta records carry no authors,
  // so the cache is not optional here. A paper with no author list is reported,
  // not skipped: an undercount would understate the concentration.
  const metaDir = path.join(ROOT, 'corpus2/.meta');
  const idx = new Map();
  for (const f of fs.readdirSync(metaDir).filter((x) => x.endsWith('.json')))
    for (const r of JSON.parse(fs.readFileSync(path.join(metaDir, f), 'utf8')).papers)
      idx.set(`${r.venue}/${r.year}/${r.slug}`, r);
  const cachePath = path.join(path.dirname(new URL(import.meta.url).pathname), '..', 'out', 'authors.json');
  const cache = fs.existsSync(cachePath) ? JSON.parse(fs.readFileSync(cachePath, 'utf8')) : {};
  const firsts = new Map(), all = new Map();
  const noAuthors = [];
  for (const p of PAPERS) {
    const k = key(p);
    const au = (idx.get(k) && idx.get(k).authors) || cache[k] || null;
    if (!au || !au.length) { noAuthors.push(k); continue; }
    const norm = (a) => String(a).trim();
    const f = norm(au[0]);
    if (!firsts.has(f)) firsts.set(f, []);
    firsts.get(f).push(k);
    for (const a of new Set(au.map(norm))) { if (!all.has(a)) all.set(a, []); all.get(a).push(k); }
  }
  console.log(`denominator: ${PAPERS.length - noAuthors.length} of ${PAPERS.length} papers have an author list.`);
  if (noAuthors.length) console.log(`  NO AUTHOR LIST (excluded, and this undercounts the concentration): ${JSON.stringify(noAuthors)}`);
  const repeatFirst = [...firsts.entries()].filter(([, v]) => v.length > 1).sort((a, b) => b[1].length - a[1].length);
  const covered = new Set(repeatFirst.flatMap(([, v]) => v));
  console.log(`\nFirst authors with more than one paper here: ${repeatFirst.length}, covering ${covered.size} of ${PAPERS.length} papers`);
  for (const [a, v] of repeatFirst) console.log(`  ${String(v.length).padStart(2)}  ${a}`);
  console.log('\nAuthors on four or more of the 31:');
  for (const [a, v] of [...all.entries()].filter(([, v]) => v.length >= 4).sort((x, y) => y[1].length - x[1].length))
    console.log(`  ${String(v.length).padStart(2)}  ${a}`);
  console.log(`\ndistinct authors across the ${PAPERS.length - noAuthors.length}: ${all.size}`);
}
 
H('B. MECHANISM MEASURED (hand map, paper-counted, multi-valued)');
{
  const c = new Map();
  for (const p of PAPERS) for (const m of MECH[p.slug]) { if (!c.has(m)) c.set(m, []); c.get(m).push(key(p)); }
  console.log(`denominator: ${PAPERS.length} papers. Multi-valued, so shares do not sum to 100%.\n`);
  for (const [m, list] of [...c.entries()].sort((a, b) => b[1].length - a[1].length))
    console.log(`  ${String(list.length).padStart(2)}  ${((list.length / PAPERS.length) * 100).toFixed(1).padStart(5)}%  ${m}`);
  console.log('\n  MECHANISMS NO PAPER MEASURES THE DEPLOYMENT OF (full-text probe, section G):');
  console.log('    BIMI       -- named in 5 corpus papers, all 5 in this population, all 5 related-work sentences');
  console.log('    TLS-RPT    -- named in 3, all 3 in this population. NOT untouched: youve-got-report');
  console.log('                  Table 4 measures provider support (2 of 8 EHPs) and its survey asks');
  console.log('                  operators whether they send reports. What nobody has done is scan a');
  console.log('                  DOMAIN POPULATION for _smtp._tls records, the way four papers scan SPF.');
  console.log('    REQUIRETLS -- named in 0 papers corpus-wide');
  if (!wikiOnly) {
    console.log('\n  papers per mechanism:');
    for (const [m, list] of [...c.entries()].sort((a, b) => b[1].length - a[1].length)) {
      console.log(`   ${m}:`);
      for (const k of list) console.log(`      ${k}`);
    }
  }
}
 
H('C. INSTRUMENT (hand map, paper-counted, multi-valued)');
{
  const c = new Map();
  for (const p of PAPERS) for (const m of METHOD[p.slug]) { if (!c.has(m)) c.set(m, []); c.get(m).push(key(p)); }
  // Mean and median are both printed and both labelled. An earlier version of
  // this line computed the mean and called it the median, and the page repeated
  // the mislabel; they differ here (2.74 vs 3).
  const per = PAPERS.map((p) => METHOD[p.slug].length).sort((a, b) => a - b);
  const mean = per.reduce((a, b) => a + b, 0) / per.length;
  const median = per[Math.floor(per.length / 2)];
  const hist = {};
  for (const n of per) hist[n] = (hist[n] || 0) + 1;
  console.log(`denominator: ${PAPERS.length} papers. Multi-valued: MEAN ${mean.toFixed(2)}, MEDIAN ${median}, range ${per[0]}-${per[per.length - 1]}, distribution ${JSON.stringify(hist)} (instruments per paper).\n`);
  for (const [m, list] of [...c.entries()].sort((a, b) => b[1].length - a[1].length))
    console.log(`  ${String(list.length).padStart(2)}  ${((list.length / PAPERS.length) * 100).toFixed(1).padStart(5)}%  ${m.padEnd(24)} ${METHODS[m]}`);
  {
    const late = PAPERS.filter((p) => p.year >= 2023);
    const newish = late.filter((p) => METHOD[p.slug].includes('operator-survey') || METHOD[p.slug].includes('software-testbed'));
    console.log(`\n  papers from 2023 onwards running an operator survey OR a software testbed: ${newish.length} of ${late.length}`);
    for (const p of newish) console.log(`      ${key(p)}  [${METHOD[p.slug].filter((m) => m === 'operator-survey' || m === 'software-testbed').join(', ')}]`);
  }
  console.log('\n  by period (does the instrument date?):');
  const periods = [['2015-2019', (y) => y <= 2019], ['2020-2022', (y) => y >= 2020 && y <= 2022], ['2023-2026', (y) => y >= 2023]];
  const all = [...c.keys()].sort();
  console.log('    ' + 'instrument'.padEnd(24) + periods.map(([n]) => n.padStart(11)).join(''));
  for (const m of all) {
    const row = periods.map(([, f]) => {
      const n = PAPERS.filter((p) => f(p.year) && METHOD[p.slug].includes(m)).length;
      const d = PAPERS.filter((p) => f(p.year)).length;
      return `${n}/${d}`.padStart(11);
    });
    console.log('    ' + m.padEnd(24) + row.join(''));
  }
  if (!wikiOnly) {
    console.log('\n  papers per instrument:');
    for (const [m, list] of [...c.entries()].sort((a, b) => b[1].length - a[1].length)) {
      console.log(`   ${m}:`);
      for (const k of list) console.log(`      ${k}`);
    }
  }
}
 
H('D. SAMPLING FRAME (population[].sourceList, folded, paper-counted)');
{
  // Fold: only the frames a domain-list scan can draw from. Residue printed.
  const FOLD = [
    [/alexa/i, 'Alexa (retired Nov 2022)'],
    [/tranco/i, 'Tranco'],
    [/majestic|umbrella|domcop|quantcast/i, 'another top list (Majestic / Umbrella / DomCop)'],
    [/zone file|verisign|internetstiftelsen|public interest registry|czds/i, 'TLD zone files'],
    [/passive dns|farsight|sie|dnsdb|openintel/i, 'passive DNS / OpenINTEL'],
    [/adobe/i, 'Adobe 2013 breach address list'],
    [/enron/i, 'Enron corpus'],
    [/gmail|coremail|smtp connection log|reception log|delivery log/i, 'operator logs'],
    [/university|times higher|the top100|institution/i, 'university lists'],
    [/custom (seed|client|list)|selected public|public software|specialized mailing|mailing list|nanog|mailop/i, 'hand-built list'],
    [/ipv4|address space|scans\.io|censys|zmap/i, 'IPv4 address space'],
  ];
  const c = new Map(); const residue = new Map();
  for (const p of PAPERS) {
    const seen = new Set();
    for (const s of p.population) {
      const v = s.sourceList;
      if (v === null || v === undefined) continue;
      const hit = FOLD.find(([re]) => re.test(v));
      if (hit) seen.add(hit[1]);
      else { if (!residue.has(v)) residue.set(v, new Set()); residue.get(v).add(key(p)); }
    }
    for (const f of seen) { if (!c.has(f)) c.set(f, new Set()); c.get(f).add(key(p)); }
  }
  const withPop = PAPERS.filter((p) => p.population.length > 0).length;
  console.log(`denominator: ${withPop} of ${PAPERS.length} papers record at least one population[].sourceList.\n`);
  for (const [f, s] of [...c.entries()].sort((a, b) => b[1].size - a[1].size)) console.log(`  ${String(s.size).padStart(2)}  ${f}`);
  // The frame moved, and it is datable. Printed per paper so the claim on the
  // page ("Alexa through 2022, Tranco after, zone files from 2023") is the
  // script's, not the prose's.
  console.log('\n  Which frame, by paper (only papers naming a domain-list frame):');
  let alexaLast = 0, trancoFirst = 9999, zoneFirst = 9999;
  for (const p of PAPERS) {
    const srcs = p.population.map((x) => x.sourceList).filter(Boolean);
    const a = srcs.some((v) => /alexa/i.test(v));
    const t = srcs.some((v) => /tranco/i.test(v));
    const z = srcs.some((v) => /zone file|verisign|internetstiftelsen|public interest registry/i.test(v));
    if (!(a || t || z)) continue;
    if (a) alexaLast = Math.max(alexaLast, p.year);
    if (t) trancoFirst = Math.min(trancoFirst, p.year);
    if (z) zoneFirst = Math.min(zoneFirst, p.year);
    console.log(`      ${p.year} ${p.venue.padEnd(8)} ${(a ? 'Alexa ' : '').padEnd(6)}${(t ? 'Tranco ' : '').padEnd(7)}${(z ? 'zone-files' : '').padEnd(10)}  ${p.slug.slice(0, 48)}`);
  }
  console.log(`\n      last Alexa use: ${alexaLast}; first Tranco use: ${trancoFirst}; first zone-file use: ${zoneFirst}`);
  console.log(`      papers from 2024 onwards using Alexa: ${PAPERS.filter((p) => p.year >= 2024 && p.population.some((x) => /alexa/i.test(x.sourceList || ''))).length}`);
 
  console.log(`\n  UNMAPPED RESIDUE: ${residue.size} distinct strings, printed in full --`);
  for (const [v, s] of [...residue.entries()].sort()) console.log(`      "${v}"  (${[...s].join(', ')})`);
}
 
H('E. FIGURES ON THE PAGE, with their denominators and their quote check');
{
  const groups = {};
  for (const f of FIGURES) (groups[f.g] ||= []).push(f);
  let bad = 0, byMode = {};
  for (const [g, list] of Object.entries(groups)) {
    console.log(`\n--- ${g} ---`);
    for (const f of list) {
      const p = bySlug.get(f.slug);
      if (!p) throw new Error(`FIGURES references a slug outside the population: ${f.slug}`);
      const modes = [['quote', locate(p, f.quote)]];
      if (f.quote2) modes.push(['quote2', locate(p, f.quote2)]);
      for (const [, m] of modes) {
        byMode[m] = (byMode[m] || 0) + 1;
        if (!['exact', 'whitespace-folded', 'punctuation-folded'].includes(m)) bad++;
      }
      console.log(`  ${f.what}`);
      console.log(`      value       : ${f.value}`);
      console.log(`      DENOMINATOR : ${f.pop}`);
      console.log(`      source      : ${key(p)}`);
      console.log(`      quote [${modes[0][1]}]: "${f.quote}"`);
      if (f.quote2) console.log(`      quote2 [${modes[1][1]}]: "${f.quote2}"`);
      if (f.spliced) console.log(`      SPLICE      : ${f.spliced} -- the page states this figure without presenting it as one continuous quotation`);
    }
  }
  console.log(`\n  ${FIGURES.length} figures, ${FIGURES.length + FIGURES.filter((f) => f.quote2).length} quote fragments. Location modes: ${JSON.stringify(byMode)}`);
  if (bad) console.log(`  *** ${bad} QUOTE(S) COULD NOT BE LOCATED -- do not publish these ***`);
  else console.log('  All figure quotes located in paper.cols.txt.');
}
 
H('F. WHAT THE 31 PAPERS REPORT ABOUT THEMSELVES (extraction enums, not hand maps)');
{
  const stated = (v) => v !== null && v !== undefined && !['not-stated', 'none-mentioned', 'not-applicable', 'unclear', 'unknown'].includes(v);
  const tally = (label, f) => {
    const c = new Map();
    for (const p of PAPERS) { const v = String(f(p)); c.set(v, (c.get(v) || 0) + 1); }
    console.log(`\n  ${label} (denominator ${PAPERS.length}):`);
    for (const [v, n] of [...c.entries()].sort((a, b) => b[1] - a[1])) console.log(`      ${String(n).padStart(2)}  ${v}`);
  };
  tally('ethics.reviewOutcome', (p) => p.ethics.reviewOutcome);
  {
    // The page used to say this population's ethics reporting is "worse than the
    // corpus-wide picture". It is not. Computed here so the comparison is the
    // script's rather than an impression.
    const withEthics = ALL.filter((p) => p.ethics !== null && p.ethics !== undefined);
    const corpusNone = withEthics.filter((p) => p.ethics.reviewOutcome === 'none-mentioned').length;
    const corpusApproved = withEthics.filter((p) => p.ethics.reviewOutcome === 'approved').length;
    const pageNone = PAPERS.filter((p) => p.ethics.reviewOutcome === 'none-mentioned').length;
    const pageApproved = PAPERS.filter((p) => p.ethics.reviewOutcome === 'approved').length;
    console.log(`\n    CORPUS COMPARISON (denominator: the ${withEthics.length} papers with an ethics object):`);
    console.log(`      none-mentioned: corpus ${corpusNone} (${((corpusNone / withEthics.length) * 100).toFixed(1)}%)  vs  this page ${pageNone}/${PAPERS.length} (${((pageNone / PAPERS.length) * 100).toFixed(1)}%)`);
    console.log(`      approved      : corpus ${corpusApproved} (${((corpusApproved / withEthics.length) * 100).toFixed(1)}%)  vs  this page ${pageApproved}/${PAPERS.length} (${((pageApproved / PAPERS.length) * 100).toFixed(1)}%)`);
    console.log(`      n=31 is small: do not read a few points either way as a difference.`);
  }
  tally('ethics.notifiedAffectedParties', (p) => p.ethics.notifiedAffectedParties);
  tally('artifacts.availability', (p) => p.artifacts.availability);
  console.log(`\n  papers whose artifacts.codeUrl is set: ${PAPERS.filter((p) => p.artifacts.codeUrl).length} of ${PAPERS.length}`);
  for (const p of PAPERS) if (p.artifacts.codeUrl) console.log(`      ${key(p)}\n          ${p.artifacts.codeUrl}`);
  console.log(`\n  statistics[].kind, papers (denominator ${PAPERS.length}, multi-valued):`);
  {
    const c = new Map();
    for (const p of PAPERS) for (const s of new Set(p.statistics.map((x) => x.kind))) c.set(s, (c.get(s) || 0) + 1);
    for (const [v, n] of [...c.entries()].sort((a, b) => b[1] - a[1])) console.log(`      ${String(n).padStart(2)}  ${v}`);
    console.log(`      ${String(PAPERS.filter((p) => p.statistics.length === 0).length).padStart(2)}  (no statistics tuple at all)`);
  }
  console.log(`\n  humanAnnotation: ${PAPERS.filter((p) => p.humanAnnotation.length > 0).length} of ${PAPERS.length} papers coded something by hand;`);
  console.log(`      of those, ${PAPERS.filter((p) => p.humanAnnotation.some((h) => stated(h.agreementMetric))).length} report an inter-rater agreement metric.`);
  console.log(`\n  participants: ${PAPERS.filter((p) => p.participants.length > 0).length} of ${PAPERS.length} recruited people.`);
  for (const p of PAPERS) for (const a of p.participants)
    console.log(`      ${key(p)}  n=${a.n} kind=${a.studyKind} recruitment=${a.recruitment}`);
}
 
H('G. FULL-TEXT PROBES: mechanisms the corpus does NOT measure');
{
  const PROBES = {
    BIMI: /\bBIMI\b|Brand Indicators for Message/g,
    'TLS-RPT': /TLS-RPT|TLSRPT|\bsmtp-tls-report/gi,
    REQUIRETLS: /REQUIRETLS/g,
    'DANE (any)': /\bTLSA\b|\bDANE\b/g,
    'MTA-STS': /MTA-STS/g,
    ARC: /Authenticated Received Chain|\bARC (chain|set|seal|header)/g,
  };
  let nofile = 0;
  const hits = {};
  for (const k of Object.keys(PROBES)) hits[k] = [];
  for (const p of ALL) {
    const t = colsText(p);
    if (t === null) { nofile++; continue; }
    const flat = t.replace(/\s+/g, ' ');
    for (const [k, re] of Object.entries(PROBES)) { re.lastIndex = 0; const m = flat.match(re); if (m) hits[k].push({ k: key(p), n: m.length, infra: INFRA_SLUGS.has(p.slug) }); }
  }
  console.log(`probe population: all ${ALL.length} extracted papers; ${nofile} have no paper.cols.txt and count as negative.\n`);
  for (const [k, list] of Object.entries(hits)) {
    const inf = list.filter((x) => x.infra).length;
    console.log(`  ${k.padEnd(12)} mentioned in ${String(list.length).padStart(4)} papers corpus-wide; ${inf} of them in this page's 31`);
  }
  {
    // The page claimed "every paper in this population cites the Informational
    // RFC 7489". The generic reviewer falsified it by grepping. Computed here so
    // the corrected sentence is the script's.
    let cites = 0; const noRfc = [];
    for (const p of PAPERS) {
      const t = colsText(p);
      if (t === null) { noRfc.push([key(p), 'NO .cols FILE']); continue; }
      if (t.includes('7489')) cites++;
      else noRfc.push([key(p), t.includes('DMARC') ? 'names DMARC, not the RFC' : 'never mentions DMARC']);
    }
    console.log(`\n  RFC 7489 (the Informational DMARC RFC that RFC 9989 obsoleted in May 2026):`);
    console.log(`    cited by ${cites} of the ${PAPERS.length}; the other ${noRfc.length} do not --`);
    for (const [k, why] of noRfc) console.log(`      ${why.padEnd(26)} ${k}`);
  }
  console.log('\n  BIMI, in full -- every corpus paper that names it:');
  for (const x of hits.BIMI.sort((a, b) => b.n - a.n)) console.log(`      ${String(x.n).padStart(2)}x  ${x.infra ? '[in the 31]' : '           '}  ${x.k}`);
  console.log('\n  A mention is not a measurement. Hand-read 2026-09-09: all five BIMI mentions are');
  console.log('  related-work or background sentences ("BIMI is built on DMARC and has not ...").');
  console.log('  No paper in these seven venues measures BIMI deployment. REQUIRETLS: 0 mentions.');
}
 
H('H. RECALL: is the 31 missing anything?');
{
  const PATS = {
    spf: /\bSPF\b|Sender Policy Framework/g, dkim: /\bDKIM\b|DomainKeys/g, dmarc: /\bDMARC\b/g,
    starttls: /STARTTLS|\bSTART ?TLS\b/g, dane: /\bTLSA\b|\bDANE\b/g, mtasts: /MTA-STS|TLS-RPT|TLSRPT/g,
    smime: /S\/MIME|SMIMEA|OPENPGPKEY/g, smtp: /\bSMTP\b|\bMTA\b|\bMX record/g,
    bimi: /\bBIMI\b|Brand Indicators/g, arc: /Authenticated Received Chain|\bARC (chain|set|seal)/g,
  };
  const mapped = new Set(Object.keys(MSG_MAP));
  const run = (name, admit) => {
    const out = [];
    for (const p of ALL) {
      const t = colsText(p); if (t === null) continue;
      const flat = t.replace(/\s+/g, ' ');
      const c = {}; for (const [k, re] of Object.entries(PATS)) { re.lastIndex = 0; const m = flat.match(re); c[k] = m ? m.length : 0; }
      const mech = c.spf + c.dkim + c.dmarc + c.dane + c.mtasts + c.bimi + c.arc;
      const tls = c.starttls + c.smime;
      if (admit(mech, tls, c.smtp)) out.push({ k: key(p), slug: p.slug, title: p.title, mech, tls, smtp: c.smtp, c });
    }
    const infra = out.filter((o) => INFRA_SLUGS.has(o.slug)).length;
    const other = out.filter((o) => mapped.has(o.slug) && !INFRA_SLUGS.has(o.slug));
    const un = out.filter((o) => !mapped.has(o.slug));
    console.log(`\n  ${name}: ${out.length} papers hit | ${infra}/31 of the population | ${other.length} already mapped to another verdict | ${un.length} unmapped`);
    return { out, other, un };
  };
  const tight = run('TIGHT  (mech>=10 or tls>=10 or (smtp>=30 and mech>=3))', (m, t, s) => m >= 10 || t >= 10 || (s >= 30 && m >= 3));
  const wide = run('WIDE   (mech>=4  or tls>=4  or (smtp>=15 and mech>=2))', (m, t, s) => m >= 4 || t >= 4 || (s >= 15 && m >= 2));
  const missed = [...INFRA_SLUGS].filter((s) => !tight.out.some((o) => o.slug === s));
  console.log(`\n  Population papers the TIGHT probe misses: ${missed.length} -- ${JSON.stringify(missed)}`);
  console.log(`  (The probe is a recall check on the hand map, not the population rule. It cannot`);
  console.log(`   find email-typosquatting, whose subject is misdirected mail and which names no`);
  console.log(`   mechanism; the hand map can. That is the argument for the hand map.)`);
  console.log('\n  Every UNMAPPED paper the WIDE probe admits, read by hand 2026-09-09:');
  for (const o of wide.un.sort((a, b) => (b.mech + b.tls) - (a.mech + a.tls)))
    console.log(`      ${String(o.mech).padStart(3)}m ${String(o.tls).padStart(3)}t  ${o.k}\n            ${o.title}`);
  console.log('\n  VERDICT: none of them measures deployed email authentication or transport security.');
  console.log('  They are DNS, PKI and TLS papers that name SPF or STARTTLS in passing (DANE for');
  console.log('  HTTPS/CDN, DNSSEC registrar studies, ALPACA, DROWN, a DNS toolkit paper).');
  console.log('  Widening the probe from TIGHT to WIDE added 0 papers to the population.');
  console.log('\n  Papers the probe hits that msg_fold gave a DIFFERENT verdict (boundary cases):');
  for (const o of wide.other.sort((a, b) => (b.mech + b.tls) - (a.mech + a.tls)))
    console.log(`      ${String(o.mech).padStart(3)}m ${String(o.tls).padStart(3)}t  [${MSG_MAP[o.slug][0]}]  ${o.k}`);
}
 
H('I. TOOLS NAMED AS USED (paper-counted, folded, residue printed)');
{
  const FOLD = [
    [/^postfix$/i, 'Postfix'], [/^exim$/i, 'Exim'], [/^sendmail$/i, 'Sendmail'], [/^qmail$/i, 'qmail'],
    [/^dovecot$/i, 'Dovecot'], [/microsoft exchange/i, 'Microsoft Exchange'],
    [/^z?map$/i, 'ZMap'], [/^zgrab/i, 'ZGrab2'], [/^xmap$/i, 'XMap'], [/^censys$/i, 'Censys'],
    [/^unbound$/i, 'Unbound'], [/^bind9?$/i, 'BIND'], [/openintel/i, 'OpenINTEL'],
    [/farsight|\bsie\b/i, 'Farsight SIE passive DNS'], [/^checkdmarc$/i, 'checkdmarc'],
    [/libspf2|pyspf|mail::spf|policyd-spf|spfmilter|python-postfix-policyd-spf/i, 'an SPF validator library'],
    [/^opendkim$|^opendmarc$|^openarc$/i, 'OpenDKIM / OpenDMARC / OpenARC'],
    [/openssl|pyopenssl/i, 'OpenSSL / pyOpenSSL'], [/gnupg|^gpg$|gpgme|enigmail|gpg suite|openkeychain|openpgp\.js/i, 'a PGP implementation'],
    [/^pkilint$/i, 'pkilint'], [/badkeys|fastgcd|factordb|pwnedkeys|hashcat/i, 'a weak-key checker'],
    [/^dnstwist$/i, 'dnstwist'], [/spamassassin|spamhaus|spamcop|rspamd|milter-greylist/i, 'a spam/blocklist service or filter'],
  ];
  const c = new Map(); const residue = new Map();
  for (const p of PAPERS) {
    const seen = new Set();
    for (const t of p.tools) {
      if (t.usedOrMentioned !== 'used') continue;
      const hit = FOLD.find(([re]) => re.test(t.name));
      if (hit) seen.add(hit[1]);
      else { if (!residue.has(t.name)) residue.set(t.name, new Set()); residue.get(t.name).add(key(p)); }
    }
    for (const f of seen) { if (!c.has(f)) c.set(f, new Set()); c.get(f).add(key(p)); }
  }
  console.log(`denominator: ${PAPERS.filter((p) => p.tools.some((t) => t.usedOrMentioned === 'used')).length} of ${PAPERS.length} papers name at least one tool as used.\n`);
  for (const [f, s] of [...c.entries()].sort((a, b) => b[1].size - a[1].size)) console.log(`  ${String(s.size).padStart(2)}  ${f}`);
  console.log(`\n  UNMAPPED RESIDUE: ${residue.size} distinct strings across ${new Set([...residue.values()].flatMap((s) => [...s])).size} papers.`);
  console.log('  Printed in full because a page that folds must show what it could not fold:');
  const cols = [...residue.keys()].sort((a, b) => a.toLowerCase().localeCompare(b.toLowerCase()));
  for (let i = 0; i < cols.length; i += 3) console.log('      ' + cols.slice(i, i + 3).map((x) => x.padEnd(34)).join(''));
}
 
H('J. QUOTE SPOT-CHECK OF THE EXTRACTION ITSELF');
{
  // Every detection[].evidence.quote in the 31, located in paper.cols.txt.
  let n = 0; const modes = {}; const bad = [];
  for (const p of PAPERS) for (const d of p.detection) {
    n++; const m = locate(p, d.evidence.quote); modes[m] = (modes[m] || 0) + 1;
    if (!['exact','whitespace-folded','punctuation-folded'].includes(m)) bad.push(`${key(p)} [${d.evidence.section}] "${d.evidence.quote}"`);
  }
  console.log(`${n} detection[].evidence.quote values across the ${PAPERS.length} papers.`);
  console.log(`location modes: ${JSON.stringify(modes)}`);
  console.log(`unlocatable: ${bad.length} (${((bad.length / n) * 100).toFixed(1)}%) --`);
  for (const b of bad) console.log(`    ${b}`);
}
 
H('K. DONE');
console.log(`figures published: ${FIGURES.length}`);
console.log(`population       : ${PAPERS.length} papers`);
console.log(`generated        : run this file, do not retype its numbers`);