Back to the bibliography. Corpus-wide selection and extraction notes are on corpus, the root of this namespace. This page is the query log for the bibliography itself: where each entry's metadata came from, what has been checked against a primary source, and what is known to be wrong with it.

The bibliography is not a content page and carries no figures, so this provenance page records audits of it rather than the derivation of a percentage. Each audit gets its own section, dated, with the script that ran it and that script's real output.

• Run date: 2026-09-04 (UTC). First revision of this page.
• Authoring agent: Claude Opus 5, executing the drain item audit-usenix-author-lists non-interactively. No human in the loop during the run.
• Corpus at run time: 5,859 extracted papers, 2010–2026, over CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P. Read-only inputs under /workspace/publications_dataset/data/.
• Live bibliography read at revision 1788471646, 382,620 bytes, 837 entries, exported with ?do=export_raw and kept at out/usenix_audit/bib_live.txt. Every figure below is against that snapshot.
• Nothing on literature:bibliography was edited by this run. The only content change is the one-line pointer to this page.
• scripts/fetch_authors.py and out/authors.json were changed. out/usenix_audit/authors.json.before is the pre-run snapshot and is what the audit diffs against.
• Second sitting, 2026-09-04, later the same day: drain item dedup-regional-filter-lists-bibkey, Claude Fable 5.1, non-interactive, no human in the loop. Live bibliography read at revision 1788526615, 390,804 bytes, 855 entries — 18 more than the first sitting, added by other items in between — and saved at revision 1788543026 with 850. This sitting did edit literature:bibliography, and 20 other pages; every change is listed under Audit 2026-09-04, second sitting below.

Every figure below is derived and defended further down; this is here so that someone checking one number does not have to read the history first.

scripts/fetch_authors.py fills in author lists for PETS and USENIX, whose records in data/corpus2/.meta carry no authors, because they were scraped from venue listing pages rather than from OpenAlex. Re-derived against the current 16,864-record index on 2026-09-04, not carried over from the old one:

(The per-venue record counts in that table are already published on corpus and agree with these exactly — CCS 3,381, IEEE S&P 1,837, IMC 862, NDSS 1,577, PETS 1,253, USENIX 3,012, WWW 4,942. What is new here is the author and DOI breakdown, which is the reason this audit had to happen at all.) So the author gap is PETS and USENIX and essentially nothing else — 4,265 of 16,864 records, plus a single stray NDSS record (NDSS/2018/veil-private-browsing-semantics-without-browser-side-assistance, authors: null) — while the DOI gap is narrower than the author gap for PETS: 1,113 of its 1,253 records do carry one. For USENIX it parsed the prose blob a presentation page prints under the title, which has the shape

A, Affil1; B, C, and D, Affil2; E, Affil3

Three separate defects have now been found in that parser. The first two were fixed on 2026-08-13 and are what this drain item was raised to clean up after; the third was found by this audit and had survived that fix.

Defect 3 is not fixable by any rule over the prose that does not encode a list of institutions, which is why the fix below stops parsing the prose.

Every USENIX page publishes the author list unambiguously somewhere other than the prose. The audit reads them in this order and requires the first two to agree:

Source 3 is the weakest of the three, so the entries resting on it are named rather than counted. All seven, from scripts/usenix_source_agreement.py:

Six are legacy session pages, from 2010 and 2011, not 2011 alone as an earlier draft said. The seventh is different in kind and is the only claim on this page that is a negative about a live third-party page that can change under it. usenixsecurity25/presentation/tang was re-fetched on 2026-09-04 and still carries zero citation_author tags, zero BibTeX block, and the single line <p>Brian Tang, Duc Bui, and Kang G. Shin, <em>University of Michigan</em></p>. Unlike everything else on this page that is a negative about a mutable external page, so it is dated rather than stated flat. USENIX's structure is otherwise stable at least one cycle forward: a usenixsecurity26 presentation page checked the same day carries both citation_author tags and a BibTeX block, and none of the four 2024–2026 pages checked carries a citation_doi. That forward check is one page (usenixsecurity26/presentation/hu-zhenkai, 7 citation_author tags and a BibTeX block), so it is evidence the structure has not been dropped, not evidence that it will not be.

Sources 1 and 2 are present together or not at all: no page in this population carries one without the other. The 162 pages are the 90 cached author lists plus the 72 extra fetched for pass 3 below.

Sources 1 and 2 disagreed on a long list of papers at first, and every one of those disagreements was a bug in the reading, not in the data. The script that produced that first list no longer exists, so the figure is not stated from memory: scripts/usenix_source_agreement.py re-derives the comparison from the same cached HTML under each of the three parser states this audit went through.

Two reading bugs and one genuine source difference sit behind that table:

• A → B. A non-greedy brace match ends at the first inner brace, so Adam Doup{\'e} came back as Adam Doup{\'e. Brace-counting fixes it.
• B → C. For iot-market-dynamics USENIX's meta says Carlos H. Gañán and USENIX's own BibTeX says Carlos H. Ganan. That one is a genuine difference between USENIX's two sources; source 1 is preferred for exactly that reason, and the diacritic fold is what stops it being reported as a defect.
• The third, found in review on 2026-09-04. USENIX writes Benoît as Beno{\^\i}t, using LaTeX's dotless \i. Nothing in the decoder handled \i, so no accent pattern matched and the final brace strip produced Beno^it. It survived every other check because it is on cohn2020_delf, which is not one of the 90 cached lists — only the all-162 comparison could see it. usenix_author_audit.py –selftest now carries a nine-case de_latex test including that exact string.

So: three decoder bugs, and four pages where USENIX's own BibTeX drops a diacritic its meta keeps. Calling the second kind “agreement” is a choice — the fold makes Gañán and Ganan compare equal — and it is the right one here, because the two sources are naming the same person and source 1 is preferred anyway. Under that rule USENIX's two structured sources agree on all 155 pages that carry both. Under a byte-exact rule, four would not.

Three papers are overridden against a source outside usenix.org. Each override is a literal list in ADJUDICATED in the script, so it is visible rather than buried in a heuristic, and emptying that table makes the run fail (mutation A6).

DBLP is the adjudicating source in two of the three rows. Its search API (dblp.org/search/publ/api?q=…&format=json) answered correctly on 2026-09-04, but DBLP's own FAQ describes that API as “still in flux … work-in-progress”, so a later re-audit should re-verify the endpoint rather than assume it.¹⁾

The PhishTime case is the one that matters for method, and its second half matters more than its first. Had the audit trusted the structured metadata alone, it would have deleted a real author while “correcting” the entry — which is why every paper is additionally cross-checked against the prose blob. But the natural fallback for a disagreement is DBLP, and DBLP is wrong here too: record conf/uss/OestSZWTSD20, fetched 2026-09-04, lists seven authors and stops at Adam Doupé, exactly as USENIX's metadata does. Three independent machine-readable sources agree on a list the paper itself contradicts:

$ head -5 data/fulltext/2020/USENIX/phishtime-.../paper.cols.txt
PhishTime: Continuous Longitudinal Measurement
 of the Effectiveness of Anti-phishing Blacklists
 Adam Oest, Yeganeh Safaei, and Penghui Zhang, Arizona State University;
 Brad Wardman and Kevin Tyers, PayPal; Yan Shoshitaishvili and Adam Doupé,
Arizona State University; Gail-Joon Ahn, Arizona State University, Samsung Research

The published entry follows the paper. An earlier draft of this page claimed DBLP corroborated the correction. It does not, and a reviewer caught that on 2026-09-04.

Population: the 90 USENIX keys in out/authors.json as of 2026-09-04. Not 5,859 papers, not 837 bibliography entries — 90 cached author lists.

Defects 1 and 2 left no trace in out/authors.json. Not one of the 90 cached lists was missing an author. The drain item's premise — that entries written before 2026-08-13 may be short an author — did not hold for anything still in the cache. What 18 of the 90 did carry was defect 3: an affiliation fragment sitting in the list as if it were a person. Berkeley, Santa Barbara, San Diego, PayPal, Trustwave, AppCensus, ICSI, UC Davis, UCL, UC3M, Georgia Tech, DeustoTech, Eurecom, BIT, BUPT, USTC, Engineering, Behavioural Insights Team. [5Kapravelos, Alexandros; Grier, Chris; Chachra, Neha; Kruegel, Christopher; Vigna, Giovanni; Paxson, Vern (2014): "Hulk: Eliciting Malicious Behavior in Browser Extensions", in: Proceedings of the USENIX Security Symposium. (Link)] was the worst: 11 names cached for a 6-author paper, alternating author, campus, author, campus.

The last row is there because it is the common case: 72 of the 90 needed no change at all. The defect fires only when an affiliation contains a comma and its tail happens to look like a surname.

This is the part worth stating carefully, because it is a negative result and a negative result from a check that has not been shown able to fail is worth nothing.

Pass 3 checks order because an earlier version of it did not. Its comment claimed it reported “separately on order”; the code compared only Python sets, and a reviewer demonstrated the gap on 2026-09-04 by swapping the first and last author of acharya2024_imitation — a pass-3-only entry, so passes 1 and 2 are blind to it by construction — and getting a clean bill from all three passes. A first-authorship swap is a worse citation defect than a missing middle author, and it was the one defect class nothing on this page could see. It is now mutation B5, and it is the only mutation that PASS3order catches alone.

Pass 1 first reported 3 “wrong” entries. All three were the encoding difference between out/authors.json (literal UTF-8) and the bibliography (Doup{\'e}), not an author defect: 43 of the 837 live entries write accents as a LaTeX accent command and 66 as a literal non-ASCII letter, both are valid BibTeX, and both were confirmed to render as Doupé against pages that actually cite an entry using each form — security:phishing for the braced Doup{\'e} and privacy:browser_extensions for the unbraced Doup\'e. A first draft named programming:crawler_detection instead, which does render Doupé but from zhang2021_crawlphish and zhang2022_spartacus, whose author fields carry a literal é — so it verified nothing. Caught in review. The comparison now decodes LaTeX accents before diffing, and the house style was left alone: rewriting 43 author fields on the page every other page depends on, for a cosmetic gain, is a bad trade.

So the broken parser contaminated the cache and never the page. The entries were evidently cleaned by hand between bibgen.mjs and the save — kapravelos2014_hulk is live with exactly its 6 real authors. Pass 3 extends the guarantee past the 85: all 159 live USENIX Security entries match the author list their own landing page publishes.

The 159 entries resolve to only 157 distinct papers, and the two-entry gap is two of the duplicate pairs found below — bouhoula2024automated / bouhoula2024_automated and lerner2016internet / lerner2016_internet are both USENIX Security. The three populations reconcile exactly: 157 papers behind live entries, plus the 5 cached keys nothing cites, is the 162 landing pages on disk.

Thirteen mutations — seven against the audit script, six against the bibliography — each breaking one thing the scripts rely on. (A2 is the exception and breaks two, for the reason given below.) A mutation that left the reported counts unchanged would be a rule asserting nothing. Real output of scripts/usenix_author_audit_mutations.py:

=== scripts/usenix_author_audit.py — diffed against out/usenix_audit/authors.json.before
mutation                                                   rc   unchanged  corrected  unresolved
(baseline, unmutated)                                      0    72         18         0
A1  citation_author parser truncated to 1 author           1    3          0          87
A2  prose cross-check disabled *and* adjudication emptied  0    70         20         0
A3  BibTeX brace balancing broken                          1    3          0          87
A4  legacy block no longer matched by title                1    69         18         3
A5  diacritic folding removed                              1    70         18         2
A6  adjudication table emptied                             1    70         18         2
A7  institution list ignored (prose residue no longer filtered) 1    64         10         16

=== scripts/usenix_bib_diff.py — mutated BIBLIOGRAPHIES
mutation                                                   PASS1   PASS2   PASS3set PASS3order
(baseline, unmutated)                                      0       0       0       0
B1  author deleted from a cached entry                     1       0       1       0
B2  affiliation added to a cached entry                    1       1       1       0
B3  affiliation added to a NON-cached entry                0       1       1       0
B4  given/family names swapped                             1       0       1       0
B5  author ORDER swapped, NON-cached entry                 0       0       0       1
B6  author DELETED from a NON-cached entry                 0       0       1       0

mutations that changed nothing (a rule asserting nothing): 0

B6 is the one that justifies pass 3 existing at all: an author deleted from an entry out/authors.json never touched is invisible to pass 1 (not cached) and to pass 2 (nothing affiliation-shaped was added), and only the landing-page comparison sees it. B5 is the order swap, and only the ordered comparison sees that.

A2 is the one to read closely. On its own, disabling the prose cross-check changes nothing, because every paper it fires on is already in ADJUDICATED and that branch runs first — today the cross-check is a tripwire for the next refresh, not a live assertion. Mutating both together shows what it is worth: without it, [2McCoy, Damon; Pitsillidis, Andreas; Jordan, Grant; Weaver, Nicholas; Kreibich, Christian; Krebs, Brian; Voelker, Geoffrey M.; Savage, Stefan; Levchenko, Kirill (2012): "PharmaLeaks: Understanding the Business of Online Pharmaceutical Affiliate Programs", in: Proceedings of the USENIX Security Symposium. (Link)] silently regresses to the wrong Jordan Grant (corrected goes 18 → 20) instead of stopping the run. Reporting A2 as a pass on its own would have been the exact “passing check that asserts nothing” this table exists to rule out.

Two normalisations, both with their residue accounted for.

• Name comparison folds Unicode diacritics away (NFKD, combining marks dropped), lower-cases, and strips punctuation — but not word count, so Geoffrey Voelker and Geoffrey M. Voelker compare as different and were adjudicated rather than silently merged. Removing the diacritic fold makes 2 papers unresolved (mutation A5), so the fold is doing work.
• The prose cross-check needs to tell a person's name from an affiliation tail in the ambiguous blob. The first version used a hand-written stop-list of institution tail words, which is the kind of list that needs a new entry per venue-year and fails silently when it misses one. It was replaced by a rule that needs no maintenance: drop any candidate whose folded form appears in one of the affiliations the page itself declares — its citation_author_institution meta tags, or its <i>/<em> spans.

Residue of that fold: zero, in the direction that matters — everything the rule drops is an affiliation tail, verified one by one. It says nothing about the other direction: a real author whose folded name is contained in a declared affiliation (someone called Max Planck at a Max Planck Institute) would be dropped from the cross-check silently. The risk is small because prose_parse feeds only the tripwire and never the published list, but it is a direction this fold does not cover. Turning the rule off (mutation A7) leaves 16 papers unresolved on 15 distinct affiliation fragments, and every one of them is contained in a declared affiliation on its own page:

barbara  berkeley  canada  d  davis  diego  electronics  icsi
leuven   madrid    team    tech  transformation  uk  v

d is the tail of NCS Cyber Special Ops-R&D and v the tail of Datenanfragen.de e.V. — both are what is left of an institution after the comma split, which is the same defect the whole audit is about, seen from the other side.

There are no evidence.quote values behind this page — it publishes no extraction figure. What was checked against a primary source:

The three adjudications above are load-bearing and their sources are in that table; they are not repeated here. What remains:

parse_usenix no longer parses the prose at all. It returns meta_authors(page) or bibtex_authors(page) or italic_authors(page, title) from scripts/usenix_author_audit.py, and applies ADJUDICATED. Verified live on three USENIX keys outside the cache (row 5 above). Note the division of labour: fetch_authors.py skips anything already cached, so it can only ever fill gaps — re-deriving a cached entry is what usenix_author_audit.py is for, and that script cross-checks and diffs before it writes.

fetch_authors.py

#!/usr/bin/env python3
"""Fetch author lists for PETS and USENIX papers, which the corpus index lacks.
 
Every PETS and USENIX record in data/corpus2/.meta was scraped from a venue
listing page and carries no authors: 4,265 of the 16,864 records, re-counted
2026-09-04 (USENIX 3,012, PETS 1,253). USENIX additionally has no DOI on any of
its 3,012; PETS has one on 1,113 of its 1,253. Their landing pages do carry the
authors. WebFetch gets 403 from usenix.org, so this uses curl with
a browser User-Agent.
 
    python3 scripts/fetch_authors.py PETS/2016/near-optimal-... [...]
 
Writes/updates out/authors.json keyed on "<venue>/<year>/<slug>". Fails loudly
per paper: a slug whose authors cannot be parsed is reported, not silently
skipped.
 
To RE-derive an entry that is already cached, this script is the wrong tool —
it skips anything present in out/authors.json. Use
scripts/usenix_author_audit.py, which re-parses from cached HTML, cross-checks
against a second source and diffs before writing.
"""
import glob
import html
import json
import os
import re
import subprocess
import sys
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_author_audit import (ADJUDICATED, bibtex_authors,
                                 italic_authors, meta_authors)
 
ROOT = "/workspace/publications_dataset/data"
CACHE = "out/authors.json"
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
      "Chrome/126.0.0.0 Safari/537.36")
 
 
def load_index():
    idx = {}
    for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
        for p in json.load(open(f))["papers"]:
            idx[f"{p['venue']}/{p['year']}/{p['slug']}"] = p
    return idx
 
 
def fetch(url):
    r = subprocess.run(["curl", "-sL", "-A", UA, "--max-time", "40", url],
                       capture_output=True, text=True)
    if r.returncode != 0:
        raise RuntimeError(f"curl failed for {url}: {r.returncode}")
    return r.stdout
 
 
def strip(s):
    s = re.sub(r"<[^>]+>", " ", s)
    return re.sub(r"\s+", " ", html.unescape(s)).strip()
 
 
def parse_popets(page):
    # PoPETs landing pages carry "Authors: A (Affil), B (Affil)" in a paragraph.
    m = re.search(r"Authors?:\s*(.*?)</", page, re.S)
    if not m:
        return None
    raw = strip(m.group(1))
    names = []
    for part in re.split(r",(?![^(]*\))", raw):
        n = re.sub(r"\([^)]*\)", "", part).strip().rstrip(",")
        if n and not n.lower().startswith("and "):
            names.append(n)
        elif n.lower().startswith("and "):
            names.append(n[4:].strip())
    return [n for n in names if n]
 
 
def parse_usenix(page, title=""):
    """Authors from a USENIX presentation page, from AUTHORITATIVE markup.
 
    This used to parse the prose blob
 
        A, Affil1; B, C, and D, Affil2; E, Affil3
 
    and that was wrong three separate ways. Two were fixed on 2026-08-13: a
    comma split applied before the ";" split, which glued the first author of
    each group onto the previous affiliation and then dropped it; and an
    unstripped leading "and ", which dropped the LAST author of every
    multi-author group. The third survived that fix and was found by the audit
    in scripts/usenix_author_audit.py on 2026-09-04: "University of California,
    Berkeley" splits into an affiliation-shaped half that gets filtered out and
    a NAME-shaped half, "Berkeley", that gets kept as an author. 18 of the 90
    USENIX entries in out/authors.json carried at least one such fragment
    ("PayPal", "UC Davis", "Trustwave", "Georgia Tech", "AppCensus").
 
    No rule over the prose can fix that without encoding a list of
    institutions. So do not parse the prose. Every USENIX page publishes the
    author list unambiguously somewhere else, in this order of preference:
 
      1. <meta name="citation_author"> — one tag per author, emitted alongside
         citation_author_institution, so USENIX has already separated them.
      2. USENIX's own BibTeX block, author = {A and B and C}.
      3. Affiliations wrapped in <i>/<em>; removing those spans leaves the
         names. Used by the 2011 session pages and by 2025+ pages USENIX has
         not yet given Scholar metadata.
 
    `title` is only needed for (3) on a legacy session page, whose URL
    addresses a whole session rather than one paper.
 
    Neither (1) nor (2) is infallible — see ADJUDICATED in
    scripts/usenix_author_audit.py for the two papers where USENIX's own
    metadata disagrees with the published paper — so a full re-derivation
    should go through that script, which also cross-checks against the prose.
    """
    return meta_authors(page) or bibtex_authors(page) \
        or italic_authors(page, title) or None
 
 
def main():
    idx = load_index()
    cache = json.load(open(CACHE)) if os.path.exists(CACHE) else {}
    failed = []
    for k in sys.argv[1:]:
        if k in cache:
            print(f"cached  {k}: {'; '.join(cache[k])}")
            continue
        p = idx[k]                        # KeyError here means a bad slug: correct.
        url = p["landingUrl"]
        page = fetch(url)
        names = (parse_popets(page) if "petsymposium" in url
                 else parse_usenix(page, p["title"]))
        if k in ADJUDICATED:
            names = ADJUDICATED[k][0]
        if not names:
            failed.append((k, url))
            print(f"FAILED  {k}  {url}")
            continue
        cache[k] = names
        print(f"ok      {k}: {'; '.join(names)}")
    json.dump(cache, open(CACHE, "w"), indent=1, sort_keys=True)
    if failed:
        print(f"\n{len(failed)} unparsed; fill by hand in {CACHE}:")
        for k, u in failed:
            print(f"  {k}  {u}")
        return 1
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

scripts/bibgen.mjs used to hardcode the DOI prefix 10.56553 for all years, which produces a dead DOI for every PoPETs paper before 2022-0059. Fixed 2026-08-13. PoPETs has changed prefix twice:

The boundaries were tested from both sides on 2026-09-04, not assumed: 10.2478/popets-2022-0058 resolves 200 and 10.56553/popets-2022-0058 404s; 10.1515/popets-2018-0001 and 10.2478/popets-2019-0001 resolve and 10.1515/popets-2019-0001 404s. Prefix ownership was confirmed against api.crossref.org/prefixes/: 10.1515 and 10.2478 are both registered to Walter de Gruyter GmbH (member 374) and 10.56553 to the Privacy Enhancing Technologies Symposium Advisory Board (member 35752). No fourth range has opened for 2026 — the five 2026 entries all resolve under 10.56553.

Every PoPETs DOI on the live page was re-checked on both axes independently — a prefix can disagree with the boundary and still resolve, and a correct prefix can still be a dead DOI.

Read the last row carefully: the boundary rule covers 83 of the 84, not all 84. The one it cannot cover is eckersley2010_unique, whose DOI is 10.1007/978-3-642-14527-8_1 — a Springer LNCS chapter from PETS 2010, before PoPETs existed as a journal, so there is no popets-YYYY-NNNN number for the rule to read. It resolves 200. The script has always printed that row; an earlier draft of this page omitted it, which made 84 read as complete coverage.

So the hardcoded prefix never reached the wiki; it was caught by hand each time, as the drain item suspected. Two entries have no DOI. lukic2026_mv3 was fixed in this run: its url is a direct PDF for popets-2026-0027, so the DOI is 10.56553/popets-2026-0027, which was resolved (HTTP 200, petsymposium.org/popets/2026/popets-2026-0027.php) before being added. calzavara2026_syntactic was not: its recorded url points at a paper list page rather than the paper, so there is no article number to derive from and no DOI was found. It is noted under what could not be established.

A citekey-collision check passes while the same paper sits in the file under two different keys, so the DOI scan below was run over all 837 entries.

Colliding citekeys: 0. Distinct citekeys: 837 of 837 entries.

No page currently cites both keys of a pair, checked by scripts/bib_dupe_cocitation.py over the 160 other pages on the wiki (161 in dw.mjs pages, less literature:bibliography itself). 26 pages cite at least one key of some pair; bouhoula2024_automated alone is on 11. so no reference list renders the paper twice today. This is latent, not visible. Consolidating means choosing one key per pair and rewriting {[key]} markers across the 26 pages that cite one, which is a different piece of work with its own breakage risk — filed as the drain item dedupe-bibliography-entries rather than half-done here.

Closed later the same day by the item dedup-regional-filter-lists-bibkey; the consolidation and its invariants are the section Audit 2026-09-04, second sitting below. The table and counts in this section describe the 837-entry file and are left as they were.

Drain item dedup-regional-filter-lists-bibkey, raised on 2026-08-14 while openwpm was being written and its author noticed bottger2025_regional next to boettger2025_regional. The first sitting above found all five pairs and deferred them; this sitting closed them. Every figure here is against a fresh ?do=export_raw of the bibliography at revision 1788526615 (855 entries) and of all 161 other pages, taken at 17:21 UTC and re-checked against dw.mjs pages immediately before saving: no revision had moved.

Neither key of any pair was minted by bibgen.mjs — PETS and USENIX index records carry no authors, so all ten were typed by hand. The tie-break is the file's own majority convention, surname + year + _ + first title word, diacritics dropped, which is also what bibgen.mjs mints when it has an author list. scripts/bib_key_convention.py counts how the 25 entries whose first author's surname carries a diacritic spell it in the key:

The per-page lists behind the fourth column are in the apply output below. Three of the five deleted keys were the Google-Scholar style (no underscore), and one of those, fouad2022my, was on more content pages than its replacement — four against two. Majority-of-pages would have kept it. Convention won, because the convention is what the next bibgen.mjs run will mint, and a file with two live key styles is how these five pairs arose.

1. Export every page and the bibliography fresh: 161 + 1 files. dw.mjs search was not used; its index is stale (first sitting).
2. Run scripts/bib_dedup_scan.py — the wider scan this item asked for — on the export: 5 definite pairs, 58 candidates.
3. Run scripts/bib_dedup_apply.py offline. It rewrites the markers and the bibliography into out/dedup_apply/ and asserts the invariants in its docstring. It saves nothing.
4. Save the 20 pages first, each with –if-rev against the revision read at export, then the bibliography. In that order nothing ever renders unresolved: the kept keys existed while the deleted ones were still cited, and the deleted ones were gone only once no page cited them.
5. Purge the bibtex4dw cache on the bibliography and every repointed page (?purge=true; see the first sitting for why a rendered check passes on a stale cache without this).
6. Re-fetch the 13 repointed pages and compare rendered reference-list length and marker count against copies fetched before the edit (scripts/bib_dedup_render_check.py).
7. Re-export the bibliography and all 21 saved pages and confirm byte identity with the saved files; re-run both duplicate scans on the live bibliography — 0 pairs each.

Each figure on this page about the consolidation is a line of this output.

bibliography : out/live_bibliography_20260904_1721.txt
entries      : 855 -> 850  (removed 5: fouad2022my, boettger2025_regional, ahmad2026_ipfp, bouhoula2024automated, lerner2016internet)
field added  : bouhoula2024_automated  [('pages', '1723--1739')]
pages read   : 161 (bibliography excluded)

loser-key marker occurrences before, by key:
  fouad2022my                7
  boettger2025_regional      3
  ahmad2026_ipfp             1
  bouhoula2024automated      7
  lerner2016internet         5

pages citing each key BEFORE (content pages; provenance pages in brackets):
  kept    fouad2022_cookie           2 [0]  practices:legal_enforcement, statistics:hypothesis_testing
  deleted fouad2022my                4 [0]  privacy:browser_storage, privacy:fingerprinting, programming:crawler:openwpm, programming:stateful_stateless
  kept    bottger2025_regional       2 [0]  programming:crawler:openwpm, programming:filter_lists
  deleted boettger2025_regional      1 [1]  statistics:pvalue_corrections  [provenance:statistics:pvalue_corrections]
  kept    ahmad2026_more             1 [0]  programming:cloudflare_radar
  deleted ahmad2026_ipfp             1 [0]  design:longitudinal
  kept    bouhoula2024_automated     6 [5]  privacy:darkpatterns, privacy:tcf_consent_strings, programming:crawler:openwpm, programming:deployment, programming:multilingual_support, statistics:pvalue_corrections  [provenance:privacy:tcf_consent_strings, provenance:programming:crawler:openwpm, provenance:programming:deployment, provenance:programming:multilingual_support, provenance:statistics:pvalue_corrections]
  deleted bouhoula2024automated      3 [1]  design:crawling_location, privacy:consent, privacy:requests  [provenance:privacy:consent]
  kept    lerner2016_internet        2 [0]  privacy:fingerprinting, statistics:biases
  deleted lerner2016internet         2 [1]  design:archives, privacy:browser_storage  [provenance:design:archives]

^ page ^ markers repointed ^ distinct keys before ^ after ^ pairs ^
| design:archives | 3 | 14 | 14 | lerner2016internet→lerner2016_internet |
| design:crawling_location | 2 | 15 | 15 | bouhoula2024automated→bouhoula2024_automated |
| design:longitudinal | 1 | 32 | 32 | ahmad2026_ipfp→ahmad2026_more |
| privacy:browser_storage | 2 | 29 | 29 | fouad2022my→fouad2022_cookie; lerner2016internet→lerner2016_internet |
| privacy:consent | 3 | 28 | 28 | bouhoula2024automated→bouhoula2024_automated |
| privacy:fingerprinting | 2 | 35 | 35 | fouad2022my→fouad2022_cookie |
| privacy:requests | 1 | 39 | 39 | bouhoula2024automated→bouhoula2024_automated |
| programming:crawler:openwpm | 1 | 26 | 26 | fouad2022my→fouad2022_cookie |
| programming:stateful_stateless | 3 | 28 | 28 | fouad2022my→fouad2022_cookie |
| provenance:design:archives | 1 | 7 | 7 | lerner2016internet→lerner2016_internet |
| provenance:privacy:consent | 1 | 3 | 3 | bouhoula2024automated→bouhoula2024_automated |
| provenance:statistics:pvalue_corrections | 2 | 18 | 18 | boettger2025_regional→bottger2025_regional |
| statistics:pvalue_corrections | 1 | 28 | 28 | boettger2025_regional→bottger2025_regional |

pages changed          : 13
markers repointed      : 23  (loser occurrences before: 23)
provenance amendments  : 10
files written          : 21 under out/dedup_apply/

markers naming a key the bibliography does not define, BEFORE: 22 on 4 key(s)
  ...  (1 page(s))
  cite  (1 page(s))
  citekey  (2 page(s))
  key  (18 page(s))
same, AFTER: 22 on 4 key(s)
  ...  (1 page(s))
  cite  (1 page(s))
  citekey  (2 page(s))
  key  (18 page(s))

deleted keys still named in PROSE (not markers), left as historical record: 26 page(s)
  provenance:design:archives                       boettger2025_regional, bouhoula2024automated, fouad2022my, lerner2016internet
  provenance:design:crawling_location              bouhoula2024automated
  provenance:design:dns                            ahmad2026_ipfp, boettger2025_regional, bouhoula2024automated, fouad2022my, lerner2016internet
  provenance:design:longitudinal                   ahmad2026_ipfp, lerner2016internet
  provenance:design:platforms                      ahmad2026_ipfp, boettger2025_regional, bouhoula2024automated, fouad2022my, lerner2016internet
  provenance:design:website_classification         ahmad2026_ipfp, boettger2025_regional, bouhoula2024automated, fouad2022my, lerner2016internet
  provenance:literature:bibliography               ahmad2026_ipfp, boettger2025_regional, bouhoula2024automated, fouad2022my, lerner2016internet
  provenance:privacy:browser_extensions            lerner2016internet
  provenance:privacy:browser_protection            ahmad2026_ipfp, boettger2025_regional, fouad2022my
  provenance:privacy:browser_storage               fouad2022my, lerner2016internet
  provenance:privacy:consent                       bouhoula2024automated
  provenance:privacy:cookie_syncing                boettger2025_regional, fouad2022my
  provenance:privacy:darkpatterns                  bouhoula2024automated
  provenance:privacy:fingerprinting                fouad2022my
  provenance:privacy:privacy_sandbox               ahmad2026_ipfp, boettger2025_regional, bouhoula2024automated, fouad2022my, lerner2016internet
  provenance:privacy:requests                      bouhoula2024automated
  provenance:privacy:server_side_tracking          fouad2022my
  provenance:privacy:tcf_consent_strings           boettger2025_regional, bouhoula2024automated, fouad2022my, lerner2016internet
  provenance:programming:crawler:openwpm           boettger2025_regional, bouhoula2024automated, fouad2022my
  provenance:programming:deployment                bouhoula2024automated
  provenance:programming:filter_lists              boettger2025_regional
  provenance:programming:stateful_stateless        boettger2025_regional, fouad2022my
  provenance:statistics:biases                     boettger2025_regional, bouhoula2024automated, lerner2016internet
  provenance:statistics:how_many_sites             lerner2016internet
  provenance:statistics:pvalue_corrections         boettger2025_regional
  provenance:statistics:regression                 boettger2025_regional, bouhoula2024automated, fouad2022my, lerner2016internet

all invariants hold

Reading it: 23 deleted-key occurrences before, 23 repointed; every page's distinct-key count is unchanged, which is the check that no page cited both keys of a pair (a page that had would have lost a reference silently); and the four “unresolved” strings — key, citekey, cite, … — are documentation examples written as literal markers in prose on 18 pages, identical before and after. They are a pre-existing residue this run did not touch and did not create.

^ page ^ references (dt) before → after ^ citekey spans before → after ^ deleted-key strings after ^ verdict ^
| design:archives | 14 → 14 | 84 → 84 | 0 | unchanged |
| design:crawling_location | 15 → 15 | 40 → 40 | 0 | unchanged |
| design:longitudinal | 32 → 32 | 90 → 90 | 0 | unchanged |
| privacy:browser_storage | 28 → 28 | 96 → 96 | 0 | unchanged |
| privacy:consent | 28 → 28 | 138 → 138 | 0 | unchanged |
| privacy:fingerprinting | 35 → 35 | 118 → 118 | 0 | unchanged |
| privacy:requests | 38 → 38 | 156 → 156 | 0 | unchanged |
| programming:crawler:openwpm | 26 → 26 | 96 → 96 | 0 | unchanged |
| programming:stateful_stateless | 28 → 28 | 150 → 150 | 0 | unchanged |
| provenance:design:archives | -1 → -1 | 46 → 46 | 5 | unchanged |
| provenance:privacy:consent | 3 → 3 | 6 → 6 | 3 | unchanged |
| provenance:statistics:pvalue_corrections | 18 → 18 | 48 → 48 | 1 | unchanged |
| statistics:pvalue_corrections | 27 → 27 | 112 → 112 | 0 | unchanged |

pages checked: 13   pages whose counts moved: 0

-1 is archives, which cites papers but carries no <bibtex bibliography> block, so it has no reference list to count. The “deleted-key strings” on the three provenance pages are their dated amendment and their historical notes, not citations — the same rows show their marker counts unchanged.

The item asked whether the transliteration mistake existed for other German-named authors. bib_dedup_scan.py adds two candidate generators to the DOI-and-title scan of the first sitting: near-identical squashed titles (difflib ratio ≥ 0.85, or one title a prefix of the other, for a dropped subtitle), and same year plus same folded first-author surname, where the fold strips diacritics and collapses oe/ue/ae/ss to o/u/a/s so that Böttger, B“ottger, Boettger and Bottger compare equal. On the 855-entry file it reports the five definite pairs and 58 candidates; on the saved 850-entry file, 0 definite and the same 58.

All 58 were read by hand and none is a duplicate. Fifty-four are one first author with two or three different papers in one year (Durumeric 2013–2015, Li 2017 and 2024, Liu 2025, Agarwal 2025 …); one is a replication and its original, caught by the title rule (bratton2019_replication / sumner2014_exaggeration); three are DuckDuckGo artefacts whose “surname” is the vendor name. No second umlaut pair exists in the file. The list is the residue of the fold and is printed in full so the judgement can be checked:

bibliography : out/live_bibliography_20260904_1721.txt
entries      : 855   distinct citekeys: 855

DEFINITE duplicate pairs (rule A or B): 5
  [ABCD] ahmad2026_ipfp  /  ahmad2026_more
       More Space, Less Privacy? Measuring the Effectiveness of IP-based Website Fingerprinting i
       More Space, Less Privacy? Measuring the Effectiveness of IP-based Website Fingerprinting i
  [ABCD] boettger2025_regional  /  bottger2025_regional
       Understanding Regional Filter Lists: Efficacy and Impact
       Understanding Regional Filter Lists: Efficacy and Impact
  [BCD] bouhoula2024_automated  /  bouhoula2024automated
       Automated Large-Scale Analysis of Cookie Notice Compliance
       Automated Large-Scale Analysis of Cookie Notice Compliance
  [ABCD] fouad2022_cookie  /  fouad2022my
       My Cookie is a phoenix: detection, measurement, and lawfulness of cookie respawning with b
       My Cookie is a phoenix: Detection, measurement, and lawfulness of cookie respawning with b
  [BCD] lerner2016_internet  /  lerner2016internet
       Internet Jones and the Raiders of the Lost Trackers: An Archaeological Study of Web Tracki
       Internet Jones and the Raiders of the lost trackers: An archaeological study of web tracki

CANDIDATE pairs (rule C or D only) — judged by hand, see the provenance page: 58
  [D] LePochat2019_tranco  /  LePochat2019_tranco_eval
       2019 Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation
       2019 Evaluating the Long-term Effects of Parameters on the Characteristics of the {Tranco} Top Sites Rank
  [D] agarwal2024_peeking  /  agarwal2024_poster
       2024 Peeking through the window: Fingerprinting Browser Extensions through Page-Visible Execution Traces 
       2024 Poster: A Comprehensive Categorization of SMS Scams
  [D] agarwal2025_dropped  /  agarwal2025_fishing
       2025 'Hey mum, I dropped my phone down the toilet': Investigating Hi Mum and Dad SMS Scams in the United 
       2025 Fishing for Smishing: Understanding SMS Phishing Infrastructure and Strategies by Mining Public User
  [D] agarwal2025_dropped  /  agarwal2025_mindsets
       2025 'Hey mum, I dropped my phone down the toilet': Investigating Hi Mum and Dad SMS Scams in the United 
       2025 "I have no idea how to make it safer": Studying Security and Privacy Mindsets of Browser Extension D
  [D] agarwal2025_fishing  /  agarwal2025_mindsets
       2025 Fishing for Smishing: Understanding SMS Phishing Infrastructure and Strategies by Mining Public User
       2025 "I have no idea how to make it safer": Studying Security and Privacy Mindsets of Browser Extension D
  [D] alroomi2023_login  /  alroomi2023_password
       2023 A Large-Scale Measurement of Website Login Policies
       2023 Measuring Website Password Creation Policies At Scale
  [D] bahrami2025_bytedefender  /  bahrami2025_cookieguard
       2025 Byte by Byte: Unmasking Browser Fingerprinting at the Function Level Using V8 Bytecode Transformers
       2025 {CookieGuard}: Characterizing and Isolating the First-Party Cookie Jar
  [D] bashir2019_adstxt  /  bashir2019_quantity
       2019 A Longitudinal Analysis of the ads.txt Standard
       2019 Quantity vs. Quality: Evaluating User Interest Profiles Using Ad Preference Managers
  [D] bhuiyan2025_digital  /  bhuiyan2025_visitors
       2025 Digital Disparities: A Comparative Web Measurement Study Across Economic Boundaries
       2025 Not All Visitors are Bilingual: A Measurement Study of the Multilingual Web from an Accessibility Pe
  [C] bratton2019_replication  /  sumner2014_exaggeration
       2019 The Association Between Exaggeration in Health-Related Science News and Academic Press Releases: A R
       2014 The Association Between Exaggeration in Health Related Science News and Academic Press Releases: Ret
  [D] chen2021_cookieswap  /  chen2021_detecting
       2021 Cookie Swap Party: Abusing First-Party Cookies for Web Tracking
       2021 Detecting Filter List Evasion with Event-Loop-Turn Granularity JavaScript Signatures
  [D] chen2025_parents  /  chen2025_semantics
       2025 Empowering Parents to Support Children's Online Security and Privacy: Findings from a Randomized Con
       2025 Semantics-Aware Cookie Purpose Compliance
  [CD] duckduckgo_tracker_radar_2026  /  duckduckgo_tracker_radar_detector_2026
       2026 DuckDuckGo Tracker Radar
       2026 DuckDuckGo Tracker Radar Detector
  [D] duckduckgo_tracker_radar_2026  /  duckduckgo_trc_2026
       2026 DuckDuckGo Tracker Radar
       2026 Tracker Radar Collector
  [D] duckduckgo_tracker_radar_detector_2026  /  duckduckgo_trc_2026
       2026 DuckDuckGo Tracker Radar Detector
       2026 Tracker Radar Collector
  [D] durumeric2013_https  /  durumeric2013_zmap
       2013 Analysis of the HTTPS certificate ecosystem
       2013 {ZMap}: Fast Internet-wide Scanning and Its Security Applications
  [D] durumeric2014_heartbleed  /  durumeric2014_view
       2014 The Matter of Heartbleed
       2014 An Internet-Wide View of Internet-Wide Scanning
  [D] durumeric2015_neither  /  durumeric2015_search
       2015 Neither Snow Nor Rain Nor MITM...: An Empirical Analysis of Email Delivery Security
       2015 A Search Engine Backed by Internet-Wide Scanning
  [D] edu2022_alexa  /  edu2022_exploring
       2022 Measuring Alexa Skill Privacy Practices across Three Years
       2022 Exploring the security and privacy risks of chatbots in messaging services
  [D] iqbal2022_khaleesi  /  iqbal2022_left
       2022 Khaleesi: Breaker of Advertising and Tracking Request Chains
       2022 Left or Right: A Peek into the Political Biases in Email Spam Filtering Algorithms During US Electio
  [D] kancherla2025_johnny  /  kancherla2025_least
       2025 Johnny Can't Revoke Consent Either: Measuring Compliance of Consent Revocation on the Web
       2025 Least Privilege Access for Persistent Storage Mechanisms in Web Browsers
  [D] kirchner2024_black  /  kirchner2024_dancer
       2024 A Black-Box Privacy Analysis of Messaging Service Providers' Chat Message Processing
       2024 Dancer in the Dark: Synthesizing and Evaluating Polyglots for Blind Cross-Site Scripting
  [D] lee2023_adtargeting  /  lee2023_track
       2023 When and Why Do People Want Ad Targeting Explanations? Evidence from a Four-Week, Mixed-Methods Fiel
       2023 Net-track: Generic Web Tracking Detection Using Packet Metadata
  [D] li2016_remedying  /  li2016_youve
       2016 Remedying Web Hijacking: Notification Effectiveness and Webmaster Comprehension
       2016 You've Got Vulnerability: Exploring Effective Vulnerability Notifications
  [D] li2017_radar  /  li2017_security
       2017 FBS-Radar: Uncovering Fake Base Stations at Scale in the Wild
       2017 A Large-Scale Empirical Study of Security Patches
  [D] li2017_radar  /  li2017_static
       2017 FBS-Radar: Uncovering Fake Base Stations at Scale in the Wild
       2017 Static analysis of Android apps: A systematic literature review
  [D] li2017_security  /  li2017_static
       2017 A Large-Scale Empirical Study of Security Patches
       2017 Static analysis of Android apps: A systematic literature review
  [D] li2024_bounce  /  li2024_wellinformed
       2024 Bounce in the Wild: A Deep Dive into Email Delivery Failures from a Large Email Service Provider
       2024 Are We Getting Well-informed? An In-depth Study of Runtime Privacy Notice Practice in Mobile Apps
  [D] li2024_bounce  /  li2024_worldwide
       2024 Bounce in the Wild: A Deep Dive into Email Delivery Failures from a Large Email Service Provider
       2024 A Worldwide View on the Reachability of Encrypted DNS Services
  [D] li2024_wellinformed  /  li2024_worldwide
       2024 Are We Getting Well-informed? An In-depth Study of Runtime Privacy Notice Practice in Mobile Apps
       2024 A Worldwide View on the Reachability of Encrypted DNS Services
  [D] liao2016_characterizing  /  liao2016_seeking
       2016 Characterizing Long-tail SEO Spam on Cloud Web Hosting Services
       2016 Seeking Nonsense, Looking for Trouble: Efficient Promotional-Infection Detection through Semantic In
  [D] lin2021_longitudinal  /  lin2021_phishpedia
       2021 A Longitudinal Study of Removed Apps in {iOS} App Store
       2021 Phishpedia: A Hybrid Deep Learning Based Approach to Visually Identify Phishing Webpages
  [D] lin2022_investigating  /  lin2022_sheep
       2022 Investigating Advertisers' Domain-changing Behaviors and Their Impacts on Ad-blocker Filter Lists
       2022 Phish in Sheep's Clothing: Exploring the Authentication Pitfalls of Browser Fingerprinting
  [D] liu2024_opted  /  liu2024_promises
       2024 Opted Out, Yet Tracked: Are Regulations Enough to Protect Your Privacy?
       2024 From Promises to Practice: Evaluating the Private Browsing Modes of Android Browser Apps
  [D] liu2025_domino  /  liu2025_fingerprinting
       2025 The DOMino Effect: Detecting and Exploiting DOM Clobbering Gadgets via Concolic Execution with Symbo
       2025 The First Early Evidence of the Use of Browser Fingerprinting for Online Tracking
  [D] liu2025_domino  /  liu2025_somesite
       2025 The DOMino Effect: Detecting and Exploiting DOM Clobbering Gadgets via Concolic Execution with Symbo
       2025 Somesite I Used To Crawl: Awareness, Agency and Efficacy in Protecting Content Creators From AI Craw
  [D] liu2025_fingerprinting  /  liu2025_somesite
       2025 The First Early Evidence of the Use of Browser Fingerprinting for Online Tracking
       2025 Somesite I Used To Crawl: Awareness, Agency and Efficacy in Protecting Content Creators From AI Craw
  [D] nguyen2025_breaking  /  nguyen2025_please
       2025 Breaking the Shield: Analyzing and Attacking Canvas Fingerprinting Defenses in the Wild
       2025 "Please don't send that bot anything": A Mixed-methods Study of Personal Impersonation Attacks Targe
  [D] nisenoff2023_awareness  /  nisenoff2023_defining
       2023 User Awareness and Behaviors Concerning Encrypted {DNS} Settings in Web Browsers
       2023 Defining "Broken": User Experiences and Remediation Tactics When Ad-Blocking or Tracking-Protection 
  [D] oest2020_phishtime  /  oest2020_sunrise
       2020 PhishTime: Continuous Longitudinal Measurement of the Effectiveness of Anti-phishing Blacklists
       2020 Sunrise to Sunset: Analyzing the End-to-end Life Cycle and Effectiveness of Phishing Attacks at Scal
  [D] papadogiannakis2025_before  /  papadogiannakis2025_darkside
       2025 Before \& After: The Effect of EU's 2022 Code of Practice on Disinformation
       2025 Welcome to the Dark Side: Analyzing the Revenue Flows of Fraud in the Online Ad Ecosystem
  [D] ruth2022_toppling  /  ruth2022_world
       2022 Toppling Top Lists: Evaluating the Accuracy of Popular Website Lists
       2022 A World Wide View of Browsing the World Wide Web
  [D] scheitle2018_long  /  scheitle2018_rise
       2018 A Long Way to the Top: Significance, Structure, and Stability of Internet Top Lists
       2018 The Rise of Certificate Transparency and Its Implications on the Internet Ecosystem
  [D] starov2017_extended  /  starov2017_xhound
       2017 Extended Tracking Powers: Measuring the Privacy Diffusion Enabled by Browser Extensions
       2017 XHOUND: Quantifying the Fingerprintability of Browser Extensions
  [D] tang2025_misuse  /  tang2025_navigating
       2025 Misuse, Misreporting, Misinterpretation of Statistical Methods in Usable Privacy and Security Papers
       2025 Navigating Cookie Consent Violations Across the Globe
  [D] utz2023_comparing  /  utz2023_rarely
       2023 Comparing Large-Scale Privacy and Security Notifications
       2023 Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websi
  [D] vastel2018_scanner  /  vastel2018_stalker
       2018 Fp-Scanner: The Privacy Implications of Browser Fingerprint Inconsistencies
       2018 FP-STALKER: Tracking Browser Fingerprint Evolutions
  [D] vekaria2025_bighelp  /  vekaria2025_soktracking
       2025 Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistan
       2025 SoK: Advances and Open Problems in Web Tracking
  [D] venkatadri2019_auditing  /  venkatadri2019_investigating
       2019 Auditing Offline Data Brokers via Facebook's Advertising Platform
       2019 Investigating sources of PII used in Facebook’s targeted advertising
  [D] wang2026_masks  /  wang2026_sipconfusion
       2026 The Masks We (Think We) Wear: Privacy Threats of Browser-Extension Wallets in the Web3 Ecosystem
       2026 SIPConfusion: Exploiting SIP Semantic Ambiguities for Caller ID and SMS Spoofing
  [D] wu2025_appprivacyreport  /  wu2025_revealing
       2025 Transparency or Information Overload? Evaluating Users’ Comprehension and Perceptions of the iOS App
       2025 Revealing the Black Box of Device Search Engine: Scanning Assets, Strategies, and Ethical Considerat
  [D] wu2026_email  /  wu2026_tracking
       2026 One Email, Many Faces: A Deep Dive into Identity Confusion in Email Aliases
       2026 Tracking the Stray Sheep: Understanding DNS Response Manipulation in the Wild
  [D] xie2024_arcanum  /  xie2024_crawling
       2024 Arcanum: Detecting and Evaluating the Privacy Risks of Browser Extensions on Web Pages and Web Conte
       2024 Crawling to the Top: An Empirical Evaluation of Top List Use
  [D] yang2022_extensive  /  yang2022_wtagraph
       2022 An Extensive Study of Residential Proxies in China
       2022 WTAGRAPH: Web Tracking and Advertising Detection using Graph Neural Networks
  [D] zhang2022_harpo  /  zhang2022_spartacus
       2022 HARPO: Learning to Subvert Online Behavioral Advertising
       2022 I'm SPARTACUS, No, I'm SPARTACUS: Proactively Protecting Users from Phishing by Intentionally Trigge
  [D] zhang2024_inbox  /  zhang2024_quic
       2024 Inbox Invasion: Exploiting MIME Ambiguities to Evade Email Attachment Detectors
       2024 QUIC is not Quick Enough over Fast Internet
  [D] zhang2025_abusability  /  zhang2025_qrcode
       2025 Abusability of Automation Apps in Intimate Partner Violence
       2025 Demystifying the (In)Security of {QR} Code-based Login in Real-world Deployments
  [D] zhu2020_label  /  zhu2020_vtset
       2020 Measuring and Modeling the Label Dynamics of Online Anti-Malware Engines
       2020 Demo: Benchmarking Label Dynamics of VirusTotal Engines

26 provenance pages name a deleted key in a run record or review log — “found and left alone”, “this page cites bouhoula2024automated”. Those are statements about the bibliography as it was when they were written and were not rewritten; the list is the last block of the apply output above. The ten provenance pages whose content page was repointed each got a dated Amendment, 2026-09-04: citekey consolidation section instead, so a reader who follows one of those statements finds the correction on the same page.

• Whether anything outside the wiki cites a deleted key — a BibTeX file someone exported from the site, a draft that copied fouad2022my. Nothing on the site does; nothing off it can be checked.
• Whether khrer2015_going and som2017_content should be re-keyed. They are the [^a-z] bug in bibgen.mjs, not duplicates, and each is cited; re-keying them is the same kind of wiki-wide rewrite as this item and was not done inside it.

• Whether defects 1 and 2 ever reached the wiki. Zero missing authors survive in out/authors.json and zero in the live bibliography, but the cache carries no history, so it cannot be shown whether a dropped author was ever cached and later corrected or was never cached at all. The live page is clean either way, which is the question that matters.
• The 42 PETS entries in out/authors.json. Out of scope for this item. They come from parse_popets, a different function against a different page shape (parenthesised affiliations, so the comma ambiguity does not arise). scripts/bib_dupe_cocitation.py scans all 42 for an affiliation-shaped or single-token name and finds none — but that is a smoke test, not the three-source audit the USENIX keys got.
• The 86 live PoPETs/PETS entries' author lists. Their DOIs were checked; their authors were not. parse_popets reads a different page shape, and PETS records do carry DOIs (1,113 of 1,253), so the venue has an independent metadata path the USENIX records lack — but that is an argument for expecting fewer defects, not evidence of none.
• The other 592 live entries. Neither USENIX Security nor PoPETs; not examined at all. Their authors come from OpenAlex/Crossref via the index, never from fetch_authors.py.
• calzavara2026_syntactic has no DOI and its recorded url is a listing page. Whether the paper has a DOI yet was not established.
• Author order for the 5 keys nothing cites. Pass 3 covers every live entry, and pass 1 covers the 85 cached keys that are cited. The 5 cached-but-uncited keys have no live entry to compare against, so their order is unverified — harmless while nothing cites them, and pass 3 will cover them the moment something does.
• Whether bibgen.mjs output was hand-cleaned, or the affected entries were written from another source. The live entries are correct and the cache was not; the intermediate step left no record. The conclusion “the parser contaminated the cache and never the page” is a statement about the two endpoints, not about what happened between them.

Committed under scripts/. Run from /workspace/artifacts/wiki. Step 1 fetches and caches HTML; every later step re-parses from disk and never re-hammers usenix.org.

usenix_author_audit_fetch.py

#!/usr/bin/env python3
"""Step 1 of the USENIX author-list audit: fetch every USENIX landing page
named in out/authors.json and cache the raw HTML under out/usenix_audit/.
 
Separate from parsing on purpose: the parser was wrong twice already, so the
HTML is kept on disk and every later pass re-parses it offline instead of
re-hammering usenix.org.
"""
import glob, json, os, subprocess, sys, time
 
ROOT = "/workspace/publications_dataset/data"
OUTDIR = "out/usenix_audit/html"
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
      "Chrome/126.0.0.0 Safari/537.36")
 
idx = {}
for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
    for p in json.load(open(f))["papers"]:
        idx[f"{p['venue']}/{p['year']}/{p['slug']}"] = p
 
keys = [k for k in json.load(open("out/authors.json")) if k.startswith("USENIX/")]
os.makedirs(OUTDIR, exist_ok=True)
print(f"{len(keys)} USENIX keys in out/authors.json")
urls = {}
for i, k in enumerate(sorted(keys)):
    dest = os.path.join(OUTDIR, k.replace("/", "__") + ".html")
    url = idx[k]["landingUrl"]          # KeyError = bad slug, correct to crash
    urls[k] = url
    if os.path.exists(dest) and os.path.getsize(dest) > 2000:
        print(f"have  {k}")
        continue
    r = subprocess.run(["curl", "-sL", "-A", UA, "--max-time", "40",
                        "-w", "%{http_code}", "-o", dest, url],
                       capture_output=True, text=True)
    code = r.stdout.strip()
    size = os.path.getsize(dest) if os.path.exists(dest) else 0
    print(f"{code}   {size:>7}  {k}")
    if code != "200":
        print(f"  !! non-200 for {url}")
    time.sleep(1.0)
json.dump(urls, open("out/usenix_audit/urls.json", "w"), indent=1, sort_keys=True)

usenix_bib_fetch_all.py

#!/usr/bin/env python3
"""Extend the USENIX author audit to EVERY live USENIX Security entry.
 
out/authors.json holds 90 keys, of which 85 are cited; literature:bibliography
carries 159 entries in a USENIX Security series. The other 74 were filled from
somewhere else, so
the prose-parser bugs cannot explain a defect in them — but a dropped author
can reach a bibliography by more than one route, and pass 2 of
scripts/usenix_bib_diff.py can only see an affiliation that got IN, never an
author that fell OUT. This fetches the landing page for the rest so pass 3 can
check the whole population.
 
Resolves each entry to a landing URL by its own url= field where it has one,
and otherwise by title-matching into data/corpus2/.meta.
"""
import glob
import json
import os
import re
import subprocess
import sys
import time
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field, squash
 
ROOT = "/workspace/publications_dataset/data"
OUTDIR = "out/usenix_audit/html"
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
      "Chrome/126.0.0.0 Safari/537.36")
 
idx = {}
for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
    for p in json.load(open(f))["papers"]:
        if p["venue"] == "USENIX":
            idx[squash(p["title"])] = p
 
text = open("out/usenix_audit/bib_live.txt", encoding="utf-8").read()
entries = re.findall(r"@\w+\{[^@]*?\n\}", text, re.S)
 
targets, unresolved = {}, []
for e in entries:
    series = (field(e, "series") or "") + " " + (field(e, "booktitle") or "")
    if not re.search(r"USENIX Security", series, re.I):
        continue
    ckey = re.match(r"@\w+\{([^,]+),", e).group(1)
    p = idx.get(squash(field(e, "title")))
    url = field(e, "url")
    if p:
        targets[ckey] = (f"{p['venue']}/{p['year']}/{p['slug']}", p["landingUrl"],
                         p["title"])
    elif url and "usenix.org" in url:
        targets[ckey] = (f"url:{ckey}", url, field(e, "title"))
    else:
        unresolved.append(ckey)
 
json.dump({k: list(v) for k, v in targets.items()},
          open("out/usenix_audit/bib_targets.json", "w"), indent=1, sort_keys=True)
print(f"live USENIX Security entries resolved to a landing page: {len(targets)}")
print(f"unresolved (no index match, no usenix.org url)         : {len(unresolved)}"
      + (f"  {unresolved}" if unresolved else ""))
 
os.makedirs(OUTDIR, exist_ok=True)
n_new = 0
for ckey, (key, url, _t) in sorted(targets.items()):
    dest = os.path.join(OUTDIR, key.replace("/", "__").replace(":", "__") + ".html")
    if os.path.exists(dest) and os.path.getsize(dest) > 2000:
        continue
    r = subprocess.run(["curl", "-sL", "-A", UA, "--max-time", "40",
                        "-w", "%{http_code}", "-o", dest, url],
                       capture_output=True, text=True)
    n_new += 1
    print(f"{r.stdout.strip()}   {os.path.getsize(dest):>7}  {key}")
    time.sleep(1.0)
print(f"newly fetched: {n_new}")

usenix_author_audit.py

#!/usr/bin/env python3
"""Step 2 of the USENIX author-list audit: re-derive every USENIX author list
from AUTHORITATIVE page markup and diff against out/authors.json.
 
Why not just re-run the fixed prose parser (scripts/fetch_authors.py).
The prose blob — "A and B, University of California, Santa Barbara; C, MIT" —
is genuinely ambiguous: an affiliation tail is not distinguishable from a
person's name by any rule that does not encode a list of institutions. It cost
two parser bugs already (a comma split that crossed ";" boundaries, and an
unstripped leading "and ") and, even after those were fixed on 2026-08-13, it
still leaked affiliation FRAGMENTS into the author list, because "University of
California, Berkeley" splits into an affiliation-looking half that gets dropped
and a name-looking half ("Berkeley") that gets kept.
 
Every USENIX page carries at least one unambiguous author list instead:
 
  modern (2012-) presentation pages
    A. <meta name="citation_author" content="..."> — one tag per author, the
       Google Scholar metadata, emitted alongside citation_author_institution
       so names and affiliations are already separated by USENIX.
    B. USENIX's own published BibTeX block, author = {A and B and C}.
    Both are read; they must agree, or the paper is reported, not guessed at.
 
  pages with no structured metadata at all
    C. Affiliations are wrapped in an italic tag: <i> on the legacy (2011)
       technical-sessions pages, <em> inside the people-text div on the modern
       pages USENIX has not yet populated with Scholar metadata (one such page
       in this population, usenixsecurity25/presentation/tang). Deleting the
       italic spans leaves exactly the names. On a legacy page the paper's own
       block is first located BY TITLE, because those URLs address a whole
       session and taking the first block returns another talk's authors.
 
USENIX's own structured metadata is NOT unconditionally trustworthy, which is
why every paper is additionally cross-checked against the prose blob:
  * PhishTime (2020) — meta AND BibTeX both omit the 8th author, Gail-Joon Ahn,
    whom the prose blob and the paper both list.
  * PharmaLeaks (2012) — meta AND BibTeX say "Jordan Grant" and drop a middle
    initial; the paper PDF and DBLP both say "Grant Jordan" and
    "Geoffrey M. Voelker".
So a structured list is accepted only when the prose blob agrees on the set of
surnames. Every mismatch is printed for a human and adjudicated in
ADJUDICATED below, with the source that settled it.
 
    python3 scripts/usenix_author_audit.py          # report only
    python3 scripts/usenix_author_audit.py --write  # rewrite out/authors.json
 
Reads the HTML cached by scripts/usenix_author_audit_fetch.py; never refetches.
"""
import glob
import html
import json
import os
import re
import sys
import unicodedata
 
HTMLDIR = "out/usenix_audit/html"
CACHE = "out/authors.json"
ROOT = "/workspace/publications_dataset/data"
 
 
def unesc(s):
    return re.sub(r"\s+", " ", html.unescape(re.sub(r"<[^>]+>", " ", s))).strip()
 
 
# ---------------------------------------------------------------- source A
def meta_authors(page):
    return [unesc(m.group(1)) for m in re.finditer(
        r'<meta\s+name="citation_author"\s+content="([^"]*)"', page)
        if unesc(m.group(1))]
 
 
# ---------------------------------------------------------------- source B
LATEX = {
    r"\'": "́", r'\"': "̈", r"\`": "̀", r"\^": "̂",
    r"\~": "̃", r"\=": "̄", r"\.": "̇", r"\c": "̧",
    r"\v": "̌", r"\u": "̆", r"\H": "̋", r"\r": "̊",
}
 
 
LETTERS = {r"\i": "i", r"\j": "j", r"\ss": "ß", r"\o": "ø", r"\O": "Ø",
           r"\aa": "å", r"\AA": "Å", r"\l": "ł", r"\L": "Ł",
           r"\ae": "æ", r"\AE": "Æ", r"\oe": "œ", r"\OE": "Œ"}
 
 
def de_latex(s):
    """Doup{\\'e} -> Doupé.  Needed because the block is real BibTeX.
 
    The dotless-i pass has to run FIRST. USENIX writes Benoît as
    Beno{\\^\\i}t, and with \\i still in place none of the accent patterns
    below match (the character after the accent command is a backslash, not a
    letter); the final brace/backslash strip then produced "Beno^it". Found
    2026-09-04 by scripts/usenix_source_agreement.py, which is the only check
    that compares the two sources across all 162 pages rather than the 90.
    """
    def one(m):
        return unicodedata.normalize("NFC", m.group(2) + LATEX[m.group(1)])
    for cmd, ch in sorted(LETTERS.items(), key=lambda kv: -len(kv[0])):
        s = re.sub(re.escape(cmd) + r"(?![A-Za-z])", ch, s)
    # {\'e}  /  {\"u}  /  \'{e}  /  \'e
    s = re.sub(r"\{(\\[\'\"`^~=.cvuHr])\s*\{?([^\W\d_])\}?\}", one, s)
    s = re.sub(r"(\\[\'\"`^~=.cvuHr])\s*\{([^\W\d_])\}", one, s)
    s = re.sub(r"(\\[\'\"`^~=.cvuHr])([^\W\d_])", one, s)
    return re.sub(r"[{}\\]", "", s)
 
 
DE_LATEX_TESTS = [
    (r"Beno{\^\i}t", "Benoît"),      # dotless i — the 2026-09-04 regression
    (r"Adam Doup{\'e}", "Adam Doupé"),
    (r"Adam Doup\'e", "Adam Doupé"),
    (r"Dominik Sch{\"u}rmann", "Dominik Schürmann"),
    (r"Mathias L{\'e}cuyer", "Mathias Lécuyer"),
    # \c is cedilla, so \c{s} decodes to U+015F (s WITH CEDILLA), not the
    # U+0219 comma-below Romanian actually uses. That is the literally correct
    # reading of the BibTeX and it does not matter here: fold() strips both.
    (r"Dumitra\c{s}", "Dumitra\u015f"),
    (r"{Luki\'c}", "Lukić"),
    (r"Gro\ss{}", "Groß"),
    (r"Plain Name", "Plain Name"),
]
 
 
def selftest():
    bad = [(a, de_latex(a), b) for a, b in DE_LATEX_TESTS if de_latex(a) != b]
    for a, got, want in bad:
        print(f"  FAIL {a!r} -> {got!r}, want {want!r}")
    print(f"de_latex self-test: {len(DE_LATEX_TESTS) - len(bad)}/"
          f"{len(DE_LATEX_TESTS)} pass")
    return 1 if bad else 0
 
 
def brace_group(s, start):
    """Return the contents of the {...} beginning at s[start], brace-balanced.
 
    A non-greedy \\{(.*?)\\} truncates every LaTeX accent — "Adam Doup{\\'e}"
    ends the match at the inner brace and yields "Adam Doup{\\'e", which is how
    an earlier version of THIS script reported 13 spurious meta/BibTeX
    disagreements. Count braces instead.
    """
    depth, i = 0, start
    while i < len(s):
        if s[i] == "{":
            depth += 1
        elif s[i] == "}":
            depth -= 1
            if depth == 0:
                return s[start + 1:i]
        i += 1
    raise ValueError("unbalanced braces in BibTeX block")
 
 
def bibtex_authors(page):
    m = re.search(r'bibtex-text-entry[^>]*>(.*?)</div>', page, re.S)
    if not m:
        return []
    block = html.unescape(re.sub(r"<br\s*/?>", "\n", m.group(1)))
    a = re.search(r"author\s*=\s*\{", block)
    if not a:
        return []
    raw = re.sub(r"\s+", " ", brace_group(block, a.end() - 1)).strip()
    return [de_latex(n).strip() for n in re.split(r"\s+and\s+", raw) if n.strip()]
 
 
# ---------------------------------------------------------------- source C
def _names_minus_italics(fragment):
    """Names are whatever is left once the <i>/<em> affiliation spans go."""
    raw = unesc(re.sub(r"<(i|em)\b.*?</\1>", " ; ", fragment, flags=re.S))
    out = []
    for part in re.split(r"[;,]", raw):
        part = re.sub(r"^\s*and\s+", "", part.strip(), flags=re.I)
        for n in re.split(r"\s+and\s+", part):
            n = n.strip(" .,")
            if n:
                out.append(n)
    return out
 
 
def italic_authors(page, title):
    """Author list from a page carrying no citation_author and no BibTeX."""
    def squash(t):
        return re.sub(r"[^a-z0-9]", "", html.unescape(t).lower())
 
    # Modern page, Scholar metadata not yet populated: the people-text div with
    # <em>affiliation</em>. Only ONE page in this population is like this
    # (usenixsecurity25/presentation/tang) — but it is also the only entry the
    # audit would otherwise have to report as unverifiable.
    m = re.search(r'field-name-field-paper-people-text.*?<p>(.*?)</p>', page, re.S)
    if m and re.search(r"<(i|em)\b", m.group(1)):
        return _names_minus_italics(m.group(1))
 
    # Legacy 2011 session page: find this paper's own block by title.
    for m in re.finditer(r'<p class="fullpaper\d*">(.*?)</p>', page, re.S):
        b = m.group(1)
        t = re.search(r"<b>(.*?)</b>", b, re.S)
        if t and squash(unesc(t.group(1))) == squash(title):
            return _names_minus_italics(b[t.end():])
    return []
 
 
# ---------------------------------------------------------------- compare
def fold(names):
    """Compare ignoring diacritics, punctuation and case, but not word count."""
    def f(s):
        s = unicodedata.normalize("NFKD", s)
        s = "".join(c for c in s if not unicodedata.combining(c))
        return re.sub(r"\s+", " ", re.sub(r"[^a-z ]", " ", s.lower())).strip()
    return [f(x) for x in names]
 
 
# The affiliation filter for the over-generating prose splitter below. Kept here
# rather than imported from scripts/fetch_authors.py: that module now imports
# the parsers below, and the dependency has to run one way.
AFFIL_RE = re.compile(
    r"universit|institut|college|inc\.|labs?\b|corporat|technolog|research|"
    r"school|academy|center|centre|gmbh|ltd|company|cispa|kaist|inria|inc\b|"
    r"microsoft|google|meta|apple|amazon|ibm|nec|eth |epfl|tu |csiro|"
    r"laborator|foundation|agency|department|academia sinica|polytechnic",
    re.I,
)
 
def page_institutions(page):
    """Every affiliation the page itself declares.
 
    Modern pages emit one <meta name="citation_author_institution"> per author;
    legacy 2011 pages wrap each affiliation in <i>. Using the page's own list
    beats a hand-maintained stop-list of institution tail words: it needs no
    maintenance, and it takes the unfiltered residue (15 fragments over 16
    papers, mutation A7) to zero.
    """
    inst = [unesc(m.group(1)) for m in re.finditer(
        r'<meta\s+name="citation_author_institution"\s+content="([^"]*)"', page)]
    inst += [unesc(m.group(1)) for m in re.finditer(r"<i>(.*?)</i>", page, re.S)]
    return " | ".join(fold(inst))
 
 
def prose_parse(blob, inst=""):
    """Candidate personal names in the prose blob. Over-generates on purpose.
 
    Anything whose folded form is contained in one of the page's own declared
    affiliations is an affiliation tail, not an author, and is dropped.
    """
    out = []
    for group in blob.split(";"):
        parts = [p.strip() for p in group.split(",") if p.strip()]
        if len(parts) > 1 and AFFIL_RE.search(parts[-1]):
            parts = parts[:-1]
        for part in parts:
            part = re.sub(r"^and\s+", "", part.strip(), flags=re.I)
            for n in re.split(r"\s+and\s+", part):
                n = re.sub(r"\s*\([^)]*\)\s*", " ", n).strip(" .,")
                n = re.sub(r"\s+", " ", n)
                if not n or AFFIL_RE.search(n) or len(n.split()) < 2:
                    continue
                if len(n.split()) > 5 or not re.match(r"^[A-ZÀ-ɏ]", n):
                    continue
                f = fold([n])[0]
                if not f or (inst and f in inst):
                    continue
                out.append(n)
    return out
 
 
 
# ------------------------------------------------- cross-check against prose
# The prose blob is the ambiguous source this audit exists to stop trusting,
# but it is an INDEPENDENT one, so it is still good enough to answer the much
# easier question "does a surname appear at all?". Any surname on one side and
# not the other is printed and adjudicated by hand.
def prose_blob(page, title):
    for pat in (r'field-name-field-paper-people-text.*?<p>(.*?)</p>',
                r'field-name-field-paper-people-text[^>]*>(.*?)<div'):
        m = re.search(pat, page, re.S)
        if m:
            return unesc(m.group(1))
    # Legacy 2011 pages address a whole SESSION: locate this paper's own block
    # by title. Taking the first <p class="fullpaper"> on the page instead made
    # all three 2011 papers report the authors of an unrelated talk.
    def squash(t):
        return re.sub(r"[^a-z0-9]", "", html.unescape(t).lower())
    for m in re.finditer(r'<p class="fullpaper\d*">(.*?)</p>', page, re.S):
        b = m.group(1)
        t = re.search(r"<b>(.*?)</b>", b, re.S)
        if t and squash(unesc(t.group(1))) == squash(title):
            return unesc(b[t.end():])
    return ""
 
 
def surnames(names):
    return {fold([n])[0].split()[-1] for n in names if fold([n])[0]}
 
 
# Papers where the structured metadata and the prose blob disagree, each
# settled against a source outside USENIX's landing page. Anything not listed
# here is reported as unresolved and blocks --write.
ADJUDICATED = {
    "USENIX/2012/pharmaleaks-understanding-the-business-of-online-pharmaceutical-affiliate-progra": (
        ["Damon McCoy", "Andreas Pitsillidis", "Grant Jordan", "Nicholas Weaver",
         "Christian Kreibich", "Brian Krebs", "Geoffrey M. Voelker",
         "Stefan Savage", "Kirill Levchenko"],
        "USENIX meta/BibTeX say 'Jordan Grant' and 'Geoffrey Voelker'. The paper "
        "PDF title block (data/fulltext/2012/USENIX/pharmaleaks-.../paper.cols.txt) "
        "and DBLP both say 'Grant Jordan' and 'Geoffrey M. Voelker'. Paper wins."),
    "USENIX/2020/phishtime-continuous-longitudinal-measurement-of-the-effectiveness-of-anti-phish": (
        ["Adam Oest", "Yeganeh Safaei", "Penghui Zhang", "Brad Wardman",
         "Kevin Tyers", "Yan Shoshitaishvili", "Adam Doup\u00e9", "Gail-Joon Ahn"],
        "USENIX meta AND BibTeX both stop at 7 authors. The prose blob on the "
        "same page and the paper's own title block carry Gail-Joon Ahn as the "
        "8th. DBLP does NOT: record conf/uss/OestSZWTSD20, checked 2026-09-04, "
        "also lists seven and stops at Adam Doupe. Three machine-readable "
        "sources agree on a list the paper contradicts. Paper wins."),
    "USENIX/2025/vulnerability-of-text-matching-in-ml-ai-conference-reviewer-assignments-to-collu": (
        ["Jhih-Yi Hsieh", "Aditi Raghunathan", "Nihar B. Shah"],
        "The paper's title block and USENIX's meta both print the nickname, "
        "'Jhih-Yi (Janet) Hsieh'. Normalised to DBLP's form: a parenthetical is "
        "not a name particle and BibTeX would parse it as part of the given "
        "name. Judgement call, not a correction of the source."),
}
 
 
def load_titles():
    t = {}
    for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
        for p in json.load(open(f))["papers"]:
            t[f"{p['venue']}/{p['year']}/{p['slug']}"] = p["title"]
    return t
 
 
def main():
    write = "--write" in sys.argv
    # --cache <path> diffs against a snapshot instead of the live file, so the
    # committed audit trail can be regenerated after --write has already run.
    # Without it, re-running after a write reports "90 unchanged" and the
    # committed -output.txt becomes a record of nothing.
    src_cache = sys.argv[sys.argv.index("--cache") + 1] \
        if "--cache" in sys.argv else CACHE
    cache = json.load(open(src_cache))
    titles = load_titles()
    rows, unchanged, problems, accepted, adjudged = [], [], [], {}, []
 
    # Iterate the CACHE keys, not the HTML directory: scripts/usenix_bib_fetch_all.py
    # drops 72 more pages in there for pass 3 of usenix_bib_diff.py, and those
    # papers are not in out/authors.json at all.
    files = sorted(k for k in cache if k.startswith("USENIX/"))
    for key in files:
        f = os.path.join(HTMLDIR, key.replace("/", "__") + ".html")
        page = open(f, encoding="utf-8", errors="replace").read()
        old = cache[key]
 
        meta, bib = meta_authors(page), bibtex_authors(page)
        if meta or bib:
            if meta and bib and fold(meta) != fold(bib):
                problems.append((key, "META/BIBTEX DISAGREE", old, meta, bib))
                continue
            # Prefer meta: it keeps diacritics USENIX's BibTeX sometimes drops
            # (Gan~an) and needs no LaTeX decoding.
            new, src = (meta, "meta+bibtex") if (meta and bib) else \
                       ((meta, "meta") if meta else (bib, "bibtex"))
        else:
            new = italic_authors(page, titles[key])
            src = "italic"
            if not new:
                problems.append((key, "NO AUTHORS IN ANY SOURCE", old, [], []))
                continue
 
        # Cross-check the structured list against the independent prose blob.
        blob = prose_blob(page, titles[key])
        fblob = fold([blob])[0].split()
        missing_from_prose = [n for n in new
                              if fold([n])[0].split()[-1] not in fblob]
        prose_only = sorted(surnames(prose_parse(blob, page_institutions(page)))
                            - surnames(new))
        # An adjudicated paper overrides the page UNCONDITIONALLY. Applying it
        # only when the prose cross-check fires made the Hsieh entry dead code:
        # the prose agrees with the metadata there, and it is the metadata
        # itself that is being overridden.
        if key in ADJUDICATED:
            new, why = ADJUDICATED[key]
            src += " +adjudicated"
            adjudged.append((key, why))
        elif missing_from_prose or prose_only:
            problems.append((key, "STRUCTURED/PROSE DISAGREE: "
                             f"not in prose={missing_from_prose} "
                             f"prose-only surnames={prose_only}",
                             old, meta, bib))
            continue
 
        if fold(old) != fold(new):
            rows.append((key, src, old, new,
                         [a for a in new if fold([a])[0] not in fold(old)],
                         [a for a in old if fold([a])[0] not in fold(new)]))
        else:
            unchanged.append(key)
        accepted[key] = new
 
    print(f"USENIX keys audited                 : {len(files)}")
    print(f"  author list unchanged             : {len(unchanged)}")
    print(f"  author list CORRECTED             : {len(rows)}")
    print(f"  overruled by the paper/DBLP       : {len(adjudged)}")
    print(f"  unresolved, needs a human         : {len(problems)}")
    print(f"    of the corrected, missing an author : "
          f"{sum(1 for r in rows if r[4])}")
    print(f"    of the corrected, carried a non-author : "
          f"{sum(1 for r in rows if r[5])}")
    print("=" * 78)
    for key, src, old, new, dropped, spurious in rows:
        print(f"\n{key}   [{src}]")
        print(f"  was : {'; '.join(old)}")
        print(f"  now : {'; '.join(new)}")
        if dropped:
            print(f"  + AUTHOR(S) RESTORED : {'; '.join(dropped)}")
        if spurious:
            print(f"  - NON-AUTHOR(S) REMOVED : {'; '.join(spurious)}")
    if adjudged:
        print("\n" + "=" * 78 + "\nADJUDICATED against a source outside usenix.org:")
        for key, why in adjudged:
            print(f"\n  {key}\n    {why}")
    if problems:
        print("\n" + "=" * 78 + "\nUNRESOLVED — needs a human:")
        for key, why, old, meta, bib in problems:
            print(f"\n  {key}\n    {why}")
            print(f"    cache : {'; '.join(old)}")
            print(f"    meta  : {'; '.join(meta)}")
            print(f"    bibtex: {'; '.join(bib)}")
 
    if write:
        if problems:
            print("\nREFUSING to write: unresolved papers above.")
            return 1
        full = json.load(open(CACHE))
        full.update(accepted)
        json.dump(full, open(CACHE, "w"), indent=1, sort_keys=True)
        print(f"\nwrote {len(accepted)} USENIX entries to {CACHE}")
    return 1 if problems else 0
 
 
if __name__ == "__main__":
    sys.exit(selftest() if "--selftest" in sys.argv else main())

Real output, diffed against the pre-run snapshot out/usenix_audit/authors.json.before:

USENIX keys audited                 : 90
  author list unchanged             : 72
  author list CORRECTED             : 18
  overruled by the paper/DBLP       : 3
  unresolved, needs a human         : 0
    of the corrected, missing an author : 0
    of the corrected, carried a non-author : 18
==============================================================================

USENIX/2012/an-evaluation-of-the-google-chrome-extension-security-architecture   [meta+bibtex]
  was : Nicholas Carlini; Adrienne Porter Felt; David Wagner; Berkeley
  now : Nicholas Carlini; Adrienne Porter Felt; David Wagner
  - NON-AUTHOR(S) REMOVED : Berkeley

USENIX/2012/privilege-separation-in-html5-applications   [meta+bibtex]
  was : Devdatta Akhawe; Prateek Saxena; Dawn Song; Berkeley
  now : Devdatta Akhawe; Prateek Saxena; Dawn Song
  - NON-AUTHOR(S) REMOVED : Berkeley

USENIX/2014/hulk-eliciting-malicious-behavior-in-browser-extensions   [meta+bibtex]
  was : Alexandros Kapravelos; Santa Barbara; Chris Grier; Berkeley; Neha Chachra; San Diego; Christopher Kruegel; Giovanni Vigna; Santa Barbara; Vern Paxson; Berkeley
  now : Alexandros Kapravelos; Chris Grier; Neha Chachra; Christopher Kruegel; Giovanni Vigna; Vern Paxson
  - NON-AUTHOR(S) REMOVED : Santa Barbara; Berkeley; San Diego; Santa Barbara; Berkeley

USENIX/2017/extension-breakdown-security-analysis-of-browsers-extension-resources-control-po   [meta+bibtex]
  was : Iskander Sanchez-Rola; Igor Santos; DeustoTech; Davide Balzarotti; Eurecom
  now : Iskander Sanchez-Rola; Igor Santos; Davide Balzarotti
  - NON-AUTHOR(S) REMOVED : DeustoTech; Eurecom

USENIX/2019/50-ways-to-leak-your-data-an-exploration-of-apps-circumvention-of-the-android-pe   [meta+bibtex]
  was : Joel Reardon; Álvaro Feal; Primal Wijesekera; U.C. Berkeley / ICSI; Amit Elazari Bar On; U.C. Berkeley; Narseo Vallina-Rodriguez; Serge Egelman
  now : Joel Reardon; Álvaro Feal; Primal Wijesekera; Amit Elazari Bar On; Narseo Vallina-Rodriguez; Serge Egelman
  - NON-AUTHOR(S) REMOVED : U.C. Berkeley / ICSI; U.C. Berkeley

USENIX/2020/measuring-and-modeling-the-label-dynamics-of-online-anti-malware-engines   [meta+bibtex]
  was : Shuofei Zhu; Jianjun Shi; BIT; Limin Yang; Boqin Qin; BUPT; Ziyi Zhang; USTC; Linhai Song; Gang Wang
  now : Shuofei Zhu; Jianjun Shi; Limin Yang; Boqin Qin; Ziyi Zhang; Linhai Song; Gang Wang
  - NON-AUTHOR(S) REMOVED : BIT; BUPT; USTC

USENIX/2020/phishtime-continuous-longitudinal-measurement-of-the-effectiveness-of-anti-phish   [meta+bibtex +adjudicated]
  was : Adam Oest; Yeganeh Safaei; Penghui Zhang; Brad Wardman; Kevin Tyers; PayPal; Yan Shoshitaishvili; Adam Doupé; Gail-Joon Ahn
  now : Adam Oest; Yeganeh Safaei; Penghui Zhang; Brad Wardman; Kevin Tyers; Yan Shoshitaishvili; Adam Doupé; Gail-Joon Ahn
  - NON-AUTHOR(S) REMOVED : PayPal

USENIX/2020/sunrise-to-sunset-analyzing-the-end-to-end-life-cycle-and-effectiveness-of-phish   [meta+bibtex]
  was : Adam Oest; Penghui Zhang; Brad Wardman; Eric Nunes; Jakub Burgis; PayPal; Ali Zand; Kurt Thomas; Adam Doupé; Gail-Joon Ahn
  now : Adam Oest; Penghui Zhang; Brad Wardman; Eric Nunes; Jakub Burgis; Ali Zand; Kurt Thomas; Adam Doupé; Gail-Joon Ahn
  - NON-AUTHOR(S) REMOVED : PayPal

USENIX/2021/phishpedia-a-hybrid-deep-learning-based-approach-to-visually-identify-phishing-w   [meta+bibtex]
  was : Yun Lin; Ruofan Liu; Dinil Mon Divakaran; Trustwave; Jun Yang Ng; Qing Zhou Chan; Yiwen Lu; Yuxuan Si; Fan Zhang; Jin Song Dong
  now : Yun Lin; Ruofan Liu; Dinil Mon Divakaran; Jun Yang Ng; Qing Zhou Chan; Yiwen Lu; Yuxuan Si; Fan Zhang; Jin Song Dong
  - NON-AUTHOR(S) REMOVED : Trustwave

USENIX/2022/inferring-phishing-intention-via-webpage-appearance-and-dynamics-a-deep-vision-b   [meta+bibtex]
  was : Ruofan Liu; Yun Lin; Xianglin Yang; Siang Hwee Ng; Dinil Mon Divakaran; Trustwave; Jin Song Dong
  now : Ruofan Liu; Yun Lin; Xianglin Yang; Siang Hwee Ng; Dinil Mon Divakaran; Jin Song Dong
  - NON-AUTHOR(S) REMOVED : Trustwave

USENIX/2023/glowing-in-the-dark-uncovering-ipv6-address-discovery-and-scanning-strategies-in   [meta+bibtex]
  was : Hammas Bin Tanveer; Rachee Singh; Paul Pearce; Georgia Tech; Rishab Nithyanand
  now : Hammas Bin Tanveer; Rachee Singh; Paul Pearce; Rishab Nithyanand
  - NON-AUTHOR(S) REMOVED : Georgia Tech

USENIX/2023/log-it-s-big-it-s-heavy-it-s-filled-with-personal-data-measuring-the-logging-of   [meta+bibtex]
  was : Allan Lyons; Julien Gamba; Austin Shawaga; Joel Reardon; AppCensus; Juan Tapiador; Universidad Carlos III de Madrid; Serge Egelman; ICSI; UC Berkeley; AppCensus; Narseo Vallina-Rodriguez; AppCensus
  now : Allan Lyons; Julien Gamba; Austin Shawaga; Joel Reardon; Juan Tapiador; Serge Egelman; Narseo Vallina-Rodriguez
  - NON-AUTHOR(S) REMOVED : AppCensus; Universidad Carlos III de Madrid; ICSI; UC Berkeley; AppCensus; AppCensus

USENIX/2023/rods-with-laser-beams-understanding-browser-fingerprinting-on-phishing-pages   [meta+bibtex]
  was : Iskander Sanchez-Rola; Leyla Bilge; Davide Balzarotti; EURECOM; Armin Buescher; Petros Efstathopoulos
  now : Iskander Sanchez-Rola; Leyla Bilge; Davide Balzarotti; Armin Buescher; Petros Efstathopoulos
  - NON-AUTHOR(S) REMOVED : EURECOM

USENIX/2023/the-ok-is-not-enough-a-large-scale-study-of-consent-dialogs-in-smartphone-applic   [meta+bibtex]
  was : Simon Koch; Benjamin Altpeter; Datenanfragen.de e.V.; Martin Johns
  now : Simon Koch; Benjamin Altpeter; Martin Johns
  - NON-AUTHOR(S) REMOVED : Datenanfragen.de e.V.

USENIX/2024/knowphish-large-language-models-meet-multimodal-knowledge-graphs-for-enhancing-r   [meta+bibtex]
  was : Yuexin Li; Chengyu Huang; Shumin Deng; Mei Lin Lock; NCS Cyber Special Ops-R&D; Tri Cao; Nay Oo; Hoon Wei Lim; NCS Cyber Special Ops-R&D; Bryan Hooi
  now : Yuexin Li; Chengyu Huang; Shumin Deng; Mei Lin Lock; Tri Cao; Nay Oo; Hoon Wei Lim; Bryan Hooi
  - NON-AUTHOR(S) REMOVED : NCS Cyber Special Ops-R&D; NCS Cyber Special Ops-R&D

USENIX/2024/the-effect-of-design-patterns-on-present-and-future-cookie-consent-decisions   [meta+bibtex]
  was : Nataliia Bielova; Laura Litvine; Anysia Nguyen; Behavioural Insights Team; Mariam Chammat; Interministerial Directorate for Public Transformation; Vincent Toubiana; Estelle Hary
  now : Nataliia Bielova; Laura Litvine; Anysia Nguyen; Mariam Chammat; Vincent Toubiana; Estelle Hary
  - NON-AUTHOR(S) REMOVED : Behavioural Insights Team; Interministerial Directorate for Public Transformation

USENIX/2025/big-help-or-big-brother-auditing-tracking-profiling-and-personalization-in-gener   [meta+bibtex]
  was : Yash Vekaria; UC Davis; Aurelio Loris Canino; UNIRC; Jonathan Levitsky; UC Davis; Alex Ciechonski; UCL; Patricia Callejo; UC3M; Anna Maria Mandalari; UCL; Zubair Shafiq; UC Davis
  now : Yash Vekaria; Aurelio Loris Canino; Jonathan Levitsky; Alex Ciechonski; Patricia Callejo; Anna Maria Mandalari; Zubair Shafiq
  - NON-AUTHOR(S) REMOVED : UC Davis; UNIRC; UC Davis; UCL; UC3M; UCL; UC Davis

USENIX/2025/privacy-law-enforcement-under-centralized-governance-a-qualitative-analysis-of-f   [meta+bibtex]
  was : Tao Jing; Engineering; Yao Li; Jingzhou Ye; Jie Wang; Engineering; Xueqiang Wang
  now : Tao Jing; Yao Li; Jingzhou Ye; Jie Wang; Xueqiang Wang
  - NON-AUTHOR(S) REMOVED : Engineering; Engineering

==============================================================================
ADJUDICATED against a source outside usenix.org:

  USENIX/2012/pharmaleaks-understanding-the-business-of-online-pharmaceutical-affiliate-progra
    USENIX meta/BibTeX say 'Jordan Grant' and 'Geoffrey Voelker'. The paper PDF title block (data/fulltext/2012/USENIX/pharmaleaks-.../paper.cols.txt) and DBLP both say 'Grant Jordan' and 'Geoffrey M. Voelker'. Paper wins.

  USENIX/2020/phishtime-continuous-longitudinal-measurement-of-the-effectiveness-of-anti-phish
    USENIX meta AND BibTeX both stop at 7 authors. The prose blob on the same page and the paper's own title block carry Gail-Joon Ahn as the 8th. DBLP does NOT: record conf/uss/OestSZWTSD20, checked 2026-09-04, also lists seven and stops at Adam Doupe. Three machine-readable sources agree on a list the paper contradicts. Paper wins.

  USENIX/2025/vulnerability-of-text-matching-in-ml-ai-conference-reviewer-assignments-to-collu
    The paper's title block and USENIX's meta both print the nickname, 'Jhih-Yi (Janet) Hsieh'. Normalised to DBLP's form: a parenthetical is not a name particle and BibTeX would parse it as part of the given name. Judgement call, not a correction of the source.

usenix_bib_diff.py

#!/usr/bin/env python3
"""Step 3 of the USENIX author-list audit: diff literature:bibliography against
the corrected out/authors.json, and flag any USENIX entry the audit cannot see.
 
Two passes, because they catch different things:
 
  PASS 1  title-match every USENIX key in out/authors.json to a live BibTeX
          entry and diff the author field. This finds entries the broken prose
          parser wrote.
  PASS 2  scan EVERY live entry in a USENIX Security series for an author name
          that is affiliation-shaped, whether or not it is in out/authors.json.
          Entries added by hand or from another source are invisible to pass 1,
          and the same class of defect can reach them.
  PASS 3  compare EVERY live USENIX Security entry against the author list its
          own landing page publishes, as an ORDERED list. Pass 2 can only see an
          affiliation that got IN; only this pass can see an author that fell
          OUT of, or was reordered within, an entry out/authors.json never
          touched. Needs the pages fetched by scripts/usenix_bib_fetch_all.py.
 
    python3 scripts/usenix_bib_diff.py [--bib out/usenix_audit/bib_live.txt]
 
Prints a corrected author line per affected entry; it does not edit the wiki.
"""
import glob
import json
import os
import re
import sys
import unicodedata
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_author_audit import (de_latex, meta_authors, bibtex_authors,
                                 italic_authors, prose_blob, prose_parse,
                                 page_institutions, surnames, ADJUDICATED)
from usenix_author_audit import fold as afold
 
CACHE = "out/authors.json"
ROOT = "/workspace/publications_dataset/data"
DEFAULT_BIB = "out/usenix_audit/bib_live.txt"
 
 
def fold(s):
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c))
    return re.sub(r"\s+", " ", re.sub(r"[^a-z0-9 ]", " ", s.lower())).strip()
 
 
def squash(s):
    return re.sub(r"[^a-z0-9]", "", fold(s))
 
 
def field(entry, name):
    m = re.search(r"\b" + name + r"\s*=\s*\{", entry)
    if not m:
        return None
    depth, i = 0, m.end() - 1
    while i < len(entry):
        if entry[i] == "{":
            depth += 1
        elif entry[i] == "}":
            depth -= 1
            if depth == 0:
                return re.sub(r"\s+", " ", entry[m.end():i]).strip()
        i += 1
    raise ValueError("unbalanced braces in " + entry[:60])
 
 
def split_bib_authors(a):
    """'Last, First and Last, First' -> ['First Last', ...]"""
    out = []
    for n in re.split(r"\s+and\s+", a):
        n = n.strip()
        if not n:
            continue
        if "," in n:
            last, first = n.split(",", 1)
            n = f"{first.strip()} {last.strip()}".strip()
        # 43 of the 837 live entries spell an accent with a LaTeX accent
        # command and 66 with a literal non-ASCII letter (see
        # scripts/bib_accent_encoding.py). Both render correctly through
        # bibtex4dw: checked on the rendered DOM of security:phishing, which
        # cites the braced "Doup{\'e}" entries, and privacy:browser_extensions,
        # which cites the one unbraced "Doup\'e" entry. Decode before diffing,
        # or those three entries drown a real defect in encoding noise.
        out.append(re.sub(r"\s+", " ", de_latex(n)))
    return out
 
 
def fmt_bib(names):
    out = []
    for a in names:
        parts = a.strip().split()
        last = parts.pop()
        out.append(f"{last}, {' '.join(parts)}" if parts else last)
    return " and ".join(out)
 
 
# An author name that is really an affiliation. Deliberately wide: every hit is
# printed in full for a human, nothing is auto-removed on the strength of it.
AFFIL_SHAPED = re.compile(
    r"universit|universidad|institut|college|\binc\b|\blabs?\b|corporat|"
    r"technolog|\bresearch\b|school|academy|center|centre|gmbh|\bltd\b|"
    r"company|cispa|kaist|inria|microsoft|\bgoogle\b|\bapple\b|\bamazon\b|"
    r"\bibm\b|\bepfl\b|\beth\b|laborator|foundation|agency|department|"
    r"polytechnic|\bpaypal\b|trustwave|appcensus|\bicsi\b|\bteam\b|\be\.?\s?v\.?$",
    re.I)
SINGLE_TOKEN_OK = {"anonymous"}
 
 
def main():
    bibpath = sys.argv[sys.argv.index("--bib") + 1] if "--bib" in sys.argv \
        else DEFAULT_BIB
    text = open(bibpath, encoding="utf-8").read()
    entries = re.findall(r"@\w+\{[^@]*?\n\}", text, re.S)
    print(f"bibliography           : {bibpath}")
    print(f"entries parsed         : {len(entries)}")
 
    by_title = {}
    for e in entries:
        t = field(e, "title")
        if t:
            by_title.setdefault(squash(t), []).append(e)
 
    titles = {}
    for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
        for p in json.load(open(f))["papers"]:
            titles[f"{p['venue']}/{p['year']}/{p['slug']}"] = p["title"]
 
    cache = json.load(open(CACHE))
    keys = sorted(k for k in cache if k.startswith("USENIX/"))
    print(f"USENIX keys in {CACHE}: {len(keys)}\n")
 
    # ---------------------------------------------------------------- PASS 1
    matched, wrong, absent = 0, [], []
    for k in keys:
        cands = by_title.get(squash(titles[k]), [])
        if not cands:
            absent.append(k)
            continue
        matched += 1
        for e in cands:
            key = re.match(r"@\w+\{([^,]+),", e).group(1)
            have = split_bib_authors(field(e, "author"))
            want = cache[k]
            if [fold(x) for x in have] != [fold(x) for x in want]:
                wrong.append((key, k, have, want))
 
    print("=" * 78)
    print(f"PASS 1  title-matched to the bibliography : {matched}/{len(keys)}")
    print(f"        author field WRONG                : {len(wrong)}")
    print(f"        key not cited on the wiki at all  : {len(absent)}")
    for key, k, have, want in wrong:
        print(f"\n  {{[{key}]}}   ({k})")
        print(f"    live      : {'; '.join(have)}")
        print(f"    corrected : {'; '.join(want)}")
        print(f"    author        = {{{fmt_bib(want)}}},")
 
    # ---------------------------------------------------------------- PASS 2
    print("\n" + "=" * 78)
    flagged = []
    us = 0
    for e in entries:
        series = (field(e, "series") or "") + " " + (field(e, "booktitle") or "")
        if not re.search(r"USENIX Security", series, re.I):
            continue
        us += 1
        names = split_bib_authors(field(e, "author") or "")
        bad = [n for n in names
               if AFFIL_SHAPED.search(n)
               or (len(n.split()) < 2 and fold(n) not in SINGLE_TOKEN_OK)]
        if bad:
            key = re.match(r"@\w+\{([^,]+),", e).group(1)
            flagged.append((key, names, bad))
    print(f"PASS 2  live entries in a USENIX Security series : {us}")
    print(f"        entries with an affiliation-shaped author: {len(flagged)}")
    for key, names, bad in flagged:
        print(f"\n  {{[{key}]}}")
        print(f"    authors : {'; '.join(names)}")
        print(f"    FLAGGED : {'; '.join(bad)}")
    # ---------------------------------------------------------------- PASS 3
    print("\n" + "=" * 78)
    tpath = "out/usenix_audit/bib_targets.json"
    if not os.path.exists(tpath):
        print("PASS 3  SKIPPED: run scripts/usenix_bib_fetch_all.py first")
    else:
        targets = json.load(open(tpath))
        by_ckey = {re.match(r"@\w+\{([^,]+),", e).group(1): e for e in entries}
        checked, p3_wrong, p3_skip = 0, [], []
        for ckey, (k, _url, title) in sorted(targets.items()):
            h = os.path.join("out/usenix_audit/html",
                             k.replace("/", "__").replace(":", "__") + ".html")
            if not os.path.exists(h):
                p3_skip.append((ckey, "page not fetched"))
                continue
            page = open(h, encoding="utf-8", errors="replace").read()
            m, b = meta_authors(page), bibtex_authors(page)
            page_authors = m or b or italic_authors(page, title)
            if not page_authors:
                p3_skip.append((ckey, "no author list on the page"))
                continue
            if k in ADJUDICATED:
                page_authors = ADJUDICATED[k][0]
            checked += 1
            have = split_bib_authors(field(by_ckey[ckey], "author"))
            hs, ps = {fold(x) for x in have}, {fold(x) for x in page_authors}
            missing = sorted(ps - hs)
            extra = sorted(hs - ps)
            # Set difference alone is NOT enough. An earlier version compared
            # only the sets, with a comment claiming it also checked order —
            # it did not, and a review on 2026-09-04 demonstrated the gap by
            # swapping first and last author on a pass-3-only entry: all three
            # passes reported clean. Author ORDER is first authorship; a swap
            # is a worse citation defect than a missing middle author.
            reordered = (not missing and not extra
                         and [fold(x) for x in have] != [fold(x) for x in page_authors])
            if missing or extra or reordered:
                p3_wrong.append((ckey, k, have, page_authors, missing, extra,
                                 reordered))
        print(f"PASS 3  live entries checked against their landing page : "
              f"{checked}/{len(targets)}")
        print(f"        author SET differs from the page               : "
              f"{sum(1 for r in p3_wrong if r[4] or r[5])}")
        print(f"        same set, WRONG ORDER                          : "
              f"{sum(1 for r in p3_wrong if r[6])}")
        print(f"        skipped                                        : "
              f"{len(p3_skip)}")
        for ckey, k, have, want, missing, extra, reordered in p3_wrong:
            print(f"\n  {{[{ckey}]}}   ({k})")
            print(f"    live : {'; '.join(have)}")
            print(f"    page : {'; '.join(want)}")
            if missing:
                print(f"    ON THE PAGE, NOT IN THE ENTRY : {'; '.join(missing)}")
            if extra:
                print(f"    IN THE ENTRY, NOT ON THE PAGE : {'; '.join(extra)}")
            if reordered:
                print(f"    SAME NAMES, DIFFERENT ORDER — check first authorship")
        for ckey, why in p3_skip:
            print(f"    skipped {ckey}: {why}")
 
    if absent:
        print("\n" + "=" * 78)
        print("Keys in out/authors.json that no live entry cites "
              "(fetched for the audit, never published):")
        for k in absent:
            print(f"  {k}")
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

bibliography           : out/usenix_audit/bib_live.txt
entries parsed         : 837
USENIX keys in out/authors.json: 90

==============================================================================
PASS 1  title-matched to the bibliography : 85/90
        author field WRONG                : 0
        key not cited on the wiki at all  : 5

==============================================================================
PASS 2  live entries in a USENIX Security series : 159
        entries with an affiliation-shaped author: 0

==============================================================================
PASS 3  live entries checked against their landing page : 159/159
        author SET differs from the page               : 0
        same set, WRONG ORDER                          : 0
        skipped                                        : 0

==============================================================================
Keys in out/authors.json that no live entry cites (fetched for the audit, never published):
  USENIX/2022/inferring-phishing-intention-via-webpage-appearance-and-dynamics-a-deep-vision-b
  USENIX/2022/ovrseen-auditing-network-traffic-and-privacy-policies-in-oculus-vr
  USENIX/2023/glowing-in-the-dark-uncovering-ipv6-address-discovery-and-scanning-strategies-in
  USENIX/2023/knowledge-expansion-and-counterfactual-interaction-for-reference-based-phishing
  USENIX/2024/knowphish-large-language-models-meet-multimodal-knowledge-graphs-for-enhancing-r

The mutated bibliographies it runs against are generated, not hand-made:

make_mutated_bibs.py

#!/usr/bin/env python3
"""Build the mutated bibliographies scripts/usenix_author_audit_mutations.py
runs against, so the mutation set is reproducible instead of a set of files
someone once made by hand.
 
Each writes out/usenix_audit/mut/<id>.bib, a copy of the live snapshot with one
defect injected:
 
  b1  an author DELETED from an entry that is in out/authors.json
  b2  an affiliation ADDED as an author, same entry
  b3  an affiliation ADDED to an entry that is NOT in out/authors.json
  b4  one author's given and family names SWAPPED
  b5  the author ORDER reversed at both ends of a NON-cached entry — the defect
      that passed every check until 2026-09-04
  b6  an author DELETED from a NON-cached entry. b1 does this to a CACHED entry,
      where pass 1 catches it; b6 is the case pass 3 exists for, and nothing
      else on this page can see it
 
    python3 scripts/make_mutated_bibs.py
"""
import json
import os
import re
import sys
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field, split_bib_authors
 
SRC = "out/usenix_audit/bib_live.txt"
DEST = "out/usenix_audit/mut"
 
CACHED = "author        = {Carlini, Nicholas and Felt, Adrienne Porter and Wagner, David},"
NONCACHED = ("author={Xie, Qinge and Tang, Shujun and Zheng, Xiaofeng and "
             "Lin, Qingran and Liu, Baojun and Duan, Haixin and Li, Frank}")
SWAP_FROM = "author        = {McCoy, Damon and Pitsillidis, Andreas and Jordan, Grant and"
B5_KEY = "acharya2024_imitation"
 
MUTS = {
    "b1": (CACHED,
           "author        = {Carlini, Nicholas and Felt, Adrienne Porter},"),
    "b2": (CACHED,
           "author        = {Carlini, Nicholas and Felt, Adrienne Porter and "
           "Wagner, David and Berkeley},"),
    "b3": (NONCACHED, NONCACHED[:-1] + " and Georgia Institute of Technology}"),
    "b4": (SWAP_FROM,
           "author        = {McCoy, Damon and Pitsillidis, Andreas and "
           "Grant, Jordan and"),
}
 
 
def main():
    bib = open(SRC, encoding="utf-8").read()
    os.makedirs(DEST, exist_ok=True)
    for name, (a, b) in MUTS.items():
        if a not in bib:
            raise SystemExit(f"{name}: anchor not in {SRC}; the snapshot moved")
        open(os.path.join(DEST, name + ".bib"), "w").write(bib.replace(a, b, 1))
        print(f"wrote {name}.bib")
 
    # b5 is derived rather than hardcoded, so it stays valid if the entry moves.
    ents = {re.match(r"@\w+\{([^,]+),", e).group(1): e
            for e in re.findall(r"@\w+\{[^@]*?\n\}", bib, re.S)}
    orig = field(ents[B5_KEY], "author")
    parts = re.split(r"\s+and\s+", orig)
    if len(parts) < 3:
        raise SystemExit(f"{B5_KEY} has too few authors to reorder")
    swapped = " and ".join([parts[-1]] + parts[1:-1] + [parts[0]])
    if "{" + orig + "}" not in bib:
        raise SystemExit(f"b5: could not locate {B5_KEY}'s author field")
    open(os.path.join(DEST, "b5.bib"), "w").write(
        bib.replace("{" + orig + "}", "{" + swapped + "}", 1))
    print(f"wrote b5.bib  ({B5_KEY}: first and last author swapped)")
    print(f"  from: {orig}")
    print(f"  to  : {swapped}")
 
    dropped = " and ".join(parts[:-1])
    open(os.path.join(DEST, "b6.bib"), "w").write(
        bib.replace("{" + orig + "}", "{" + dropped + "}", 1))
    print(f"wrote b6.bib  ({B5_KEY}: last author deleted — {parts[-1]})")
 
 
if __name__ == "__main__":
    sys.exit(main())

usenix_author_audit_mutations.py

#!/usr/bin/env python3
"""Mutation-test scripts/usenix_author_audit.py and scripts/usenix_bib_diff.py.
 
Both scripts report "0 unresolved" and "0 defects", and a check that passes is
worth nothing until it has been shown it can fail. Each mutation below breaks
exactly one load-bearing rule; a mutation that does NOT change the reported
counts is a rule that is asserting nothing.
 
The audit mutations diff against the PRE-FIX snapshot
(out/usenix_audit/authors.json.before), because against the already-corrected
out/authors.json every run reports 90 unchanged and no mutation can move it.
 
    python3 scripts/usenix_author_audit_mutations.py
"""
import os
import re
import subprocess
import sys
 
BEFORE = "out/usenix_audit/authors.json.before"
LIVEBIB = "out/usenix_audit/bib_live.txt"
MUTBIB = "out/usenix_audit/mut"
TMP = "scripts/_mut_tmp.py"
 
AUDIT = "scripts/usenix_author_audit.py"
AUDIT_MUTS = [
    ("A1  citation_author parser truncated to 1 author",
     'return [unesc(m.group(1)) for m in re.finditer(\n'
     '        r\'<meta\\s+name="citation_author"\\s+content="([^"]*)"\', page)\n'
     '        if unesc(m.group(1))]',
     'return [unesc(m.group(1)) for m in list(re.finditer(\n'
     '        r\'<meta\\s+name="citation_author"\\s+content="([^"]*)"\', page))[:1]\n'
     '        if unesc(m.group(1))]'),
    # A2 must ALSO empty the adjudication table. On its own it changes nothing,
    # because every paper the cross-check fires on is already adjudicated and
    # the adjudication branch runs first — the check is a tripwire for the next
    # corpus refresh, not a live assertion today. Emptying both shows what it
    # is worth: without it, PharmaLeaks silently regresses to the wrong
    # "Jordan Grant" (corrected goes 18 -> 20) instead of stopping the run.
    ("A2  prose cross-check disabled *and* adjudication emptied",
     [("        elif missing_from_prose or prose_only:",
       "        elif False and (missing_from_prose or prose_only):"),
      ('ADJUDICATED = {\n    "USENIX/2012/pharmaleaks',
       'ADJUDICATED = {}\n_UNUSED = {\n    "USENIX/2012/pharmaleaks')], None),
    ("A3  BibTeX brace balancing broken",
     "                return s[start + 1:i]",
     "                return s[start + 1:start + 8]"),
    ("A4  legacy block no longer matched by title",
     "        if t and squash(unesc(t.group(1))) == squash(title):",
     "        if t:"),
    ("A5  diacritic folding removed",
     '        s = "".join(c for c in s if not unicodedata.combining(c))',
     "        pass"),
    ("A6  adjudication table emptied",
     'ADJUDICATED = {\n    "USENIX/2012/pharmaleaks',
     'ADJUDICATED = {}\n_UNUSED = {\n    "USENIX/2012/pharmaleaks'),
    ("A7  institution list ignored (prose residue no longer filtered)",
     "                if not f or (inst and f in inst):",
     "                if not f:"),
]
 
# The mutated bibliographies under out/usenix_audit/mut/ are built by
# scripts/make_mutated_bibs.py, so the mutation set is reproducible rather than
# a set of files someone once made by hand.
 
 
def field(out, pat):
    m = re.search(pat, out)
    return m.group(1) if m else "?"
 
 
def run_audit(path, extra=()):
    r = subprocess.run([sys.executable, path, "--cache", BEFORE, *extra],
                       capture_output=True, text=True)
    return (r.returncode,
            field(r.stdout, r"unchanged +: (\d+)"),
            field(r.stdout, r"CORRECTED +: (\d+)"),
            field(r.stdout, r"unresolved, needs a human +: (\d+)"))
 
 
def run_diff(bib):
    r = subprocess.run([sys.executable, "scripts/usenix_bib_diff.py",
                        "--bib", bib], capture_output=True, text=True)
    return (field(r.stdout, r"author field WRONG +: (\d+)"),
            field(r.stdout, r"affiliation-shaped author: (\d+)"),
            field(r.stdout, r"author SET differs from the page +: (\d+)"),
            field(r.stdout, r"same set, WRONG ORDER +: (\d+)"))
 
 
def main():
    src = open(AUDIT).read()
    print(f"=== {AUDIT} — diffed against {BEFORE}")
    print("%-58s %-4s %-10s %-10s %s"
          % ("mutation", "rc", "unchanged", "corrected", "unresolved"))
    rc, u, c, p = run_audit(AUDIT)
    print("%-58s %-4s %-10s %-10s %s" % ("(baseline, unmutated)", rc, u, c, p))
    bad = []
    for name, a, b in AUDIT_MUTS:
        steps = a if isinstance(a, list) else [(a, b)]
        mutated, stale = src, False
        for fro, to in steps:
            if fro not in mutated:
                stale = True
                break
            mutated = mutated.replace(fro, to, 1)
        if stale:
            print(f"{name}: PATTERN NOT FOUND — mutation is stale")
            bad.append(name)
            continue
        open(TMP, "w").write(mutated)
        rc2, u2, c2, p2 = run_audit(TMP)
        print("%-58s %-4s %-10s %-10s %s" % (name, rc2, u2, c2, p2))
        if (u2, c2, p2) == (u, c, p):
            bad.append(name)
    if os.path.exists(TMP):
        os.remove(TMP)
 
    print(f"\n=== scripts/usenix_bib_diff.py — mutated BIBLIOGRAPHIES")
    print("%-58s %-7s %-7s %-7s %s"
          % ("mutation", "PASS1", "PASS2", "PASS3set", "PASS3order"))
    base = run_diff(LIVEBIB)
    print("%-58s %-7s %-7s %-7s %s" % ("(baseline, unmutated)", *base))
    for name, f in [("B1  author deleted from a cached entry", "b1"),
                    ("B2  affiliation added to a cached entry", "b2"),
                    ("B3  affiliation added to a NON-cached entry", "b3"),
                    ("B4  given/family names swapped", "b4"),
                    ("B5  author ORDER swapped, NON-cached entry", "b5"),
                    ("B6  author DELETED from a NON-cached entry", "b6")]:
        path = os.path.join(MUTBIB, f + ".bib")
        if not os.path.exists(path):
            print(f"{name}: {path} missing")
            bad.append(name)
            continue
        got = run_diff(path)
        print("%-58s %-7s %-7s %-7s %s" % (name, *got))
        if got == base:
            bad.append(name)
 
    print(f"\nmutations that changed nothing (a rule asserting nothing): "
          f"{len(bad)}" + (f"  {bad}" if bad else ""))
    return 1 if bad else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

usenix_source_agreement.py

#!/usr/bin/env python3
"""Measure how often USENIX's two structured author sources disagree, under the
three parser states this audit went through.
 
The provenance page claimed "13 of the first 90 disagreed" from a run whose
script no longer exists, which is not a reproducible figure. This reconstructs
the comparison from the same cached HTML under each state, so the number on the
page is one anybody can re-derive:
 
  A  non-greedy brace match, no diacritic fold  — the audit's first, buggy state
  B  brace-balanced, no diacritic fold          — after the brace fix only
  C  brace-balanced, diacritic-folded           — what ships
 
    python3 scripts/usenix_source_agreement.py
"""
import glob
import html
import json
import os
import re
import sys
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_author_audit import (bibtex_authors, de_latex, fold, meta_authors)
 
HTMLDIR = "out/usenix_audit/html"
 
 
def bibtex_authors_nongreedy(page):
    """The original bug: \\{(.*?)\\} ends at the first inner brace."""
    m = re.search(r"bibtex-text-entry[^>]*>(.*?)</div>", page, re.S)
    if not m:
        return []
    block = html.unescape(re.sub(r"<br\s*/?>", "\n", m.group(1)))
    a = re.search(r"author\s*=\s*\{(.*?)\}", block, re.S)
    if not a:
        return []
    raw = re.sub(r"\s+", " ", a.group(1)).strip()
    return [de_latex(n).strip() for n in re.split(r"\s+and\s+", raw) if n.strip()]
 
 
def main():
    cached = sorted(k for k in json.load(open("out/authors.json"))
                    if k.startswith("USENIX/"))
    allpages = sorted(glob.glob(os.path.join(HTMLDIR, "*.html")))
    pops = [("the 90 cached author lists",
             [os.path.join(HTMLDIR, k.replace("/", "__") + ".html")
              for k in cached]),
            ("all 162 fetched landing pages", allpages)]
 
    for label, files in pops:
        rows = {"A  non-greedy brace, no fold": 0,
                "B  brace-balanced, no fold": 0,
                "C  brace-balanced + diacritic fold (ships)": 0}
        both = 0
        cases = {k: [] for k in rows}
        for f in files:
            page = open(f, encoding="utf-8", errors="replace").read()
            m = meta_authors(page)
            if not m:
                continue
            bg, bn = bibtex_authors(page), bibtex_authors_nongreedy(page)
            if not bg:
                continue
            both += 1
            key = os.path.basename(f)[:-5].replace("__", "/")
            if [x.strip() for x in m] != [x.strip() for x in bn]:
                rows["A  non-greedy brace, no fold"] += 1
                cases["A  non-greedy brace, no fold"].append(key)
            if [x.strip() for x in m] != [x.strip() for x in bg]:
                rows["B  brace-balanced, no fold"] += 1
                cases["B  brace-balanced, no fold"].append(key)
            if fold(m) != fold(bg):
                rows["C  brace-balanced + diacritic fold (ships)"] += 1
                cases["C  brace-balanced + diacritic fold (ships)"].append(key)
        print(f"=== {label}: {both} carry BOTH citation_author and a BibTeX block")
        for k, v in rows.items():
            print(f"    {k:<46} disagree on {v}")
        rem = cases["C  brace-balanced + diacritic fold (ships)"]
        print(f"    still disagreeing under C: {rem if rem else 'none'}")
        onlyB = [x for x in cases["B  brace-balanced, no fold"] if x not in rem]
        print(f"    fixed by the diacritic fold alone: "
              f"{[x.split('/')[-1][:40] for x in onlyB] or 'none'}")
        print()
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

=== the 90 cached author lists: 87 carry BOTH citation_author and a BibTeX block
    A  non-greedy brace, no fold                   disagree on 11
    B  brace-balanced, no fold                     disagree on 2
    C  brace-balanced + diacritic fold (ships)     disagree on 0
    still disagreeing under C: none
    fixed by the diacritic fold alone: ['iot-market-dynamics-an-analysis-of-devic', 'addressing-the-address-books-interdepend']

=== all 162 fetched landing pages: 155 carry BOTH citation_author and a BibTeX block
    A  non-greedy brace, no fold                   disagree on 17
    B  brace-balanced, no fold                     disagree on 4
    C  brace-balanced + diacritic fold (ships)     disagree on 0
    still disagreeing under C: none
    fixed by the diacritic fold alone: ['vulnerability-disclosure-in-the-age-of-s', 'dissecting-privacy-perspectives-of-websi', 'iot-market-dynamics-an-analysis-of-devic', 'addressing-the-address-books-interdepend']

bib_accent_encoding.py

#!/usr/bin/env python3
"""Count how literature:bibliography spells accented author names.
 
The audit needs this because the author-field diff has to decode LaTeX accents
before comparing, and the reason it has to is that the file uses BOTH
conventions. That was asserted in prose on the provenance page with no script
behind it; three independent counters then produced three different splits
(44/67, 43/66, 45/66), because "has a LaTeX escape" and "has a non-ASCII letter"
are not complements and the edge cases sit in neither or both. This defines the
buckets explicitly and prints the edge cases instead of hiding them.
 
Only the AUTHOR field is counted: an accent in a title has no bearing on the
author diff.
 
    python3 scripts/bib_accent_encoding.py [--bib out/usenix_audit/bib_live.txt]
"""
import os
import re
import sys
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field
 
# A LaTeX ACCENT command: \'e, \"u, \c{s}, \v{c}, \~a, \^i, \=o, \.z, \u{a},
# \H{o}, \r{a}, braced or not. Deliberately NOT \ss, \o, \aa, \O, \AA — those
# are ligature/letter commands, not accents, and they are reported separately
# because they are exactly what made the three earlier counts disagree.
ACCENT = re.compile(r"\\[\'\"`^~=.]|\\[cvuHr]\s*\{")
LETTER_CMD = re.compile(r"\\(ss|o|O|aa|AA|l|L|ae|AE|oe|OE)\b|\\\{?(ss|o|O|aa|AA)\}")
NONASCII = re.compile(r"[^\x00-\x7f]")
 
 
def main():
    bib = sys.argv[sys.argv.index("--bib") + 1] if "--bib" in sys.argv \
        else "out/usenix_audit/bib_live.txt"
    entries = re.findall(r"@\w+\{[^@]*?\n\}",
                         open(bib, encoding="utf-8").read(), re.S)
    n = len(entries)
    acc, uni, cmd, both, acc_cmd, neither = [], [], [], [], [], 0
    for e in entries:
        key = re.match(r"@\w+\{([^,]+),", e).group(1)
        a = field(e, "author") or ""
        A, U, C = bool(ACCENT.search(a)), bool(NONASCII.search(a)), \
            bool(LETTER_CMD.search(a))
        if A:
            acc.append(key)
        if U:
            uni.append(key)
        if C:
            cmd.append(key)
        if A and U:
            both.append(key)
        if A and C:
            acc_cmd.append(key)
        if not (A or U or C):
            neither += 1
 
    print(f"bibliography                                  : {bib}")
    print(f"entries                                       : {n}")
    print(f"AUTHOR field has a LaTeX accent command       : {len(acc)}")
    print(f"AUTHOR field has a literal non-ASCII letter   : {len(uni)}")
    print(f"AUTHOR field has a LaTeX LETTER command      : {len(cmd)}  "
          f"(\\ss, \\o, \\aa — not accents)")
    print(f"AUTHOR field has BOTH accent and non-ASCII    : {len(both)}")
    print(f"AUTHOR field has BOTH accent and letter command: {len(acc_cmd)}")
    print(f"AUTHOR field is plain ASCII, no commands      : {neither}")
    print(f"AUTHOR field is plain ASCII, no commands      : {neither}"
          if False else "", end="")
    print(f"\nThe buckets OVERLAP — do not add the rows up. Distinct entries "
          f"touched by any of the three: {len(set(acc) | set(uni) | set(cmd))}")
    print(f"Distinct entries using an accent command or a non-ASCII letter: "
          f"{len(set(acc) | set(uni))}")
    if both:
        print("\nMIXED — these entries spell one name each way:")
        for k in both:
            print(f"  {k}")
    if cmd:
        print("\nLaTeX LETTER commands (the reason three earlier hand counts "
              "disagreed — they are neither accents nor non-ASCII):")
        for k in cmd:
            e = next(x for x in entries if x.startswith("@") and
                     re.match(r"@\w+\{([^,]+),", x).group(1) == k)
            m = LETTER_CMD.search(field(e, "author"))
            print(f"  {k:<32} {m.group(0)!r}")
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

bibliography                                  : out/usenix_audit/bib_live.txt
entries                                       : 837
AUTHOR field has a LaTeX accent command       : 43
AUTHOR field has a literal non-ASCII letter   : 66
AUTHOR field has a LaTeX LETTER command      : 3  (\ss, \o, \aa — not accents)
AUTHOR field has BOTH accent and non-ASCII    : 0
AUTHOR field has BOTH accent and letter command: 1
AUTHOR field is plain ASCII, no commands      : 726

The buckets OVERLAP — do not add the rows up. Distinct entries touched by any of the three: 111
Distinct entries using an accent command or a non-ASCII letter: 109

LaTeX LETTER commands (the reason three earlier hand counts disagreed — they are neither accents nor non-ASCII):
  demir2023_similarity             '\\ss'
  mackinnon2023_clusterrobust      '\\O'
  coopamootoo2017_forgotten        '\\ss'

popets_doi_check.py

#!/usr/bin/env python3
"""Check every PoPETs DOI in literature:bibliography against the correct prefix
boundary, and resolve each one.
 
scripts/bibgen.mjs used to hardcode 10.56553 for all years. PoPETs has changed
DOI prefix twice:
 
    2015 - 2018        10.1515    (De Gruyter Open)
    2019 - 2022-0058   10.2478    (De Gruyter / Sciendo)
    2022-0059 onward   10.56553   (PoPETs' own prefix)
 
so the hardcoded prefix produced a dead DOI for every PoPETs paper before
2022-0059. Fixed 2026-08-13. This checks what actually reached the wiki:
a prefix that disagrees with the boundary is reported whether or not it
resolves, and a DOI that does not resolve is reported whether or not the prefix
is right — the two failures are independent.
 
    python3 scripts/popets_doi_check.py [--bib out/usenix_audit/bib_live.txt]
                                        [--no-network]
"""
import os
import re
import subprocess
import sys
import time
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field
 
DEFAULT_BIB = "out/usenix_audit/bib_live.txt"
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
      "Chrome/126.0.0.0 Safari/537.36")
 
 
def expected_prefix(year, article):
    y, n = int(year), int(article)
    if y <= 2018:
        return "10.1515"
    if y < 2022 or (y == 2022 and n <= 58):
        return "10.2478"
    return "10.56553"
 
 
def resolve(doi):
    """HTTP status of https://doi.org/<doi>, following redirects."""
    r = subprocess.run(
        ["curl", "-sIL", "-A", UA, "--max-time", "30",
         "-o", "/dev/null", "-w", "%{http_code} %{url_effective}",
         f"https://doi.org/{doi}"], capture_output=True, text=True)
    code, _, url = r.stdout.strip().partition(" ")
    return code, url
 
 
def main():
    bib = sys.argv[sys.argv.index("--bib") + 1] if "--bib" in sys.argv \
        else DEFAULT_BIB
    network = "--no-network" not in sys.argv
    entries = re.findall(r"@\w+\{[^@]*?\n\}",
                         open(bib, encoding="utf-8").read(), re.S)
 
    rows, no_doi = [], []
    for e in entries:
        series = (field(e, "series") or "") + " " + (field(e, "booktitle") or "")
        if not re.search(r"PoPETs|Privacy Enhancing Technolog", series, re.I):
            continue
        key = re.match(r"@\w+\{([^,]+),", e).group(1)
        doi = field(e, "doi")
        if not doi:
            no_doi.append((key, field(e, "url") or "no url either"))
            continue
        doi = re.sub(r"^https?://(dx\.)?doi\.org/", "", doi)
        m = re.search(r"popets-(\d{4})-(\d{4})", doi)
        exp = expected_prefix(*m.groups()) if m else None
        rows.append([key, doi, exp, doi.split("/")[0], "", ""])
 
    print(f"bibliography              : {bib}")
    print(f"PoPETs entries with a DOI : {len(rows)}")
    print(f"PoPETs entries without    : {len(no_doi)}")
 
    if network:
        for r in rows:
            r[4], r[5] = resolve(r[1])
            time.sleep(0.3)
 
    bad_prefix = [r for r in rows if r[2] and r[2] != r[3]]
    unparsed = [r for r in rows if not r[2]]
    dead = [r for r in rows if network and not r[4].startswith(("2", "3"))]
 
    print(f"  prefix disagrees with the boundary : {len(bad_prefix)}")
    print(f"  DOI not in popets-YYYY-NNNN form   : {len(unparsed)}")
    print(f"  does not resolve                   : "
          f"{len(dead) if network else 'not checked (--no-network)'}")
    print()
    print("%-34s %-26s %-9s %-5s %s"
          % ("citekey", "doi", "expected", "http", "resolves to"))
    for key, doi, exp, got, code, url in sorted(rows):
        flag = "  <-- PREFIX" if exp and exp != got else ""
        print("%-34s %-26s %-9s %-5s %s%s"
              % (key, doi, exp or "n/a", code or "-", url[:60], flag))
    for key, url in no_doi:
        print(f"{key:<34} NO DOI                     -         -     {url}")
    return 1 if (bad_prefix or dead) else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

Real output, abridged here to its counters — the full 86-row table is in scripts/popets_doi_check-output.txt:

bibliography              : out/usenix_audit/bib_live.txt
PoPETs entries with a DOI : 84
PoPETs entries without    : 2
  prefix disagrees with the boundary : 0
  DOI not in popets-YYYY-NNNN form   : 1
  does not resolve                   : 0

citekey                            doi                        expected  http  resolves to

bib_doi_duplicates.py

#!/usr/bin/env python3
"""Scan literature:bibliography for two entries describing the same paper.
 
A citekey-collision check passes while the same paper sits in the file twice
under different keys, and the bibtex4dw plugin will happily render both in the
reference list. Match on DOI first (exact), then on squashed title for the
entries that have no DOI.
 
    python3 scripts/bib_doi_duplicates.py [--bib out/usenix_audit/bib_live.txt]
"""
import os
import re
import sys
from collections import defaultdict
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field, squash
 
bib = sys.argv[sys.argv.index("--bib") + 1] if "--bib" in sys.argv \
    else "out/usenix_audit/bib_live.txt"
entries = re.findall(r"@\w+\{[^@]*?\n\}", open(bib, encoding="utf-8").read(), re.S)
 
by_doi, by_title, keys = defaultdict(list), defaultdict(list), []
for e in entries:
    k = re.match(r"@\w+\{([^,]+),", e).group(1)
    keys.append(k)
    doi = field(e, "doi")
    if doi:
        by_doi[re.sub(r"^https?://(dx\.)?doi\.org/", "", doi).lower()].append(k)
    t = field(e, "title")
    if t:
        by_title[squash(t)].append(k)
 
print(f"bibliography     : {bib}")
print(f"entries          : {len(entries)}")
print(f"distinct citekeys: {len(set(keys))}")
dupkeys = sorted({k for k in keys if keys.count(k) > 1})
print(f"COLLIDING citekeys                  : {len(dupkeys)}"
      + (f"  {dupkeys}" if dupkeys else ""))
 
d_doi = {k: v for k, v in by_doi.items() if len(v) > 1}
d_tit = {k: v for k, v in by_title.items()
         if len(v) > 1 and not any(set(v) <= set(x) for x in d_doi.values())}
print(f"same DOI under different citekeys   : {len(d_doi)}")
for doi, ks in sorted(d_doi.items()):
    print(f"  {doi}\n      {', '.join(ks)}")
print(f"same TITLE, no shared DOI           : {len(d_tit)}")
for t, ks in sorted(d_tit.items()):
    print(f"  {t[:70]}\n      {', '.join(ks)}")
sys.exit(1 if (dupkeys or d_doi or d_tit) else 0)

bibliography     : out/usenix_audit/bib_live.txt
entries          : 837
distinct citekeys: 837
COLLIDING citekeys                  : 0
same DOI under different citekeys   : 3
  10.56553/popets-2022-0063
      fouad2022my, fouad2022_cookie
  10.56553/popets-2025-0063
      bottger2025_regional, boettger2025_regional
  10.56553/popets-2026-0109
      ahmad2026_ipfp, ahmad2026_more
same TITLE, no shared DOI           : 2
  automatedlargescaleanalysisofcookienoticecompliance
      bouhoula2024automated, bouhoula2024_automated
  internetjonesandtheraidersofthelosttrackersanarchaeologicalstudyofwebt
      lerner2016internet, lerner2016_internet

bib_dupe_cocitation.py

#!/usr/bin/env python3
"""Two small checks the provenance page was asserting without a script.
 
  1. For each duplicate pair from scripts/bib_doi_duplicates.py, does any wiki
     page cite BOTH keys? If one does, its rendered reference list shows the
     same paper twice — that is the difference between a latent duplicate and a
     visible one, and the page's "nothing renders wrong today" rests on it.
  2. The 42 PETS entries in out/authors.json come from a different parser
     (parse_popets) against a different page shape, so they are out of scope for
     the USENIX audit. This is the smoke test that says so honestly: an
     affiliation-shaped or single-token name in any of them. It is NOT the
     three-source audit the USENIX keys got, and it is labelled that way.
 
Reads the page exports under out/usenix_audit/pages/, made with
`node scripts/dw.mjs pages` + `?do=export_raw` per page.
 
    python3 scripts/bib_dupe_cocitation.py
"""
import glob
import json
import os
import re
import sys
 
PAGES = "out/usenix_audit/pages"
BIBPAGE = "literature_bibliography"
 
PAIRS = [("fouad2022my", "fouad2022_cookie"),
         ("bottger2025_regional", "boettger2025_regional"),
         ("ahmad2026_ipfp", "ahmad2026_more"),
         ("bouhoula2024automated", "bouhoula2024_automated"),
         ("lerner2016internet", "lerner2016_internet")]
 
AFFIL = re.compile(
    r"universi|institut|college|inc\.|labs?\b|corporat|technolog|research|"
    r"school|center|centre|gmbh|ltd|cispa|kaist|inria|microsoft|google|meta|"
    r"apple|amazon|ibm|eth |epfl|tu |laborator|foundation|agency|department|"
    r"polytechnic|\bteam\b|e\.?\s?v\.?$", re.I)
 
 
def main():
    files = [f for f in sorted(glob.glob(os.path.join(PAGES, "*.txt")))
             if BIBPAGE not in os.path.basename(f)]
    pages = {os.path.basename(f)[:-4]: open(f, encoding="utf-8",
                                            errors="replace").read()
             for f in files}
    print(f"page exports read (literature:bibliography excluded) : {len(pages)}")
 
    worst, touched = 0, set()
    print("\n^ duplicate pair ^ pages citing key A ^ citing key B ^ citing BOTH ^")
    for a, b in PAIRS:
        ca = {p for p, t in pages.items() if "{[" + a + "]}" in t}
        cb = {p for p, t in pages.items() if "{[" + b + "]}" in t}
        both = sorted(ca & cb)
        worst = max(worst, len(both))
        touched |= ca | cb
        print(f"| {a} / {b} | {len(ca)} | {len(cb)} | "
              f"{'**' + ', '.join(both) + '**' if both else '0'} |")
    print(f"\npages citing at least one key of any pair : {len(touched)}")
    print(f"pages citing BOTH keys of any pair       : {worst}")
    print("A page citing both would render the same paper twice in its "
          "reference list.")
 
    cache = json.load(open("out/authors.json"))
    pets = {k: v for k, v in cache.items() if k.startswith("PETS/")}
    flagged = {k: [n for n in v
                   if AFFIL.search(n) or len(n.split()) < 2]
               for k, v in pets.items()}
    flagged = {k: v for k, v in flagged.items() if v}
    print(f"\nPETS entries in out/authors.json                : {len(pets)}")
    print(f"  with an affiliation-shaped or single-token name: {len(flagged)}")
    for k, v in sorted(flagged.items()):
        print(f"    {k}: {v}")
    print("  This is a smoke test over parse_popets output, not the "
          "three-source audit the USENIX keys got.")
    return 1 if (worst or flagged) else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

page exports read (literature:bibliography excluded) : 160

^ duplicate pair ^ pages citing key A ^ citing key B ^ citing BOTH ^
| fouad2022my / fouad2022_cookie | 4 | 2 | 0 |
| bottger2025_regional / boettger2025_regional | 2 | 2 | 0 |
| ahmad2026_ipfp / ahmad2026_more | 1 | 1 | 0 |
| bouhoula2024automated / bouhoula2024_automated | 4 | 11 | 0 |
| lerner2016internet / lerner2016_internet | 3 | 2 | 0 |

pages citing at least one key of any pair : 26
pages citing BOTH keys of any pair       : 0
A page citing both would render the same paper twice in its reference list.

PETS entries in out/authors.json                : 42
  with an affiliation-shaped or single-token name: 0
  This is a smoke test over parse_popets output, not the three-source audit the USENIX keys got.

Run on the 855-entry export before the change (output above, under the audit) and on the saved 850-entry page:

bib_dedup_scan.py

#!/usr/bin/env python3
"""Find every pair of literature:bibliography entries that may describe one paper.
 
scripts/bib_doi_duplicates.py matches on exact DOI and exact squashed title.
That found five pairs on 2026-09-04, and one of them was an umlaut
transliteration in the citekey (bottger / boettger). The same slip can also
land in the TITLE (a subtitle dropped, "{IPv6}" braced) or leave no shared DOI,
so this scan casts wider and prints its candidates for a human to judge:
 
  A  same DOI                                   (definite)
  B  same squashed title                        (definite)
  C  near-identical title, difflib ratio >= 0.85 on the squashed form,
     or one squashed title a prefix of the other (subtitle dropped)
  D  same year AND same folded first-author surname — the fold strips
     diacritics and collapses the German digraphs oe/ue/ae/ss to o/u/a/s, so
     Böttger, B\"ottger, Boettger and Bottger all compare equal
 
C and D are candidate generators, not verdicts. Every C/D pair that is not
already in A/B is printed with both titles so the residue is visible; the
verdicts recorded on provenance:literature:bibliography are hand judgements
over that printed list, not the script's.
 
    python3 scripts/bib_dedup_scan.py --bib out/live_bibliography_YYYYMMDD.txt
"""
import difflib
import os
import re
import sys
import unicodedata
from collections import defaultdict
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field, squash
 
LATEX = {r'\"o': 'oe', r'\"u': 'ue', r'\"a': 'ae', r'\"O': 'Oe', r'\"U': 'Ue',
         r'\"A': 'Ae', r'\ss': 'ss'}
 
 
def fold_surname(author_field):
    """First author's surname, lower-cased, diacritics stripped, German
    digraphs collapsed. Handles 'Last, First' and 'First Last'."""
    first = re.split(r"\s+and\s+", author_field)[0]
    s = first
    for k, v in LATEX.items():
        s = s.replace(k, v)
    s = re.sub(r"\\[\'`^~vcH=.]\{?(\w)\}?", r"\1", s)        # \'e, \v{c}, {\'e}
    s = re.sub(r"[{}]", "", s)
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c))
    last = s.split(",")[0].strip() if "," in s else s.strip().split()[-1]
    last = last.lower()
    last = re.sub(r"[^a-z]", "", last)
    for dg, one in (("oe", "o"), ("ue", "u"), ("ae", "a"), ("ss", "s")):
        last = last.replace(dg, one)
    return last
 
 
def main():
    bib = sys.argv[sys.argv.index("--bib") + 1]
    text = open(bib, encoding="utf-8").read()
    entries = re.findall(r"@\w+\{[^@]*?\n\}", text, re.S)
    rec = {}
    for e in entries:
        k = re.match(r"@\w+\{([^,]+),", e).group(1)
        if k in rec:
            raise SystemExit(f"colliding citekey {k}")
        doi = field(e, "doi") or ""
        doi = re.sub(r"^https?://(dx\.)?doi\.org/", "", doi).lower()
        title = field(e, "title") or ""
        author = field(e, "author") or ""
        year = field(e, "year") or ""
        rec[k] = dict(doi=doi, title=title, sq=squash(title), year=year,
                      surname=fold_surname(author) if author else "")
    print(f"bibliography : {bib}")
    print(f"entries      : {len(entries)}   distinct citekeys: {len(rec)}")
 
    pairs = {}  # frozenset(k1,k2) -> set of rules
 
    def add(a, b, rule):
        pairs.setdefault(frozenset((a, b)), set()).add(rule)
 
    by = defaultdict(list)
    for k, r in rec.items():
        if r["doi"]:
            by[("A", r["doi"])].append(k)
        if r["sq"]:
            by[("B", r["sq"])].append(k)
        if r["year"] and r["surname"]:
            by[("D", r["year"], r["surname"])].append(k)
    for key, ks in by.items():
        for i in range(len(ks)):
            for j in range(i + 1, len(ks)):
                add(ks[i], ks[j], key[0])
    keys = sorted(rec)
    for i in range(len(keys)):
        a = rec[keys[i]]["sq"]
        if len(a) < 20:
            continue
        for j in range(i + 1, len(keys)):
            b = rec[keys[j]]["sq"]
            if len(b) < 20:
                continue
            if a.startswith(b) or b.startswith(a) or \
               difflib.SequenceMatcher(None, a, b).ratio() >= 0.85:
                add(keys[i], keys[j], "C")
 
    definite = {p: r for p, r in pairs.items() if r & {"A", "B"}}
    candidates = {p: r for p, r in pairs.items() if not r & {"A", "B"}}
    print(f"\nDEFINITE duplicate pairs (rule A or B): {len(definite)}")
    for p, rules in sorted(definite.items(), key=lambda x: sorted(x[0])):
        a, b = sorted(p)
        print(f"  [{''.join(sorted(rules))}] {a}  /  {b}")
        print(f"       {rec[a]['title'][:90]}")
        print(f"       {rec[b]['title'][:90]}")
    print(f"\nCANDIDATE pairs (rule C or D only) — judged by hand, see the "
          f"provenance page: {len(candidates)}")
    for p, rules in sorted(candidates.items(), key=lambda x: sorted(x[0])):
        a, b = sorted(p)
        print(f"  [{''.join(sorted(rules))}] {a}  /  {b}")
        print(f"       {rec[a]['year']} {rec[a]['title'][:100]}")
        print(f"       {rec[b]['year']} {rec[b]['title'][:100]}")
    return 1 if definite else 0
 
 
if __name__ == "__main__":
    sys.exit(main())