Back to the bibliography. Corpus-wide selection and extraction notes belong on corpus (not yet written). This page is the query log for the bibliography itself: where each entry's metadata came from, what has been checked against a primary source, and what is known to be wrong with it.

The bibliography is not a content page and carries no figures, so this provenance page records audits of it rather than the derivation of a percentage. Each audit gets its own section, dated, with the script that ran it and that script's real output.

• Run date: 2026-09-04 (UTC). First revision of this page.
• Authoring agent: Claude Opus 5, executing the drain item audit-usenix-author-lists non-interactively. No human in the loop during the run.
• Corpus at run time: 5,859 extracted papers, 2010–2026, over CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P. Read-only inputs under /workspace/publications_dataset/data/.
• Live bibliography read at revision 1788471646, 382,620 bytes, 837 entries, exported with ?do=export_raw and kept at out/usenix_audit/bib_live.txt. Every figure below is against that snapshot.
• Nothing on literature:bibliography was edited by this run. The only content change is the one-line pointer to this page.
• scripts/fetch_authors.py and out/authors.json were changed. out/usenix_audit/authors.json.before is the pre-run snapshot and is what the audit diffs against.

Every figure below is derived and defended further down; this is here so that someone checking one number does not have to read the history first.

scripts/fetch_authors.py fills in author lists for PETS and USENIX, whose records in data/corpus2/.meta carry no authors, because they were scraped from venue listing pages rather than from OpenAlex. Re-derived against the current 16,864-record index on 2026-09-04, not carried over from the old one:

(That table is corpus-level and belongs on corpus once it exists; it is here because the author gap is the reason this audit had to happen at all.) So the author gap is PETS and USENIX and essentially nothing else — 4,265 of 16,864 records, plus a single stray NDSS record (NDSS/2018/veil-private-browsing-semantics-without-browser-side-assistance, authors: null) — while the DOI gap is narrower than the author gap for PETS: 1,113 of its 1,253 records do carry one. For USENIX it parsed the prose blob a presentation page prints under the title, which has the shape

A, Affil1; B, C, and D, Affil2; E, Affil3

Three separate defects have now been found in that parser. The first two were fixed on 2026-08-13 and are what this drain item was raised to clean up after; the third was found by this audit and had survived that fix.

Defect 3 is not fixable by any rule over the prose that does not encode a list of institutions, which is why the fix below stops parsing the prose.

Every USENIX page publishes the author list unambiguously somewhere other than the prose. The audit reads them in this order and requires the first two to agree:

Source 3 is the weakest of the three, so the entries resting on it are named rather than counted. All seven, from scripts/usenix_source_agreement.py:

Six are legacy session pages, from 2010 and 2011, not 2011 alone as an earlier draft said. The seventh is different in kind and is the only claim on this page that is a negative about a live third-party page that can change under it. usenixsecurity25/presentation/tang was re-fetched on 2026-09-04 and still carries zero citation_author tags, zero BibTeX block, and the single line <p>Brian Tang, Duc Bui, and Kang G. Shin, <em>University of Michigan</em></p>. Unlike everything else on this page that is a negative about a mutable external page, so it is dated rather than stated flat. USENIX's structure is otherwise stable at least one cycle forward: a usenixsecurity26 presentation page checked the same day carries both citation_author tags and a BibTeX block, and none of the four 2024–2026 pages checked carries a citation_doi. That forward check is one page (usenixsecurity26/presentation/hu-zhenkai, 7 citation_author tags and a BibTeX block), so it is evidence the structure has not been dropped, not evidence that it will not be.

Sources 1 and 2 are present together or not at all: no page in this population carries one without the other. The 162 pages are the 90 cached author lists plus the 72 extra fetched for pass 3 below.

Sources 1 and 2 disagreed on a long list of papers at first, and every one of those disagreements was a bug in the reading, not in the data. The script that produced that first list no longer exists, so the figure is not stated from memory: scripts/usenix_source_agreement.py re-derives the comparison from the same cached HTML under each of the three parser states this audit went through.

Two reading bugs and one genuine source difference sit behind that table:

• A → B. A non-greedy brace match ends at the first inner brace, so Adam Doup{\'e} came back as Adam Doup{\'e. Brace-counting fixes it.
• B → C. For iot-market-dynamics USENIX's meta says Carlos H. Gañán and USENIX's own BibTeX says Carlos H. Ganan. That one is a genuine difference between USENIX's two sources; source 1 is preferred for exactly that reason, and the diacritic fold is what stops it being reported as a defect.
• The third, found in review on 2026-09-04. USENIX writes Benoît as Beno{\^\i}t, using LaTeX's dotless \i. Nothing in the decoder handled \i, so no accent pattern matched and the final brace strip produced Beno^it. It survived every other check because it is on cohn2020_delf, which is not one of the 90 cached lists — only the all-162 comparison could see it. usenix_author_audit.py –selftest now carries a nine-case de_latex test including that exact string.

So: three decoder bugs, and four pages where USENIX's own BibTeX drops a diacritic its meta keeps. Calling the second kind “agreement” is a choice — the fold makes Gañán and Ganan compare equal — and it is the right one here, because the two sources are naming the same person and source 1 is preferred anyway. Under that rule USENIX's two structured sources agree on all 155 pages that carry both. Under a byte-exact rule, four would not.

Three papers are overridden against a source outside usenix.org. Each override is a literal list in ADJUDICATED in the script, so it is visible rather than buried in a heuristic, and emptying that table makes the run fail (mutation A6).

DBLP is the adjudicating source in two of the three rows. Its search API (dblp.org/search/publ/api?q=…&format=json) answered correctly on 2026-09-04, but DBLP's own FAQ describes that API as “still in flux … work-in-progress”, so a later re-audit should re-verify the endpoint rather than assume it.¹⁾

The PhishTime case is the one that matters for method, and its second half matters more than its first. Had the audit trusted the structured metadata alone, it would have deleted a real author while “correcting” the entry — which is why every paper is additionally cross-checked against the prose blob. But the natural fallback for a disagreement is DBLP, and DBLP is wrong here too: record conf/uss/OestSZWTSD20, fetched 2026-09-04, lists seven authors and stops at Adam Doupé, exactly as USENIX's metadata does. Three independent machine-readable sources agree on a list the paper itself contradicts:

$ head -5 data/fulltext/2020/USENIX/phishtime-.../paper.cols.txt
PhishTime: Continuous Longitudinal Measurement
 of the Effectiveness of Anti-phishing Blacklists
 Adam Oest, Yeganeh Safaei, and Penghui Zhang, Arizona State University;
 Brad Wardman and Kevin Tyers, PayPal; Yan Shoshitaishvili and Adam Doupé,
Arizona State University; Gail-Joon Ahn, Arizona State University, Samsung Research

The published entry follows the paper. An earlier draft of this page claimed DBLP corroborated the correction. It does not, and a reviewer caught that on 2026-09-04.

Population: the 90 USENIX keys in out/authors.json as of 2026-09-04. Not 5,859 papers, not 837 bibliography entries — 90 cached author lists.

Defects 1 and 2 left no trace in out/authors.json. Not one of the 90 cached lists was missing an author. The drain item's premise — that entries written before 2026-08-13 may be short an author — did not hold for anything still in the cache. What 18 of the 90 did carry was defect 3: an affiliation fragment sitting in the list as if it were a person. Berkeley, Santa Barbara, San Diego, PayPal, Trustwave, AppCensus, ICSI, UC Davis, UCL, UC3M, Georgia Tech, DeustoTech, Eurecom, BIT, BUPT, USTC, Engineering, Behavioural Insights Team. [5Kapravelos, Alexandros; Grier, Chris; Chachra, Neha; Kruegel, Christopher; Vigna, Giovanni; Paxson, Vern (2014): "Hulk: Eliciting Malicious Behavior in Browser Extensions", in: Proceedings of the USENIX Security Symposium. (Link)] was the worst: 11 names cached for a 6-author paper, alternating author, campus, author, campus.

The last row is there because it is the common case: 72 of the 90 needed no change at all. The defect fires only when an affiliation contains a comma and its tail happens to look like a surname.

This is the part worth stating carefully, because it is a negative result and a negative result from a check that has not been shown able to fail is worth nothing.

Pass 3 checks order because an earlier version of it did not. Its comment claimed it reported “separately on order”; the code compared only Python sets, and a reviewer demonstrated the gap on 2026-09-04 by swapping the first and last author of acharya2024_imitation — a pass-3-only entry, so passes 1 and 2 are blind to it by construction — and getting a clean bill from all three passes. A first-authorship swap is a worse citation defect than a missing middle author, and it was the one defect class nothing on this page could see. It is now mutation B5, and it is the only mutation that PASS3order catches alone.

Pass 1 first reported 3 “wrong” entries. All three were the encoding difference between out/authors.json (literal UTF-8) and the bibliography (Doup{\'e}), not an author defect: 43 of the 837 live entries write accents as a LaTeX accent command and 66 as a literal non-ASCII letter, both are valid BibTeX, and both were confirmed to render as Doupé against pages that actually cite an entry using each form — security:phishing for the braced Doup{\'e} and privacy:browser_extensions for the unbraced Doup\'e. A first draft named programming:crawler_detection instead, which does render Doupé but from zhang2021_crawlphish and zhang2022_spartacus, whose author fields carry a literal é — so it verified nothing. Caught in review. The comparison now decodes LaTeX accents before diffing, and the house style was left alone: rewriting 43 author fields on the page every other page depends on, for a cosmetic gain, is a bad trade.

So the broken parser contaminated the cache and never the page. The entries were evidently cleaned by hand between bibgen.mjs and the save — kapravelos2014_hulk is live with exactly its 6 real authors. Pass 3 extends the guarantee past the 85: all 159 live USENIX Security entries match the author list their own landing page publishes.

The 159 entries resolve to only 157 distinct papers, and the two-entry gap is two of the duplicate pairs found below — bouhoula2024automated / bouhoula2024_automated and lerner2016internet / lerner2016_internet are both USENIX Security. The three populations reconcile exactly: 157 papers behind live entries, plus the 5 cached keys nothing cites, is the 162 landing pages on disk.

Thirteen mutations — seven against the audit script, six against the bibliography — each breaking one thing the scripts rely on. (A2 is the exception and breaks two, for the reason given below.) A mutation that left the reported counts unchanged would be a rule asserting nothing. Real output of scripts/usenix_author_audit_mutations.py:

=== scripts/usenix_author_audit.py — diffed against out/usenix_audit/authors.json.before
mutation                                                   rc   unchanged  corrected  unresolved
(baseline, unmutated)                                      0    72         18         0
A1  citation_author parser truncated to 1 author           1    3          0          87
A2  prose cross-check disabled *and* adjudication emptied  0    70         20         0
A3  BibTeX brace balancing broken                          1    3          0          87
A4  legacy block no longer matched by title                1    69         18         3
A5  diacritic folding removed                              1    70         18         2
A6  adjudication table emptied                             1    70         18         2
A7  institution list ignored (prose residue no longer filtered) 1    64         10         16

=== scripts/usenix_bib_diff.py — mutated BIBLIOGRAPHIES
mutation                                                   PASS1   PASS2   PASS3set PASS3order
(baseline, unmutated)                                      0       0       0       0
B1  author deleted from a cached entry                     1       0       1       0
B2  affiliation added to a cached entry                    1       1       1       0
B3  affiliation added to a NON-cached entry                0       1       1       0
B4  given/family names swapped                             1       0       1       0
B5  author ORDER swapped, NON-cached entry                 0       0       0       1
B6  author DELETED from a NON-cached entry                 0       0       1       0

mutations that changed nothing (a rule asserting nothing): 0

B6 is the one that justifies pass 3 existing at all: an author deleted from an entry out/authors.json never touched is invisible to pass 1 (not cached) and to pass 2 (nothing affiliation-shaped was added), and only the landing-page comparison sees it. B5 is the order swap, and only the ordered comparison sees that.

A2 is the one to read closely. On its own, disabling the prose cross-check changes nothing, because every paper it fires on is already in ADJUDICATED and that branch runs first — today the cross-check is a tripwire for the next refresh, not a live assertion. Mutating both together shows what it is worth: without it, [2McCoy, Damon; Pitsillidis, Andreas; Jordan, Grant; Weaver, Nicholas; Kreibich, Christian; Krebs, Brian; Voelker, Geoffrey M.; Savage, Stefan; Levchenko, Kirill (2012): "PharmaLeaks: Understanding the Business of Online Pharmaceutical Affiliate Programs", in: Proceedings of the USENIX Security Symposium. (Link)] silently regresses to the wrong Jordan Grant (corrected goes 18 → 20) instead of stopping the run. Reporting A2 as a pass on its own would have been the exact “passing check that asserts nothing” this table exists to rule out.

Two normalisations, both with their residue accounted for.

• Name comparison folds Unicode diacritics away (NFKD, combining marks dropped), lower-cases, and strips punctuation — but not word count, so Geoffrey Voelker and Geoffrey M. Voelker compare as different and were adjudicated rather than silently merged. Removing the diacritic fold makes 2 papers unresolved (mutation A5), so the fold is doing work.
• The prose cross-check needs to tell a person's name from an affiliation tail in the ambiguous blob. The first version used a hand-written stop-list of institution tail words, which is the kind of list that needs a new entry per venue-year and fails silently when it misses one. It was replaced by a rule that needs no maintenance: drop any candidate whose folded form appears in one of the affiliations the page itself declares — its citation_author_institution meta tags, or its <i>/<em> spans.

Residue of that fold: zero, in the direction that matters — everything the rule drops is an affiliation tail, verified one by one. It says nothing about the other direction: a real author whose folded name is contained in a declared affiliation (someone called Max Planck at a Max Planck Institute) would be dropped from the cross-check silently. The risk is small because prose_parse feeds only the tripwire and never the published list, but it is a direction this fold does not cover. Turning the rule off (mutation A7) leaves 16 papers unresolved on 15 distinct affiliation fragments, and every one of them is contained in a declared affiliation on its own page:

barbara  berkeley  canada  d  davis  diego  electronics  icsi
leuven   madrid    team    tech  transformation  uk  v

d is the tail of NCS Cyber Special Ops-R&D and v the tail of Datenanfragen.de e.V. — both are what is left of an institution after the comma split, which is the same defect the whole audit is about, seen from the other side.

There are no evidence.quote values behind this page — it publishes no extraction figure. What was checked against a primary source:

The three adjudications above are load-bearing and their sources are in that table; they are not repeated here. What remains:

parse_usenix no longer parses the prose at all. It returns meta_authors(page) or bibtex_authors(page) or italic_authors(page, title) from scripts/usenix_author_audit.py, and applies ADJUDICATED. Verified live on three USENIX keys outside the cache (row 5 above). Note the division of labour: fetch_authors.py skips anything already cached, so it can only ever fill gaps — re-deriving a cached entry is what usenix_author_audit.py is for, and that script cross-checks and diffs before it writes.

fetch_authors.py

#!/usr/bin/env python3
"""Fetch author lists for PETS and USENIX papers, which the corpus index lacks.
 
Every PETS and USENIX record in data/corpus2/.meta was scraped from a venue
listing page and carries no authors: 4,265 of the 16,864 records, re-counted
2026-09-04 (USENIX 3,012, PETS 1,253). USENIX additionally has no DOI on any of
its 3,012; PETS has one on 1,113 of its 1,253. Their landing pages do carry the
authors. WebFetch gets 403 from usenix.org, so this uses curl with
a browser User-Agent.
 
    python3 scripts/fetch_authors.py PETS/2016/near-optimal-... [...]
 
Writes/updates out/authors.json keyed on "<venue>/<year>/<slug>". Fails loudly
per paper: a slug whose authors cannot be parsed is reported, not silently
skipped.
 
To RE-derive an entry that is already cached, this script is the wrong tool —
it skips anything present in out/authors.json. Use
scripts/usenix_author_audit.py, which re-parses from cached HTML, cross-checks
against a second source and diffs before writing.
"""
import glob
import html
import json
import os
import re
import subprocess
import sys
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_author_audit import (ADJUDICATED, bibtex_authors,
                                 italic_authors, meta_authors)
 
ROOT = "/workspace/publications_dataset/data"
CACHE = "out/authors.json"
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
      "Chrome/126.0.0.0 Safari/537.36")
 
 
def load_index():
    idx = {}
    for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
        for p in json.load(open(f))["papers"]:
            idx[f"{p['venue']}/{p['year']}/{p['slug']}"] = p
    return idx
 
 
def fetch(url):
    r = subprocess.run(["curl", "-sL", "-A", UA, "--max-time", "40", url],
                       capture_output=True, text=True)
    if r.returncode != 0:
        raise RuntimeError(f"curl failed for {url}: {r.returncode}")
    return r.stdout
 
 
def strip(s):
    s = re.sub(r"<[^>]+>", " ", s)
    return re.sub(r"\s+", " ", html.unescape(s)).strip()
 
 
def parse_popets(page):
    # PoPETs landing pages carry "Authors: A (Affil), B (Affil)" in a paragraph.
    m = re.search(r"Authors?:\s*(.*?)</", page, re.S)
    if not m:
        return None
    raw = strip(m.group(1))
    names = []
    for part in re.split(r",(?![^(]*\))", raw):
        n = re.sub(r"\([^)]*\)", "", part).strip().rstrip(",")
        if n and not n.lower().startswith("and "):
            names.append(n)
        elif n.lower().startswith("and "):
            names.append(n[4:].strip())
    return [n for n in names if n]
 
 
def parse_usenix(page, title=""):
    """Authors from a USENIX presentation page, from AUTHORITATIVE markup.
 
    This used to parse the prose blob
 
        A, Affil1; B, C, and D, Affil2; E, Affil3
 
    and that was wrong three separate ways. Two were fixed on 2026-08-13: a
    comma split applied before the ";" split, which glued the first author of
    each group onto the previous affiliation and then dropped it; and an
    unstripped leading "and ", which dropped the LAST author of every
    multi-author group. The third survived that fix and was found by the audit
    in scripts/usenix_author_audit.py on 2026-09-04: "University of California,
    Berkeley" splits into an affiliation-shaped half that gets filtered out and
    a NAME-shaped half, "Berkeley", that gets kept as an author. 18 of the 90
    USENIX entries in out/authors.json carried at least one such fragment
    ("PayPal", "UC Davis", "Trustwave", "Georgia Tech", "AppCensus").
 
    No rule over the prose can fix that without encoding a list of
    institutions. So do not parse the prose. Every USENIX page publishes the
    author list unambiguously somewhere else, in this order of preference:
 
      1. <meta name="citation_author"> — one tag per author, emitted alongside
         citation_author_institution, so USENIX has already separated them.
      2. USENIX's own BibTeX block, author = {A and B and C}.
      3. Affiliations wrapped in <i>/<em>; removing those spans leaves the
         names. Used by the 2011 session pages and by 2025+ pages USENIX has
         not yet given Scholar metadata.
 
    `title` is only needed for (3) on a legacy session page, whose URL
    addresses a whole session rather than one paper.
 
    Neither (1) nor (2) is infallible — see ADJUDICATED in
    scripts/usenix_author_audit.py for the two papers where USENIX's own
    metadata disagrees with the published paper — so a full re-derivation
    should go through that script, which also cross-checks against the prose.
    """
    return meta_authors(page) or bibtex_authors(page) \
        or italic_authors(page, title) or None
 
 
def main():
    idx = load_index()
    cache = json.load(open(CACHE)) if os.path.exists(CACHE) else {}
    failed = []
    for k in sys.argv[1:]:
        if k in cache:
            print(f"cached  {k}: {'; '.join(cache[k])}")
            continue
        p = idx[k]                        # KeyError here means a bad slug: correct.
        url = p["landingUrl"]
        page = fetch(url)
        names = (parse_popets(page) if "petsymposium" in url
                 else parse_usenix(page, p["title"]))
        if k in ADJUDICATED:
            names = ADJUDICATED[k][0]
        if not names:
            failed.append((k, url))
            print(f"FAILED  {k}  {url}")
            continue
        cache[k] = names
        print(f"ok      {k}: {'; '.join(names)}")
    json.dump(cache, open(CACHE, "w"), indent=1, sort_keys=True)
    if failed:
        print(f"\n{len(failed)} unparsed; fill by hand in {CACHE}:")
        for k, u in failed:
            print(f"  {k}  {u}")
        return 1
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

scripts/bibgen.mjs used to hardcode the DOI prefix 10.56553 for all years, which produces a dead DOI for every PoPETs paper before 2022-0059. Fixed 2026-08-13. PoPETs has changed prefix twice:

The boundaries were tested from both sides on 2026-09-04, not assumed: 10.2478/popets-2022-0058 resolves 200 and 10.56553/popets-2022-0058 404s; 10.1515/popets-2018-0001 and 10.2478/popets-2019-0001 resolve and 10.1515/popets-2019-0001 404s. Prefix ownership was confirmed against api.crossref.org/prefixes/: 10.1515 and 10.2478 are both registered to Walter de Gruyter GmbH (member 374) and 10.56553 to the Privacy Enhancing Technologies Symposium Advisory Board (member 35752). No fourth range has opened for 2026 — the five 2026 entries all resolve under 10.56553.

Every PoPETs DOI on the live page was re-checked on both axes independently — a prefix can disagree with the boundary and still resolve, and a correct prefix can still be a dead DOI.

Read the last row carefully: the boundary rule covers 83 of the 84, not all 84. The one it cannot cover is eckersley2010_unique, whose DOI is 10.1007/978-3-642-14527-8_1 — a Springer LNCS chapter from PETS 2010, before PoPETs existed as a journal, so there is no popets-YYYY-NNNN number for the rule to read. It resolves 200. The script has always printed that row; an earlier draft of this page omitted it, which made 84 read as complete coverage.

So the hardcoded prefix never reached the wiki; it was caught by hand each time, as the drain item suspected. Two entries have no DOI. lukic2026_mv3 was fixed in this run: its url is a direct PDF for popets-2026-0027, so the DOI is 10.56553/popets-2026-0027, which was resolved (HTTP 200, petsymposium.org/popets/2026/popets-2026-0027.php) before being added. calzavara2026_syntactic was not: its recorded url points at a paper list page rather than the paper, so there is no article number to derive from and no DOI was found. It is noted under what could not be established.

A citekey-collision check passes while the same paper sits in the file under two different keys, so the DOI scan below was run over all 837 entries.

Colliding citekeys: 0. Distinct citekeys: 837 of 837 entries.

No page currently cites both keys of a pair, checked by scripts/bib_dupe_cocitation.py over the 160 other pages on the wiki (161 in dw.mjs pages, less literature:bibliography itself). 26 pages cite at least one key of some pair; bouhoula2024_automated alone is on 11. so no reference list renders the paper twice today. This is latent, not visible. Consolidating means choosing one key per pair and rewriting {[key]} markers across the 26 pages that cite one, which is a different piece of work with its own breakage risk — filed as the drain item dedupe-bibliography-entries rather than half-done here.

• Whether defects 1 and 2 ever reached the wiki. Zero missing authors survive in out/authors.json and zero in the live bibliography, but the cache carries no history, so it cannot be shown whether a dropped author was ever cached and later corrected or was never cached at all. The live page is clean either way, which is the question that matters.
• The 42 PETS entries in out/authors.json. Out of scope for this item. They come from parse_popets, a different function against a different page shape (parenthesised affiliations, so the comma ambiguity does not arise). scripts/bib_dupe_cocitation.py scans all 42 for an affiliation-shaped or single-token name and finds none — but that is a smoke test, not the three-source audit the USENIX keys got.
• The 86 live PoPETs/PETS entries' author lists. Their DOIs were checked; their authors were not. parse_popets reads a different page shape, and PETS records do carry DOIs (1,113 of 1,253), so the venue has an independent metadata path the USENIX records lack — but that is an argument for expecting fewer defects, not evidence of none.
• The other 592 live entries. Neither USENIX Security nor PoPETs; not examined at all. Their authors come from OpenAlex/Crossref via the index, never from fetch_authors.py.
• calzavara2026_syntactic has no DOI and its recorded url is a listing page. Whether the paper has a DOI yet was not established.
• Author order for the 5 keys nothing cites. Pass 3 covers every live entry, and pass 1 covers the 85 cached keys that are cited. The 5 cached-but-uncited keys have no live entry to compare against, so their order is unverified — harmless while nothing cites them, and pass 3 will cover them the moment something does.
• Whether bibgen.mjs output was hand-cleaned, or the affected entries were written from another source. The live entries are correct and the cache was not; the intermediate step left no record. The conclusion “the parser contaminated the cache and never the page” is a statement about the two endpoints, not about what happened between them.

Committed under scripts/. Run from /workspace/artifacts/wiki. Step 1 fetches and caches HTML; every later step re-parses from disk and never re-hammers usenix.org.

usenix_author_audit_fetch.py

#!/usr/bin/env python3
"""Step 1 of the USENIX author-list audit: fetch every USENIX landing page
named in out/authors.json and cache the raw HTML under out/usenix_audit/.
 
Separate from parsing on purpose: the parser was wrong twice already, so the
HTML is kept on disk and every later pass re-parses it offline instead of
re-hammering usenix.org.
"""
import glob, json, os, subprocess, sys, time
 
ROOT = "/workspace/publications_dataset/data"
OUTDIR = "out/usenix_audit/html"
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
      "Chrome/126.0.0.0 Safari/537.36")
 
idx = {}
for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
    for p in json.load(open(f))["papers"]:
        idx[f"{p['venue']}/{p['year']}/{p['slug']}"] = p
 
keys = [k for k in json.load(open("out/authors.json")) if k.startswith("USENIX/")]
os.makedirs(OUTDIR, exist_ok=True)
print(f"{len(keys)} USENIX keys in out/authors.json")
urls = {}
for i, k in enumerate(sorted(keys)):
    dest = os.path.join(OUTDIR, k.replace("/", "__") + ".html")
    url = idx[k]["landingUrl"]          # KeyError = bad slug, correct to crash
    urls[k] = url
    if os.path.exists(dest) and os.path.getsize(dest) > 2000:
        print(f"have  {k}")
        continue
    r = subprocess.run(["curl", "-sL", "-A", UA, "--max-time", "40",
                        "-w", "%{http_code}", "-o", dest, url],
                       capture_output=True, text=True)
    code = r.stdout.strip()
    size = os.path.getsize(dest) if os.path.exists(dest) else 0
    print(f"{code}   {size:>7}  {k}")
    if code != "200":
        print(f"  !! non-200 for {url}")
    time.sleep(1.0)
json.dump(urls, open("out/usenix_audit/urls.json", "w"), indent=1, sort_keys=True)

usenix_bib_fetch_all.py

#!/usr/bin/env python3
"""Extend the USENIX author audit to EVERY live USENIX Security entry.
 
out/authors.json holds 90 keys, of which 85 are cited; literature:bibliography
carries 159 entries in a USENIX Security series. The other 74 were filled from
somewhere else, so
the prose-parser bugs cannot explain a defect in them — but a dropped author
can reach a bibliography by more than one route, and pass 2 of
scripts/usenix_bib_diff.py can only see an affiliation that got IN, never an
author that fell OUT. This fetches the landing page for the rest so pass 3 can
check the whole population.
 
Resolves each entry to a landing URL by its own url= field where it has one,
and otherwise by title-matching into data/corpus2/.meta.
"""
import glob
import json
import os
import re
import subprocess
import sys
import time
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field, squash
 
ROOT = "/workspace/publications_dataset/data"
OUTDIR = "out/usenix_audit/html"
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
      "Chrome/126.0.0.0 Safari/537.36")
 
idx = {}
for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
    for p in json.load(open(f))["papers"]:
        if p["venue"] == "USENIX":
            idx[squash(p["title"])] = p
 
text = open("out/usenix_audit/bib_live.txt", encoding="utf-8").read()
entries = re.findall(r"@\w+\{[^@]*?\n\}", text, re.S)
 
targets, unresolved = {}, []
for e in entries:
    series = (field(e, "series") or "") + " " + (field(e, "booktitle") or "")
    if not re.search(r"USENIX Security", series, re.I):
        continue
    ckey = re.match(r"@\w+\{([^,]+),", e).group(1)
    p = idx.get(squash(field(e, "title")))
    url = field(e, "url")
    if p:
        targets[ckey] = (f"{p['venue']}/{p['year']}/{p['slug']}", p["landingUrl"],
                         p["title"])
    elif url and "usenix.org" in url:
        targets[ckey] = (f"url:{ckey}", url, field(e, "title"))
    else:
        unresolved.append(ckey)
 
json.dump({k: list(v) for k, v in targets.items()},
          open("out/usenix_audit/bib_targets.json", "w"), indent=1, sort_keys=True)
print(f"live USENIX Security entries resolved to a landing page: {len(targets)}")
print(f"unresolved (no index match, no usenix.org url)         : {len(unresolved)}"
      + (f"  {unresolved}" if unresolved else ""))
 
os.makedirs(OUTDIR, exist_ok=True)
n_new = 0
for ckey, (key, url, _t) in sorted(targets.items()):
    dest = os.path.join(OUTDIR, key.replace("/", "__").replace(":", "__") + ".html")
    if os.path.exists(dest) and os.path.getsize(dest) > 2000:
        continue
    r = subprocess.run(["curl", "-sL", "-A", UA, "--max-time", "40",
                        "-w", "%{http_code}", "-o", dest, url],
                       capture_output=True, text=True)
    n_new += 1
    print(f"{r.stdout.strip()}   {os.path.getsize(dest):>7}  {key}")
    time.sleep(1.0)
print(f"newly fetched: {n_new}")

usenix_author_audit.py

#!/usr/bin/env python3
"""Step 2 of the USENIX author-list audit: re-derive every USENIX author list
from AUTHORITATIVE page markup and diff against out/authors.json.
 
Why not just re-run the fixed prose parser (scripts/fetch_authors.py).
The prose blob — "A and B, University of California, Santa Barbara; C, MIT" —
is genuinely ambiguous: an affiliation tail is not distinguishable from a
person's name by any rule that does not encode a list of institutions. It cost
two parser bugs already (a comma split that crossed ";" boundaries, and an
unstripped leading "and ") and, even after those were fixed on 2026-08-13, it
still leaked affiliation FRAGMENTS into the author list, because "University of
California, Berkeley" splits into an affiliation-looking half that gets dropped
and a name-looking half ("Berkeley") that gets kept.
 
Every USENIX page carries at least one unambiguous author list instead:
 
  modern (2012-) presentation pages
    A. <meta name="citation_author" content="..."> — one tag per author, the
       Google Scholar metadata, emitted alongside citation_author_institution
       so names and affiliations are already separated by USENIX.
    B. USENIX's own published BibTeX block, author = {A and B and C}.
    Both are read; they must agree, or the paper is reported, not guessed at.
 
  pages with no structured metadata at all
    C. Affiliations are wrapped in an italic tag: <i> on the legacy (2011)
       technical-sessions pages, <em> inside the people-text div on the modern
       pages USENIX has not yet populated with Scholar metadata (one such page
       in this population, usenixsecurity25/presentation/tang). Deleting the
       italic spans leaves exactly the names. On a legacy page the paper's own
       block is first located BY TITLE, because those URLs address a whole
       session and taking the first block returns another talk's authors.
 
USENIX's own structured metadata is NOT unconditionally trustworthy, which is
why every paper is additionally cross-checked against the prose blob:
  * PhishTime (2020) — meta AND BibTeX both omit the 8th author, Gail-Joon Ahn,
    whom the prose blob and the paper both list.
  * PharmaLeaks (2012) — meta AND BibTeX say "Jordan Grant" and drop a middle
    initial; the paper PDF and DBLP both say "Grant Jordan" and
    "Geoffrey M. Voelker".
So a structured list is accepted only when the prose blob agrees on the set of
surnames. Every mismatch is printed for a human and adjudicated in
ADJUDICATED below, with the source that settled it.
 
    python3 scripts/usenix_author_audit.py          # report only
    python3 scripts/usenix_author_audit.py --write  # rewrite out/authors.json
 
Reads the HTML cached by scripts/usenix_author_audit_fetch.py; never refetches.
"""
import glob
import html
import json
import os
import re
import sys
import unicodedata
 
HTMLDIR = "out/usenix_audit/html"
CACHE = "out/authors.json"
ROOT = "/workspace/publications_dataset/data"
 
 
def unesc(s):
    return re.sub(r"\s+", " ", html.unescape(re.sub(r"<[^>]+>", " ", s))).strip()
 
 
# ---------------------------------------------------------------- source A
def meta_authors(page):
    return [unesc(m.group(1)) for m in re.finditer(
        r'<meta\s+name="citation_author"\s+content="([^"]*)"', page)
        if unesc(m.group(1))]
 
 
# ---------------------------------------------------------------- source B
LATEX = {
    r"\'": "́", r'\"': "̈", r"\`": "̀", r"\^": "̂",
    r"\~": "̃", r"\=": "̄", r"\.": "̇", r"\c": "̧",
    r"\v": "̌", r"\u": "̆", r"\H": "̋", r"\r": "̊",
}
 
 
LETTERS = {r"\i": "i", r"\j": "j", r"\ss": "ß", r"\o": "ø", r"\O": "Ø",
           r"\aa": "å", r"\AA": "Å", r"\l": "ł", r"\L": "Ł",
           r"\ae": "æ", r"\AE": "Æ", r"\oe": "œ", r"\OE": "Œ"}
 
 
def de_latex(s):
    """Doup{\\'e} -> Doupé.  Needed because the block is real BibTeX.
 
    The dotless-i pass has to run FIRST. USENIX writes Benoît as
    Beno{\\^\\i}t, and with \\i still in place none of the accent patterns
    below match (the character after the accent command is a backslash, not a
    letter); the final brace/backslash strip then produced "Beno^it". Found
    2026-09-04 by scripts/usenix_source_agreement.py, which is the only check
    that compares the two sources across all 162 pages rather than the 90.
    """
    def one(m):
        return unicodedata.normalize("NFC", m.group(2) + LATEX[m.group(1)])
    for cmd, ch in sorted(LETTERS.items(), key=lambda kv: -len(kv[0])):
        s = re.sub(re.escape(cmd) + r"(?![A-Za-z])", ch, s)
    # {\'e}  /  {\"u}  /  \'{e}  /  \'e
    s = re.sub(r"\{(\\[\'\"`^~=.cvuHr])\s*\{?([^\W\d_])\}?\}", one, s)
    s = re.sub(r"(\\[\'\"`^~=.cvuHr])\s*\{([^\W\d_])\}", one, s)
    s = re.sub(r"(\\[\'\"`^~=.cvuHr])([^\W\d_])", one, s)
    return re.sub(r"[{}\\]", "", s)
 
 
DE_LATEX_TESTS = [
    (r"Beno{\^\i}t", "Benoît"),      # dotless i — the 2026-09-04 regression
    (r"Adam Doup{\'e}", "Adam Doupé"),
    (r"Adam Doup\'e", "Adam Doupé"),
    (r"Dominik Sch{\"u}rmann", "Dominik Schürmann"),
    (r"Mathias L{\'e}cuyer", "Mathias Lécuyer"),
    # \c is cedilla, so \c{s} decodes to U+015F (s WITH CEDILLA), not the
    # U+0219 comma-below Romanian actually uses. That is the literally correct
    # reading of the BibTeX and it does not matter here: fold() strips both.
    (r"Dumitra\c{s}", "Dumitra\u015f"),
    (r"{Luki\'c}", "Lukić"),
    (r"Gro\ss{}", "Groß"),
    (r"Plain Name", "Plain Name"),
]
 
 
def selftest():
    bad = [(a, de_latex(a), b) for a, b in DE_LATEX_TESTS if de_latex(a) != b]
    for a, got, want in bad:
        print(f"  FAIL {a!r} -> {got!r}, want {want!r}")
    print(f"de_latex self-test: {len(DE_LATEX_TESTS) - len(bad)}/"
          f"{len(DE_LATEX_TESTS)} pass")
    return 1 if bad else 0
 
 
def brace_group(s, start):
    """Return the contents of the {...} beginning at s[start], brace-balanced.
 
    A non-greedy \\{(.*?)\\} truncates every LaTeX accent — "Adam Doup{\\'e}"
    ends the match at the inner brace and yields "Adam Doup{\\'e", which is how
    an earlier version of THIS script reported 13 spurious meta/BibTeX
    disagreements. Count braces instead.
    """
    depth, i = 0, start
    while i < len(s):
        if s[i] == "{":
            depth += 1
        elif s[i] == "}":
            depth -= 1
            if depth == 0:
                return s[start + 1:i]
        i += 1
    raise ValueError("unbalanced braces in BibTeX block")
 
 
def bibtex_authors(page):
    m = re.search(r'bibtex-text-entry[^>]*>(.*?)</div>', page, re.S)
    if not m:
        return []
    block = html.unescape(re.sub(r"<br\s*/?>", "\n", m.group(1)))
    a = re.search(r"author\s*=\s*\{", block)
    if not a:
        return []
    raw = re.sub(r"\s+", " ", brace_group(block, a.end() - 1)).strip()
    return [de_latex(n).strip() for n in re.split(r"\s+and\s+", raw) if n.strip()]
 
 
# ---------------------------------------------------------------- source C
def _names_minus_italics(fragment):
    """Names are whatever is left once the <i>/<em> affiliation spans go."""
    raw = unesc(re.sub(r"<(i|em)\b.*?</\1>", " ; ", fragment, flags=re.S))
    out = []
    for part in re.split(r"[;,]", raw):
        part = re.sub(r"^\s*and\s+", "", part.strip(), flags=re.I)
        for n in re.split(r"\s+and\s+", part):
            n = n.strip(" .,")
            if n:
                out.append(n)
    return out
 
 
def italic_authors(page, title):
    """Author list from a page carrying no citation_author and no BibTeX."""
    def squash(t):
        return re.sub(r"[^a-z0-9]", "", html.unescape(t).lower())
 
    # Modern page, Scholar metadata not yet populated: the people-text div with
    # <em>affiliation</em>. Only ONE page in this population is like this
    # (usenixsecurity25/presentation/tang) — but it is also the only entry the
    # audit would otherwise have to report as unverifiable.
    m = re.search(r'field-name-field-paper-people-text.*?<p>(.*?)</p>', page, re.S)
    if m and re.search(r"<(i|em)\b", m.group(1)):
        return _names_minus_italics(m.group(1))
 
    # Legacy 2011 session page: find this paper's own block by title.
    for m in re.finditer(r'<p class="fullpaper\d*">(.*?)</p>', page, re.S):
        b = m.group(1)
        t = re.search(r"<b>(.*?)</b>", b, re.S)
        if t and squash(unesc(t.group(1))) == squash(title):
            return _names_minus_italics(b[t.end():])
    return []
 
 
# ---------------------------------------------------------------- compare
def fold(names):
    """Compare ignoring diacritics, punctuation and case, but not word count."""
    def f(s):
        s = unicodedata.normalize("NFKD", s)
        s = "".join(c for c in s if not unicodedata.combining(c))
        return re.sub(r"\s+", " ", re.sub(r"[^a-z ]", " ", s.lower())).strip()
    return [f(x) for x in names]
 
 
# The affiliation filter for the over-generating prose splitter below. Kept here
# rather than imported from scripts/fetch_authors.py: that module now imports
# the parsers below, and the dependency has to run one way.
AFFIL_RE = re.compile(
    r"universit|institut|college|inc\.|labs?\b|corporat|technolog|research|"
    r"school|academy|center|centre|gmbh|ltd|company|cispa|kaist|inria|inc\b|"
    r"microsoft|google|meta|apple|amazon|ibm|nec|eth |epfl|tu |csiro|"
    r"laborator|foundation|agency|department|academia sinica|polytechnic",
    re.I,
)
 
def page_institutions(page):
    """Every affiliation the page itself declares.
 
    Modern pages emit one <meta name="citation_author_institution"> per author;
    legacy 2011 pages wrap each affiliation in <i>. Using the page's own list
    beats a hand-maintained stop-list of institution tail words: it needs no
    maintenance, and it takes the unfiltered residue (15 fragments over 16
    papers, mutation A7) to zero.
    """
    inst = [unesc(m.group(1)) for m in re.finditer(
        r'<meta\s+name="citation_author_institution"\s+content="([^"]*)"', page)]
    inst += [unesc(m.group(1)) for m in re.finditer(r"<i>(.*?)</i>", page, re.S)]
    return " | ".join(fold(inst))
 
 
def prose_parse(blob, inst=""):
    """Candidate personal names in the prose blob. Over-generates on purpose.
 
    Anything whose folded form is contained in one of the page's own declared
    affiliations is an affiliation tail, not an author, and is dropped.
    """
    out = []
    for group in blob.split(";"):
        parts = [p.strip() for p in group.split(",") if p.strip()]
        if len(parts) > 1 and AFFIL_RE.search(parts[-1]):
            parts = parts[:-1]
        for part in parts:
            part = re.sub(r"^and\s+", "", part.strip(), flags=re.I)
            for n in re.split(r"\s+and\s+", part):
                n = re.sub(r"\s*\([^)]*\)\s*", " ", n).strip(" .,")
                n = re.sub(r"\s+", " ", n)
                if not n or AFFIL_RE.search(n) or len(n.split()) < 2:
                    continue
                if len(n.split()) > 5 or not re.match(r"^[A-ZÀ-ɏ]", n):
                    continue
                f = fold([n])[0]
                if not f or (inst and f in inst):
                    continue
                out.append(n)
    return out
 
 
 
# ------------------------------------------------- cross-check against prose
# The prose blob is the ambiguous source this audit exists to stop trusting,
# but it is an INDEPENDENT one, so it is still good enough to answer the much
# easier question "does a surname appear at all?". Any surname on one side and
# not the other is printed and adjudicated by hand.
def prose_blob(page, title):
    for pat in (r'field-name-field-paper-people-text.*?<p>(.*?)</p>',
                r'field-name-field-paper-people-text[^>]*>(.*?)<div'):
        m = re.search(pat, page, re.S)
        if m:
            return unesc(m.group(1))
    # Legacy 2011 pages address a whole SESSION: locate this paper's own block
    # by title. Taking the first <p class="fullpaper"> on the page instead made
    # all three 2011 papers report the authors of an unrelated talk.
    def squash(t):
        return re.sub(r"[^a-z0-9]", "", html.unescape(t).lower())
    for m in re.finditer(r'<p class="fullpaper\d*">(.*?)</p>', page, re.S):
        b = m.group(1)
        t = re.search(r"<b>(.*?)</b>", b, re.S)
        if t and squash(unesc(t.group(1))) == squash(title):
            return unesc(b[t.end():])
    return ""
 
 
def surnames(names):
    return {fold([n])[0].split()[-1] for n in names if fold([n])[0]}
 
 
# Papers where the structured metadata and the prose blob disagree, each
# settled against a source outside USENIX's landing page. Anything not listed
# here is reported as unresolved and blocks --write.
ADJUDICATED = {
    "USENIX/2012/pharmaleaks-understanding-the-business-of-online-pharmaceutical-affiliate-progra": (
        ["Damon McCoy", "Andreas Pitsillidis", "Grant Jordan", "Nicholas Weaver",
         "Christian Kreibich", "Brian Krebs", "Geoffrey M. Voelker",
         "Stefan Savage", "Kirill Levchenko"],
        "USENIX meta/BibTeX say 'Jordan Grant' and 'Geoffrey Voelker'. The paper "
        "PDF title block (data/fulltext/2012/USENIX/pharmaleaks-.../paper.cols.txt) "
        "and DBLP both say 'Grant Jordan' and 'Geoffrey M. Voelker'. Paper wins."),
    "USENIX/2020/phishtime-continuous-longitudinal-measurement-of-the-effectiveness-of-anti-phish": (
        ["Adam Oest", "Yeganeh Safaei", "Penghui Zhang", "Brad Wardman",
         "Kevin Tyers", "Yan Shoshitaishvili", "Adam Doup\u00e9", "Gail-Joon Ahn"],
        "USENIX meta AND BibTeX both stop at 7 authors. The prose blob on the "
        "same page and the paper's own title block carry Gail-Joon Ahn as the "
        "8th. DBLP does NOT: record conf/uss/OestSZWTSD20, checked 2026-09-04, "
        "also lists seven and stops at Adam Doupe. Three machine-readable "
        "sources agree on a list the paper contradicts. Paper wins."),
    "USENIX/2025/vulnerability-of-text-matching-in-ml-ai-conference-reviewer-assignments-to-collu": (
        ["Jhih-Yi Hsieh", "Aditi Raghunathan", "Nihar B. Shah"],
        "The paper's title block and USENIX's meta both print the nickname, "
        "'Jhih-Yi (Janet) Hsieh'. Normalised to DBLP's form: a parenthetical is "
        "not a name particle and BibTeX would parse it as part of the given "
        "name. Judgement call, not a correction of the source."),
}
 
 
def load_titles():
    t = {}
    for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
        for p in json.load(open(f))["papers"]:
            t[f"{p['venue']}/{p['year']}/{p['slug']}"] = p["title"]
    return t
 
 
def main():
    write = "--write" in sys.argv
    # --cache <path> diffs against a snapshot instead of the live file, so the
    # committed audit trail can be regenerated after --write has already run.
    # Without it, re-running after a write reports "90 unchanged" and the
    # committed -output.txt becomes a record of nothing.
    src_cache = sys.argv[sys.argv.index("--cache") + 1] \
        if "--cache" in sys.argv else CACHE
    cache = json.load(open(src_cache))
    titles = load_titles()
    rows, unchanged, problems, accepted, adjudged = [], [], [], {}, []
 
    # Iterate the CACHE keys, not the HTML directory: scripts/usenix_bib_fetch_all.py
    # drops 72 more pages in there for pass 3 of usenix_bib_diff.py, and those
    # papers are not in out/authors.json at all.
    files = sorted(k for k in cache if k.startswith("USENIX/"))
    for key in files:
        f = os.path.join(HTMLDIR, key.replace("/", "__") + ".html")
        page = open(f, encoding="utf-8", errors="replace").read()
        old = cache[key]
 
        meta, bib = meta_authors(page), bibtex_authors(page)
        if meta or bib:
            if meta and bib and fold(meta) != fold(bib):
                problems.append((key, "META/BIBTEX DISAGREE", old, meta, bib))
                continue
            # Prefer meta: it keeps diacritics USENIX's BibTeX sometimes drops
            # (Gan~an) and needs no LaTeX decoding.
            new, src = (meta, "meta+bibtex") if (meta and bib) else \
                       ((meta, "meta") if meta else (bib, "bibtex"))
        else:
            new = italic_authors(page, titles[key])
            src = "italic"
            if not new:
                problems.append((key, "NO AUTHORS IN ANY SOURCE", old, [], []))
                continue
 
        # Cross-check the structured list against the independent prose blob.
        blob = prose_blob(page, titles[key])
        fblob = fold([blob])[0].split()
        missing_from_prose = [n for n in new
                              if fold([n])[0].split()[-1] not in fblob]
        prose_only = sorted(surnames(prose_parse(blob, page_institutions(page)))
                            - surnames(new))
        # An adjudicated paper overrides the page UNCONDITIONALLY. Applying it
        # only when the prose cross-check fires made the Hsieh entry dead code:
        # the prose agrees with the metadata there, and it is the metadata
        # itself that is being overridden.
        if key in ADJUDICATED:
            new, why = ADJUDICATED[key]
            src += " +adjudicated"
            adjudged.append((key, why))
        elif missing_from_prose or prose_only:
            problems.append((key, "STRUCTURED/PROSE DISAGREE: "
                             f"not in prose={missing_from_prose} "
                             f"prose-only surnames={prose_only}",
                             old, meta, bib))
            continue
 
        if fold(old) != fold(new):
            rows.append((key, src, old, new,
                         [a for a in new if fold([a])[0] not in fold(old)],
                         [a for a in old if fold([a])[0] not in fold(new)]))
        else:
            unchanged.append(key)
        accepted[key] = new
 
    print(f"USENIX keys audited                 : {len(files)}")
    print(f"  author list unchanged             : {len(unchanged)}")
    print(f"  author list CORRECTED             : {len(rows)}")
    print(f"  overruled by the paper/DBLP       : {len(adjudged)}")
    print(f"  unresolved, needs a human         : {len(problems)}")
    print(f"    of the corrected, missing an author : "
          f"{sum(1 for r in rows if r[4])}")
    print(f"    of the corrected, carried a non-author : "
          f"{sum(1 for r in rows if r[5])}")
    print("=" * 78)
    for key, src, old, new, dropped, spurious in rows:
        print(f"\n{key}   [{src}]")
        print(f"  was : {'; '.join(old)}")
        print(f"  now : {'; '.join(new)}")
        if dropped:
            print(f"  + AUTHOR(S) RESTORED : {'; '.join(dropped)}")
        if spurious:
            print(f"  - NON-AUTHOR(S) REMOVED : {'; '.join(spurious)}")
    if adjudged:
        print("\n" + "=" * 78 + "\nADJUDICATED against a source outside usenix.org:")
        for key, why in adjudged:
            print(f"\n  {key}\n    {why}")
    if problems:
        print("\n" + "=" * 78 + "\nUNRESOLVED — needs a human:")
        for key, why, old, meta, bib in problems:
            print(f"\n  {key}\n    {why}")
            print(f"    cache : {'; '.join(old)}")
            print(f"    meta  : {'; '.join(meta)}")
            print(f"    bibtex: {'; '.join(bib)}")
 
    if write:
        if problems:
            print("\nREFUSING to write: unresolved papers above.")
            return 1
        full = json.load(open(CACHE))
        full.update(accepted)
        json.dump(full, open(CACHE, "w"), indent=1, sort_keys=True)
        print(f"\nwrote {len(accepted)} USENIX entries to {CACHE}")
    return 1 if problems else 0
 
 
if __name__ == "__main__":
    sys.exit(selftest() if "--selftest" in sys.argv else main())

Real output, diffed against the pre-run snapshot out/usenix_audit/authors.json.before:

USENIX keys audited                 : 90
  author list unchanged             : 72
  author list CORRECTED             : 18
  overruled by the paper/DBLP       : 3
  unresolved, needs a human         : 0
    of the corrected, missing an author : 0
    of the corrected, carried a non-author : 18
==============================================================================

USENIX/2012/an-evaluation-of-the-google-chrome-extension-security-architecture   [meta+bibtex]
  was : Nicholas Carlini; Adrienne Porter Felt; David Wagner; Berkeley
  now : Nicholas Carlini; Adrienne Porter Felt; David Wagner
  - NON-AUTHOR(S) REMOVED : Berkeley

USENIX/2012/privilege-separation-in-html5-applications   [meta+bibtex]
  was : Devdatta Akhawe; Prateek Saxena; Dawn Song; Berkeley
  now : Devdatta Akhawe; Prateek Saxena; Dawn Song
  - NON-AUTHOR(S) REMOVED : Berkeley

USENIX/2014/hulk-eliciting-malicious-behavior-in-browser-extensions   [meta+bibtex]
  was : Alexandros Kapravelos; Santa Barbara; Chris Grier; Berkeley; Neha Chachra; San Diego; Christopher Kruegel; Giovanni Vigna; Santa Barbara; Vern Paxson; Berkeley
  now : Alexandros Kapravelos; Chris Grier; Neha Chachra; Christopher Kruegel; Giovanni Vigna; Vern Paxson
  - NON-AUTHOR(S) REMOVED : Santa Barbara; Berkeley; San Diego; Santa Barbara; Berkeley

USENIX/2017/extension-breakdown-security-analysis-of-browsers-extension-resources-control-po   [meta+bibtex]
  was : Iskander Sanchez-Rola; Igor Santos; DeustoTech; Davide Balzarotti; Eurecom
  now : Iskander Sanchez-Rola; Igor Santos; Davide Balzarotti
  - NON-AUTHOR(S) REMOVED : DeustoTech; Eurecom

USENIX/2019/50-ways-to-leak-your-data-an-exploration-of-apps-circumvention-of-the-android-pe   [meta+bibtex]
  was : Joel Reardon; Álvaro Feal; Primal Wijesekera; U.C. Berkeley / ICSI; Amit Elazari Bar On; U.C. Berkeley; Narseo Vallina-Rodriguez; Serge Egelman
  now : Joel Reardon; Álvaro Feal; Primal Wijesekera; Amit Elazari Bar On; Narseo Vallina-Rodriguez; Serge Egelman
  - NON-AUTHOR(S) REMOVED : U.C. Berkeley / ICSI; U.C. Berkeley

USENIX/2020/measuring-and-modeling-the-label-dynamics-of-online-anti-malware-engines   [meta+bibtex]
  was : Shuofei Zhu; Jianjun Shi; BIT; Limin Yang; Boqin Qin; BUPT; Ziyi Zhang; USTC; Linhai Song; Gang Wang
  now : Shuofei Zhu; Jianjun Shi; Limin Yang; Boqin Qin; Ziyi Zhang; Linhai Song; Gang Wang
  - NON-AUTHOR(S) REMOVED : BIT; BUPT; USTC

USENIX/2020/phishtime-continuous-longitudinal-measurement-of-the-effectiveness-of-anti-phish   [meta+bibtex +adjudicated]
  was : Adam Oest; Yeganeh Safaei; Penghui Zhang; Brad Wardman; Kevin Tyers; PayPal; Yan Shoshitaishvili; Adam Doupé; Gail-Joon Ahn
  now : Adam Oest; Yeganeh Safaei; Penghui Zhang; Brad Wardman; Kevin Tyers; Yan Shoshitaishvili; Adam Doupé; Gail-Joon Ahn
  - NON-AUTHOR(S) REMOVED : PayPal

USENIX/2020/sunrise-to-sunset-analyzing-the-end-to-end-life-cycle-and-effectiveness-of-phish   [meta+bibtex]
  was : Adam Oest; Penghui Zhang; Brad Wardman; Eric Nunes; Jakub Burgis; PayPal; Ali Zand; Kurt Thomas; Adam Doupé; Gail-Joon Ahn
  now : Adam Oest; Penghui Zhang; Brad Wardman; Eric Nunes; Jakub Burgis; Ali Zand; Kurt Thomas; Adam Doupé; Gail-Joon Ahn
  - NON-AUTHOR(S) REMOVED : PayPal

USENIX/2021/phishpedia-a-hybrid-deep-learning-based-approach-to-visually-identify-phishing-w   [meta+bibtex]
  was : Yun Lin; Ruofan Liu; Dinil Mon Divakaran; Trustwave; Jun Yang Ng; Qing Zhou Chan; Yiwen Lu; Yuxuan Si; Fan Zhang; Jin Song Dong
  now : Yun Lin; Ruofan Liu; Dinil Mon Divakaran; Jun Yang Ng; Qing Zhou Chan; Yiwen Lu; Yuxuan Si; Fan Zhang; Jin Song Dong
  - NON-AUTHOR(S) REMOVED : Trustwave

USENIX/2022/inferring-phishing-intention-via-webpage-appearance-and-dynamics-a-deep-vision-b   [meta+bibtex]
  was : Ruofan Liu; Yun Lin; Xianglin Yang; Siang Hwee Ng; Dinil Mon Divakaran; Trustwave; Jin Song Dong
  now : Ruofan Liu; Yun Lin; Xianglin Yang; Siang Hwee Ng; Dinil Mon Divakaran; Jin Song Dong
  - NON-AUTHOR(S) REMOVED : Trustwave

USENIX/2023/glowing-in-the-dark-uncovering-ipv6-address-discovery-and-scanning-strategies-in   [meta+bibtex]
  was : Hammas Bin Tanveer; Rachee Singh; Paul Pearce; Georgia Tech; Rishab Nithyanand
  now : Hammas Bin Tanveer; Rachee Singh; Paul Pearce; Rishab Nithyanand
  - NON-AUTHOR(S) REMOVED : Georgia Tech

USENIX/2023/log-it-s-big-it-s-heavy-it-s-filled-with-personal-data-measuring-the-logging-of   [meta+bibtex]
  was : Allan Lyons; Julien Gamba; Austin Shawaga; Joel Reardon; AppCensus; Juan Tapiador; Universidad Carlos III de Madrid; Serge Egelman; ICSI; UC Berkeley; AppCensus; Narseo Vallina-Rodriguez; AppCensus
  now : Allan Lyons; Julien Gamba; Austin Shawaga; Joel Reardon; Juan Tapiador; Serge Egelman; Narseo Vallina-Rodriguez
  - NON-AUTHOR(S) REMOVED : AppCensus; Universidad Carlos III de Madrid; ICSI; UC Berkeley; AppCensus; AppCensus

USENIX/2023/rods-with-laser-beams-understanding-browser-fingerprinting-on-phishing-pages   [meta+bibtex]
  was : Iskander Sanchez-Rola; Leyla Bilge; Davide Balzarotti; EURECOM; Armin Buescher; Petros Efstathopoulos
  now : Iskander Sanchez-Rola; Leyla Bilge; Davide Balzarotti; Armin Buescher; Petros Efstathopoulos
  - NON-AUTHOR(S) REMOVED : EURECOM

USENIX/2023/the-ok-is-not-enough-a-large-scale-study-of-consent-dialogs-in-smartphone-applic   [meta+bibtex]
  was : Simon Koch; Benjamin Altpeter; Datenanfragen.de e.V.; Martin Johns
  now : Simon Koch; Benjamin Altpeter; Martin Johns
  - NON-AUTHOR(S) REMOVED : Datenanfragen.de e.V.

USENIX/2024/knowphish-large-language-models-meet-multimodal-knowledge-graphs-for-enhancing-r   [meta+bibtex]
  was : Yuexin Li; Chengyu Huang; Shumin Deng; Mei Lin Lock; NCS Cyber Special Ops-R&D; Tri Cao; Nay Oo; Hoon Wei Lim; NCS Cyber Special Ops-R&D; Bryan Hooi
  now : Yuexin Li; Chengyu Huang; Shumin Deng; Mei Lin Lock; Tri Cao; Nay Oo; Hoon Wei Lim; Bryan Hooi
  - NON-AUTHOR(S) REMOVED : NCS Cyber Special Ops-R&D; NCS Cyber Special Ops-R&D

USENIX/2024/the-effect-of-design-patterns-on-present-and-future-cookie-consent-decisions   [meta+bibtex]
  was : Nataliia Bielova; Laura Litvine; Anysia Nguyen; Behavioural Insights Team; Mariam Chammat; Interministerial Directorate for Public Transformation; Vincent Toubiana; Estelle Hary
  now : Nataliia Bielova; Laura Litvine; Anysia Nguyen; Mariam Chammat; Vincent Toubiana; Estelle Hary
  - NON-AUTHOR(S) REMOVED : Behavioural Insights Team; Interministerial Directorate for Public Transformation

USENIX/2025/big-help-or-big-brother-auditing-tracking-profiling-and-personalization-in-gener   [meta+bibtex]
  was : Yash Vekaria; UC Davis; Aurelio Loris Canino; UNIRC; Jonathan Levitsky; UC Davis; Alex Ciechonski; UCL; Patricia Callejo; UC3M; Anna Maria Mandalari; UCL; Zubair Shafiq; UC Davis
  now : Yash Vekaria; Aurelio Loris Canino; Jonathan Levitsky; Alex Ciechonski; Patricia Callejo; Anna Maria Mandalari; Zubair Shafiq
  - NON-AUTHOR(S) REMOVED : UC Davis; UNIRC; UC Davis; UCL; UC3M; UCL; UC Davis

USENIX/2025/privacy-law-enforcement-under-centralized-governance-a-qualitative-analysis-of-f   [meta+bibtex]
  was : Tao Jing; Engineering; Yao Li; Jingzhou Ye; Jie Wang; Engineering; Xueqiang Wang
  now : Tao Jing; Yao Li; Jingzhou Ye; Jie Wang; Xueqiang Wang
  - NON-AUTHOR(S) REMOVED : Engineering; Engineering

==============================================================================
ADJUDICATED against a source outside usenix.org:

  USENIX/2012/pharmaleaks-understanding-the-business-of-online-pharmaceutical-affiliate-progra
    USENIX meta/BibTeX say 'Jordan Grant' and 'Geoffrey Voelker'. The paper PDF title block (data/fulltext/2012/USENIX/pharmaleaks-.../paper.cols.txt) and DBLP both say 'Grant Jordan' and 'Geoffrey M. Voelker'. Paper wins.

  USENIX/2020/phishtime-continuous-longitudinal-measurement-of-the-effectiveness-of-anti-phish
    USENIX meta AND BibTeX both stop at 7 authors. The prose blob on the same page and the paper's own title block carry Gail-Joon Ahn as the 8th. DBLP does NOT: record conf/uss/OestSZWTSD20, checked 2026-09-04, also lists seven and stops at Adam Doupe. Three machine-readable sources agree on a list the paper contradicts. Paper wins.

  USENIX/2025/vulnerability-of-text-matching-in-ml-ai-conference-reviewer-assignments-to-collu
    The paper's title block and USENIX's meta both print the nickname, 'Jhih-Yi (Janet) Hsieh'. Normalised to DBLP's form: a parenthetical is not a name particle and BibTeX would parse it as part of the given name. Judgement call, not a correction of the source.

usenix_bib_diff.py

#!/usr/bin/env python3
"""Step 3 of the USENIX author-list audit: diff literature:bibliography against
the corrected out/authors.json, and flag any USENIX entry the audit cannot see.
 
Two passes, because they catch different things:
 
  PASS 1  title-match every USENIX key in out/authors.json to a live BibTeX
          entry and diff the author field. This finds entries the broken prose
          parser wrote.
  PASS 2  scan EVERY live entry in a USENIX Security series for an author name
          that is affiliation-shaped, whether or not it is in out/authors.json.
          Entries added by hand or from another source are invisible to pass 1,
          and the same class of defect can reach them.
  PASS 3  compare EVERY live USENIX Security entry against the author list its
          own landing page publishes, as an ORDERED list. Pass 2 can only see an
          affiliation that got IN; only this pass can see an author that fell
          OUT of, or was reordered within, an entry out/authors.json never
          touched. Needs the pages fetched by scripts/usenix_bib_fetch_all.py.
 
    python3 scripts/usenix_bib_diff.py [--bib out/usenix_audit/bib_live.txt]
 
Prints a corrected author line per affected entry; it does not edit the wiki.
"""
import glob
import json
import os
import re
import sys
import unicodedata
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_author_audit import (de_latex, meta_authors, bibtex_authors,
                                 italic_authors, prose_blob, prose_parse,
                                 page_institutions, surnames, ADJUDICATED)
from usenix_author_audit import fold as afold
 
CACHE = "out/authors.json"
ROOT = "/workspace/publications_dataset/data"
DEFAULT_BIB = "out/usenix_audit/bib_live.txt"
 
 
def fold(s):
    s = unicodedata.normalize("NFKD", s)
    s = "".join(c for c in s if not unicodedata.combining(c))
    return re.sub(r"\s+", " ", re.sub(r"[^a-z0-9 ]", " ", s.lower())).strip()
 
 
def squash(s):
    return re.sub(r"[^a-z0-9]", "", fold(s))
 
 
def field(entry, name):
    m = re.search(r"\b" + name + r"\s*=\s*\{", entry)
    if not m:
        return None
    depth, i = 0, m.end() - 1
    while i < len(entry):
        if entry[i] == "{":
            depth += 1
        elif entry[i] == "}":
            depth -= 1
            if depth == 0:
                return re.sub(r"\s+", " ", entry[m.end():i]).strip()
        i += 1
    raise ValueError("unbalanced braces in " + entry[:60])
 
 
def split_bib_authors(a):
    """'Last, First and Last, First' -> ['First Last', ...]"""
    out = []
    for n in re.split(r"\s+and\s+", a):
        n = n.strip()
        if not n:
            continue
        if "," in n:
            last, first = n.split(",", 1)
            n = f"{first.strip()} {last.strip()}".strip()
        # 43 of the 837 live entries spell an accent with a LaTeX accent
        # command and 66 with a literal non-ASCII letter (see
        # scripts/bib_accent_encoding.py). Both render correctly through
        # bibtex4dw: checked on the rendered DOM of security:phishing, which
        # cites the braced "Doup{\'e}" entries, and privacy:browser_extensions,
        # which cites the one unbraced "Doup\'e" entry. Decode before diffing,
        # or those three entries drown a real defect in encoding noise.
        out.append(re.sub(r"\s+", " ", de_latex(n)))
    return out
 
 
def fmt_bib(names):
    out = []
    for a in names:
        parts = a.strip().split()
        last = parts.pop()
        out.append(f"{last}, {' '.join(parts)}" if parts else last)
    return " and ".join(out)
 
 
# An author name that is really an affiliation. Deliberately wide: every hit is
# printed in full for a human, nothing is auto-removed on the strength of it.
AFFIL_SHAPED = re.compile(
    r"universit|universidad|institut|college|\binc\b|\blabs?\b|corporat|"
    r"technolog|\bresearch\b|school|academy|center|centre|gmbh|\bltd\b|"
    r"company|cispa|kaist|inria|microsoft|\bgoogle\b|\bapple\b|\bamazon\b|"
    r"\bibm\b|\bepfl\b|\beth\b|laborator|foundation|agency|department|"
    r"polytechnic|\bpaypal\b|trustwave|appcensus|\bicsi\b|\bteam\b|\be\.?\s?v\.?$",
    re.I)
SINGLE_TOKEN_OK = {"anonymous"}
 
 
def main():
    bibpath = sys.argv[sys.argv.index("--bib") + 1] if "--bib" in sys.argv \
        else DEFAULT_BIB
    text = open(bibpath, encoding="utf-8").read()
    entries = re.findall(r"@\w+\{[^@]*?\n\}", text, re.S)
    print(f"bibliography           : {bibpath}")
    print(f"entries parsed         : {len(entries)}")
 
    by_title = {}
    for e in entries:
        t = field(e, "title")
        if t:
            by_title.setdefault(squash(t), []).append(e)
 
    titles = {}
    for f in glob.glob(os.path.join(ROOT, "corpus2/.meta/*.json")):
        for p in json.load(open(f))["papers"]:
            titles[f"{p['venue']}/{p['year']}/{p['slug']}"] = p["title"]
 
    cache = json.load(open(CACHE))
    keys = sorted(k for k in cache if k.startswith("USENIX/"))
    print(f"USENIX keys in {CACHE}: {len(keys)}\n")
 
    # ---------------------------------------------------------------- PASS 1
    matched, wrong, absent = 0, [], []
    for k in keys:
        cands = by_title.get(squash(titles[k]), [])
        if not cands:
            absent.append(k)
            continue
        matched += 1
        for e in cands:
            key = re.match(r"@\w+\{([^,]+),", e).group(1)
            have = split_bib_authors(field(e, "author"))
            want = cache[k]
            if [fold(x) for x in have] != [fold(x) for x in want]:
                wrong.append((key, k, have, want))
 
    print("=" * 78)
    print(f"PASS 1  title-matched to the bibliography : {matched}/{len(keys)}")
    print(f"        author field WRONG                : {len(wrong)}")
    print(f"        key not cited on the wiki at all  : {len(absent)}")
    for key, k, have, want in wrong:
        print(f"\n  {{[{key}]}}   ({k})")
        print(f"    live      : {'; '.join(have)}")
        print(f"    corrected : {'; '.join(want)}")
        print(f"    author        = {{{fmt_bib(want)}}},")
 
    # ---------------------------------------------------------------- PASS 2
    print("\n" + "=" * 78)
    flagged = []
    us = 0
    for e in entries:
        series = (field(e, "series") or "") + " " + (field(e, "booktitle") or "")
        if not re.search(r"USENIX Security", series, re.I):
            continue
        us += 1
        names = split_bib_authors(field(e, "author") or "")
        bad = [n for n in names
               if AFFIL_SHAPED.search(n)
               or (len(n.split()) < 2 and fold(n) not in SINGLE_TOKEN_OK)]
        if bad:
            key = re.match(r"@\w+\{([^,]+),", e).group(1)
            flagged.append((key, names, bad))
    print(f"PASS 2  live entries in a USENIX Security series : {us}")
    print(f"        entries with an affiliation-shaped author: {len(flagged)}")
    for key, names, bad in flagged:
        print(f"\n  {{[{key}]}}")
        print(f"    authors : {'; '.join(names)}")
        print(f"    FLAGGED : {'; '.join(bad)}")
    # ---------------------------------------------------------------- PASS 3
    print("\n" + "=" * 78)
    tpath = "out/usenix_audit/bib_targets.json"
    if not os.path.exists(tpath):
        print("PASS 3  SKIPPED: run scripts/usenix_bib_fetch_all.py first")
    else:
        targets = json.load(open(tpath))
        by_ckey = {re.match(r"@\w+\{([^,]+),", e).group(1): e for e in entries}
        checked, p3_wrong, p3_skip = 0, [], []
        for ckey, (k, _url, title) in sorted(targets.items()):
            h = os.path.join("out/usenix_audit/html",
                             k.replace("/", "__").replace(":", "__") + ".html")
            if not os.path.exists(h):
                p3_skip.append((ckey, "page not fetched"))
                continue
            page = open(h, encoding="utf-8", errors="replace").read()
            m, b = meta_authors(page), bibtex_authors(page)
            page_authors = m or b or italic_authors(page, title)
            if not page_authors:
                p3_skip.append((ckey, "no author list on the page"))
                continue
            if k in ADJUDICATED:
                page_authors = ADJUDICATED[k][0]
            checked += 1
            have = split_bib_authors(field(by_ckey[ckey], "author"))
            hs, ps = {fold(x) for x in have}, {fold(x) for x in page_authors}
            missing = sorted(ps - hs)
            extra = sorted(hs - ps)
            # Set difference alone is NOT enough. An earlier version compared
            # only the sets, with a comment claiming it also checked order —
            # it did not, and a review on 2026-09-04 demonstrated the gap by
            # swapping first and last author on a pass-3-only entry: all three
            # passes reported clean. Author ORDER is first authorship; a swap
            # is a worse citation defect than a missing middle author.
            reordered = (not missing and not extra
                         and [fold(x) for x in have] != [fold(x) for x in page_authors])
            if missing or extra or reordered:
                p3_wrong.append((ckey, k, have, page_authors, missing, extra,
                                 reordered))
        print(f"PASS 3  live entries checked against their landing page : "
              f"{checked}/{len(targets)}")
        print(f"        author SET differs from the page               : "
              f"{sum(1 for r in p3_wrong if r[4] or r[5])}")
        print(f"        same set, WRONG ORDER                          : "
              f"{sum(1 for r in p3_wrong if r[6])}")
        print(f"        skipped                                        : "
              f"{len(p3_skip)}")
        for ckey, k, have, want, missing, extra, reordered in p3_wrong:
            print(f"\n  {{[{ckey}]}}   ({k})")
            print(f"    live : {'; '.join(have)}")
            print(f"    page : {'; '.join(want)}")
            if missing:
                print(f"    ON THE PAGE, NOT IN THE ENTRY : {'; '.join(missing)}")
            if extra:
                print(f"    IN THE ENTRY, NOT ON THE PAGE : {'; '.join(extra)}")
            if reordered:
                print(f"    SAME NAMES, DIFFERENT ORDER — check first authorship")
        for ckey, why in p3_skip:
            print(f"    skipped {ckey}: {why}")
 
    if absent:
        print("\n" + "=" * 78)
        print("Keys in out/authors.json that no live entry cites "
              "(fetched for the audit, never published):")
        for k in absent:
            print(f"  {k}")
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

bibliography           : out/usenix_audit/bib_live.txt
entries parsed         : 837
USENIX keys in out/authors.json: 90

==============================================================================
PASS 1  title-matched to the bibliography : 85/90
        author field WRONG                : 0
        key not cited on the wiki at all  : 5

==============================================================================
PASS 2  live entries in a USENIX Security series : 159
        entries with an affiliation-shaped author: 0

==============================================================================
PASS 3  live entries checked against their landing page : 159/159
        author SET differs from the page               : 0
        same set, WRONG ORDER                          : 0
        skipped                                        : 0

==============================================================================
Keys in out/authors.json that no live entry cites (fetched for the audit, never published):
  USENIX/2022/inferring-phishing-intention-via-webpage-appearance-and-dynamics-a-deep-vision-b
  USENIX/2022/ovrseen-auditing-network-traffic-and-privacy-policies-in-oculus-vr
  USENIX/2023/glowing-in-the-dark-uncovering-ipv6-address-discovery-and-scanning-strategies-in
  USENIX/2023/knowledge-expansion-and-counterfactual-interaction-for-reference-based-phishing
  USENIX/2024/knowphish-large-language-models-meet-multimodal-knowledge-graphs-for-enhancing-r

The mutated bibliographies it runs against are generated, not hand-made:

make_mutated_bibs.py

#!/usr/bin/env python3
"""Build the mutated bibliographies scripts/usenix_author_audit_mutations.py
runs against, so the mutation set is reproducible instead of a set of files
someone once made by hand.
 
Each writes out/usenix_audit/mut/<id>.bib, a copy of the live snapshot with one
defect injected:
 
  b1  an author DELETED from an entry that is in out/authors.json
  b2  an affiliation ADDED as an author, same entry
  b3  an affiliation ADDED to an entry that is NOT in out/authors.json
  b4  one author's given and family names SWAPPED
  b5  the author ORDER reversed at both ends of a NON-cached entry — the defect
      that passed every check until 2026-09-04
  b6  an author DELETED from a NON-cached entry. b1 does this to a CACHED entry,
      where pass 1 catches it; b6 is the case pass 3 exists for, and nothing
      else on this page can see it
 
    python3 scripts/make_mutated_bibs.py
"""
import json
import os
import re
import sys
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field, split_bib_authors
 
SRC = "out/usenix_audit/bib_live.txt"
DEST = "out/usenix_audit/mut"
 
CACHED = "author        = {Carlini, Nicholas and Felt, Adrienne Porter and Wagner, David},"
NONCACHED = ("author={Xie, Qinge and Tang, Shujun and Zheng, Xiaofeng and "
             "Lin, Qingran and Liu, Baojun and Duan, Haixin and Li, Frank}")
SWAP_FROM = "author        = {McCoy, Damon and Pitsillidis, Andreas and Jordan, Grant and"
B5_KEY = "acharya2024_imitation"
 
MUTS = {
    "b1": (CACHED,
           "author        = {Carlini, Nicholas and Felt, Adrienne Porter},"),
    "b2": (CACHED,
           "author        = {Carlini, Nicholas and Felt, Adrienne Porter and "
           "Wagner, David and Berkeley},"),
    "b3": (NONCACHED, NONCACHED[:-1] + " and Georgia Institute of Technology}"),
    "b4": (SWAP_FROM,
           "author        = {McCoy, Damon and Pitsillidis, Andreas and "
           "Grant, Jordan and"),
}
 
 
def main():
    bib = open(SRC, encoding="utf-8").read()
    os.makedirs(DEST, exist_ok=True)
    for name, (a, b) in MUTS.items():
        if a not in bib:
            raise SystemExit(f"{name}: anchor not in {SRC}; the snapshot moved")
        open(os.path.join(DEST, name + ".bib"), "w").write(bib.replace(a, b, 1))
        print(f"wrote {name}.bib")
 
    # b5 is derived rather than hardcoded, so it stays valid if the entry moves.
    ents = {re.match(r"@\w+\{([^,]+),", e).group(1): e
            for e in re.findall(r"@\w+\{[^@]*?\n\}", bib, re.S)}
    orig = field(ents[B5_KEY], "author")
    parts = re.split(r"\s+and\s+", orig)
    if len(parts) < 3:
        raise SystemExit(f"{B5_KEY} has too few authors to reorder")
    swapped = " and ".join([parts[-1]] + parts[1:-1] + [parts[0]])
    if "{" + orig + "}" not in bib:
        raise SystemExit(f"b5: could not locate {B5_KEY}'s author field")
    open(os.path.join(DEST, "b5.bib"), "w").write(
        bib.replace("{" + orig + "}", "{" + swapped + "}", 1))
    print(f"wrote b5.bib  ({B5_KEY}: first and last author swapped)")
    print(f"  from: {orig}")
    print(f"  to  : {swapped}")
 
    dropped = " and ".join(parts[:-1])
    open(os.path.join(DEST, "b6.bib"), "w").write(
        bib.replace("{" + orig + "}", "{" + dropped + "}", 1))
    print(f"wrote b6.bib  ({B5_KEY}: last author deleted — {parts[-1]})")
 
 
if __name__ == "__main__":
    sys.exit(main())

usenix_author_audit_mutations.py

#!/usr/bin/env python3
"""Mutation-test scripts/usenix_author_audit.py and scripts/usenix_bib_diff.py.
 
Both scripts report "0 unresolved" and "0 defects", and a check that passes is
worth nothing until it has been shown it can fail. Each mutation below breaks
exactly one load-bearing rule; a mutation that does NOT change the reported
counts is a rule that is asserting nothing.
 
The audit mutations diff against the PRE-FIX snapshot
(out/usenix_audit/authors.json.before), because against the already-corrected
out/authors.json every run reports 90 unchanged and no mutation can move it.
 
    python3 scripts/usenix_author_audit_mutations.py
"""
import os
import re
import subprocess
import sys
 
BEFORE = "out/usenix_audit/authors.json.before"
LIVEBIB = "out/usenix_audit/bib_live.txt"
MUTBIB = "out/usenix_audit/mut"
TMP = "scripts/_mut_tmp.py"
 
AUDIT = "scripts/usenix_author_audit.py"
AUDIT_MUTS = [
    ("A1  citation_author parser truncated to 1 author",
     'return [unesc(m.group(1)) for m in re.finditer(\n'
     '        r\'<meta\\s+name="citation_author"\\s+content="([^"]*)"\', page)\n'
     '        if unesc(m.group(1))]',
     'return [unesc(m.group(1)) for m in list(re.finditer(\n'
     '        r\'<meta\\s+name="citation_author"\\s+content="([^"]*)"\', page))[:1]\n'
     '        if unesc(m.group(1))]'),
    # A2 must ALSO empty the adjudication table. On its own it changes nothing,
    # because every paper the cross-check fires on is already adjudicated and
    # the adjudication branch runs first — the check is a tripwire for the next
    # corpus refresh, not a live assertion today. Emptying both shows what it
    # is worth: without it, PharmaLeaks silently regresses to the wrong
    # "Jordan Grant" (corrected goes 18 -> 20) instead of stopping the run.
    ("A2  prose cross-check disabled *and* adjudication emptied",
     [("        elif missing_from_prose or prose_only:",
       "        elif False and (missing_from_prose or prose_only):"),
      ('ADJUDICATED = {\n    "USENIX/2012/pharmaleaks',
       'ADJUDICATED = {}\n_UNUSED = {\n    "USENIX/2012/pharmaleaks')], None),
    ("A3  BibTeX brace balancing broken",
     "                return s[start + 1:i]",
     "                return s[start + 1:start + 8]"),
    ("A4  legacy block no longer matched by title",
     "        if t and squash(unesc(t.group(1))) == squash(title):",
     "        if t:"),
    ("A5  diacritic folding removed",
     '        s = "".join(c for c in s if not unicodedata.combining(c))',
     "        pass"),
    ("A6  adjudication table emptied",
     'ADJUDICATED = {\n    "USENIX/2012/pharmaleaks',
     'ADJUDICATED = {}\n_UNUSED = {\n    "USENIX/2012/pharmaleaks'),
    ("A7  institution list ignored (prose residue no longer filtered)",
     "                if not f or (inst and f in inst):",
     "                if not f:"),
]
 
# The mutated bibliographies under out/usenix_audit/mut/ are built by
# scripts/make_mutated_bibs.py, so the mutation set is reproducible rather than
# a set of files someone once made by hand.
 
 
def field(out, pat):
    m = re.search(pat, out)
    return m.group(1) if m else "?"
 
 
def run_audit(path, extra=()):
    r = subprocess.run([sys.executable, path, "--cache", BEFORE, *extra],
                       capture_output=True, text=True)
    return (r.returncode,
            field(r.stdout, r"unchanged +: (\d+)"),
            field(r.stdout, r"CORRECTED +: (\d+)"),
            field(r.stdout, r"unresolved, needs a human +: (\d+)"))
 
 
def run_diff(bib):
    r = subprocess.run([sys.executable, "scripts/usenix_bib_diff.py",
                        "--bib", bib], capture_output=True, text=True)
    return (field(r.stdout, r"author field WRONG +: (\d+)"),
            field(r.stdout, r"affiliation-shaped author: (\d+)"),
            field(r.stdout, r"author SET differs from the page +: (\d+)"),
            field(r.stdout, r"same set, WRONG ORDER +: (\d+)"))
 
 
def main():
    src = open(AUDIT).read()
    print(f"=== {AUDIT} — diffed against {BEFORE}")
    print("%-58s %-4s %-10s %-10s %s"
          % ("mutation", "rc", "unchanged", "corrected", "unresolved"))
    rc, u, c, p = run_audit(AUDIT)
    print("%-58s %-4s %-10s %-10s %s" % ("(baseline, unmutated)", rc, u, c, p))
    bad = []
    for name, a, b in AUDIT_MUTS:
        steps = a if isinstance(a, list) else [(a, b)]
        mutated, stale = src, False
        for fro, to in steps:
            if fro not in mutated:
                stale = True
                break
            mutated = mutated.replace(fro, to, 1)
        if stale:
            print(f"{name}: PATTERN NOT FOUND — mutation is stale")
            bad.append(name)
            continue
        open(TMP, "w").write(mutated)
        rc2, u2, c2, p2 = run_audit(TMP)
        print("%-58s %-4s %-10s %-10s %s" % (name, rc2, u2, c2, p2))
        if (u2, c2, p2) == (u, c, p):
            bad.append(name)
    if os.path.exists(TMP):
        os.remove(TMP)
 
    print(f"\n=== scripts/usenix_bib_diff.py — mutated BIBLIOGRAPHIES")
    print("%-58s %-7s %-7s %-7s %s"
          % ("mutation", "PASS1", "PASS2", "PASS3set", "PASS3order"))
    base = run_diff(LIVEBIB)
    print("%-58s %-7s %-7s %-7s %s" % ("(baseline, unmutated)", *base))
    for name, f in [("B1  author deleted from a cached entry", "b1"),
                    ("B2  affiliation added to a cached entry", "b2"),
                    ("B3  affiliation added to a NON-cached entry", "b3"),
                    ("B4  given/family names swapped", "b4"),
                    ("B5  author ORDER swapped, NON-cached entry", "b5"),
                    ("B6  author DELETED from a NON-cached entry", "b6")]:
        path = os.path.join(MUTBIB, f + ".bib")
        if not os.path.exists(path):
            print(f"{name}: {path} missing")
            bad.append(name)
            continue
        got = run_diff(path)
        print("%-58s %-7s %-7s %-7s %s" % (name, *got))
        if got == base:
            bad.append(name)
 
    print(f"\nmutations that changed nothing (a rule asserting nothing): "
          f"{len(bad)}" + (f"  {bad}" if bad else ""))
    return 1 if bad else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

usenix_source_agreement.py

#!/usr/bin/env python3
"""Measure how often USENIX's two structured author sources disagree, under the
three parser states this audit went through.
 
The provenance page claimed "13 of the first 90 disagreed" from a run whose
script no longer exists, which is not a reproducible figure. This reconstructs
the comparison from the same cached HTML under each state, so the number on the
page is one anybody can re-derive:
 
  A  non-greedy brace match, no diacritic fold  — the audit's first, buggy state
  B  brace-balanced, no diacritic fold          — after the brace fix only
  C  brace-balanced, diacritic-folded           — what ships
 
    python3 scripts/usenix_source_agreement.py
"""
import glob
import html
import json
import os
import re
import sys
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_author_audit import (bibtex_authors, de_latex, fold, meta_authors)
 
HTMLDIR = "out/usenix_audit/html"
 
 
def bibtex_authors_nongreedy(page):
    """The original bug: \\{(.*?)\\} ends at the first inner brace."""
    m = re.search(r"bibtex-text-entry[^>]*>(.*?)</div>", page, re.S)
    if not m:
        return []
    block = html.unescape(re.sub(r"<br\s*/?>", "\n", m.group(1)))
    a = re.search(r"author\s*=\s*\{(.*?)\}", block, re.S)
    if not a:
        return []
    raw = re.sub(r"\s+", " ", a.group(1)).strip()
    return [de_latex(n).strip() for n in re.split(r"\s+and\s+", raw) if n.strip()]
 
 
def main():
    cached = sorted(k for k in json.load(open("out/authors.json"))
                    if k.startswith("USENIX/"))
    allpages = sorted(glob.glob(os.path.join(HTMLDIR, "*.html")))
    pops = [("the 90 cached author lists",
             [os.path.join(HTMLDIR, k.replace("/", "__") + ".html")
              for k in cached]),
            ("all 162 fetched landing pages", allpages)]
 
    for label, files in pops:
        rows = {"A  non-greedy brace, no fold": 0,
                "B  brace-balanced, no fold": 0,
                "C  brace-balanced + diacritic fold (ships)": 0}
        both = 0
        cases = {k: [] for k in rows}
        for f in files:
            page = open(f, encoding="utf-8", errors="replace").read()
            m = meta_authors(page)
            if not m:
                continue
            bg, bn = bibtex_authors(page), bibtex_authors_nongreedy(page)
            if not bg:
                continue
            both += 1
            key = os.path.basename(f)[:-5].replace("__", "/")
            if [x.strip() for x in m] != [x.strip() for x in bn]:
                rows["A  non-greedy brace, no fold"] += 1
                cases["A  non-greedy brace, no fold"].append(key)
            if [x.strip() for x in m] != [x.strip() for x in bg]:
                rows["B  brace-balanced, no fold"] += 1
                cases["B  brace-balanced, no fold"].append(key)
            if fold(m) != fold(bg):
                rows["C  brace-balanced + diacritic fold (ships)"] += 1
                cases["C  brace-balanced + diacritic fold (ships)"].append(key)
        print(f"=== {label}: {both} carry BOTH citation_author and a BibTeX block")
        for k, v in rows.items():
            print(f"    {k:<46} disagree on {v}")
        rem = cases["C  brace-balanced + diacritic fold (ships)"]
        print(f"    still disagreeing under C: {rem if rem else 'none'}")
        onlyB = [x for x in cases["B  brace-balanced, no fold"] if x not in rem]
        print(f"    fixed by the diacritic fold alone: "
              f"{[x.split('/')[-1][:40] for x in onlyB] or 'none'}")
        print()
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

=== the 90 cached author lists: 87 carry BOTH citation_author and a BibTeX block
    A  non-greedy brace, no fold                   disagree on 11
    B  brace-balanced, no fold                     disagree on 2
    C  brace-balanced + diacritic fold (ships)     disagree on 0
    still disagreeing under C: none
    fixed by the diacritic fold alone: ['iot-market-dynamics-an-analysis-of-devic', 'addressing-the-address-books-interdepend']

=== all 162 fetched landing pages: 155 carry BOTH citation_author and a BibTeX block
    A  non-greedy brace, no fold                   disagree on 17
    B  brace-balanced, no fold                     disagree on 4
    C  brace-balanced + diacritic fold (ships)     disagree on 0
    still disagreeing under C: none
    fixed by the diacritic fold alone: ['vulnerability-disclosure-in-the-age-of-s', 'dissecting-privacy-perspectives-of-websi', 'iot-market-dynamics-an-analysis-of-devic', 'addressing-the-address-books-interdepend']

bib_accent_encoding.py

#!/usr/bin/env python3
"""Count how literature:bibliography spells accented author names.
 
The audit needs this because the author-field diff has to decode LaTeX accents
before comparing, and the reason it has to is that the file uses BOTH
conventions. That was asserted in prose on the provenance page with no script
behind it; three independent counters then produced three different splits
(44/67, 43/66, 45/66), because "has a LaTeX escape" and "has a non-ASCII letter"
are not complements and the edge cases sit in neither or both. This defines the
buckets explicitly and prints the edge cases instead of hiding them.
 
Only the AUTHOR field is counted: an accent in a title has no bearing on the
author diff.
 
    python3 scripts/bib_accent_encoding.py [--bib out/usenix_audit/bib_live.txt]
"""
import os
import re
import sys
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field
 
# A LaTeX ACCENT command: \'e, \"u, \c{s}, \v{c}, \~a, \^i, \=o, \.z, \u{a},
# \H{o}, \r{a}, braced or not. Deliberately NOT \ss, \o, \aa, \O, \AA — those
# are ligature/letter commands, not accents, and they are reported separately
# because they are exactly what made the three earlier counts disagree.
ACCENT = re.compile(r"\\[\'\"`^~=.]|\\[cvuHr]\s*\{")
LETTER_CMD = re.compile(r"\\(ss|o|O|aa|AA|l|L|ae|AE|oe|OE)\b|\\\{?(ss|o|O|aa|AA)\}")
NONASCII = re.compile(r"[^\x00-\x7f]")
 
 
def main():
    bib = sys.argv[sys.argv.index("--bib") + 1] if "--bib" in sys.argv \
        else "out/usenix_audit/bib_live.txt"
    entries = re.findall(r"@\w+\{[^@]*?\n\}",
                         open(bib, encoding="utf-8").read(), re.S)
    n = len(entries)
    acc, uni, cmd, both, acc_cmd, neither = [], [], [], [], [], 0
    for e in entries:
        key = re.match(r"@\w+\{([^,]+),", e).group(1)
        a = field(e, "author") or ""
        A, U, C = bool(ACCENT.search(a)), bool(NONASCII.search(a)), \
            bool(LETTER_CMD.search(a))
        if A:
            acc.append(key)
        if U:
            uni.append(key)
        if C:
            cmd.append(key)
        if A and U:
            both.append(key)
        if A and C:
            acc_cmd.append(key)
        if not (A or U or C):
            neither += 1
 
    print(f"bibliography                                  : {bib}")
    print(f"entries                                       : {n}")
    print(f"AUTHOR field has a LaTeX accent command       : {len(acc)}")
    print(f"AUTHOR field has a literal non-ASCII letter   : {len(uni)}")
    print(f"AUTHOR field has a LaTeX LETTER command      : {len(cmd)}  "
          f"(\\ss, \\o, \\aa — not accents)")
    print(f"AUTHOR field has BOTH accent and non-ASCII    : {len(both)}")
    print(f"AUTHOR field has BOTH accent and letter command: {len(acc_cmd)}")
    print(f"AUTHOR field is plain ASCII, no commands      : {neither}")
    print(f"AUTHOR field is plain ASCII, no commands      : {neither}"
          if False else "", end="")
    print(f"\nThe buckets OVERLAP — do not add the rows up. Distinct entries "
          f"touched by any of the three: {len(set(acc) | set(uni) | set(cmd))}")
    print(f"Distinct entries using an accent command or a non-ASCII letter: "
          f"{len(set(acc) | set(uni))}")
    if both:
        print("\nMIXED — these entries spell one name each way:")
        for k in both:
            print(f"  {k}")
    if cmd:
        print("\nLaTeX LETTER commands (the reason three earlier hand counts "
              "disagreed — they are neither accents nor non-ASCII):")
        for k in cmd:
            e = next(x for x in entries if x.startswith("@") and
                     re.match(r"@\w+\{([^,]+),", x).group(1) == k)
            m = LETTER_CMD.search(field(e, "author"))
            print(f"  {k:<32} {m.group(0)!r}")
    return 0
 
 
if __name__ == "__main__":
    sys.exit(main())

bibliography                                  : out/usenix_audit/bib_live.txt
entries                                       : 837
AUTHOR field has a LaTeX accent command       : 43
AUTHOR field has a literal non-ASCII letter   : 66
AUTHOR field has a LaTeX LETTER command      : 3  (\ss, \o, \aa — not accents)
AUTHOR field has BOTH accent and non-ASCII    : 0
AUTHOR field has BOTH accent and letter command: 1
AUTHOR field is plain ASCII, no commands      : 726

The buckets OVERLAP — do not add the rows up. Distinct entries touched by any of the three: 111
Distinct entries using an accent command or a non-ASCII letter: 109

LaTeX LETTER commands (the reason three earlier hand counts disagreed — they are neither accents nor non-ASCII):
  demir2023_similarity             '\\ss'
  mackinnon2023_clusterrobust      '\\O'
  coopamootoo2017_forgotten        '\\ss'

popets_doi_check.py

#!/usr/bin/env python3
"""Check every PoPETs DOI in literature:bibliography against the correct prefix
boundary, and resolve each one.
 
scripts/bibgen.mjs used to hardcode 10.56553 for all years. PoPETs has changed
DOI prefix twice:
 
    2015 - 2018        10.1515    (De Gruyter Open)
    2019 - 2022-0058   10.2478    (De Gruyter / Sciendo)
    2022-0059 onward   10.56553   (PoPETs' own prefix)
 
so the hardcoded prefix produced a dead DOI for every PoPETs paper before
2022-0059. Fixed 2026-08-13. This checks what actually reached the wiki:
a prefix that disagrees with the boundary is reported whether or not it
resolves, and a DOI that does not resolve is reported whether or not the prefix
is right — the two failures are independent.
 
    python3 scripts/popets_doi_check.py [--bib out/usenix_audit/bib_live.txt]
                                        [--no-network]
"""
import os
import re
import subprocess
import sys
import time
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field
 
DEFAULT_BIB = "out/usenix_audit/bib_live.txt"
UA = ("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) "
      "Chrome/126.0.0.0 Safari/537.36")
 
 
def expected_prefix(year, article):
    y, n = int(year), int(article)
    if y <= 2018:
        return "10.1515"
    if y < 2022 or (y == 2022 and n <= 58):
        return "10.2478"
    return "10.56553"
 
 
def resolve(doi):
    """HTTP status of https://doi.org/<doi>, following redirects."""
    r = subprocess.run(
        ["curl", "-sIL", "-A", UA, "--max-time", "30",
         "-o", "/dev/null", "-w", "%{http_code} %{url_effective}",
         f"https://doi.org/{doi}"], capture_output=True, text=True)
    code, _, url = r.stdout.strip().partition(" ")
    return code, url
 
 
def main():
    bib = sys.argv[sys.argv.index("--bib") + 1] if "--bib" in sys.argv \
        else DEFAULT_BIB
    network = "--no-network" not in sys.argv
    entries = re.findall(r"@\w+\{[^@]*?\n\}",
                         open(bib, encoding="utf-8").read(), re.S)
 
    rows, no_doi = [], []
    for e in entries:
        series = (field(e, "series") or "") + " " + (field(e, "booktitle") or "")
        if not re.search(r"PoPETs|Privacy Enhancing Technolog", series, re.I):
            continue
        key = re.match(r"@\w+\{([^,]+),", e).group(1)
        doi = field(e, "doi")
        if not doi:
            no_doi.append((key, field(e, "url") or "no url either"))
            continue
        doi = re.sub(r"^https?://(dx\.)?doi\.org/", "", doi)
        m = re.search(r"popets-(\d{4})-(\d{4})", doi)
        exp = expected_prefix(*m.groups()) if m else None
        rows.append([key, doi, exp, doi.split("/")[0], "", ""])
 
    print(f"bibliography              : {bib}")
    print(f"PoPETs entries with a DOI : {len(rows)}")
    print(f"PoPETs entries without    : {len(no_doi)}")
 
    if network:
        for r in rows:
            r[4], r[5] = resolve(r[1])
            time.sleep(0.3)
 
    bad_prefix = [r for r in rows if r[2] and r[2] != r[3]]
    unparsed = [r for r in rows if not r[2]]
    dead = [r for r in rows if network and not r[4].startswith(("2", "3"))]
 
    print(f"  prefix disagrees with the boundary : {len(bad_prefix)}")
    print(f"  DOI not in popets-YYYY-NNNN form   : {len(unparsed)}")
    print(f"  does not resolve                   : "
          f"{len(dead) if network else 'not checked (--no-network)'}")
    print()
    print("%-34s %-26s %-9s %-5s %s"
          % ("citekey", "doi", "expected", "http", "resolves to"))
    for key, doi, exp, got, code, url in sorted(rows):
        flag = "  <-- PREFIX" if exp and exp != got else ""
        print("%-34s %-26s %-9s %-5s %s%s"
              % (key, doi, exp or "n/a", code or "-", url[:60], flag))
    for key, url in no_doi:
        print(f"{key:<34} NO DOI                     -         -     {url}")
    return 1 if (bad_prefix or dead) else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

Real output, abridged here to its counters — the full 86-row table is in scripts/popets_doi_check-output.txt:

bibliography              : out/usenix_audit/bib_live.txt
PoPETs entries with a DOI : 84
PoPETs entries without    : 2
  prefix disagrees with the boundary : 0
  DOI not in popets-YYYY-NNNN form   : 1
  does not resolve                   : 0

citekey                            doi                        expected  http  resolves to

bib_doi_duplicates.py

#!/usr/bin/env python3
"""Scan literature:bibliography for two entries describing the same paper.
 
A citekey-collision check passes while the same paper sits in the file twice
under different keys, and the bibtex4dw plugin will happily render both in the
reference list. Match on DOI first (exact), then on squashed title for the
entries that have no DOI.
 
    python3 scripts/bib_doi_duplicates.py [--bib out/usenix_audit/bib_live.txt]
"""
import os
import re
import sys
from collections import defaultdict
 
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from usenix_bib_diff import field, squash
 
bib = sys.argv[sys.argv.index("--bib") + 1] if "--bib" in sys.argv \
    else "out/usenix_audit/bib_live.txt"
entries = re.findall(r"@\w+\{[^@]*?\n\}", open(bib, encoding="utf-8").read(), re.S)
 
by_doi, by_title, keys = defaultdict(list), defaultdict(list), []
for e in entries:
    k = re.match(r"@\w+\{([^,]+),", e).group(1)
    keys.append(k)
    doi = field(e, "doi")
    if doi:
        by_doi[re.sub(r"^https?://(dx\.)?doi\.org/", "", doi).lower()].append(k)
    t = field(e, "title")
    if t:
        by_title[squash(t)].append(k)
 
print(f"bibliography     : {bib}")
print(f"entries          : {len(entries)}")
print(f"distinct citekeys: {len(set(keys))}")
dupkeys = sorted({k for k in keys if keys.count(k) > 1})
print(f"COLLIDING citekeys                  : {len(dupkeys)}"
      + (f"  {dupkeys}" if dupkeys else ""))
 
d_doi = {k: v for k, v in by_doi.items() if len(v) > 1}
d_tit = {k: v for k, v in by_title.items()
         if len(v) > 1 and not any(set(v) <= set(x) for x in d_doi.values())}
print(f"same DOI under different citekeys   : {len(d_doi)}")
for doi, ks in sorted(d_doi.items()):
    print(f"  {doi}\n      {', '.join(ks)}")
print(f"same TITLE, no shared DOI           : {len(d_tit)}")
for t, ks in sorted(d_tit.items()):
    print(f"  {t[:70]}\n      {', '.join(ks)}")
sys.exit(1 if (dupkeys or d_doi or d_tit) else 0)

bibliography     : out/usenix_audit/bib_live.txt
entries          : 837
distinct citekeys: 837
COLLIDING citekeys                  : 0
same DOI under different citekeys   : 3
  10.56553/popets-2022-0063
      fouad2022my, fouad2022_cookie
  10.56553/popets-2025-0063
      bottger2025_regional, boettger2025_regional
  10.56553/popets-2026-0109
      ahmad2026_ipfp, ahmad2026_more
same TITLE, no shared DOI           : 2
  automatedlargescaleanalysisofcookienoticecompliance
      bouhoula2024automated, bouhoula2024_automated
  internetjonesandtheraidersofthelosttrackersanarchaeologicalstudyofwebt
      lerner2016internet, lerner2016_internet

bib_dupe_cocitation.py

#!/usr/bin/env python3
"""Two small checks the provenance page was asserting without a script.
 
  1. For each duplicate pair from scripts/bib_doi_duplicates.py, does any wiki
     page cite BOTH keys? If one does, its rendered reference list shows the
     same paper twice — that is the difference between a latent duplicate and a
     visible one, and the page's "nothing renders wrong today" rests on it.
  2. The 42 PETS entries in out/authors.json come from a different parser
     (parse_popets) against a different page shape, so they are out of scope for
     the USENIX audit. This is the smoke test that says so honestly: an
     affiliation-shaped or single-token name in any of them. It is NOT the
     three-source audit the USENIX keys got, and it is labelled that way.
 
Reads the page exports under out/usenix_audit/pages/, made with
`node scripts/dw.mjs pages` + `?do=export_raw` per page.
 
    python3 scripts/bib_dupe_cocitation.py
"""
import glob
import json
import os
import re
import sys
 
PAGES = "out/usenix_audit/pages"
BIBPAGE = "literature_bibliography"
 
PAIRS = [("fouad2022my", "fouad2022_cookie"),
         ("bottger2025_regional", "boettger2025_regional"),
         ("ahmad2026_ipfp", "ahmad2026_more"),
         ("bouhoula2024automated", "bouhoula2024_automated"),
         ("lerner2016internet", "lerner2016_internet")]
 
AFFIL = re.compile(
    r"universi|institut|college|inc\.|labs?\b|corporat|technolog|research|"
    r"school|center|centre|gmbh|ltd|cispa|kaist|inria|microsoft|google|meta|"
    r"apple|amazon|ibm|eth |epfl|tu |laborator|foundation|agency|department|"
    r"polytechnic|\bteam\b|e\.?\s?v\.?$", re.I)
 
 
def main():
    files = [f for f in sorted(glob.glob(os.path.join(PAGES, "*.txt")))
             if BIBPAGE not in os.path.basename(f)]
    pages = {os.path.basename(f)[:-4]: open(f, encoding="utf-8",
                                            errors="replace").read()
             for f in files}
    print(f"page exports read (literature:bibliography excluded) : {len(pages)}")
 
    worst, touched = 0, set()
    print("\n^ duplicate pair ^ pages citing key A ^ citing key B ^ citing BOTH ^")
    for a, b in PAIRS:
        ca = {p for p, t in pages.items() if "{[" + a + "]}" in t}
        cb = {p for p, t in pages.items() if "{[" + b + "]}" in t}
        both = sorted(ca & cb)
        worst = max(worst, len(both))
        touched |= ca | cb
        print(f"| {a} / {b} | {len(ca)} | {len(cb)} | "
              f"{'**' + ', '.join(both) + '**' if both else '0'} |")
    print(f"\npages citing at least one key of any pair : {len(touched)}")
    print(f"pages citing BOTH keys of any pair       : {worst}")
    print("A page citing both would render the same paper twice in its "
          "reference list.")
 
    cache = json.load(open("out/authors.json"))
    pets = {k: v for k, v in cache.items() if k.startswith("PETS/")}
    flagged = {k: [n for n in v
                   if AFFIL.search(n) or len(n.split()) < 2]
               for k, v in pets.items()}
    flagged = {k: v for k, v in flagged.items() if v}
    print(f"\nPETS entries in out/authors.json                : {len(pets)}")
    print(f"  with an affiliation-shaped or single-token name: {len(flagged)}")
    for k, v in sorted(flagged.items()):
        print(f"    {k}: {v}")
    print("  This is a smoke test over parse_popets output, not the "
          "three-source audit the USENIX keys got.")
    return 1 if (worst or flagged) else 0
 
 
if __name__ == "__main__":
    sys.exit(main())

page exports read (literature:bibliography excluded) : 160

^ duplicate pair ^ pages citing key A ^ citing key B ^ citing BOTH ^
| fouad2022my / fouad2022_cookie | 4 | 2 | 0 |
| bottger2025_regional / boettger2025_regional | 2 | 2 | 0 |
| ahmad2026_ipfp / ahmad2026_more | 1 | 1 | 0 |
| bouhoula2024automated / bouhoula2024_automated | 4 | 11 | 0 |
| lerner2016internet / lerner2016_internet | 3 | 2 | 0 |

pages citing at least one key of any pair : 26
pages citing BOTH keys of any pair       : 0
A page citing both would render the same paper twice in its reference list.

PETS entries in out/authors.json                : 42
  with an affiliation-shaped or single-token name: 0
  This is a smoke test over parse_popets output, not the three-source audit the USENIX keys got.

Four reviewers, all told explicitly that the authoring context may not be exhaustive, and all handed the page text, every script and its real output. Findings and dispositions are recorded below, including the two that were rejected or downgraded.

Handed the frozen draft, every script and every committed output, and told to re-run everything rather than read it.

Second pass, after the fixes above. The same reviewer re-ran everything and confirmed all seven fixes, including that the byte-identity guard fires when the generator ordering is reverted, and that de_latex decodes every LaTeX-escaped author field in the bibliography and in all 162 cached BibTeX blocks with no residual artefact. It found three more:

Everything on the page's external surface was re-fetched. Nothing was found stale, which for a page whose external claims are mostly about DOI prefixes and one vendor's HTML is the expected result.

Writing usenix_source_agreement.py — which exists only because a reviewer objected to the unreproducible “13” — surfaced a third decoder bug neither the reviewers nor the original audit had seen. USENIX writes Benoît as Beno{\^\i}t with LaTeX's dotless \i, which no accent pattern matched, so the final brace strip produced Beno^it. It sat on cohn2020_delf, one of the 72 pages that is not among the 90 cached lists, so only a comparison across all 162 could reach it. The live entry was always correct and no published figure moved; de_latex now has a nine-case self-test (usenix_author_audit.py –selftest) whose first case is that string.

Of the four reviewer findings that led to new code rather than new prose, two were requests to make an already-correct-looking figure reproducible.

Ran with no checklist after the focused findings were applied.

Two findings from the earlier passes were not accepted as stated:

• “programming:crawler_detection cites no Doupé-authored paper at all.” It cites two, zhang2021_crawlphish and zhang2022_spartacus. The reviewer conclusion was right for a different reason — both spell the name with a literal é, so that page verified nothing about LaTeX escapes — and the fix was made on that basis, not the one offered.
• “5,859 extracted papers is wrong; there are 5,869.” Two different populations: 5,859 records in extractions.jsonl, 5,869 papers with full text on disk. Recorded as a clarification, not a correction; the run record now gives both and says neither is used here.

[1]

Tang, Brian; Bui, Duc; Shin, Kang G. (2025): "Navigating Cookie Consent Violations Across the Globe", in: Proceedings of the USENIX Security Symposium. (Link)

[2]

McCoy, Damon; Pitsillidis, Andreas; Jordan, Grant; Weaver, Nicholas; Kreibich, Christian; Krebs, Brian; Voelker, Geoffrey M.; Savage, Stefan; Levchenko, Kirill (2012): "PharmaLeaks: Understanding the Business of Online Pharmaceutical Affiliate Programs", in: Proceedings of the USENIX Security Symposium. (Link)

[3]

Oest, Adam; Safaei, Yeganeh; Zhang, Penghui; Wardman, Brad; Tyers, Kevin; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2020): "PhishTime: Continuous Longitudinal Measurement of the Effectiveness of Anti-phishing Blacklists", in: Proceedings of the USENIX Security Symposium. (Link)

[4]

Hsieh, Jhih-Yi; Raghunathan, Aditi; Shah, Nihar B. (2025): "Vulnerability of Text-Matching in ML/AI Conference Reviewer Assignments to Collusions", in: Proceedings of the USENIX Security Symposium. (Link)

[5]

Kapravelos, Alexandros; Grier, Chris; Chachra, Neha; Kruegel, Christopher; Vigna, Giovanni; Paxson, Vern (2014): "Hulk: Eliciting Malicious Behavior in Browser Extensions", in: Proceedings of the USENIX Security Symposium. (Link)

[6]

Lyons, Allan; Gamba, Julien; Shawaga, Austin; Reardon, Joel; Tapiador, Juan; Egelman, Serge; Vallina-Rodriguez, Narseo (2023): "Log: It's Big, It's Heavy, It's Filled with Personal Data! Measuring the Logging of Sensitive Information in the Android Ecosystem", in: Proceedings of the USENIX Security Symposium. (Link)

[7]

Vekaria, Yash; Canino, Aurelio Loris; Levitsky, Jonathan; Ciechonski, Alex; Callejo, Patricia; Mandalari, Anna Maria; Shafiq, Zubair (2025): "Big Help or Big Brother? Auditing Tracking, Profiling, and Personalization in Generative AI Assistants", in: Proceedings of the USENIX Security Symposium. (Link)

[8]

Bielova, Nataliia; Litvine, Laura; Nguyen, Anysia; Chammat, Mariam; Toubiana, Vincent; Hary, Estelle (2024): "The Effect of Design Patterns on (Present and Future) Cookie Consent Decisions", in: Proceedings of the USENIX Security Symposium. (Link)

[9]

Carlini, Nicholas; Felt, Adrienne Porter; Wagner, David (2012): "An Evaluation of the Google Chrome Extension Security Architecture", in: Proceedings of the USENIX Security Symposium. (Link)

[10]

Trickel, Erik; Starov, Oleksii; Kapravelos, Alexandros; Nikiforakis, Nick; Doupé, Adam (2019): "Everyone is Different: Client-side Diversification for Defending Against Extension Fingerprinting", in: Proceedings of the USENIX Security Symposium. (Link)