Working log behind corpus. This page holds the working notes for that page's own figures; the corpus-level caveats every other provenance: page inherits — the venue scope, the funnel, the provisional years — are on corpus itself and are not restated here.

Its whole content, as of 2026-09-04, is one audit: the corpus-wide diff of an independent PDF rendering against the stored text renderings, which estimates what share of §6's 1.3% unlocatable-quote rate is a text-extraction artefact rather than an extraction defect. Estimates, not settles: it rests on one alternative PDF reader whose own failure rate is worse, and on a page-layout detector with no negative control. Both limits are in §7. corpus's other sections were written on 2026-08-12 with their notes inline, in its own §9 and §11; this page does not duplicate them.

hypothesis_testing published, on 2026-08-13, that four of Mai et al.'s methodology quotes (PoPETs 2025, More and Scammier Ads) were present in the stored paper.pdf and in none of paper.cols.txt, paper.norm.txt or paper.txt, concluded that “the extraction was faithful; the stored rendering is deficient”, and logged a corpus-wide diff as the highest-value follow-up on that page.

The premise was wrong. Checked in Python on 2026-09-04, both sentences the content page quotes are in paper.cols.txt verbatim after the normalisation every checker on this site applies, and Friedman, Conover, omnibus, 172.47 and all ten authors are in all three renderings:

                                                        paper.txt  norm  cols
one-way repeated measures ANOVA as the omnibus test           no     no   YES
  since the ad load data follow a normal distribution
for the hypothesis on predatory ad rates, since the           no     no   YES
  rates do not approximately follow a normal
  distribution, we use Friedman test as the omnibus test
Friedman (7x) / Conover (6x) / omnibus / 172.47              yes    yes   yes
Kieserman / Matsumoto / Andreou / Greenstadt / McCoy         yes    yes   yes

The two full sentences are missing from paper.txt and paper.norm.txt for the ordinary reason: those renderings keep the columns side by side, so no sentence spanning a line is contiguous in them. That is the entire reason paper.cols.txt exists.

pdftotext writes U+0000 wherever a PDF's font encoding maps a glyph to nothing. That paper's paper.cols.txt has 181 of them, so it is a binary file to any grep. What happens next depends on which grep, and the one you get by typing grep in this agent's shell is not the one installed on the machine:

$ type -a grep
grep is a shell function from /home/dev/.claude/shell-snapshots/snapshot-zsh-....sh
grep is /usr/bin/grep

$ typeset -f grep     # abridged
grep () { ...
  exec -a ugrep "$CLAUDE_CODE_EXECPATH" -G --ignore-files --hidden -I        --exclude-dir=.git ... "$@"
}

The agent harness installs a shell function that re-execs its own binary under the argv0 ugrep, with a fixed set of flags — and -I is one of them. -I means treat a binary file as containing no match. So on this file:

$ cd data/fulltext/2025/PETS/more-and-scammier-ads-.../
$ grep -c Friedman paper.cols.txt        # the function: no output, exit 1
$ grep    Friedman paper.cols.txt        # no output, exit 1
$ grep -o Friedman paper.cols.txt        # no output, exit 1
$ grep -a -c Friedman paper.cols.txt     # -a overrides the hardcoded -I
7

The real grep on this machine gets it right. /usr/bin/grep is GNU grep 3.8, it is not what the shell function runs, and it neither hides nor miscounts:

$ /usr/bin/grep --version | head -1
grep (GNU grep) 3.8
$ /usr/bin/grep -c Friedman paper.cols.txt
7
$ /usr/bin/grep Friedman paper.cols.txt
/usr/bin/grep: paper.cols.txt: binary file matches

So the trap is narrower and nastier than “grep hides binary matches”. It is: the grep an agent types is a wrapper with -I baked in, and no output plus exit 1 is indistinguishable from a real negative. It applies to the interactive shell an agent works in, not to the machine, and not to any script that shells out to grep itself — those get GNU grep 3.8. It was still enough to put a false finding on a published page.

¹⁾

The exposure, from scripts/audit_binary_detect.mjs:

90 papers, 74,772 NUL bytes, and 2,156 of the 135,025 evidence quotes (1.6%) sit in them. They cluster in the recent years and in traffic-classification and ML papers — NDSS 35, WWW 20, CCS 15, IEEE S&P 10, USENIX 5, IMC 4, PoPETs 1; 2020 onwards holds 89 of the 90. None of this audit's scripts shell out to grep: they test for NUL with fs.readFileSync(f).includes(0) in process.

python3 scripts/audit_cols_vs_pdf.py --out out/colsaudit/per_paper.jsonl \
        --workers 2 --cache out/colsaudit/pdftext
node    scripts/audit_decolumn_pages.mjs --out out/colsaudit/per_page.jsonl
python3 scripts/audit_page_geometry.py --sample out/colsaudit/per_page.jsonl \
        --n 660 --out out/colsaudit/geometry_sample.jsonl
node    scripts/audit_binary_detect.mjs
node    scripts/report_cols_vs_pdf.mjs --examples 12 --both-fail 15
node    scripts/report_cols_mechanism.mjs

pdftotext is not installed in this container, so the stored paper.txt could not be regenerated and the comparison had to use a different reader rather than the same one. pypdf 6.16.2 was used — the current release on PyPI as of 2026-09-04. pdfminer, PyMuPDF, pdfplumber, mutool, qpdf and ghostscript are all absent too, so there is no third rendering and no majority vote: every “present in the PDF” verdict on this page rests on one reader.

pypdf refuses to decompress a stream over 75 MB and refuses an array-based stream whose parts total over 75 MB, and five of this corpus's PDFs are legitimately that large. On the first run that left 151 quotes with no second rendering. Both limits are raised in audit_cols_vs_pdf.py and pdftext.py:

import pypdf.filters as _f
_f.ZLIB_MAX_OUTPUT_LENGTH = 0                    # 0 disables, per its own docstring
_f.LZW_MAX_OUTPUT_LENGTH = 0
_f.RUN_LENGTH_MAX_OUTPUT_LENGTH = 10**12
_f.MAX_ARRAY_BASED_STREAM_OUTPUT_LENGTH = 10**12

Both constants are in pypdf.filters, including the array-based one. pypdf.generic._data_structures imports it from there at call time, so setting the name in _data_structures has no effect — which cost one wrong “still fails after raising the limit” conclusion during this run. With the raise, 5 of the 6 failures go away; the sixth needs the cryptography package, which pip refuses to install here under PEP 668 (see §8).

Denominators that are easy to mix up here. 135,025 is every evidence quote; 135,004 is the subset with a verdict on both sides, and the 21-quote difference is exactly the one paper that still cannot be rendered. 97,505 was the page count before the phantom trailing segment was removed; 91,509 is the real one. 5,859 is every record; 5,858 have a cached rendering, 5,856 have a comparable character count, 5,853 have both a quote and a page verdict. The 1,806 / 931 / 541 / 118 / 216 figures quoted from hypothesis_testing are hypothesis-test quotes only, scored by ht_quotecheck.mjs's much looser 60%-of-5-word-windows rule — not by the four-verdict rule everything else here uses, and not comparable to it.

report_cols_vs_pdf.mjs does not approximate report_corpus.mjs –quotes: its four-verdict function is copied from it, including the fragmented anchor search. Run on the same day, on the same records:

15 + 6 + 0 + 0 = 21, the quotes in the one paper pypdf still cannot render. No quote changed bucket. Without that check the pdf column of every table below would be uninterpretable, because a difference could as easily be two verdict functions disagreeing as two renderings disagreeing.

Six, with scripts/audit_quote_handcheck.py. All six are present in the paper; none is a fabrication.

Two different rules produce two different “rescued” populations, and mixing them up is easy. The four-verdict rule this page's tables use (exact / 8-word run / fragmented / not found) is the one report_corpus.mjs uses, and it is lenient: a spliced page usually keeps an 8-word run, so the quote still counts as located. quote_check.mjs and ht_quotecheck.mjs use a stricter per-page rule — 60% of the quote's 5-word windows — and a quote can be below that threshold while being comfortably run8 under the four-verdict rule. The last two rows below are of that second kind, and an earlier draft of this page labelled them as if they belonged to the 1,708. The citations-and-quotes reviewer caught it.

The examples printed by report_cols_vs_pdf.mjs are spread evenly through the corpus order, not the first N. The first twelve rescued quotes are all from 2010, for the same reason corpus's five were all from 2010–2012: the list is filled in sorted-key order. That is a property of the tool, and it produced a criticised sample once already.

audit_quote_handcheck.py tests exact containment only, so a quote whose PDF verdict is run8 or fragmented prints in pdf: False. Read the printed neighbourhood, not the boolean — the AR/VR and Chameleoscan rows above are both of that kind.

Nothing here is a free-text name fold, so there is no alias list. There are three normalisations, and each one hides something, so each is stated with what it hides.

The residue of the character-ratio comparison, in full. 34 of 5,856 papers fall below 0.95. Four are artefacts of the comparison — two whose extractor input was paper.mistral.md rather than paper.cols.txt (so the ratio compared a file nothing read), two where pypdf glyph names survived the strip because they are under the 10% threshold. The other 30 are listed in Appendix B and none was hand-read; they are candidates for real loss, they are 0.5% of the corpus, and the worst holds 58% of the PDF's characters rather than none of it.

Two residues that do not exist and should. The 458 quotes neither rendering locates are characterised from 12 read examples, not classified exhaustively. And the 31 sampled pages the geometry detector called too-little-text were not adjudicated at all — they are excluded from every share, which is the conservative direction for the 0.0–0.1 band and the anti-conservative direction nowhere.

Six found before review, all the same shape — a metric that looked clean and was measuring the wrong thing — and all found by re-reading an output rather than by re-reading the code.

1. The first sentence-recall metric measured pypdf's defects, not the stored rendering's. Exact-sentence containment scored 65% recall on a 60-paper probe, and reading the misses showed pypdf inserting spaces inside words. Fixed by comparing with all spacing removed, then by replaying the real quotes instead of synthetic sentences.
2. The first two-column ground-truth detector called a hand-verified two-column page single. It projected each text run onto the x axis using len(text) × font_size × 0.5 and looked for an empty band; the width estimate over-shot and filled the gutter. Replaced with the x-histogram of run starts, which needs no width estimate — and then validated on the 60 reflowed pages, where it is 60 for 60.
3. Splitting paper.txt on the form feed produced 5,996 phantom pages. Files end with a form feed, so the last segment is empty. They were counted as untouched single-column pages, put the corpus page total at 97,505 instead of 91,509, and diluted every share computed from the lowest score band. Found by noticing that 37 of 60 sampled band-0.0 pages did not exist in their PDF.
4. The character-ratio comparison read paper.cols.txt for papers whose extractor input was paper.mistral.md. Three of the five worst apparent content losses were that. It is the same bug report_corpus.mjs had already had to fix once, reintroduced in a new script.
5. Refactoring ht_quotecheck.mjs silently moved its published exact count from 931 to 939. Pulling the window-fraction test into a shared helper made it return 1 for a quote whose every 5-word window is present but which is not contiguous, folding 8 such quotes into exact. Caught because the number had to reproduce a published figure and did not. exact now means verbatim and is tested separately.
6. The pypdf array-stream limit appeared unraisable. Setting MAX_ARRAY_BASED_STREAM_OUTPUT_LENGTH in pypdf.generic._data_structures, where the error is raised, changes nothing: that module imports the name from pypdf.filters at call time. One run was written off as “still fails” on that basis before the import was read.

One more, which cost 10 minutes of a 37-minute run: pypdf returns unpaired surrogates for a malformed ToUnicode map, writing one raises UnicodeEncodeError, and inside a multiprocessing.Pool worker that takes the whole run down. The restart is why the script now reuses its cache.

Four passes, all given the page text, the scripts, their output and these notes, and all told explicitly that the author's context might not be exhaustive. Three ran in parallel first; their findings were fixed and the figures re-derived before the fourth.

Run after the three above were fixed, with no checklist.

• corpus — the page these notes are for.
• hypothesis_testing — the retraction this audit produced, and the page that logged the audit as a TODO.
• hypothesis_testing — the only content page whose quote-check figures have been refreshed.

report_cols_vs_pdf-output.txt

=== Quote verdicts: the stored rendering vs an independent one =========
 
papers with a cached pypdf rendering   5,858 of 5,859
quotes checked against BOTH renderings 135,004
quotes skipped, no extractor source    0
quotes skipped, no cached PDF text     21
 
Marginals. The `source` column is report_corpus.mjs --quotes reproduced on
the same quotes; `pdf` is the same verdict function against pypdf.
 
  verdict                    source            pdf
  present exactly            78435   58.1%     79326   58.8%
  an 8-word run survives     49419   36.6%     48595   36.0%
  fragmented                  5442    4.0%      3963    2.9%
  not found                   1708    1.3%      3120    2.3%
 
Cross-tab, rows = stored rendering, columns = PDF rendering:
 
                               exact      run8      frag  notfound     total
  present exactly              60452     15306      1228      1449     78435
  an 8-word run survives       16315     31700       536       868     49419
  fragmented                    2048       987      2062       345      5442
  not found                      511       602       137       458      1708
 
NOT FOUND in the stored rendering                 1,708 (1.3% of quotes)
  ...of which the PDF rendering DOES contain      1,250 (73.2% of them)
  ...verbatim ("exact")                           511
  ...as an 8-word run                             602
  ...fragmented only                              137
  ...not found in either rendering                458 (0.3% of quotes)
 
The reverse, as a noise floor: quotes the stored rendering finds and the
PDF does not                                      2,662 (2.0% of quotes)
 
Distinct papers holding at least one rescued quote: 862 of 5,859 (14.7%)
 
By venue:
  Venue           notfound   rescued    share
  CCS                  202       136    67.3%
  IEEE-SP              148        97    65.5%
  IMC                  151       123    81.5%
  NDSS                 178       136    76.4%
  PETS                 256       207    80.9%
  USENIX               471       369    78.3%
  WWW                  302       182    60.3%
 
By year:
  Year            notfound   rescued    share
  2010                  21        15    71.4%
  2011                  21        15    71.4%
  2012                  34        28    82.4%
  2013                  20        12    60.0%
  2014                  40        27    67.5%
  2015                  49        38    77.6%
  2016                  53        46    86.8%
  2017                  80        64    80.0%
  2018                  79        62    78.5%
  2019                 103        76    73.8%
  2020                  92        64    69.6%
  2021                  96        71    74.0%
  2022                 166       123    74.1%
  2023                 245       133    54.3%
  2024                 208       151    72.6%
  2025                 243       196    80.7%
  2026                 158       129    81.6%
 
By relation family:
  Family          notfound   rescued    share
  artifacts             98        54    55.1%
  classification       153       109    71.2%
  crawlConfig            7         6    85.7%
  detection            247       161    65.2%
  ethics                70        43    61.4%
  humanAnnotation        64        59    92.2%
  legal                 16        12    75.0%
  participants          22        18    81.8%
  population           169       122    72.2%
  statistics           147       111    75.5%
  temporal             105        75    71.4%
  tools                566       447    79.0%
  vantage               44        33    75.0%
 
=== The mechanism's prediction, tested ================================
 
A page whose two columns were never separated keeps every word and
destroys the order, so a quote from it should lose its `exact` verdict and
land in `an 8-word run survives` — a pdftotext line holds roughly ten
words, which is why the strict `not found` bucket only catches the tail.
If that is the mechanism, `exact` must fall monotonically with the share
of a paper's pages left unrepaired, and `run8` must rise.
 
  unrepaired   papers   quotes    exact     run8     frag  notfound
  0%              585    12486    70.6%    26.7%     2.2%      0.6%
  0–10%           951    22898    65.1%    31.2%     2.9%      0.7%
  10–20%         2009    47855    59.5%    35.3%     4.0%      1.3%
  20–35%         1846    42335    52.5%    40.9%     4.9%      1.6%
  35–50%          353     7652    43.7%    48.6%     5.8%      1.9%
  50%+            109     1734    36.6%    56.8%     5.2%      1.4%
 
12 of the 458 quotes NEITHER rendering can locate, evenly spread through the corpus, for hand-reading:
 
  IMC/2010/measurement-of-loss-pairs-in-network-paths  [vantage]
    between eight local universities in Hong Kong ... and the 14 PlanetLab nodes located at eight countries
  USENIX/2014/jigsaw-protecting-resource-access-by-inferring-programmer-expectations  [tools]
    Figure 8: Implementation of JIGSAW's testing framework.
  PETS/2018/panoptispy-characterizing-audio-and-video-exfiltration-from-android-applications  [tools]
    For apps that prevent TLS interception via certificate pinning, we use JustTrustMe [13], which modifies Android to bypass certificate pinning
  CCS/2020/enhancing-state-of-the-art-classifiers-with-api-semantics-to-detect-evolved-andr  [tools]
    our API embedding and clustering is built with TensorFlow [44] and sklearn [41] respectively.
  PETS/2021/website-fingerprinting-in-the-age-of-quic  [tools]
    We selected Wireguard [24] as a representative VPN technology.
  USENIX/2022/a-large-scale-investigation-into-geodifferences-in-mobile-apps  [tools]
    For app signature, we combine results from keytool [68] and apksigner [13].
  USENIX/2023/animatedead-debloating-web-applications-using-concolic-execution  [detection]
    We debloat popular PHP applications and demonstrate ... that ... web applications [are] 25-69% smaller than their original versions
  WWW/2023/the-chameleon-on-the-web-an-empirical-study-of-the-insidious-proactive-web-defac  [tools]
    none of the 154 control scripts were filtered by either EasyList or EasyPrivacy.
  WWW/2023/longitudinal-assessment-of-reference-quality-on-wikipedia  [statistics]
    T-test was used to measure the effect of expertise on reference quality
  USENIX/2024/remote-keylogging-attacks-in-multi-user-vr-applications  [artifacts]
    The full version can be found at https://arxiv.org/abs/2405.14036.
  NDSS/2025/mineshark-cryptomining-traffic-detection-at-scale  [vantage]
    MineShark is deployed at our campus gateway to monitor cryptomining incidents.
  USENIX/2025/data-duplication-a-novel-multi-purpose-attack-paradigm-in-machine-unlearning  [temporal]
    Datasets and Model Architectures. CIFAR10 ... MNIST ... SVHN ... FaceScrub ...
 
12 of the 1250 rescued quotes, evenly spread through the corpus, for hand-checking:
 
  IMC/2010/measurement-of-loss-pairs-in-network-paths  [tools]  pdf verdict: run8
    To augment the path measurement with route information, tcptraceroute [49] was performed at both the sources and destinations.
  USENIX/2015/raptor-routing-attacks-on-privacy-in-tor  [temporal]  pdf verdict: run8
    612+ million BGP updates pertaining to 550,000 IP prefixes collected by six RIPE-maintained BGP Looking Glass... in January 2015.
  PETS/2017/social-engineering-attacks-on-government-opponents-target-perspectives  [statistics]  pdf verdict: run8
    We interviewed thirty subjects (randomly assigned identifiers S1-S30) over a two year period between March 2014 and March 2016.
  CCS/2019/tokenscope-automatically-detecting-inconsistent-behaviors-of-cryptocurrency-toke  [tools]  pdf verdict: run8
    To demonstrate the feasibility, we develop a tool named TokenFuzzer that integrates TokenScope with ContractFuzzer.
  PETS/2020/the-road-not-taken-re-thinking-the-feasibility-of-voice-calling-over-tor  [tools]  pdf verdict: frag
    available bandwidth and RTT using iperf and ping, respectively.
  CCS/2022/phishing-url-detection-a-network-based-approach-robust-to-evasion  [tools]  pdf verdict: exact
    We use Node2Vec [16] and DeepWalk [41].
  WWW/2022/using-web-data-to-reveal-22-year-history-of-sneaker-designs  [population]  pdf verdict: run8
    We found information for approximately 23,492 sneakers from past years.
  USENIX/2023/what-are-the-chances-explaining-the-epsilon-parameter-in-differential-privacy  [artifacts]  pdf verdict: run8
    Data and analysis code available on OSF. The full survey text is available on OSF4.
  USENIX/2024/it-doesnt-look-like-anything-to-me-using-diffusion-model-to-subvert-visual-phish  [detection]  pdf verdict: exact
    users have a slightly better performance in detecting adversarial phishing pages (TPR=0.59) than in detecting unperturbed phishing pages (TPR=0.45).
  PETS/2025/what-are-they-gonna-do-with-my-data-privacy-expectations-concerns-and-behaviors  [tools]  pdf verdict: exact
    The interviews were audio-recorded and transcribed using Whisper [73].
  NDSS/2026/side-channel-inference-of-user-activities-in-ar-vr-using-gpu-profiling  [tools]  pdf verdict: run8
    We employ LinearRegression from the Python scikit-learn library [67] and separately compute Pearson correlation coefficient.
  USENIX/2025/a-framework-for-abusability-analysis-the-case-of-passkeys-in-interpersonal-threa  [artifacts]  pdf verdict: run8
    All data pertaining to this work is available online at https://doi.org/10.5281/zenodo.14745290.

report_cols_mechanism-output.txt

=== 1. What the two-column repair did, page by page =====================
 
papers with a paper.txt on disk        5,859 of 5,859
pages                                  91,509
  reflowed (gutter found, rewritten)   62,162 (67.9%)
  rejected by the word-multiset check  0 (0.0%)
  left untouched (no gutter found)     29,347 (32.1%)
 
An untouched page is only a defect if it really was two-column. The looser
detector in audit_decolumn_pages.mjs scores each untouched page again;
audit_page_geometry.py then calibrates that score against the PDF's own
geometry, because a text-shape heuristic cannot referee a text-shape
heuristic. Distribution of the loose score over untouched pages:
 
  0.0–0.1    4410   15.0%
  0.1–0.2    2371    8.1%
  0.2–0.3    4111   14.0%
  0.3–0.4    6931   23.6%
  0.4–0.5   10527   35.9%
  0.5–0.6     957    3.3%
  0.6–0.7      25    0.1%
  0.7–0.8       6    0.0%
  0.8–0.9       6    0.0%
  0.9–1.0       3    0.0%
 
=== 2. Does the untouched-page rate predict the failed-quote rate? ======
 
"Below threshold" here is quote_check.mjs's rule — fewer than 60% of the
quote's 5-word windows present — NOT report_corpus.mjs's "not found"
bucket, which is much stricter. The two are not comparable and the numbers
below are several times larger than the 1.3% on literature:corpus for that
reason alone. What matters here is the gradient, not the level.
 
papers with >=1 evidence quote and both audits complete  5,853
 
  untouched≥0.35   papers   quotes   below thr.       rescued
  0–0%               585    12486     980    7.8%      508    4.1%
  0–10%              951    22898    2464   10.8%     1498    6.5%
  10–20%            2009    47855    7031   14.7%     4632    9.7%
  20–35%            1846    42335    7893   18.6%     5573   13.2%
  35–50%             353     7652    1826   23.9%     1324   17.3%
  50–101%            109     1734     436   25.1%      374   21.6%
 
=== 3. Is anything actually LOST, or only re-ordered? ===================
 
papers compared                        5,856
ds_ratio = de-spaced characters in the stored rendering / in the PDF
rendering. Both sides stripped to [a-z0-9], so spacing, hyphenation and
ligature differences cannot move it. 1.00 means nothing is missing.
 
  p05 1.000   p25 1.003   median 1.005   p75 1.008   p95 1.015
  papers below 0.95 (stored text materially shorter) 34 (0.6%)
  papers above 1.05 (pypdf materially shorter)       11 (0.2%)
 
Every paper below 0.95, classified. The first two classes are artefacts of
the comparison, not of the stored rendering, and both were found by reading
the outliers rather than by trusting the ratio:
 
ocr-repaired: 2 papers — textSource is not paper.cols.txt — the extractor read paper.mistral.md, so comparing paper.cols.txt to the PDF compares a file nothing read
  0.502  src   13278  pdf   26444  CCS/2013/graphical-password-using-object-based-image-ranking
  0.520  src   64652  pdf  124234  USENIX/2024/you-can-obfuscate-but-you-cannot-hide-crosspoint-attacks-against-network-topolog
 
pypdf-glyph-noise: 2 papers — pypdf emitted PostScript glyph names (/SolidCircle …) for a plot's markers, inflating its own side of the ratio
  0.689  src   67259  pdf   97628  WWW/2020/complex-factoid-question-answering-with-a-free-text-knowledge-graph
  0.785  src   83411  pdf  106197  IMC/2017/ethical-issues-in-research-using-datasets-of-illicit-origin
 
RESIDUE: 30 papers — neither excuse applies — these are the candidates for real loss and every one is listed
  0.577  src   59828  pdf  103672  WWW/2026/towards-multi-label-text-interpretation-with-chain-of-thought-prompting-and-cont
  0.643  src   58151  pdf   90451  IMC/2010/estimating-and-sampling-graphs-with-multidimensional-random-walks
  0.722  src   79007  pdf  109404  USENIX/2023/hidden-reality-caution-your-hand-gesture-inputs-in-the-immersive-virtual-world-a
  0.733  src   72467  pdf   98827  WWW/2021/cookie-swap-party-abusing-first-party-cookies-for-web-tracking
  0.769  src   55630  pdf   72354  IEEE-SP/2012/dont-trust-satellite-phones-a-security-analysis-of-two-satphone-standards
  0.778  src   63161  pdf   81209  IEEE-SP/2016/domain-z-28-registrations-later-measuring-the-exploitation-of-residual-trust-in
  0.825  src   84730  pdf  102744  IEEE-SP/2025/code-speaks-louder-exploring-security-and-privacy-relevant-regional-variations-i
  0.831  src   74611  pdf   89806  WWW/2019/characterizing-speed-and-scale-of-cryptocurrency-discussion-spread-on-reddit
  0.834  src   71004  pdf   85149  WWW/2025/roles-of-network-and-identity-in-hashtag-diffusion
  0.850  src  109545  pdf  128898  WWW/2021/an-empirical-study-of-real-world-webassembly-binaries-security-languages-use-cas
  0.880  src   65297  pdf   74205  IEEE-SP/2013/the-crossfire-attack
  0.889  src  113065  pdf  127216  NDSS/2023/no-grammar-no-problem-towards-fuzzing-the-linux-kernel-without-system-call-descriptions
  0.892  src   74698  pdf   83760  IEEE-SP/2015/controlled-channel-attacks-deterministic-side-channels-for-untrusted-operating-s
  0.893  src   60800  pdf   68076  WWW/2025/semantics-aware-cookie-purpose-compliance
  0.900  src   49372  pdf   54855  IEEE-SP/2013/on-limitations-of-friendly-jamming-for-confidentiality
  0.903  src  117365  pdf  130016  WWW/2021/chinese-wall-or-swiss-cheese-keyword-filtering-in-the-great-firewall-of-china
  0.906  src   84461  pdf   93256  NDSS/2026/bacnet-or-badnet-on-the-insecurity-of-implicitly-reserved-fields-in-bacnet
  0.915  src   59521  pdf   65030  USENIX/2013/alice-in-warningland-a-large-scale-field-study-of-browser-security-warning-effec
  0.924  src   73074  pdf   79108  CCS/2023/dont-leak-your-keys-understanding-measuring-and-exploiting-the-appsecret-leaks-i
  0.928  src  120609  pdf  129999  IEEE-SP/2021/which-privacy-and-security-attributes-most-impact-consumers-risk-perception-and
  0.930  src   68017  pdf   73126  NDSS/2026/cot-dpg-a-co-training-based-dynamic-password-guessing-method
  0.930  src   82299  pdf   88462  CCS/2023/uncovering-and-exploiting-hidden-apis-in-mobile-super-apps
  0.940  src   46634  pdf   49597  WWW/2013/is-this-app-safe-for-children-a-comparison-study-of-maturity-ratings-on-android
  0.942  src   62999  pdf   66869  WWW/2019/dealing-with-interdependencies-and-uncertainty-in-multi-channel-advertising-camp
  0.943  src   84759  pdf   89901  IEEE-SP/2018/sok-keylogging-side-channels
  0.943  src   86269  pdf   91478  CCS/2024/inbox-invasion-exploiting-mime-ambiguities-to-evade-email-attachment-detectors
  0.943  src   65928  pdf   69889  IEEE-SP/2013/cookieless-monster-exploring-the-ecosystem-of-web-based-device-fingerprinting
  0.946  src   35712  pdf   37752  IMC/2017/shortcuts-through-colocation-facilities
  0.947  src   47809  pdf   50486  WWW/2026/webgeoinfer-structure-free-multi-stage-framework-for-geolocation-inference-from
  0.949  src   95554  pdf  100672  USENIX/2025/malicious-llm-based-conversational-ai-makes-users-reveal-personal-information
 
records with no paper.cols.txt at all: 0
 
records the PDF rendering could not be produced for: 1
  pypdf-error  NDSS/2025/mtzk-testing-and-exploring-bugs-in-zero-knowledge-zk-compilers  DependencyError: cryptography>=3.1 is required for AES algorithm
 
=== 4. How many untouched pages were really two-column? ================
 
audit_page_geometry.py classifies a stratified sample of pages from the
PDF's own text-positioning operators — the x histogram of run starts —
which shares no evidence with either text-shape heuristic. Pages that
decolumn.mjs DID reflow are the control: if the geometry detector does not
call those two-column, nothing below it means anything.
 
  band        n   2col  single  thin  n/a    share   95% CI
  0.0–0.1    55      7      27    21    5    12.7%  6–24%
  0.1–0.2    59     41      15     3    1    69.5%  57–80%
  0.2–0.3    60     48      10     2    0    80.0%  68–88%
  0.3–0.4    60     53       6     1    0    88.3%  78–94%
  0.4–0.5    60     57       3     0    0    95.0%  86–98%
  0.5–0.6    60     57       3     0    0    95.0%  86–98%
  0.6–1.0    40     19      17     4    0    47.5%  33–63%
  reflowed   60     60       0     0    0   100.0%  94–100%
 
untouched pages covered by the calibration      29,347 of 29,347
estimated genuinely two-column, left unrepaired 22,475 pages
  95% CI from the band-wise Wilson intervals    19,701–24,302
as a share of all 91,509 pages in the corpus        24.6% (21.5%–26.6%)
 
That is the size of the defect: pages whose two columns were never
separated, so every line splices the left column to the right, and no
quote taken from them can be located in the stored rendering.

audit_binary_detect-output.txt

=== Renderings a shell grep treats as binary ============================
 
  paper.txt          5859 files,   90 with a NUL byte (1.5%)
  paper.norm.txt     5859 files,   90 with a NUL byte (1.5%)
  paper.cols.txt     5855 files,   90 with a NUL byte (1.5%)
 
papers whose EXTRACTOR SOURCE file carries a NUL   90 of 5859 (1.5%)
NUL bytes in those files, total                    74,772
evidence quotes sitting in those papers            2,156 of 135,025 (1.6%)
 
Those quotes are not wrong. They are the quotes for which a `grep` check in
this container returns a silent false negative, and therefore the quotes a
by-hand check is most likely to misreport as fabricated.
 
by venue: NDSS 35, WWW 20, CCS 15, IEEE-SP 10, USENIX 5, IMC 4, PETS 1
by year:  2012 1, 2020 8, 2021 2, 2022 6, 2023 13, 2024 20, 2025 24, 2026 16
 
worst 10 by NUL count:
   13643 NUL  21 quotes  CCS/2025/training-robust-classifiers-for-classifying-encrypted-traffic-under-dynamic-netw
    4892 NUL  33 quotes  NDSS/2023/detecting-unknown-encrypted-malicious-traffic-in-real-time-via-flow-interaction-graph-analysis
    4630 NUL  34 quotes  CCS/2025/training-with-only-1-0-samples-malicious-traffic-detection-via-cross-modality-fe
    3177 NUL  23 quotes  WWW/2024/exploring-unconfirmed-transactions-for-effective-bitcoin-address-clustering
    2913 NUL  14 quotes  IEEE-SP/2026/the-battle-of-metasurfaces-understanding-security-in-smart-radio-environments
    2658 NUL  16 quotes  CCS/2023/devil-in-disguise-breaching-graph-neural-networks-privacy-through-infiltration
    2180 NUL  25 quotes  NDSS/2025/clibe-detecting-dynamic-backdoors-in-transformer-based-nlp-models
    1977 NUL  25 quotes  IEEE-SP/2023/mmecho-a-mmwave-based-acoustic-eavesdropping-method
    1927 NUL  17 quotes  NDSS/2026/time-will-tell-large-scale-de-anonymization-of-hidden-i2p-services-via-live-behavior-alignment
    1785 NUL  19 quotes  USENIX/2022/neither-access-nor-control-a-longitudinal-investigation-of-the-efficacy-of-user