Working log behind Deployment: running a measurement for weeks. Every figure on that page, the query that produced it, its denominator, the hand maps and their verdicts, the quotes checked, the external sources verified and rejected, what could not be established, and the review. Corpus-wide caveats — the venue scope, the selection funnel, the stability of each field — are on corpus and are not restated here.

Voice is a working log, not prose. It is for someone checking a number.

Create, extend or overlap? Create. Research journey already listed “Deployment issues / Executing long experiments / Monitoring experiments” as its own execution step with no page behind it, and no existing page covered it. The nearest neighbours were checked and none was a better host:

Every figure on the page uses one of these. None of them is 5859.

All 1120 crawled papers have a readable paper.cols.txt. No paper was dropped from a probe for a missing file.

One denominator on the page is a complement, not a sentinel row, and the difference is 44 papers. vantage[].infrastructure is multi-valued: the not-stated row covers 688 papers (61.4%), but 44 of those also carry a stated value in another tuple. The number the page needs for “says nothing” is the complement of the 445 that state something — 675 of 1,120 (60.3%). The first draft published 61.4%. report_deployment.mjs now prints all three figures and the overlap beside them.

temporal[].mode == “live-crawl” does not mean the span is wall-clock time. Read against the tuples' own evidence quotes, the 230 day-precision spans split four ways, and only campaign answers “how long was the apparatus running”:

• campaign — a crawler, scraper or collector was running between those dates, continuously or on a cadence.
• content — the span is the date range of the collected content. The apparatus may have run for an afternoon. The longest “crawl” in the corpus is 9,028 days and its sentence is “90,153 disputes decided between December 1999 and August 2024”.
• separated — two or more disjoint episodes with the gap counted in (“two periods, 26 August 2018 to 9 November 2018, and 20 April 2019 to 14 May 2019”).
• retrieval — a list-fetch or “as of” date. One is a bibliography access date (“Accessed: 2026-3-19”) that the extraction read as a crawl span.

Including all 230 would move the median from 30 days to 40 days and put four multi-year “crawls” in the top ten that were never crawls. Every excluded span is listed below with the sentence that decided it — the whole point of publishing the map is that the reading is checkable.

The rule was written before the counts: the span counts if the tuple's own quote says an apparatus was running between those dates. Borderline calls, and how they went:

A reasonable person could put the Strava and PropertyRoom rows the other way. Both are excluded, which makes the published distribution slightly shorter than the most generous reading. That is the conservative direction for a page arguing that long runs are normal — but it does not make the whole procedure conservative, because a second rule pushes the other way: where a paper has several day-precision spans the report keeps the longest, so every figure describes the longest run each paper reports, not a typical one. The page now says so in its population paragraph. The two rules were not chosen to cancel and there is no reason to think they do.

Full-text sweeps read data/fulltext/<year>/<venue>/<slug>/paper.cols.txt with whitespace collapsed (.replace(/\s+/g, “ ”)), because a phrase that spans a column break does not match otherwise. Paper-counted, never tuple-counted. Every probe rate on the page is an upper bound: it counts papers that used a word.

Probe width is load-bearing and the page prints both widths for the one that moves most: the bare word “retry” matches 45 papers, “retry” within 160 characters of a crawl noun matches 21. Neither is a practice rate.

Probes run and rejected:

The cost probe was widened after reading, and that is the most important correction in this run. The first version required a crawl-ish subject near the cost word and returned 8 candidates. Reading Englehardt and Narayanan [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] for a different figure turned up “This virtual machine costs around $300 per month using price estimates from May 2016” — the single most-cited crawl-infrastructure sentence in this corpus, and the probe did not see it, because “virtual machine” was not in the subject list. Adding virtual machine, VM, server and instance took the candidate set from 8 to 19 and the own-infrastructure count from 2 to 5. Had the page shipped on the first regex it would have said “2 of 1120 papers state an infrastructure bill” and been wrong by more than half. The lesson is not “widen your regexes”; it is that a fold you have not read the residue of is not finished.

Four hand classifications, all in scripts/dep_fold.mjs. Three of them are checked against their candidate sweep at run time in both directions — a sweep hit with no verdict and a verdict for a paper the sweep does not return both make report_deployment.mjs exit 1. The span map is checked one way (a named slug must be in the span population); a span paper with no entry defaults to campaign by design, because campaign is the majority class and the map lists the exceptions.

Two verdicts are worth defending because a reasonable person would differ:

• lend-me-your-ear-passive-remote-physical-side-channels-on-pcs → own-infrastructure. Its “$1.3” is the authors' own EC2 bill, but for a key-extraction evaluation, not a crawl. It is counted because the question the page asks is “does the paper say what its own compute cost”, and it does. It is not quoted in the page's cost table, which lists only the five and describes them as infrastructure bills — this one is the least crawl-like of the five and a reader following the citation would be surprised. Flagged rather than hidden.
• digging-into-browser-based-crypto-mining → lost-days. The whole disclosure is a figure legend, “Black parts mark outages of our infrastructure.” That is a complete and honest disclosure and it is also five words in a caption. It counts.

The crawl-numeric display in the report prefers a numeric match over the candidate match, because the candidate regex also matches the number-free phrasing: Acar et al. [2Acar, Gunes; Eubank, Christian; Englehardt, Steven; Juarez, Marc; Narayanan, Arvind; Díaz, Claudia (2014): "The Web Never Forgets: Persistent Tracking Mechanisms in the Wild", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)] say “multiple Firefox instances in parallel” fifteen lines before they say “up to 30 browsers in parallel on a 4-core 8GB desktop machine”, and the first draft of the report printed the wrong sentence beside the verdict. One paper in the crawl-numeric class has its number split by a column break in paper.cols.txt — before-and-after-gdpr, “four vanilla Mozilla Firefox … browsers in parallel” — and it is not quoted on the page. An earlier draft of this log said the same of the-web-never-forgets; a reviewer checked and it is wrong twice over. That sentence matches contiguously once whitespace is collapsed, and it is quoted on the page, as the Acar et al. row of the concurrency table and as needle 55 of the verifier.

/\b(successfully (crawl|visit|load|reach|retriev|scrap|analy)[a-z]*|...)\b[^.]{0,110}?(number or percent)/i returns 75 of 1120 papers (6.7%). A reproducible every-6th sample of 12 was read by hand: 6 report crawl attrition and 6 do not — they report a fuzzer's sink coverage, a mini-program analyser's success rate, how many times each domain was visited, a pull-request retrieval count. Precision is about 50%, so the honest statement is “of the order of 38 papers”, and the page publishes no rate at all from this probe: it publishes the seven papers whose attrition figures were read and verified individually. The hand verdicts are in report_deployment.mjs section D2 and the report exits 1 if the sample composition changes under it.

scripts/verify_deployment_figures.mjs checks 52 literal per-paper figures and quoted sentences against the paper each is attributed to, with whitespace collapsed on both sides, and flags any needle under 20 characters as capable of matching by accident. Result: 52 of 52 found verbatim, 0 not found, 0 flagged weak. Its full output is below.

One figure is verified by a different route, and the verifier really runs it. Senol et al.'s [3Senol, Asuman; Acar, Gunes; Humbert, Mathias; Zuiderveen Borgesius, Frederik (2022): "Leaky Forms: A Study of Email and Password Exfiltration Before Form Submission", in: 31st USENIX Security Symposium (USENIX Security 22). (Link)] “increased the successfully visited websites from 94,427 (EU pilot crawl) to 99,380 (EU final crawl)” appears in paper.cols.txt as 94, 427 and 99, 380 — the .cols rendering inserts a space after the thousands comma. Re-extracting paper.pdf with pypdf gives the numbers with no space, which is what the page quotes; checking that quote against .cols alone would have scored a faithful quote as a fabrication. The first version of the verifier only printed the recipe for that check, which a reviewer pointed out verifies nothing on a re-run. It now spawns pypdf and fails the script if the needle is absent. Making it real immediately found a second problem: the PDF sets “final” with an fi ligature, so the de-ligatured form the page quotes did not match until the check normalised U+FB00–U+FB04.

Two denominators were softened after checking. The page first wrote Annamalai et al.'s [4Annamalai, Meenatchi Sundaram Muthu Selva; Bilogrevic, Igor; Cristofaro, Emiliano De (2024): "FP-Fed: Privacy-Preserving Federated Detection of Browser Fingerprinting", in: Proceedings of the Network and Distributed System Security Symposium. (Link)] population as “20,000 sites”; the paper says “the 20k websites”, so the cell now quotes the paper. It first wrote Demir et al.'s [5Demir, Nurullah; Urban, Tobias; Pohlmann, Norbert; Wressnegger, Christian (2024): "A Large-Scale Study of Cookie Banner Interaction Tools and their Impact on Users' Privacy", in: Proceedings on Privacy Enhancing Technologies, pp. 5-20. (DOI)] as “~30,000 sites”; the paper gives 29,660 (99%) but no total in that sentence, so the cell now says “a Tranco sample” and the inferred total is not published.

Rejected external sources.

pages/crawl_queue.py is stdlib-only Python (sqlite3), talks to no network, and is published on the page in a <file> block with its real --demo output beside it. Round-tripped after publication: the block was downloaded from /_export/code/programming:deployment?codeblock=0, diffed line-by-line against the file (identical, 388 lines), executed, and its output diffed line-by-line against the published output block (identical).

The demo is a self-test, not a demonstration. It runs the same 200-URL fixture twice — once clean, once killed on the 87th visit and resumed — and asserts that the two runs end with the same successful set and the same failure set. It also asserts the two reclaim paths behave differently: reclaim_expired() returns 0 because the 300-second lease has not run out, and reclaim_worker(“w1”) returns 1. That asymmetry is the thing the code exists to teach, so it is asserted rather than described.

The first version of the demo failed four of its own checks, because run() called reclaim_expired() and nothing else: the stranded row stayed stranded and the resumed run ended with 139 done against 140. Rather than shorten the lease to make the demo pass — which would have published a queue that steals live workers' rows — the row was given a worker id and reclaim_worker() was added. The failing check found a real design hole.

• How often crawls actually break. Everything on this page measures how often papers say something. 10 of 1120 papers disclose losing time; the true rate is certainly far higher and this corpus cannot bound it. The page says so.
• How much disk a crawl needs. The obvious probe (a TB/GB figure anywhere in the text) is 31.8% of crawled papers and almost none of those figures are the crawl's own output size. Answering this needs a hand read of a few hundred papers, or a different data source. Not attempted.
• Whether retry policies changed over time. The retry probe returns 21 papers under the narrow width. That is too few to cut by year, and the page does not try.
• What a crawl costs today, from the literature. Five papers, spread over 2010–2026, on four different pricing models. The page gives the arithmetic instead and labels the one live price as a single fetch that will be wrong soon.
• Sub-day durations. 27 campaigns are recorded as one day. Some of those really took hours — Bui et al. [6Bui, Duc; Tang, Brian; Shin, Kang G. (2023): "Detection of Inconsistencies in Privacy Practices of Browser Extensions", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] say their collection “took 18 hours to complete” — but temporal has no sub-day resolution, so a two-hour crawl and a twenty-hour crawl are the same row.
• A pre-existing bibliography duplicate, noticed and left alone. bibliography contains the same paper (Bouhoula et al., Automated Large-Scale Analysis of Cookie Notice Compliance [7Bouhoula, Ahmed; Kubicek, Karel; Zac, Amit; Cotrini, Carlos; Basin, David (2024): "Automated Large-Scale Analysis of Cookie Notice Compliance", in: Proceedings of the USENIX Security Symposium. (Link)]) under two keys, bouhoula2024automated and bouhoula2024_automated. It predates this run. This page cites bouhoula2024_automated. Merging them would silently break whichever other pages cite the other spelling, so nothing was changed; it is recorded here so the next run does not rediscover it.

1. A hand map over 230 spans rather than a headline from the raw field. The raw day-precision span distribution has a median of 40 days and a 9,028-day maximum, and would have been publishable-looking and wrong. Reading 230 evidence quotes was the largest single cost in this run and it is what the duration section rests on.
2. campaign includes repeated waves inside one window. A crawler that runs every ten minutes for 25 months is one campaign, not 100,000 crawls, because the page is about the process surviving. That decision moves the distribution long. longitudinal is where the wave-comparability question belongs and the page delegates to it.
3. No year-by-year trend anywhere. The corpus reaches 2026 and 2025–2026 are provisional by construction. Every year-sliced version of these figures was left out rather than starred, because none of them was load-bearing.
4. The monitoring section cites almost nothing. 16 of 1120 papers name a monitoring stack, so there is no literature to systematise. The section is written as a list of counters derived from the failure modes the ten lost-days papers describe, and it says in its first sentence that there is no literature behind it. The alternative — omitting it — would have left the page unable to answer its own question.
5. One anchor price, not a survey. See the rejected sources above.

cd /workspace/artifacts/wiki
node scripts/report_deployment.mjs                 # every figure with its denominator
node scripts/report_deployment.mjs --wiki          # the same as DokuWiki tables
node scripts/report_deployment.mjs --list campaign # the 182 campaign papers
node scripts/report_deployment.mjs --quotes '<regex>'
node scripts/verify_deployment_figures.mjs         # per-paper figures against the papers
python3 pages/crawl_queue.py --demo                # the published code's self-test
python3 scripts/ec2_price.py c7i.2xlarge c7i.4xlarge

report_deployment.mjs exits 1 if the corpus contract has moved (not 7 venues) or if any hand map and its sweep diverge in either direction. It exited 0 for the run below.

report_deployment-output.txt

=== A. POPULATION AND CORPUS CONTRACT ===
 
Quantity                     Value                                     
venues                       CCS, IEEE-SP, IMC, NDSS, PETS, USENIX, WWW
years                        2010-2026                                 
all papers                   5859                                      
crawled (page denominator)   1120                                      
crawled with readable .cols  1120                                      
 
=== B. HOW LONG A CRAWL ACTUALLY RUNS ===
 
Population: crawled papers with at least one temporal[] tuple whose mode is
live-crawl and whose spanStart AND spanEnd are both day-precision (YYYY-MM-DD).
Duration is inclusive of both endpoints. Per paper we keep the LONGEST such span.
 
Quantity                                    Papers / tuples  Share of 1,120 crawled
live-crawl tuples, both ends day-precision  298              —                     
... month-precision only (excluded)         267              —                     
... year-precision only (excluded)          22               —                     
... non-positive duration (excluded)        0                —                     
papers with a day-precision span            230              20.5%                 
 
Span role  Papers  Share of 230
campaign   182     79.1%       
content    25      10.9%       
separated  16      7.0%        
retrieval  7       3.0%        
 
Campaign spans, N = 182 papers.
Statistic        Days
minimum          1   
25th percentile  8   
median           30  
75th percentile  98  
90th percentile  239 
maximum          1254
 
Counterfactual: if the hand map were skipped and all 230 day-precision spans
were treated as crawl duration, the median would be 40 days and the maximum 9028.
 
Campaign length     Papers  Share of 182
1 day               27      14.8%       
2-7 days            17      9.3%        
8-30 days           48      26.4%       
31-90 days          37      20.3%       
91-365 days         45      24.7%       
more than 365 days  8       4.4%        
 
longer than 7 days:  138 of 182 (75.8%)
longer than 30 days: 90 of 182 (49.5%)
longer than 90 days: 53 of 182 (29.1%)
 
Ten longest campaign spans:
Days  Paper                                                                                         Span                  
1254  CCS/2014/a-nearly-four-year-longitudinal-study-of-search-engine-poisoning                     2010-04-12..2013-09-16
931   WWW/2025/whats-in-phishers-a-longitudinal-study-of-security-configurations-in-phishing-we     2021-07-15..2024-01-31
752   WWW/2024/phishing-vs-legit-comparative-analysis-of-client-side-resources-of-phishing-and      2021-07-10..2023-07-31
648   WWW/2019/hack-for-hire-exploring-the-emerging-market-for-account-hijacking                    2017-01-01..2018-10-10
488   IMC/2025/unmasking-the-shadow-economy-a-deep-dive-into-drainer-as-a-service-phishing-on-e     2023-12-01..2025-04-01
486   WWW/2021/a-longitudinal-study-of-removed-apps-in-ios-app-store                                2019-01-01..2020-04-30
447   IMC/2021/who-you-gonna-call-an-empirical-evaluation-of-website-security-txt-deployment        2020-01-29..2021-04-19
442   IMC/2019/a-longitudinal-analysis-of-the-ads-txt-standard                                      2018-01-15..2019-04-01
365   USENIX/2016/investigating-commercial-pay-per-install-and-the-distribution-of-unwanted-softwa  2015-01-08..2016-01-07
346   USENIX/2024/unpacking-privacy-labels-a-measurement-and-developer-perspective-on-googles-data  2022-06-20..2023-05-31
 
Every excluded span, with the quote that decided it:
 
-- content --
 9028 NDSS/2026/repairing-trust-in-domain-name-disputes-practices-insights-from-a-quarter-centurys-worth-of-squabbles
      Specifically, drawing on data from 90,153 disputes decided between December 1999 and August 2024
 5198 USENIX/2024/moderating-illicit-online-image-promotion-for-unsafe-user-generated-content-game
      These stories span from July 9, 2009, to October 1, 2023.
 3991 WWW/2023/misbehavior-and-account-suspension-in-an-online-financial-communication-platform
      The articles collected were published between Sep. 5th, 2011, and Aug. 8th, 2022.
 2618 USENIX/2011/dirty-jobs-the-role-of-freelance-labor-in-web-service-abuse
      Starting with the earliest project posted on February 5, 2004 at 12:28 EST, we collected data through April 6th, 2011
 2270 WWW/2022/identification-of-disease-or-symptom-terms-in-reddit-to-improve-health-mention-c
      Using the PRAW API, we retrieved all available posts from these subreddits between January 1, 2015, and March 19, 2021.
 1461 CCS/2022/clues-in-tweets-twitter-guided-discovery-and-analysis-of-sms-spam
      In total, the collector found 40,998 tweets with 50,545 image attachments between Jan 1, 2018 and Dec 31, 2021.
 1145 NDSS/2022/auto-draft-246
      Focusing on "Fixed bugs", we crawl the entire queue from Syzbot dashboard. Our dataset summarized in Table I covers all the corresponding crash report
  731 IMC/2022/is-my-internet-down-sifting-through-user-affecting-outages-with-google-trends
      Time range: 01 Jan 2020 - 31 Dec 2021
  726 IEEE-SP/2023/blue-is-the-new-black-market-privacy-leaks-and-re-victimization-from-police-auct
      We crawled the PropertyRoom website to obtain information about each auction spanning approximately two years: March 5, 2020 [32] to February 28, 2022
  486 IEEE-SP/2025/hey-your-secrets-leaked-detecting-and-characterizing-secret-leakage-in-the-wild
      we collected packages from January 1, 2023, to April 30, 2024
  438 WWW/2013/google-or-google-dissecting-the-evolution-of-the-new-osn-in-its-first-year
      collect all the public posts and their associated reactions for all LCC-Sep users between G+ release date (Jun 28th 2011) and the date our measurement
  382 WWW/2013/real-time-recommendation-of-diverse-related-articles
      We crawled 13K news articles published by Reuters between December 4th , 2010 and December 20th , 2011.
  366 CCS/2025/quantifying-security-training-in-organizations-through-the-analysis-of-u-s-sec-1
      our 2024 dataset includes all 10-K filings submitted between January 1st and December 31st, 2024.
  365 USENIX/2025/assessing-the-aftermath-the-effects-of-a-global-takedown-against-ddos-for-hire-s
      We collected 34 438 messages ... in 52 channels ... from 1 October 2022 to 30 September 2023.
  349 WWW/2025/causal-insights-into-parlers-content-moderation-shift-effects-on-toxicity-and-fa
      Our dataset consists of parleys from February 1st, 2021 to January 15th, 2022.
  206 IEEE-SP/2021/a-first-look-at-zoombombing
      We developed a custom crawler ... and collected all posts between January 1st, 2020, and July 24th, 2020.
  198 IMC/2023/ethereums-proposer-builder-separation-promises-and-realities
      our data ranges from block 15,537,394 to block 16,950,602, i.e., from the first block after the merge on 15 September 2022 to the last block on 31 Mar
  184 WWW/2018/tagvisor-a-privacy-advisor-for-sharing-hashtags
      we collect its publicly available posts' metadata (user ID, time and hashtags) in the second half of 2015 (2015.7.1-2015.12.31).
  181 WWW/2024/getting-bored-of-cyberwar-exploring-the-role-of-low-level-cybercrime-actors-in-t
      We use several quantitative datasets collected regularly and separately, spanning 1 January to 30 June 2022
  170 NDSS/2026/beyond-jailbreak-unveiling-risks-in-llm-applications-arising-from-blurred-capability-boundaries
      GPTs (until Apr 13, 2024), ... Coze (until Sep 29, 2024), ... AgentBuilder (until Sep 29, 2024), and Poe (until Sep 16, 2024).
  166 IEEE-SP/2014/hunting-the-red-fox-online-understanding-and-detection-of-mass-redirect-script-i
      our data set shows that the RedKit network had been there at least from August 4th, 2012 to Jan 16th, 2013
  123 IEEE-SP/2019/stealthy-porn-understanding-real-world-adversarial-images-for-illicit-online-pro
      we totally obtain 3,813,888 unique images from 648,621 posts on Tieba from 03/15/2018 to 07/15/2018
   31 USENIX/2018/analysis-of-privacy-protections-in-fitness-tracking-social-networks-or-you-can-r
      Using the above methodology, we collected a month worth of Strava activities beginning on May 1, 2016.
   31 WWW/2024/navigating-the-post-api-dilemma
      we repeated each query setting the date restriction for one day-at-a-time for each day in January 2023 thereby matching the timeframe from Reddit data
   20 WWW/2022/screenshots-symbols-and-personal-thoughts-the-role-of-instagram-for-social-activ
      we collect and study Instagram data generated between June 1 and June 20 2019, but focus on two significant protests occurring on June 9 and June 16.
 
-- separated --
  759 NDSS/2025/the-unusual-suspects-studying-reasons-for-lacking-updates-in-wordpress
      First, we conducted a new crawl of outdated WP-based sites, using the same method as described for Sampling Round 1 (May 9-19, 2023).
  365 WWW/2018/exposing-search-and-advertisement-abuse-tactics-and-infrastructure-of-technical
      the results discussed in this section are based on data that was collected over a period of 8 months in two distinct time windows, April 1 to August 3
  336 IMC/2019/an-empirical-study-of-the-cost-of-dns-over-https
      We initially retrieved all information in this section on 10 October 2018. We then verified it and, where necessary, updated entries in both tables ag
  244 WWW/2019/a-large-scale-study-on-the-risks-of-the-html5-webapi-for-mobile-sensor-based-att
      Our crawling experiments took place between 03/24/2018-09/03/2018 and 11/11/2018-11/22/2018
  225 USENIX/2022/experimental-security-analysis-of-the-app-model-in-business-collaboration-platfo
      We collected 2,460 apps from the Slack [11] on April 7, 2021 and 1,304 apps from Microsoft Teams [13] on November 17, 2021.
  164 WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion
      We collected ads once per day through May 9, 2022. Starting on June 24, 2022, we collected ads on a weekly basis. In this paper, we report on data col
  155 WWW/2020/towards-ip-based-geolocation-via-fine-grained-and-stable-webcam-landmarks
      We illustrate their dynamic changes in Figure 5 using the datasets collected on May 11, 2019 and October 5, 2019, as well as the most recent datasets 
   92 CCS/2016/cspautogen-black-box-enforcement-of-content-security-policy-upon-real-world-webs
      we train templates based on the contents of each website on 01/01/2016. Then we use these templates to match the same website captured on 02/01/2016, 
   76 NDSS/2020/encrypted-dns-privacy-a-traffic-analysis-perspective
      We collected data during two periods, from 26 August 2018 to 9 November 2018, and from 20 April 2019 to 14 May 2019.
   49 PETS/2026/overcoming-language-barriers-multilingual-analysis-of-the-2023-swiss-privacy-law
      Each website was scraped once one week before the entry into force ... (August 23-28, 2023) and once one month thereafter (October 4-10, 2023).
   47 USENIX/2025/when-llms-go-online-the-emerging-threat-of-web-enabled-llms
      We utilize the latest models as of July 17, 2024. Our attacks were last confirmed to be valid on September 1, 2024.
   41 IEEE-SP/2022/towards-automated-auditing-for-account-and-session-management-flaws-in-single-si
      We tracked changes in Single Sign-On support across two rounds of data collection that were 50 days apart.
   31 WWW/2025/assessing-compliance-in-digital-advertising-a-deep-dive-into-acceptable-ads-stan
      Table 3: Counts of non-compliant ads using CacceptableAds at two distinct time stamps.
   11 PETS/2024/a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users
      The three locations that we used for our repeated control experiments ... 04/21/23 04/24/23 ... 04/28/23 05/01/23
    9 USENIX/2022/the-security-lottery-measuring-client-side-web-security-inconsistencies
      we ran three more confirmation crawls (January 6, 10, and 14, 2022).
    6 IMC/2018/how-tracking-companies-circumvented-ad-blockers-using-websockets
      Two crawls were performed just prior to the release of Chrome 58 ... between April 2-5 and April 11-16, 2017.
 
-- retrieval --
    1 USENIX/2018/o-single-sign-off-where-art-thou-an-empirical-analysis-of-single-sign-on-account
      the top 1M websites according to Alexa (as reported on September 14, 2017)
    1 IEEE-SP/2019/dangerous-skills-understanding-and-mitigating-security-risks-of-voice-controlled
      Up to November 11th, 2017, we gathered 23,758 skills
    1 IMC/2020/hiding-in-plain-site-detecting-javascript-obfuscation-through-concealed-browser
      http://s3.amazonaws.com/alexa-static/top-1m.csv.zip - as retrieved on Sep 24, 2019
    1 WWW/2020/apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web
      The domain lists were fetched on 25t h April 2019.
    1 IEEE-SP/2026/parasites-in-the-toolchain-a-large-scale-analysis-of-attacks-on-the-mcp-ecosyste
      Accessed: 2026-3-19.
    1 PETS/2026/the-masks-we-think-we-wear-privacy-threats-of-browser-extension-wallets-in-the-w
      we collected the 30 most popular Ethereum dApps from DappRadar's Ethereum category [9] as of 14 November 2025.
    1 PETS/2026/from-syntactic-matching-to-taint-tracking-and-back-a-comparative-study-of-web-tr
      the Tranco Top 10k [31] generated on February 8, 2026.
 
=== C. WHERE THE MEASUREMENT RUNS ===
 
Population: 1120 crawled papers. vantage[].infrastructure is an enum;
a paper with no vantage[] tuple at all is its own row, not folded into not-stated.
 
vantage[].infrastructure   Papers  Share of 1120
not-stated                 688     61.4%        
university-network         134     12.0%        
cloud-provider             123     11.0%        
research-testbed           122     10.9%        
commercial-vpn             48      4.3%         
volunteer-devices          29      2.6%         
residential                19      1.7%         
proxy-service              18      1.6%         
tor                        15      1.3%         
mobile-network             9       0.8%         
(no vantage tuple at all)  31      2.8%         
 
States some non-sentinel infrastructure: 445 of 1120 (39.7%)
Names NOTHING (the complement, and the right figure for "says nothing"): 675 of 1120 (60.3%)
The not-stated ROW is larger (688) because vantage[] is multi-valued:
44 papers have both a not-stated tuple and a stated one.
Subtracting a sentinel row from the population is the wrong way to get this.
 
=== D. WHAT THE PAPERS SAY ABOUT OPERATING THE RUN ===
 
Full-text sweeps over paper.cols.txt, whitespace collapsed, paper-counted.
Population: 1120 crawled papers, 1120 readable.
Every row is an UPPER BOUND on the practice: a match is a mention, not a use.
 
Probe (full text, crawled = 1120)      Papers  Share
checkpoint / resume a crawl            13      1.2% 
a task or job queue by name            18      1.6% 
cluster orchestration by name          21      1.9% 
a monitoring or logging stack by name  16      1.4% 
a supervisor for a long process        8       0.7% 
retrying a failed page or request      21      1.9% 
a page-load timeout                    226     20.2%
pages or sites that failed to load     134     12.0%
 
Queue probe, with "redis" in the alternation: 34. Without: 18.
16 papers match ONLY on "redis", which in this corpus is a data store,
a vulnerability target and a studied software corpus far more often than a work queue.
The published figure is the narrower one, for the same reason Elasticsearch was
dropped from the monitoring probe. --list queue:redis-only prints the excluded papers.
 
Probe width is load-bearing. The same subject, two widths:
 
Probe                                     Papers  Share of 1120
the word "retry" anywhere                 45      4.0%         
"retry" within 160 chars of a crawl noun  21      1.9%         
 
=== D2. HOW MUCH OF THE LIST YOU ACTUALLY GET ===
 
Probe: 75 of 1120 crawled papers (6.7%).
This is an UPPER BOUND and a weak one. A reproducible every-6th sample of the
hits was read by hand; the precision for "reports how much of its own list it
reached" is printed below. Do not publish the 6.7% as a reporting rate.
 
Hand-read sample: 6 of 12 report crawl attrition (50.0% precision).
Implied order of magnitude: about 38 of 1120 crawled papers.
 
=== E. PAPERS THAT SAY THEIR OWN RUN LOST TIME ===
 
Candidate sweep: 21 of 1120 crawled papers (1.9%).
Verdict      Papers  Share of 21 hits  Share of 1120 crawled
lost-days    10      47.6%             0.9%                 
offtopic     6       28.6%             0.5%                 
target-down  3       14.3%             0.3%                 
planned      1       4.8%              0.1%                 
no-downtime  1       4.8%              0.1%                 
 
The lost-days papers and the sentence that says so:
 
WWW/2013/two-years-of-short-urls-internet-measurement-security-threats-and-countermeasure
      ...because of dynamic IP-assignment policies employed by ISPs. Our system experienced three outages due to database failures throughout one year: in late December 2010, in late January 2011, and between late February and March 2011. Nevertheless, we col...
 
CCS/2014/a-nearly-four-year-longitudinal-study-of-search-engine-poisoning
      ...d to glitches with the measurement equipment (e.g. power or network outage), or upgrades to the measurement infrastructure. Out of the 1 254 days in the measurement period, we have complete measurements for 1 004 days. Combining the datasets Since, i...
 
USENIX/2015/vulnerability-disclosure-in-the-age-of-social-media-exploiting-twitter-for-predi
      ...se (as the API signals rate limiting), we conclude that our collection contains all references to CVEs, except during the periods of downtime for our infrastructure. We collect data over a period of one year, from February 2014 to January 2015. Out o...
 
IMC/2016/anatomy-of-a-personalized-livestreaming-system
      ...est were from anonymous views on the Periscope webpage. Our crawler was interrupted briefly between August 7-9, 2015 due to a communication bug with the Periscope server. As a result, our dataset is missing roughly 4.5% of the broadcasts during this ...
 
IMC/2018/digging-into-browser-based-crypto-mining
      ...ain utilize the RuleSpace categories work. Black parts mark outages of our infrastructure. to manually classify those 10 domains. As the table shows, most links point to streaming and filesharing services. Top Categories. We employ the RuleSpace engi...
 
IEEE-SP/2018/the-spyware-used-in-intimate-partner-violence
      ...d a mechanism for pruning apps that are not IPS-relevant. 3 Scans were not performed on Nov 07 and Nov 08 due to a power failure. 445 Authorized licensed use limited to: ETH BIBLIOTHEK ZURICH. Downloaded on August 10,2026 at 10:32:19 UTC from IEEE Xp...
 
IMC/2021/who-you-gonna-call-an-empirical-evaluation-of-website-security-txt-deployment
      ...from January 29, 2020 to April 19, 2021 (we had a 1.5-month measurement interruption throughout November and the first half of December, 2020). For each measurement instance, we first downloaded the latest Alexa top list snapshot, and crawled only th...
 
WWW/2021/tls-1-3-in-practice-how-tls-1-3-contributes-to-the-internet
      ...od, around 84% of the websites were consistently collected. There were network outages for 17 days, which are pruned out from the dataset. 4 Handshake Messages (D2). To analyze the TLS 1.3 features supported in the TLS 1.3 web servers (399K on Dec. 3...
 
USENIX/2023/timeless-timing-attacks-and-preload-defenses-in-tors-dns-cache
      ...ur ically triggers multiple domain lookups [13] that affect our exits experienced downtime, visible as dips in lookups in both global counters. A collection interval of 15 minutes should figures (at times fewer than 1,000 lookups, as noted in Secthus...
 
WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion
      ...ple, we found a cluster with war and veterans as keywords 1 Our crawler crashed on March 12th in both Ukraine and Russia, and again on March 20th and April 18th in Ukraine after running for some time. 2 Before topic modeling, we pre-processed the tex...
 
=== F. HOW MANY WORKERS AT ONCE ===
 
Candidate sweep: 44 of 1120 crawled papers (3.9%).
Verdict        Papers  Share of 44 hits  Share of 1120 crawled
crawl-numeric  29      65.9%             2.6%                 
offtopic       6       13.6%             0.5%                 
analysis       5       11.4%             0.4%                 
crawl-vague    4       9.1%              0.4%                 
 
Every crawl-numeric paper with its matched phrase. The candidate regex also
matches the number-free phrasing, so display prefers a numeric match where the
paper has one (CCS/2014 says "multiple Firefox instances" before it says 30).
 
CCS/2014/the-web-never-forgets-persistent-tracking-mechanisms-in-the-wild
      ... basic optimizations and a naive load limiting check, we were able to run up to 30 browsers in parallel on a 4-core 8GB desktop machine running GNU/Linux operating system. The modified browsers were run in a chroot jail ...
 
CCS/2016/online-tracking-a-1-million-site-measurement-and-analysis
      ... can support a higher number of instances. On the same machine we can run 20 browser instances in parallel if the browser state is cleared after each page load. Generality. The platform minimizes code duplication both ac...
 
CCS/2017/poster-probing-tor-hidden-service-with-dockers
      ...l of We explain the Tor hidden service crawler in each Docker con- 20 Containers in parallel. The experiment was conducted over 9 tainer in Fig. 1. The Tor crawler running on the Docker consists days from August 2, 2017 ...
 
CCS/2018/minesweeper-an-in-depth-look-into-drive-by-cryptocurrency-mining-and-its-defense
      ...ners running on Kubernetes in an unfiltered network. We ran 50 Docker containers in parallel for one week mid-March 2018 to collect data from Alexa's Top 1 Million websites (as of February 28, 2018). Around 1% of the web...
 
IMC/2018/beyond-google-play-a-large-scale-comparative-study-of-chinese-android-app-market
      ...tally4 . GooglePlay Tencent Baidu 360 OPPO Xiaomi We launched several crawlers in parallel via 50 Aliyun Cloud Meizu Huawei Lenovo MM 25PP Category Wandoujia HiAPK Servers [37] between August 15 and August 30, 2017. Howe...
 
PETS/2018/i-never-signed-up-for-this-privacy-implications-of-email-tracking
      ...ted in a captcha. Since our primary fo- OpenWPM to run 15 measurement instances in parallel. cus is mailing lists, we leave the evaluation of complex Each email is loaded twice in its own measurement inand captcha-protec...
 
WWW/2018/an-automated-approach-to-auditing-disclosure-of-third-party-data-collection-in-w
      ... for this study. The first task required is to determine the machine, 64 parallel instances of PhantomJS can be easily run. third-parties which collect data on a given website and if the data A computer located at an aca...
 
USENIX/2019/inadvertently-making-cyber-criminals-rich-a-comprehensive-study-of-cryptojacking
      ...rs on 60 servers within the university network, each running 8 Docker instances in parallel. The crawl started on the December 24, 2018 and completed on January 9, 2019. In total, 1,769,183 websites have been successfull...
 
WWW/2019/before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private
      ...x Report 2017 [24]. For countries not covered in the Reuter's report, browsers in parallel, meaning that we did not in any case register we use the Alexa web service13 . For websites in other categories or log in with a ...
 
WWW/2019/pythia-a-framework-for-the-automated-analysis-of-web-hosting-environments
      ...econds before switching to the next one. Each chunk is analyzed using 20 parallel instances of our Crawler module, which uses a maximum timeout of 60 seconds while waiting for a webpage to finish loading. Note that when ...
 
WWW/2020/the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing
      ...cedure was repeated 4 times. The crawlers were configured to run with 3 parallel browser sessions running-that is, 3 browsers shared the crawl list and ran 1/3 each. These crawls used our alternative orchestration toolin...
 
PETS/2021/website-fingerprinting-in-the-age-of-quic
      ...d was provisioned with 1 GB of memory and 2 CPUs. We further utilised 15 simultaneous instances of the Chromium browser on a RedHat Linux virtual machine with thirty-two 2.7GHz processors and 20GB of RAM. We deployed eac...
 
USENIX/2021/scalable-detection-of-promotional-website-defacements-in-black-hat-seo-campaigns
      ...Memory, and a 400 Mbps network connection. A machine is configured to run 7 instances of crawlers in parallel. Knowing a defaced website may, by design, serve different pages to visitors originated from different geograp...
 
USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild
      ...ccording to the Tranco list [33] generated on 21 Dec 2020. We started 80 parallel crawlers using Chromium 87.0.4280 on 22 Dec and finished the crawl three days later. On each page, our crawler waits up to 30 seconds for ...
 
PETS/2022/who-knows-i-like-jelly-beans-an-investigation-into-search-privacy
      ...ational power, the underlying system was configured with 64 CPU cores running 128 threads and 800 GB RAM running Ubuntu 18.04LTS and Linux kernel version 4.15.0-72-generic. This setup was used to handle more than 100 con...
 
WWW/2022/et-tu-brute-privacy-analysis-of-government-websites-and-mobile-apps
      ...onfigure the OpenWPM [24] web privacy measurement framework to launch 15 parallel browser instances in headless mode. To simulate the first visit to a website, we clear the browser profile after each URL visit. We use tw...
 
IMC/2023/on-the-similarity-of-web-measurements-under-different-experimental-setups
      ...hich might introduce some bias into our results. Furthermore, each VM runs 15 browser instances in parallel. We configured a timeout of 30 seconds for each page visit (similar to previous works [14, 34, 35]). Other works...
 
IEEE-SP/2023/the-leaky-web-automated-discovery-of-cross-site-information-leaks-in-browsers-an
      ...iment was 13 days for Chromium and Firefox on a Linux server with 100 browsers in parallel and 47 days for WebKit on three Apple machines with a total of 15 browsers in parallel. The number of all tested observation chan...
 
USENIX/2024/automated-large-scale-analysis-of-cookie-notice-compliance
      ...n AMD Ryzen 9 5950X CPU. This 16-core CPU allows us to run 30 OpenWPM browsers in parallel, achieving a crawling speed of 300 websites per hour. The crawl is not networkconstrained and we only transfer 10 Mbit/s on avera...
 
PETS/2024/fp-tracer-fine-grained-browser-fingerprinting-detection-via-taint-tracking-and-e
      ...(GDPR). We started our crawl on 31/03/23 and ran it for 18 days using up to 12 parallel workers on a CentOS Linux server with 16 CPUs and 32GB of RAM. To evaluate that additional overhead caused by FP-tracer, we ran dedi...
 
USENIX/2024/dancer-in-the-dark-synthesizing-and-evaluating-polyglots-for-blind-cross-site-sc
      ...ork requests. We conducted our study over the course of 20 days using 60 parallel crawler instances on Ubuntu 20.04.5 LTS. Our monitoring was online during this time to allow observing 33rd USENIX Security Symposium 6733...
 
USENIX/2024/sok-state-of-the-krawlers-evaluating-the-effectiveness-of-crawling-algorithms-fo
      ...are and software configuration - We run Arachnarium with a maximum of 12 parallel workers, a wait time after page reloads of 500 ms, and an interval between actions of 500 ms. We run our experiments on a GNU/Linux Debian...
 
WWW/2024/detecting-and-understanding-self-deleting-javascript-code
      ...web security research. To automate the measurement, we drive 20 JSRay instances in parallel using Selenium 4 to visit the main page of each website on the list. In our experiment, we try to load a web page within a 30-se...
 
IEEE-SP/2024/the-great-request-robbery-an-empirical-study-of-client-side-request-hijacking-vu
      ...tion. TABLE 3: Summary of the collected data and preprocessing steps. 100 parallel browser instances. To ensure comprehensive coverage, we made up to three repeated attempts for each failed crawling website, followed by ...
 
CCS/2025/in-the-dom-we-trust-exploring-the-hidden-dangers-of-reading-from-the-dom-on-the
      ... to remove all regular content and ing infrastructure uses 𝑤 = 100 workers in parallel while minimizreplace it with a frameset element. This allows to "delete" prior ing disk I/O by leveraging RAMFS for concurrent wri...
 
IMC/2025/a-permissions-odyssey-a-systematic-study-of-browser-permissions-on-modern-websit
      ...d the crawl between August 23, 2024, and September 1, 2024, utilizing 40 parallel crawlers. Each website was crawled once, and the entire process was executed from the same server and Autonomous System Number (ASN) in Ge...
 
USENIX/2025/double-edged-shield-on-the-fingerprintability-of-customized-ad-blockers
      ... which do not require any prior consent from the servers. We use only 10 parallel crawlers to constrain the additional internet traffic volume. Also, we only load the pages passively by loading the page and any associate...
 
WWW/2026/tgnn-enhancing-pixel-tracking-detection-via-llm-driven-annotation-and-gat-powere
      ...ites/. In our data collection phase, we used Firefox v130.0.1 and ran 20 concurrent browser instances with a 10-minute per-site browsing time cap; additionally, we employed proxies to fix the IP address for improved repr...
 
PETS/2026/privacy-vs-profit-the-impact-of-googles-manifest-version-3-mv3-update-on-ad-bloc
      ...ent as they would for a standard visitor. We ran ten separate browser instances in parallel, each one representing a unique user-one without an ad blocker and the others with a selection of ad blockers from Adblock Plus,...
 
=== G. WHAT THE MEASUREMENT COST ===
 
Candidate sweep: 19 of 1120 crawled papers (1.7%).
Verdict             Papers  Share of 1120 crawled
subject             11      1.0%                 
own-infrastructure  5       0.4%                 
own-data            3       0.3%                 
 
States a cost the authors themselves paid: 8 of 1120 (0.7%)
States an infrastructure bill: 5 of 1120 (0.4%)
 
IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users [own-infrastructure]
      ...ut 7,4 million files that contain 39,156,580 group IDs. The crawling service cost us $18.47 and we received the data after five days. To enumerate Facebook's group members, we extracted the group IDs from the group directory, and then used our custom crawler t...
 
IEEE-SP/2011/design-and-evaluation-of-a-real-time-url-spam-filtering-service [own-infrastructure]
      ...wn of the time spent processing a single URL. Component AWS Infrastructure Monthly Cost URL aggregation 1 Extra Large $178 Feature collection 20 High-CPU Medium $882 Feature extraction - $0 Classification 50 Double Extra Large $527 Storage 700GB on EBS $70 Tot...
 
CCS/2016/online-tracking-a-1-million-site-measurement-and-analysis [own-infrastructure]
      ...ances on an Amazon EC2 "c4.2xlarge" virtual machine9 . This virtual machine costs around $300 per month using price estimates from May 2016. Due to Firefox's memory consumption, stateful parallel measurements are memory-limited while stateless parallel measure...
 
USENIX/2022/lend-me-your-ear-passive-remote-physical-side-channels-on-pcs [own-infrastructure]
      ...ng these in parallel on an Amazon EC2 c5.9xlarge 36-threads instance, the correct key was extracted within 20 minutes, at a total computational cost of less than $1.3 Parametrization. An important attack parameter is the number of traces acquired. The required...
 
CCS/2023/under-the-dark-a-systematical-study-of-stealthy-mining-pools-ab-use-in-the-wild [own-infrastructure]
      ... is $1,269.717 on average. Considering the renting price of servers for deploying taken-over pools, our overall purchase cost is $44.538 for one month ($40 for renting servers and $4.538 for registering domains). By taking over the mining pool, we can earn $1,...
 
IEEE-SP/2023/deepfake-text-detection-limitations-and-opportunities [own-data]
      ...eration-as-a-service platforms and from Internet forums. We spent $586 to collect the articles from the services. All the services we study claim that the synthetic articles can be used for white hat SEO. Regardless of the legal status of these services, the b...
 
USENIX/2023/calpric-inclusive-and-fine-grain-labeling-of-privacy-policies-with-crowdsourcing [own-data]
      ..., they reported a cost of $60 to label each privacy policy. Our system is able to further reduce the average cost of $13.5. Zimmeck et al. [48] also explored crowdsourcing, but did not train their ML models on these labels. They did, however, bring up an impor...
 
PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms [own-data]
      ...l user traffic to meet these requirements is expensive. Our experiments show that acquiring 1GB of authentic human browsing data costs approximately $35, primarily due to participant compensation and lab infrastructure. In comparison, 690 Redefining Website Fi...
 
=== Y. FIGURES THE PAGE DERIVES BY ARITHMETIC ===
 
Figure                                                  From     
182 campaign papers of 230 day-precision spans = 79.1%  section B
138 of 182 campaigns longer than a week = 75.8%         section B
10 lost-days of 1120 crawled = 0.9%                     section E
29 crawl-numeric of 1120 crawled = 2.6%                 section F
 
OK: all hand maps agree with their sweeps.

verify_deployment_figures-output.txt

ok   CCS/2014/a-nearly-four-year-longitudinal-study-of-search-engine-poisoning
     Out of the 1 254 days in the measurement period, we have complete measurements for 1 004 days
ok   IMC/2016/anatomy-of-a-personalized-livestreaming-system
     Our crawler was interrupted briefly between August 7-9, 2015 due to a communication bug with the Periscope server
ok   IMC/2016/anatomy-of-a-personalized-livestreaming-system
     our dataset is missing roughly 4.5% of the broadcasts
ok   IEEE-SP/2018/the-spyware-used-in-intimate-partner-violence
     Scans were not performed on Nov 07 and Nov 08 due to a power failure
ok   WWW/2021/tls-1-3-in-practice-how-tls-1-3-contributes-to-the-internet
     There were network outages for 17 days, which are pruned out from the dataset
ok   WWW/2021/tls-1-3-in-practice-how-tls-1-3-contributes-to-the-internet
     around 84% of the websites were consistently collected
ok   WWW/2013/two-years-of-short-urls-internet-measurement-security-threats-and-countermeasure
     Our system experienced three outages due to database failures throughout one year
ok   IMC/2018/digging-into-browser-based-crypto-mining
     Black parts mark outages of our infrastructure
ok   USENIX/2015/vulnerability-disclosure-in-the-age-of-social-media-exploiting-twitter-for-predi
     except during the periods of downtime for our infrastructure
ok   NDSS/2015/seven-months-worth-of-mistakes-a-longitudinal-study-of-typosquatting-abuse
     to accommodate for a planned power interruption of our crawling machines on August 27 and 28
ok   WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion
     Our crawler crashed on March 12th in both Ukraine and Russia, and again on March 20th and April 18th in Ukraine
ok   IMC/2021/who-you-gonna-call-an-empirical-evaluation-of-website-security-txt-deployment
     we had a 1.5-month measurement interruption throughout November and the first half of December, 2020
ok   NDSS/2017/dial-one-for-scam-a-large-scale-analysis-of-technical-support-scams
     none of the three ROBOVIC instances experienced any downtime during our monitored period
ok   USENIX/2025/websites-global-privacy-control-compliance-at-scale-and-over-time
     nearly 40% of sites that failed to load on the first attempt were loaded and successfully analyzed
ok   USENIX/2025/websites-global-privacy-control-compliance-at-scale-and-over-time
     a site failed to load in 35 seconds, had an insecure certificate, or led to an error page
ok   WWW/2021/websocket-adoption-and-the-landscape-of-the-real-time-web
     After retrying each failed crawl a second time, we obtained data for a total of 88.1% of websites in the top million
ok   WWW/2021/websocket-adoption-and-the-landscape-of-the-real-time-web
     7.7% of the listed domain names failed to resolve, and the remaining 4.2% had web servers which failed to respond
ok   USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild
     In another 4%, the server returned an HTTP error code and the remaining 3% failed to load before our 30 seconds timeout hit
ok   NDSS/2024/fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting
     Out of the 20k websites, we successfully visit 18,300 (91.5%)
ok   NDSS/2024/fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting
     the overwhelming majority (64.3%) of them due to HTTP 403 Forbidden errors
ok   PETS/2025/tracker-installations-are-not-created-equal-understanding-tracker-configuration
     Out of the 42,481 websites visited, we visited 40,150 (94.51%) successfully
ok   PETS/2024/a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users
     we successfully crawled 29,660 (99%) sites
ok   WWW/2017/security-challenges-in-an-increasingly-tangled-web
     15K domains did not resolve, 13K timed out
ok   PETS/2017/towards-seamless-tracking-free-web-improved-detection-of-trackers-via-one-class
     we reran the crawler on the webpage by increasing the waiting time until the webpage is rendered
ok   IEEE-SP/2024/the-great-request-robbery-an-empirical-study-of-client-side-request-hijacking-vu
     we made up to three repeated attempts for each failed crawling
ok   USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission
     This minor change increased the successfully visited websites from 94, 427 (EU pilot crawl) to 99, 380 (EU final crawl)
ok   WWW/2017/security-challenges-in-an-increasingly-tangled-web
     page for 944,000 sites
ok   NDSS/2024/fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting
     Out of the 20k websites
ok   IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users
     the dictionary consisted of 7.1GB of HTML data
ok   PETS/2025/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and
     16,296 pages were successfully loaded, 692 triggered
ok   CCS/2016/online-tracking-a-1-million-site-measurement-and-analysis
     which currently allocates 8 vCPUs and 15 GiB of memory per machine. With this configuration we are able to run 20 browser instances in parallel
ok   USENIX/2024/automated-large-scale-analysis-of-cookie-notice-compliance
     This 16-core CPU allows us to run 30 OpenWPM browsers in parallel, achieving a crawling speed of 300 websites per hour
ok   CCS/2014/the-web-never-forgets-persistent-tracking-mechanisms-in-the-wild
     we were able to run up to 30 browsers in parallel on a 4-core 8GB desktop machine
ok   USENIX/2019/inadvertently-making-cyber-criminals-rich-a-comprehensive-study-of-cryptojacking
     We deployed the crawler in Docker containers on 60 servers within the university network, each running 8 Docker instances in parallel
ok   CCS/2018/minesweeper-an-in-depth-look-into-drive-by-cryptocurrency-mining-and-its-defense
     We ran 50 Docker containers in parallel for one week
ok   IEEE-SP/2023/the-leaky-web-automated-discovery-of-cross-site-information-leaks-in-browsers-an
     13 days for Chromium and Firefox on a Linux server with 100 browsers in parallel
ok   PETS/2020/in-depth-evaluation-of-redirect-tracking-and-link-usage
     execute them in individual Docker containers in parallel so that a problem in one of the tasks
ok   IMC/2025/a-permissions-odyssey-a-systematic-study-of-browser-permissions-on-modern-websit
     utilizing 40 parallel crawlers
ok   IEEE-SP/2011/design-and-evaluation-of-a-real-time-url-spam-filtering-service
     URL aggregation 1 Extra Large $178 Feature collection 20 High-CPU Medium $882
ok   IEEE-SP/2011/design-and-evaluation-of-a-real-time-url-spam-filtering-service
     Storage 700GB on EBS $70 Total $1,587
ok   CCS/2016/online-tracking-a-1-million-site-measurement-and-analysis
     This virtual machine costs around $300 per month using price estimates from May 2016
ok   IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users
     The crawling service cost us $18.47
ok   IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users
     The costs for the crawling were $6.57
ok   IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users
     we estimate overall costs of about $88 for crawling all 40
ok   USENIX/2022/lend-me-your-ear-passive-remote-physical-side-channels-on-pcs
     at a total computational cost of less than $1.3
ok   USENIX/2023/calpric-inclusive-and-fine-grain-labeling-of-privacy-policies-with-crowdsourcing
     reported a cost of $60 to label each privacy policy. Our system is able to further reduce the average cost of $13.5
ok   IEEE-SP/2023/deepfake-text-detection-limitations-and-opportunities
     We spent $586 to collect the articles from the services
ok   IMC/2021/knock-and-talk-investigating-local-network-communications-on-websites
     Nearly 90% of the failures were due to DNS resolution errors (NAME_NOT_RESOLVED)
ok   WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion
     and again on March 20th and April 18th in Ukraine after running for some time
ok   CCS/2014/a-nearly-four-year-longitudinal-study-of-search-engine-poisoning
     complete measurements for 1 004 days
ok   CCS/2023/under-the-dark-a-systematical-study-of-stealthy-mining-pools-ab-use-in-the-wild
     our overall purchase cost is $44.538 for one month
ok   PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms
     acquiring 1GB of authentic human browsing data costs approximately $35
 
52 needles: 52 found verbatim (whitespace collapsed), 0 not found, 0 flagged weak.
 
--- PDF ROUTE (run against paper.pdf with pypdf, not paper.cols.txt) ---
FOUND  USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission
     increased the successfully visited websites from 94,427 (EU pilot crawl) to 99,380 (EU final crawl)
     paper.cols.txt renders these as "94, 427" and "99, 380"; the PDF has no space.
     The PDF sets "final" with an fi ligature; the check de-ligatures before matching,
     and the page quotes the de-ligatured form.
 
--- ARITHMETIC ---
3,300 machine-hours  1,000,000 sites / 300 sites per machine-hour = 3,333, rounded to 3,300
5 days on 28 machines  3,300 machine-hours / (24 h * 5 d) = 27.5, rounded to 28
a month on four machines  3,300 / (24 * 30) = 4.6, so four machines is about five weeks; stated as "a month on four"
about 10 sites per browser-hour  300 sites per hour / 30 parallel browsers = 10
7,200 sites a day per machine  300 sites per hour * 24 hours
USD 2,700 for one pass  3,300 machine-hours * USD 0.8148/hour (c7i.4xlarge) = 2,689, rounded to 2,700
480 concurrent (Bijmans et al.)  60 servers * 8 Docker instances each
84.6% (Musch and Johns)  about 846k sites of the Tranco top 1M = 84.6%. The paper states the numerator and names the list; the percentage is ours. It is also the sum of its own failure classes: ~8% network/DNS + 4% HTTP error + 3% timeout = 15%.
the 85-94% range  the low end is Musch and Johns at 84.6%, rounded to 85; the high end is Kumar et al. at 944,000/1,000,000 = 94.4%, rounded to 94. Both endpoints are in NEEDLES.
the 90-99% range  the low end is Kuchhal and Li at ~90%; the high end is Demir et al. at 99%. Both in NEEDLES.
$88 to crawl all 40 million profiles  Wondracek et al.'s own linear-cost estimate from their $6.57 LinkedIn crawl, quoted from the paper. It is NOT derived from the $18.47 Facebook-groups figure; a reviewer caught the page implying it was.
1,004 of the 1,254 days  the page writes Leontiadis et al.'s figures with thousands commas; the paper writes "1 004" and "1 254" with thin spaces
 
--- EXTERNAL FIGURES (not from the corpus) ---
431 MB  Content-Length of the eu-central-1 EC2 offer file on 2026-08-28: 431,220,398 bytes
0.4074  AWS EC2 on-demand USD/hour, c7i.2xlarge, eu-central-1, Linux, shared tenancy, bulk price list version 20260828175757, fetched 2026-08-28 by scripts/ec2_price.py
0.8148  same source, c7i.4xlarge
7  venue count of this corpus
64  Docker default /dev/shm in MiB, documented on programming:docker
 
--- NOT FIGURES ---
1.62  Playwright image tag on programming:docker, not a figure here
300  sites per hour, a corpus figure (Bouhoula et al.), listed in NEEDLES
300.0  crawl_queue.py default lease_seconds, a code literal
404  HTTP status in the error-class names http_404 / TERMINAL_ERRORS
410  HTTP status in TERMINAL_ERRORS
403  HTTP status; the Forbidden figure itself is in NEEDLES
200  crawl_queue.py demo fixture size, printed in the quoted demo output
87  crawl_queue.py demo crash point, printed in the quoted demo output
256  sha256 in the seed fingerprint
9  the c5.9xlarge instance name in the Genkin et al. row
1.3  Genkin et al.: USD, in NEEDLES
13.5  Qiu et al.: USD per policy, in NEEDLES
586  Pu et al.: USD, in NEEDLES
6.57  Wondracek et al.: USD, in NEEDLES
20  crawl_queue.py demo ordering fixture size, printed in the quoted demo output
301  crawl_queue.py demo clock advance in seconds, printed in the quoted demo output
301.0  the same value as a float literal in the demo source
1000000  crawl_queue.py demo fake clock origin

dep_fold.mjs

// Hand classification for programming:deployment.
//
// There is no name fold on this page: everything it counts is a date, an enum,
// or a published regex. The work is one hand map.
//
// SPAN_ROLE — what a temporal[] tuple's (spanStart, spanEnd) pair actually
// measures. `temporal[].mode == "live-crawl"` does NOT mean the span is the
// wall-clock time an apparatus was running. Read against the tuple's own
// evidence quote, the 230 crawled papers that give both ends at day precision
// split four ways:
//
//   campaign  — the span is machine time: a crawler, scraper or collector was
//               running between those dates, continuously or on a cadence.
//               This is the only class the page's duration figures use.
//   content   — the span is the date range of the *collected content* (posts
//               published 2015-2021, disputes decided 1999-2024). The machine
//               may have run for an afternoon. Counting these as crawl
//               duration is what puts a 9,028-day "crawl" in the corpus.
//   separated — two or more disjoint episodes with the gap between them inside
//               the span ("two periods, Aug-Nov 2018 and Apr-May 2019").
//               Real machine time, but not a continuous run.
//   retrieval — the span is an artefact of a retrieval date: a list-fetch date,
//               an "as of" date, or in one case a bibliography access date.
//
// Verdicts are keyed on slug (unique across the 230) and were made by reading
// each tuple's evidence quote; report_deployment.mjs prints the quote beside
// every verdict so the reading is auditable. Anything not named here is
// `campaign`; the report exits 1 if a slug here is not in the population.
 
export const SPAN_ROLE_DEFAULT = 'campaign';
 
export const SPAN_CONTENT = [
  'screenshots-symbols-and-personal-thoughts-the-role-of-instagram-for-social-activ',
  'stealthy-porn-understanding-real-world-adversarial-images-for-illicit-online-pro',
  'hunting-the-red-fox-online-understanding-and-detection-of-mass-redirect-script-i',
  'beyond-jailbreak-unveiling-risks-in-llm-applications-arising-from-blurred-capability-boundaries',
  'getting-bored-of-cyberwar-exploring-the-role-of-low-level-cybercrime-actors-in-t',
  'tagvisor-a-privacy-advisor-for-sharing-hashtags',
  'ethereums-proposer-builder-separation-promises-and-realities',
  'a-first-look-at-zoombombing',
  'quantifying-security-training-in-organizations-through-the-analysis-of-u-s-sec-1',
  'real-time-recommendation-of-diverse-related-articles',
  'google-or-google-dissecting-the-evolution-of-the-new-osn-in-its-first-year',
  'hey-your-secrets-leaked-detecting-and-characterizing-secret-leakage-in-the-wild',
  'causal-insights-into-parlers-content-moderation-shift-effects-on-toxicity-and-fa',
  'assessing-the-aftermath-the-effects-of-a-global-takedown-against-ddos-for-hire-s',
  'is-my-internet-down-sifting-through-user-affecting-outages-with-google-trends',
  'auto-draft-246',
  'clues-in-tweets-twitter-guided-discovery-and-analysis-of-sms-spam',
  'identification-of-disease-or-symptom-terms-in-reddit-to-improve-health-mention-c',
  'dirty-jobs-the-role-of-freelance-labor-in-web-service-abuse',
  'misbehavior-and-account-suspension-in-an-online-financial-communication-platform',
  'moderating-illicit-online-image-promotion-for-unsafe-user-generated-content-game',
  'repairing-trust-in-domain-name-disputes-practices-insights-from-a-quarter-centurys-worth-of-squabbles',
  'navigating-the-post-api-dilemma',
  'analysis-of-privacy-protections-in-fitness-tracking-social-networks-or-you-can-r',
  'blue-is-the-new-black-market-privacy-leaks-and-re-victimization-from-police-auct',
];
 
export const SPAN_SEPARATED = [
  'towards-ip-based-geolocation-via-fine-grained-and-stable-webcam-landmarks',
  'experimental-security-analysis-of-the-app-model-in-business-collaboration-platfo',
  'towards-automated-auditing-for-account-and-session-management-flaws-in-single-si',
  'encrypted-dns-privacy-a-traffic-analysis-perspective',
  'a-large-scale-study-on-the-risks-of-the-html5-webapi-for-mobile-sensor-based-att',
  'exposing-search-and-advertisement-abuse-tactics-and-infrastructure-of-technical',
  'an-empirical-study-of-the-cost-of-dns-over-https',
  'a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users',
  'overcoming-language-barriers-multilingual-analysis-of-the-2023-swiss-privacy-law',
  'the-security-lottery-measuring-client-side-web-security-inconsistencies',
  'how-tracking-companies-circumvented-ad-blockers-using-websockets',
  'when-llms-go-online-the-emerging-threat-of-web-enabled-llms',
  'assessing-compliance-in-digital-advertising-a-deep-dive-into-acceptable-ads-stan',
  'the-unusual-suspects-studying-reasons-for-lacking-updates-in-wordpress',
  'cspautogen-black-box-enforcement-of-content-security-policy-upon-real-world-webs',
  'online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion',
];
 
export const SPAN_RETRIEVAL = [
  'o-single-sign-off-where-art-thou-an-empirical-analysis-of-single-sign-on-account',
  'dangerous-skills-understanding-and-mitigating-security-risks-of-voice-controlled',
  'hiding-in-plain-site-detecting-javascript-obfuscation-through-concealed-browser',
  'apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web',
  'parasites-in-the-toolchain-a-large-scale-analysis-of-attacks-on-the-mcp-ecosyste',
  'the-masks-we-think-we-wear-privacy-threats-of-browser-extension-wallets-in-the-w',
  'from-syntactic-matching-to-taint-tracking-and-back-a-comparative-study-of-web-tr',
];
 
const ROLE_OF = new Map();
for (const s of SPAN_CONTENT) ROLE_OF.set(s, 'content');
for (const s of SPAN_SEPARATED) ROLE_OF.set(s, 'separated');
for (const s of SPAN_RETRIEVAL) ROLE_OF.set(s, 'retrieval');
 
if (ROLE_OF.size !== SPAN_CONTENT.length + SPAN_SEPARATED.length + SPAN_RETRIEVAL.length) {
  throw new Error('dep_fold: a slug appears in more than one SPAN_ list');
}
 
export function spanRole(slug) {
  return ROLE_OF.get(slug) ?? SPAN_ROLE_DEFAULT;
}
 
export function namedSlugs() {
  return [...ROLE_OF.keys()];
}
 
// ---------------------------------------------------------------------------
// INTERRUPTION — every hit of the CANDIDATE_INTERRUPTION sweep gets a verdict.
// There is no default: report_deployment.mjs exits 1 on a hit with no verdict
// and on a verdict whose paper is not a hit. Roles:
//   lost-days  — the paper's own measurement infrastructure lost time.
//   planned    — the authors knew about the outage in advance and worked round it.
//   no-downtime— an explicit claim that nothing was lost (the negative is a
//                reporting decision too).
//   target-down— what went down was the measured site/network, not the crawler.
//   offtopic   — outages are the paper's subject, or the word is a CPU interrupt.
export const CANDIDATE_INTERRUPTION = new RegExp([
  '(measurement|crawl|crawler|crawling|scan|scans|scanning|collection|experiment|monitoring)[a-z]{0,4}\\b[^.]{0,90}\\b(interrupt|outage|power failure|downtime|went down|was down|not performed|missing days|incomplete (measurements|days|data)|gaps? in (our |the )?(measurement|data|coverage))',
  '\\b(power|network|hardware|machine|server|disk)\\s+(outage|failure|interruption)[^.]{0,90}(measurement|crawl|scan|collection|experiment)',
  'there (were|was) (a |an )?(\\d+ )?(network|power|server|machine|infrastructure)?\\s?outages?',
  'our (crawler|crawlers|infrastructure|servers?|machines?|system|exits?|nodes?|pipeline)[^.]{0,70}(went down|was down|crashed|downtime|outage|stopped (working|running))',
  '(downtime|outage|interruption)[^.]{0,60}\\b(our|we)\\b[^.]{0,40}(infrastructure|crawler|measurement|collection|machines?|servers?)',
].join('|'), 'i');
 
export const INTERRUPTION = {
  'a-nearly-four-year-longitudinal-study-of-search-engine-poisoning': 'lost-days',
  'vulnerability-disclosure-in-the-age-of-social-media-exploiting-twitter-for-predi': 'lost-days',
  'anatomy-of-a-personalized-livestreaming-system': 'lost-days',
  'the-spyware-used-in-intimate-partner-violence': 'lost-days',
  'who-you-gonna-call-an-empirical-evaluation-of-website-security-txt-deployment': 'lost-days',
  'timeless-timing-attacks-and-preload-defenses-in-tors-dns-cache': 'lost-days',
  'two-years-of-short-urls-internet-measurement-security-threats-and-countermeasure': 'lost-days',
  'digging-into-browser-based-crypto-mining': 'lost-days',
  'tls-1-3-in-practice-how-tls-1-3-contributes-to-the-internet': 'lost-days',
  'online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion': 'lost-days',
  'seven-months-worth-of-mistakes-a-longitudinal-study-of-typosquatting-abuse': 'planned',
  'dial-one-for-scam-a-large-scale-analysis-of-technical-support-scams': 'no-downtime',
  'traveling-the-silk-road-a-measurement-analysis-of-a-large-anonymous-online-marke': 'target-down',
  'more-space-less-privacy-measuring-the-effectiveness-of-ip-based-website-fingerpr': 'target-down',
  'challenges-in-the-decentralised-web-the-mastodon-case': 'target-down',
  'opening-the-blackbox-of-virustotal-analyzing-online-phishing-scan-engines': 'offtopic',
  'inadvertently-making-cyber-criminals-rich-a-comprehensive-study-of-cryptojacking': 'offtopic',
  'is-my-internet-down-sifting-through-user-affecting-outages-with-google-trends': 'offtopic',
  'access-denied-assessing-physical-risks-to-internet-access-networks': 'offtopic',
  'idleleak-exploiting-idle-state-side-effects-for-information-leakage': 'offtopic',
  'from-mirai-to-gorilla-deep-dive-into-a-long-lasting-ddos-for-hire-botnet': 'offtopic',
};
 
// ---------------------------------------------------------------------------
// OWN_COST — a currency figure near a cost word in a measurement context is
// usually the *adversary's* economics, not the authors' bill. Roles:
//   own-infrastructure  — what the authors paid to run the measurement.
//   own-data            — what the authors paid to acquire data (services,
//                         participants, annotation), which is a real study cost
//                         but not a machine bill.
//   subject             — the price being reported belongs to what was measured.
// Widened on 2026-08-28 after reading: the first version named only crawl-ish
// subjects and therefore missed Englehardt and Narayanan's own line, "This
// virtual machine costs around $300 per month using price estimates from May
// 2016" — the OpenWPM 1M-site paper, which is the single most-cited crawl
// infrastructure sentence in this corpus. Adding machine/VM/server/instance as
// cost subjects took the candidate set from 8 to 19 and the own-infrastructure
// count from 2 to 5.
export const CANDIDATE_COST =
  /((crawl|crawling|measurement|scan|scanning|experiment|data collection|infrastructure|compute|storage|virtual machine|\bVM\b|server|instance|our (setup|pipeline|system|crawler))[^.]{0,120}\b(cost|costs|costed|expense|expenses|budget|price estimates)\b[^.]{0,60}(\$|USD|EUR|€|dollar)|\b(cost|costs|spent|paid)\b[^.]{0,40}(\$|USD|EUR|€)[0-9][^.]{0,90}\b(to (run|crawl|collect|scan)|for (the |our )?(crawl|measurement|experiment|data collection)|crawling))/i;
 
export const OWN_COST = {
  'a-practical-attack-to-de-anonymize-social-network-users': 'own-infrastructure',
  'design-and-evaluation-of-a-real-time-url-spam-filtering-service': 'own-infrastructure',
  'deepfake-text-detection-limitations-and-opportunities': 'own-data',
  'calpric-inclusive-and-fine-grain-labeling-of-privacy-policies-with-crowdsourcing': 'own-data',
  'redefining-website-fingerprinting-attacks-with-multi-agent-llms': 'own-data',
  'online-tracking-a-1-million-site-measurement-and-analysis': 'own-infrastructure',
  'lend-me-your-ear-passive-remote-physical-side-channels-on-pcs': 'own-infrastructure',
  'under-the-dark-a-systematical-study-of-stealthy-mining-pools-ab-use-in-the-wild': 'own-infrastructure',
  'measurement-and-analysis-of-traffic-exchange-services': 'subject',
  'practical-censorship-evasion-leveraging-content-delivery-networks': 'subject',
  'adbudgetkiller-online-advertising-budget-draining-attack': 'subject',
  'ready-raider-one-exploring-the-misuse-of-cloud-gaming-services': 'subject',
  'mistrust-plugins-you-must-a-large-scale-study-of-malicious-plugins-in-wordpress': 'subject',
  'understanding-and-analyzing-appraisal-systems-in-the-underground-marketplaces': 'subject',
  'evaluating-llm-based-personal-information-extraction-and-countermeasures': 'subject',
  'dns-flare-a-flush-reload-attack-on-dns-forwarders': 'subject',
  'show-me-the-money-characterizing-spam-advertised-revenue': 'subject',
  'an-empirical-analysis-of-the-commercial-vpn-ecosystem': 'subject',
  'polls-clickbait-and-commemorative-2-bills-problematic-political-advertising-on-n': 'subject',
};
 
// ---------------------------------------------------------------------------
// CONCURRENCY — how many workers ran at once. Roles:
//   crawl-numeric — a number of parallel browsers/crawlers/containers used for
//                   the measurement itself. This is the only role the page counts.
//   crawl-vague   — the measurement was parallel, no number given.
//   analysis      — the parallelism is in a post-hoc analysis or fuzzing stage,
//                   not in the crawl.
//   offtopic      — parallelism of the measured phenomenon, of a future-work
//                   proposal, or of HTTP/CPU internals.
export const CANDIDATE_CONCURRENCY =
  /\b(\d{1,4})\s*(parallel|concurrent|simultaneous)\s+(browser|crawler|instance|worker|thread|container|process|node|machine|vm|virtual machine)[a-z]*|\b(browser|crawler|worker|instance|container|process)[a-z]*\s+in\s+parallel|\brun(ning)?\s+(up to\s+)?\d{1,4}\s+(browser|crawler|instance|worker|thread|container|process)[a-z]*\s+(in parallel|concurrently|simultaneously)/i;
 
export const CONCURRENCY = {
  'the-web-never-forgets-persistent-tracking-mechanisms-in-the-wild': 'crawl-numeric',
  'online-tracking-a-1-million-site-measurement-and-analysis': 'crawl-numeric',
  'poster-probing-tor-hidden-service-with-dockers': 'crawl-numeric',
  'minesweeper-an-in-depth-look-into-drive-by-cryptocurrency-mining-and-its-defense': 'crawl-numeric',
  'beyond-google-play-a-large-scale-comparative-study-of-chinese-android-app-market': 'crawl-numeric',
  'i-never-signed-up-for-this-privacy-implications-of-email-tracking': 'crawl-numeric',
  'an-automated-approach-to-auditing-disclosure-of-third-party-data-collection-in-w': 'crawl-numeric',
  'inadvertently-making-cyber-criminals-rich-a-comprehensive-study-of-cryptojacking': 'crawl-numeric',
  'before-and-after-gdpr-the-changes-in-third-party-presence-at-public-and-private': 'crawl-numeric',
  'pythia-a-framework-for-the-automated-analysis-of-web-hosting-environments': 'crawl-numeric',
  'the-representativeness-of-automated-web-crawls-as-a-surrogate-for-human-browsing': 'crawl-numeric',
  'website-fingerprinting-in-the-age-of-quic': 'crawl-numeric',
  'scalable-detection-of-promotional-website-defacements-in-black-hat-seo-campaigns': 'crawl-numeric',
  'u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild': 'crawl-numeric',
  'who-knows-i-like-jelly-beans-an-investigation-into-search-privacy': 'crawl-numeric',
  'et-tu-brute-privacy-analysis-of-government-websites-and-mobile-apps': 'crawl-numeric',
  'on-the-similarity-of-web-measurements-under-different-experimental-setups': 'crawl-numeric',
  'automated-large-scale-analysis-of-cookie-notice-compliance': 'crawl-numeric',
  'fp-tracer-fine-grained-browser-fingerprinting-detection-via-taint-tracking-and-e': 'crawl-numeric',
  'dancer-in-the-dark-synthesizing-and-evaluating-polyglots-for-blind-cross-site-sc': 'crawl-numeric',
  'sok-state-of-the-krawlers-evaluating-the-effectiveness-of-crawling-algorithms-fo': 'crawl-numeric',
  'detecting-and-understanding-self-deleting-javascript-code': 'crawl-numeric',
  'in-the-dom-we-trust-exploring-the-hidden-dangers-of-reading-from-the-dom-on-the': 'crawl-numeric',
  'a-permissions-odyssey-a-systematic-study-of-browser-permissions-on-modern-websit': 'crawl-numeric',
  'double-edged-shield-on-the-fingerprintability-of-customized-ad-blockers': 'crawl-numeric',
  'tgnn-enhancing-pixel-tracking-detection-via-llm-driven-annotation-and-gat-powere': 'crawl-numeric',
  'privacy-vs-profit-the-impact-of-googles-manifest-version-3-mv3-update-on-ad-bloc': 'crawl-numeric',
  'the-great-request-robbery-an-empirical-study-of-client-side-request-hijacking-vu': 'crawl-numeric',
  'the-leaky-web-automated-discovery-of-cross-site-information-leaks-in-browsers-an': 'crawl-numeric',
  'understanding-the-evolution-of-mobile-app-ecosystems-a-longitudinal-measurement': 'crawl-vague',
  'mnemosyne-an-effective-and-efficient-postmortem-watering-hole-attack-investigati': 'crawl-vague',
  'in-depth-evaluation-of-redirect-tracking-and-link-usage': 'crawl-vague',
  'omnicrawl-comprehensive-measurement-of-web-tracking-with-real-desktop-and-mobile': 'crawl-vague',
  'searching-the-searchers-with-searchaudit': 'analysis',
  'mystique-uncovering-information-leakage-from-browser-extensions': 'analysis',
  'iframes-popups-are-dangerous-in-mobile-webview-studying-and-mitigating-different': 'analysis',
  'animatedead-debloating-web-applications-using-concolic-execution': 'analysis',
  'do-not-follow-the-white-rabbit-challenging-the-myth-of-harmless-open-redirection': 'analysis',
  'your-online-interests-pwned-a-pollution-attack-against-targeted-advertising': 'offtopic',
  'auditing-the-personalization-and-composition-of-politically-related-search-engin': 'offtopic',
  'minos-a-lightweight-real-time-cryptojacking-detection-system': 'offtopic',
  'deepfake-text-detection-limitations-and-opportunities': 'offtopic',
  'evocrawl-exploring-web-application-code-and-state-using-evolutionary-search': 'offtopic',
  'time-and-time-again-leveraging-tcp-timestamps-to-improve-remote-timing-attacks': 'offtopic',
};

report_deployment.mjs

#!/usr/bin/env node
// Every figure on programming:deployment, with its denominator.
//
//   node scripts/report_deployment.mjs            plain text
//   node scripts/report_deployment.mjs --wiki     DokuWiki tables
//   node scripts/report_deployment.mjs --list <k> the papers behind a figure
//   node scripts/report_deployment.mjs --quotes <regex>
//
// Exits 1 if a hand map and its sweep diverge in either direction, or if the
// corpus contract (7 venues, the OVERVIEW.md populations) has moved.
 
import { loadExtractions, POPULATIONS, pct, isSentinel } from './lib.mjs';
import { fulltext } from './sweep.mjs';
import {
  spanRole, namedSlugs,
  CANDIDATE_INTERRUPTION, INTERRUPTION,
  CANDIDATE_COST, OWN_COST,
  CANDIDATE_CONCURRENCY, CONCURRENCY,
} from './dep_fold.mjs';
 
const WIKI = process.argv.includes('--wiki');
const LIST = process.argv.includes('--list') ? process.argv[process.argv.indexOf('--list') + 1] : null;
const QUOTES = process.argv.includes('--quotes') ? process.argv[process.argv.indexOf('--quotes') + 1] : null;
 
const rows = loadExtractions();
const crawled = rows.filter(POPULATIONS.crawled);
const key = (p) => `${p.venue}/${p.year}/${p.slug}`;
const LISTS = new Map();
const remember = (name, papers) => { LISTS.set(name, papers.map(key).sort()); return papers; };
 
let failures = 0;
const fail = (msg) => { console.log(`FAILURE: ${msg}`); failures += 1; };
 
function table(headers, rowsIn) {
  if (WIKI) {
    console.log(`^ ${headers.join(' ^ ')} ^`);
    for (const r of rowsIn) console.log(`| ${r.join(' | ')} |`);
  } else {
    const w = headers.map((h, i) => Math.max(h.length, ...rowsIn.map((r) => String(r[i]).length)));
    console.log(headers.map((h, i) => h.padEnd(w[i])).join('  '));
    for (const r of rowsIn) console.log(r.map((c, i) => String(c).padEnd(w[i])).join('  '));
  }
  console.log('');
}
 
const flatCache = new Map();
function flat(p) {
  const k = key(p);
  if (!flatCache.has(k)) {
    const t = fulltext(p);
    flatCache.set(k, t === null ? null : t.replace(/\s+/g, ' '));
  }
  return flatCache.get(k);
}
function sweepFlat(papers, re) {
  const hits = []; let missing = 0;
  for (const p of papers) {
    const t = flat(p);
    if (t === null) { missing += 1; continue; }
    if (re.test(t)) hits.push(p);
  }
  return { hits, missing };
}
 
// ---------------------------------------------------------------- A. contract
console.log('=== A. POPULATION AND CORPUS CONTRACT ===\n');
const venues = [...new Set(rows.map((p) => p.venue))].sort();
if (venues.length !== 7) fail(`expected 7 venues, got ${venues.length}: ${venues.join(', ')}`);
if (rows.length !== 5859) console.log(`NOTE: corpus size is ${rows.length}, not the 5,859 the page was written against`);
if (crawled.length !== 1120) console.log(`NOTE: crawled is ${crawled.length}, not the 1,120 the page was written against`);
const years = rows.map((p) => p.year);
table(['Quantity', 'Value'], [
  ['venues', venues.join(', ')],
  ['years', `${Math.min(...years)}-${Math.max(...years)}`],
  ['all papers', rows.length],
  ['crawled (page denominator)', crawled.length],
  ['crawled with readable .cols', crawled.filter((p) => flat(p) !== null).length],
]);
 
// ------------------------------------------------------- B. how long they run
console.log('=== B. HOW LONG A CRAWL ACTUALLY RUNS ===\n');
console.log('Population: crawled papers with at least one temporal[] tuple whose mode is');
console.log('live-crawl and whose spanStart AND spanEnd are both day-precision (YYYY-MM-DD).');
console.log('Duration is inclusive of both endpoints. Per paper we keep the LONGEST such span.\n');
 
const pd = (s) => {
  if (typeof s !== 'string') return null;
  const m = s.match(/^(\d{4})-(\d{2})-(\d{2})$/);
  if (!m) return null;
  const d = new Date(Date.UTC(+m[1], +m[2] - 1, +m[3]));
  return Number.isNaN(+d) ? null : d;
};
const DAY = 86400000;
const spans = new Map();
let dayTuples = 0, monthTuples = 0, yearTuples = 0, nonPositive = 0;
for (const p of crawled) {
  for (const t of p.temporal) {
    if (t.mode !== 'live-crawl') continue;
    const a = pd(t.spanStart), b = pd(t.spanEnd);
    if (!a || !b) {
      if (typeof t.spanStart === 'string' && typeof t.spanEnd === 'string') {
        if (/^\d{4}-\d{2}$/.test(t.spanStart) && /^\d{4}-\d{2}$/.test(t.spanEnd)) monthTuples += 1;
        else if (/^\d{4}$/.test(t.spanStart) && /^\d{4}$/.test(t.spanEnd)) yearTuples += 1;
      }
      continue;
    }
    dayTuples += 1;
    const d = Math.round((b - a) / DAY) + 1;
    if (d <= 0) { nonPositive += 1; continue; }
    const prev = spans.get(key(p));
    if (!prev || prev.days < d) {
      spans.set(key(p), { days: d, paper: p, raw: `${t.spanStart}..${t.spanEnd}`, quote: (t.evidence?.quote || '').replace(/\s+/g, ' ') });
    }
  }
}
const spanPapers = [...spans.values()];
table(['Quantity', 'Papers / tuples', 'Share of 1,120 crawled'], [
  ['live-crawl tuples, both ends day-precision', dayTuples, '—'],
  ['... month-precision only (excluded)', monthTuples, '—'],
  ['... year-precision only (excluded)', yearTuples, '—'],
  ['... non-positive duration (excluded)', nonPositive, '—'],
  ['papers with a day-precision span', spanPapers.length, pct(spanPapers.length, crawled.length)],
]);
 
// hand map integrity
const spanSlugs = new Set(spanPapers.map((s) => s.paper.slug));
for (const s of namedSlugs()) if (!spanSlugs.has(s)) fail(`SPAN_ hand map names ${s}, which is not in the span population`);
 
const byRole = new Map();
for (const s of spanPapers) {
  const r = spanRole(s.paper.slug);
  if (!byRole.has(r)) byRole.set(r, []);
  byRole.get(r).push(s);
}
table(['Span role', 'Papers', 'Share of ' + spanPapers.length], [...byRole.entries()]
  .sort((a, b) => b[1].length - a[1].length)
  .map(([r, v]) => [r, v.length, pct(v.length, spanPapers.length)]));
 
const campaigns = byRole.get('campaign') ?? [];
remember('campaign', campaigns.map((s) => s.paper));
for (const [r, v] of byRole) remember(`span-${r}`, v.map((s) => s.paper));
const ds = campaigns.map((s) => s.days).sort((a, b) => a - b);
const q = (f) => ds[Math.floor(f * (ds.length - 1))];
console.log(`Campaign spans, N = ${ds.length} papers.`);
table(['Statistic', 'Days'], [
  ['minimum', ds[0]], ['25th percentile', q(0.25)], ['median', q(0.5)],
  ['75th percentile', q(0.75)], ['90th percentile', q(0.9)], ['maximum', ds[ds.length - 1]],
]);
const allDs = spanPapers.map((s) => s.days).sort((a, b) => a - b);
const qAll = (f) => allDs[Math.floor(f * (allDs.length - 1))];
console.log(`Counterfactual: if the hand map were skipped and all ${allDs.length} day-precision spans`);
console.log(`were treated as crawl duration, the median would be ${qAll(0.5)} days and the maximum ${allDs[allDs.length - 1]}.\n`);
const BUCKETS = [
  ['1 day', (d) => d <= 1],
  ['2-7 days', (d) => d > 1 && d <= 7],
  ['8-30 days', (d) => d > 7 && d <= 30],
  ['31-90 days', (d) => d > 30 && d <= 90],
  ['91-365 days', (d) => d > 90 && d <= 365],
  ['more than 365 days', (d) => d > 365],
];
table(['Campaign length', 'Papers', `Share of ${ds.length}`],
  BUCKETS.map(([n, f]) => [n, ds.filter(f).length, pct(ds.filter(f).length, ds.length)]));
const over7 = ds.filter((d) => d > 7).length;
const over30 = ds.filter((d) => d > 30).length;
const over90 = ds.filter((d) => d > 90).length;
console.log(`longer than 7 days:  ${over7} of ${ds.length} (${pct(over7, ds.length)})`);
console.log(`longer than 30 days: ${over30} of ${ds.length} (${pct(over30, ds.length)})`);
console.log(`longer than 90 days: ${over90} of ${ds.length} (${pct(over90, ds.length)})\n`);
console.log('Ten longest campaign spans:');
table(['Days', 'Paper', 'Span'], campaigns.slice().sort((a, b) => b.days - a.days).slice(0, 10)
  .map((s) => [s.days, key(s.paper), s.raw]));
console.log('Every excluded span, with the quote that decided it:\n');
for (const r of ['content', 'separated', 'retrieval']) {
  console.log(`-- ${r} --`);
  for (const s of (byRole.get(r) ?? []).sort((a, b) => b.days - a.days)) {
    console.log(`${String(s.days).padStart(5)} ${key(s.paper)}\n      ${s.quote.slice(0, 150)}`);
  }
  console.log('');
}
 
// ------------------------------------------------------------ C. where it runs
console.log('=== C. WHERE THE MEASUREMENT RUNS ===\n');
console.log(`Population: ${crawled.length} crawled papers. vantage[].infrastructure is an enum;`);
console.log('a paper with no vantage[] tuple at all is its own row, not folded into not-stated.\n');
const infra = new Map();
let noVantage = 0;
for (const p of crawled) {
  if (p.vantage.length === 0) { noVantage += 1; continue; }
  for (const v of new Set(p.vantage.map((v) => v.infrastructure))) {
    if (!infra.has(v)) infra.set(v, new Set());
    infra.get(v).add(key(p));
  }
}
const infraRows = [...infra.entries()].sort((a, b) => b[1].size - a[1].size)
  .map(([v, s]) => [v, s.size, pct(s.size, crawled.length)]);
infraRows.push(['(no vantage tuple at all)', noVantage, pct(noVantage, crawled.length)]);
table(['vantage[].infrastructure', 'Papers', 'Share of ' + crawled.length], infraRows);
const statedInfra = new Set();
for (const [v, s] of infra) if (!isSentinel(v)) for (const k of s) statedInfra.add(k);
console.log(`States some non-sentinel infrastructure: ${statedInfra.size} of ${crawled.length} (${pct(statedInfra.size, crawled.length)})`);
const namesNothing = crawled.length - statedInfra.size;
console.log(`Names NOTHING (the complement, and the right figure for "says nothing"): ${namesNothing} of ${crawled.length} (${pct(namesNothing, crawled.length)})`);
console.log(`The not-stated ROW is larger (${(infra.get('not-stated') ?? new Set()).size}) because vantage[] is multi-valued:`);
console.log(`${(infra.get('not-stated') ?? new Set()).size + statedInfra.size - crawled.length + noVantage} papers have both a not-stated tuple and a stated one.`);
console.log('Subtracting a sentinel row from the population is the wrong way to get this.\n');
 
// ------------------------------------------------------ D. the operating gaps
console.log('=== D. WHAT THE PAPERS SAY ABOUT OPERATING THE RUN ===\n');
console.log(`Full-text sweeps over paper.cols.txt, whitespace collapsed, paper-counted.`);
console.log(`Population: ${crawled.length} crawled papers, ${crawled.filter((p) => flat(p) !== null).length} readable.`);
console.log('Every row is an UPPER BOUND on the practice: a match is a mention, not a use.\n');
const PROBES = [
  ['checkpoint / resume a crawl', /\bcheckpoint(ed|ing|s)?\b|\bresum(e|ed|ing)\s+(the\s+)?(crawl|scan|measurement|experiment|collection)\b/i],
  ['a task or job queue by name', /\b(rabbitmq|celery|kafka|beanstalkd?|sidekiq|amazon sqs|\bsqs\b|task queue|job queue|work queue)\b/i],
  ['cluster orchestration by name', /\b(kubernetes|k8s|docker swarm|slurm|nomad|ansible|terraform|htcondor|openstack)\b/i],
  ['a monitoring or logging stack by name', /\b(prometheus|grafana|nagios|zabbix|datadog|kibana|logstash|opensearch dashboards|sentry\.io)\b/i],
  ['a supervisor for a long process', /\b(tmux|nohup|GNU screen|systemd (unit|service|timer)|supervisord|cron ?job|crontab)\b/i],
  ['retrying a failed page or request', /\bretr(y|ies|ied|ying)\b[^.]{0,160}\b(crawl|visit|page|site|domain|url|request|fetch|load)|\b(crawl|visit|page|site|domain|url|request|fetch|load)[^.]{0,160}\bretr(y|ies|ied|ying)\b/i],
  ['a page-load timeout', /\btime[- ]?outs?\b|\btimed out\b/i],
  ['pages or sites that failed to load', /\bfail(ed|ure|ures)?\s+to\s+(load|resolve|respond|connect)\b|\bcould not be (loaded|reached|resolved)\b|\bunreachable\b/i],
];
table(['Probe (full text, crawled = ' + crawled.length + ')', 'Papers', 'Share'],
  PROBES.map(([n, re]) => {
    const { hits } = sweepFlat(crawled, re);
    remember(`probe:${n}`, hits);
    return [n, hits.length, pct(hits.length, crawled.length)];
  }));
 
const QUEUE_WITH_REDIS = /\b(rabbitmq|celery|redis|kafka|beanstalkd?|sidekiq|amazon sqs|\bsqs\b|task queue|job queue|work queue)\b/i;
const QUEUE_NO_REDIS = PROBES.find(([n]) => n.startsWith('a task or job queue'))[1];
const REDIS_ONLY = /\bredis\b/i;
const withRedis = sweepFlat(crawled, QUEUE_WITH_REDIS).hits;
const noRedis = sweepFlat(crawled, QUEUE_NO_REDIS).hits;
const redisOnlySet = new Set(noRedis.map(key));
const redisOnly = withRedis.filter((p) => !redisOnlySet.has(key(p)));
remember('queue:redis-only', redisOnly);
console.log(`Queue probe, with "redis" in the alternation: ${withRedis.length}. Without: ${noRedis.length}.`);
console.log(`${redisOnly.length} papers match ONLY on "redis", which in this corpus is a data store,`);
console.log('a vulnerability target and a studied software corpus far more often than a work queue.');
console.log('The published figure is the narrower one, for the same reason Elasticsearch was');
console.log('dropped from the monitoring probe. --list queue:redis-only prints the excluded papers.\n');
 
console.log('Probe width is load-bearing. The same subject, two widths:\n');
const RETRY_BARE = /\bretr(y|ies|ied|ying)\b/i;
const RETRY_NEAR = PROBES.find(([n]) => n.startsWith('retrying'))[1];
const bare = sweepFlat(crawled, RETRY_BARE).hits.length;
const near = sweepFlat(crawled, RETRY_NEAR).hits.length;
table(['Probe', 'Papers', 'Share of ' + crawled.length], [
  ['the word "retry" anywhere', bare, pct(bare, crawled.length)],
  ['"retry" within 160 chars of a crawl noun', near, pct(near, crawled.length)],
]);
 
// ---------------------------------------------------- D2. attrition reporting
console.log('=== D2. HOW MUCH OF THE LIST YOU ACTUALLY GET ===\n');
const ATTRITION = /\b(successfully (crawl|visit|load|reach|retriev|scrap|analy)[a-z]*|(crawl|visit|load|reach|retriev|scrap)[a-z]*\s+successfully)\b[^.]{0,110}?([0-9][0-9,.]*\s*%|\([0-9.]{1,5}\s?%\)|[0-9][0-9,.]{2,})/i;
const att = sweepFlat(crawled, ATTRITION);
remember('attrition', att.hits);
console.log(`Probe: ${att.hits.length} of ${crawled.length} crawled papers (${pct(att.hits.length, crawled.length)}).`);
console.log('This is an UPPER BOUND and a weak one. A reproducible every-6th sample of the');
console.log('hits was read by hand; the precision for "reports how much of its own list it');
console.log('reached" is printed below. Do not publish the 6.7% as a reporting rate.\n');
const step = Math.max(1, Math.floor(att.hits.length / 12));
const sample = att.hits.filter((_, i) => i % step === 0).slice(0, 12);
// Hand verdicts on that sample, made by reading the matched sentence (see the
// provenance page). true = the sentence reports crawl/collection attrition.
const SAMPLE_VERDICT = {
  'vanity-cracks-and-malware-insights-into-the-anti-copy-protection-ecosystem': true,
  'security-challenges-in-an-increasingly-tangled-web': true,
  'pythia-a-framework-for-the-automated-analysis-of-web-hosting-environments': false,
  'understanding-the-performance-costs-and-benefits-of-privacy-focused-browser-exte': false,
  'u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild': true,
  'leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission': true,
  'minicat-understanding-and-detecting-cross-page-request-forgery-vulnerabilities-i': false,
  'fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting': true,
  'a-practical-attack-to-de-anonymize-social-network-users': false,
  'intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and': true,
  'beyond-exploit-scanning-a-functional-change-driven-approach-to-remote-software-v': false,
  'zelda-feedback-driven-closed-box-fuzzing-for-identifying-web-application-vulnera': false,
};
let ok = 0;
for (const p of sample) {
  if (!(p.slug in SAMPLE_VERDICT)) fail(`attrition sample changed: ${key(p)} has no hand verdict`);
  else if (SAMPLE_VERDICT[p.slug]) ok += 1;
}
for (const s of Object.keys(SAMPLE_VERDICT)) {
  if (!sample.some((p) => p.slug === s)) fail(`attrition sample verdict for a paper no longer in the sample: ${s}`);
}
console.log(`Hand-read sample: ${ok} of ${sample.length} report crawl attrition (${pct(ok, sample.length)} precision).`);
console.log(`Implied order of magnitude: about ${Math.round((att.hits.length * ok) / sample.length)} of ${crawled.length} crawled papers.\n`);
 
// ------------------------------------------------------------ E. interruption
console.log('=== E. PAPERS THAT SAY THEIR OWN RUN LOST TIME ===\n');
const intr = sweepFlat(crawled, CANDIDATE_INTERRUPTION);
console.log(`Candidate sweep: ${intr.hits.length} of ${crawled.length} crawled papers (${pct(intr.hits.length, crawled.length)}).`);
const intrSlugs = new Set(intr.hits.map((p) => p.slug));
for (const p of intr.hits) if (!(p.slug in INTERRUPTION)) fail(`interruption sweep hit with no verdict: ${key(p)}`);
for (const s of Object.keys(INTERRUPTION)) if (!intrSlugs.has(s)) fail(`INTERRUPTION verdict for a paper the sweep does not return: ${s}`);
const intrRoles = new Map();
for (const p of intr.hits) {
  const r = INTERRUPTION[p.slug] ?? 'UNVERDICTED';
  if (!intrRoles.has(r)) intrRoles.set(r, []);
  intrRoles.get(r).push(p);
}
for (const [r, v] of intrRoles) remember(`interruption:${r}`, v);
table(['Verdict', 'Papers', `Share of ${intr.hits.length} hits`, `Share of ${crawled.length} crawled`],
  [...intrRoles.entries()].sort((a, b) => b[1].length - a[1].length)
    .map(([r, v]) => [r, v.length, pct(v.length, intr.hits.length), pct(v.length, crawled.length)]));
console.log('The lost-days papers and the sentence that says so:\n');
for (const p of (intrRoles.get('lost-days') ?? []).sort((a, b) => a.year - b.year)) {
  const t = flat(p);
  const m = new RegExp(CANDIDATE_INTERRUPTION.source, 'i').exec(t);
  console.log(`${key(p)}\n      ...${t.slice(Math.max(0, m.index - 60), m.index + 190)}...\n`);
}
 
// -------------------------------------------------------------- F. concurrency
console.log('=== F. HOW MANY WORKERS AT ONCE ===\n');
const conc = sweepFlat(crawled, CANDIDATE_CONCURRENCY);
console.log(`Candidate sweep: ${conc.hits.length} of ${crawled.length} crawled papers (${pct(conc.hits.length, crawled.length)}).`);
const concSlugs = new Set(conc.hits.map((p) => p.slug));
for (const p of conc.hits) if (!(p.slug in CONCURRENCY)) fail(`concurrency sweep hit with no verdict: ${key(p)}`);
for (const s of Object.keys(CONCURRENCY)) if (!concSlugs.has(s)) fail(`CONCURRENCY verdict for a paper the sweep does not return: ${s}`);
const concRoles = new Map();
for (const p of conc.hits) {
  const r = CONCURRENCY[p.slug] ?? 'UNVERDICTED';
  if (!concRoles.has(r)) concRoles.set(r, []);
  concRoles.get(r).push(p);
}
for (const [r, v] of concRoles) remember(`concurrency:${r}`, v);
table(['Verdict', 'Papers', `Share of ${conc.hits.length} hits`, `Share of ${crawled.length} crawled`],
  [...concRoles.entries()].sort((a, b) => b[1].length - a[1].length)
    .map(([r, v]) => [r, v.length, pct(v.length, conc.hits.length), pct(v.length, crawled.length)]));
console.log('Every crawl-numeric paper with its matched phrase. The candidate regex also');
console.log('matches the number-free phrasing, so display prefers a numeric match where the');
console.log('paper has one (CCS/2014 says "multiple Firefox instances" before it says 30).\n');
const NUMERIC_PHRASE = /\b(?:up to\s+)?\d{1,4}\s*(?:parallel|concurrent|simultaneous)?\s*(?:browser|crawler|instance|worker|thread|container|process|node|machine|vm|virtual machine)[a-z]*\s*(?:in parallel|concurrently|simultaneously|of crawlers in parallel)?/i;
for (const p of (concRoles.get('crawl-numeric') ?? []).sort((a, b) => a.year - b.year)) {
  const t = flat(p);
  const numeric = new RegExp('\\b(?:up to\\s+)?\\d{1,4}\\s*(?:parallel|concurrent|simultaneous)\\s+(?:browser|crawler|instance|worker|thread|container|process|node|machine|vm)[a-z]*|\\brun[a-z]*\\s+(?:up to\\s+)?\\d{1,4}\\s+(?:browser|crawler|instance|worker|thread|container|process)[a-z]*', 'i').exec(t);
  const m = numeric ?? new RegExp(CANDIDATE_CONCURRENCY.source, 'i').exec(t);
  console.log(`${key(p)}\n      ...${t.slice(Math.max(0, m.index - 70), m.index + 150)}...\n`);
}
 
// -------------------------------------------------------------------- G. cost
console.log('=== G. WHAT THE MEASUREMENT COST ===\n');
const cost = sweepFlat(crawled, CANDIDATE_COST);
console.log(`Candidate sweep: ${cost.hits.length} of ${crawled.length} crawled papers (${pct(cost.hits.length, crawled.length)}).`);
const costSlugs = new Set(cost.hits.map((p) => p.slug));
for (const p of cost.hits) if (!(p.slug in OWN_COST)) fail(`cost sweep hit with no verdict: ${key(p)}`);
for (const s of Object.keys(OWN_COST)) if (!costSlugs.has(s)) fail(`OWN_COST verdict for a paper the sweep does not return: ${s}`);
const costRoles = new Map();
for (const p of cost.hits) {
  const r = OWN_COST[p.slug] ?? 'UNVERDICTED';
  if (!costRoles.has(r)) costRoles.set(r, []);
  costRoles.get(r).push(p);
}
for (const [r, v] of costRoles) remember(`cost:${r}`, v);
table(['Verdict', 'Papers', `Share of ${crawled.length} crawled`],
  [...costRoles.entries()].sort((a, b) => b[1].length - a[1].length)
    .map(([r, v]) => [r, v.length, pct(v.length, crawled.length)]));
const own = [...(costRoles.get('own-infrastructure') ?? []), ...(costRoles.get('own-data') ?? [])];
console.log(`States a cost the authors themselves paid: ${own.length} of ${crawled.length} (${pct(own.length, crawled.length)})`);
console.log(`States an infrastructure bill: ${(costRoles.get('own-infrastructure') ?? []).length} of ${crawled.length} (${pct((costRoles.get('own-infrastructure') ?? []).length, crawled.length)})\n`);
for (const p of own.sort((a, b) => a.year - b.year)) {
  const t = flat(p);
  const m = new RegExp(CANDIDATE_COST.source, 'i').exec(t);
  console.log(`${key(p)} [${OWN_COST[p.slug]}]\n      ...${t.slice(Math.max(0, m.index - 60), m.index + 200)}...\n`);
}
 
// --------------------------------------------------------------- H. arithmetic
console.log('=== Y. FIGURES THE PAGE DERIVES BY ARITHMETIC ===\n');
const conc29 = (concRoles.get('crawl-numeric') ?? []).length;
table(['Figure', 'From'], [
  [`${ds.length} campaign papers of ${spanPapers.length} day-precision spans = ${pct(ds.length, spanPapers.length)}`, 'section B'],
  [`${over7} of ${ds.length} campaigns longer than a week = ${pct(over7, ds.length)}`, 'section B'],
  [`${(intrRoles.get('lost-days') ?? []).length} lost-days of ${crawled.length} crawled = ${pct((intrRoles.get('lost-days') ?? []).length, crawled.length)}`, 'section E'],
  [`${conc29} crawl-numeric of ${crawled.length} crawled = ${pct(conc29, crawled.length)}`, 'section F'],
]);
 
if (LIST) {
  console.log(`=== LIST ${LIST} ===`);
  const l = LISTS.get(LIST);
  if (!l) { console.log(`no such list. available: ${[...LISTS.keys()].join(' | ')}`); }
  else l.forEach((k) => console.log(k));
  console.log('');
}
if (QUOTES) {
  const re = new RegExp(QUOTES, 'i');
  console.log(`=== QUOTES matching /${QUOTES}/i ===`);
  for (const p of crawled) {
    const t = flat(p);
    if (t === null) continue;
    const g = new RegExp(re.source, 'gi');
    let m, shown = 0;
    while ((m = g.exec(t)) !== null && shown < 2) {
      console.log(`${key(p)}\n      ...${t.slice(Math.max(0, m.index - 120), m.index + 220)}...`);
      shown += 1;
    }
  }
  console.log('');
}
 
console.log(failures === 0 ? 'OK: all hand maps agree with their sweeps.' : `FAILURES: ${failures}`);
process.exit(failures === 0 ? 0 : 1);

verify_deployment_figures.mjs

#!/usr/bin/env node
// Every literal per-paper figure and quoted sentence on programming:deployment,
// checked against the paper it is attributed to. Whitespace is collapsed on both
// sides, because a quote that spans a column break does not grep otherwise.
// Needles shorter than 20 characters are flagged: they can match by accident.
//
// Also carries the blocks check_page_numbers.mjs needs so it can run whole-page:
// EXTERNAL (numbers from outside the corpus, with their source) and NOT_FIGURES
// (digit fragments that are not figures).
 
import fs from 'node:fs';
import path from 'node:path';
import { spawnSync } from 'node:child_process';
import { dataRoot } from './lib.mjs';
 
const NEEDLES = [
  // --- the run lost time ---
  ['CCS/2014/a-nearly-four-year-longitudinal-study-of-search-engine-poisoning', 'Out of the 1 254 days in the measurement period, we have complete measurements for 1 004 days'],
  ['IMC/2016/anatomy-of-a-personalized-livestreaming-system', 'Our crawler was interrupted briefly between August 7-9, 2015 due to a communication bug with the Periscope server'],
  ['IMC/2016/anatomy-of-a-personalized-livestreaming-system', 'our dataset is missing roughly 4.5% of the broadcasts'],
  ['IEEE-SP/2018/the-spyware-used-in-intimate-partner-violence', 'Scans were not performed on Nov 07 and Nov 08 due to a power failure'],
  ['WWW/2021/tls-1-3-in-practice-how-tls-1-3-contributes-to-the-internet', 'There were network outages for 17 days, which are pruned out from the dataset'],
  ['WWW/2021/tls-1-3-in-practice-how-tls-1-3-contributes-to-the-internet', 'around 84% of the websites were consistently collected'],
  ['WWW/2013/two-years-of-short-urls-internet-measurement-security-threats-and-countermeasure', 'Our system experienced three outages due to database failures throughout one year'],
  ['IMC/2018/digging-into-browser-based-crypto-mining', 'Black parts mark outages of our infrastructure'],
  ['USENIX/2015/vulnerability-disclosure-in-the-age-of-social-media-exploiting-twitter-for-predi', 'except during the periods of downtime for our infrastructure'],
  ['NDSS/2015/seven-months-worth-of-mistakes-a-longitudinal-study-of-typosquatting-abuse', 'to accommodate for a planned power interruption of our crawling machines on August 27 and 28'],
  ['WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion', 'Our crawler crashed on March 12th in both Ukraine and Russia, and again on March 20th and April 18th in Ukraine'],
  ['IMC/2021/who-you-gonna-call-an-empirical-evaluation-of-website-security-txt-deployment', 'we had a 1.5-month measurement interruption throughout November and the first half of December, 2020'],
  ['NDSS/2017/dial-one-for-scam-a-large-scale-analysis-of-technical-support-scams', 'none of the three ROBOVIC instances experienced any downtime during our monitored period'],
 
  // --- retries and attrition ---
  ['USENIX/2025/websites-global-privacy-control-compliance-at-scale-and-over-time', 'nearly 40% of sites that failed to load on the first attempt were loaded and successfully analyzed'],
  ['USENIX/2025/websites-global-privacy-control-compliance-at-scale-and-over-time', 'a site failed to load in 35 seconds, had an insecure certificate, or led to an error page'],
  ['WWW/2021/websocket-adoption-and-the-landscape-of-the-real-time-web', 'After retrying each failed crawl a second time, we obtained data for a total of 88.1% of websites in the top million'],
  ['WWW/2021/websocket-adoption-and-the-landscape-of-the-real-time-web', '7.7% of the listed domain names failed to resolve, and the remaining 4.2% had web servers which failed to respond'],
  ['USENIX/2021/u-cant-debug-this-detecting-javascript-anti-debugging-techniques-in-the-wild', 'In another 4%, the server returned an HTTP error code and the remaining 3% failed to load before our 30 seconds timeout hit'],
  ['NDSS/2024/fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting', 'Out of the 20k websites, we successfully visit 18,300 (91.5%)'],
  ['NDSS/2024/fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting', 'the overwhelming majority (64.3%) of them due to HTTP 403 Forbidden errors'],
  ['PETS/2025/tracker-installations-are-not-created-equal-understanding-tracker-configuration', 'Out of the 42,481 websites visited, we visited 40,150 (94.51%) successfully'],
  ['PETS/2024/a-large-scale-study-of-cookie-banner-interaction-tools-and-their-impact-on-users', 'we successfully crawled 29,660 (99%) sites'],
  ['WWW/2017/security-challenges-in-an-increasingly-tangled-web', '15K domains did not resolve, 13K timed out'],
  ['PETS/2017/towards-seamless-tracking-free-web-improved-detection-of-trackers-via-one-class', 'we reran the crawler on the webpage by increasing the waiting time until the webpage is rendered'],
  ['IEEE-SP/2024/the-great-request-robbery-an-empirical-study-of-client-side-request-hijacking-vu', 'we made up to three repeated attempts for each failed crawling'],
  ['USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission', 'This minor change increased the successfully visited websites from 94, 427 (EU pilot crawl) to 99, 380 (EU final crawl)'],
  // NOTE: paper.cols.txt renders those two numbers with a space after the comma
  // ("94, 427"). The page quotes the PDF form; see PDF_ROUTE below.
  ['WWW/2017/security-challenges-in-an-increasingly-tangled-web', 'page for 944,000 sites'],
  ['NDSS/2024/fp-fed-privacy-preserving-federated-detection-of-browser-fingerprinting', 'Out of the 20k websites'],
  ['IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users', 'the dictionary consisted of 7.1GB of HTML data'],
  ['PETS/2025/intractable-cookie-crumbs-unveiling-the-nexus-of-stateful-banner-interaction-and', '16,296 pages were successfully loaded, 692 triggered'],
 
  // --- concurrency and throughput ---
  ['CCS/2016/online-tracking-a-1-million-site-measurement-and-analysis', 'which currently allocates 8 vCPUs and 15 GiB of memory per machine. With this configuration we are able to run 20 browser instances in parallel'],
  ['USENIX/2024/automated-large-scale-analysis-of-cookie-notice-compliance', 'This 16-core CPU allows us to run 30 OpenWPM browsers in parallel, achieving a crawling speed of 300 websites per hour'],
  ['CCS/2014/the-web-never-forgets-persistent-tracking-mechanisms-in-the-wild', 'we were able to run up to 30 browsers in parallel on a 4-core 8GB desktop machine'],
  ['USENIX/2019/inadvertently-making-cyber-criminals-rich-a-comprehensive-study-of-cryptojacking', 'We deployed the crawler in Docker containers on 60 servers within the university network, each running 8 Docker instances in parallel'],
  ['CCS/2018/minesweeper-an-in-depth-look-into-drive-by-cryptocurrency-mining-and-its-defense', 'We ran 50 Docker containers in parallel for one week'],
  ['IEEE-SP/2023/the-leaky-web-automated-discovery-of-cross-site-information-leaks-in-browsers-an', '13 days for Chromium and Firefox on a Linux server with 100 browsers in parallel'],
  ['PETS/2020/in-depth-evaluation-of-redirect-tracking-and-link-usage', 'execute them in individual Docker containers in parallel so that a problem in one of the tasks'],
  ['IMC/2025/a-permissions-odyssey-a-systematic-study-of-browser-permissions-on-modern-websit', 'utilizing 40 parallel crawlers'],
 
  // --- cost ---
  ['IEEE-SP/2011/design-and-evaluation-of-a-real-time-url-spam-filtering-service', 'URL aggregation 1 Extra Large $178 Feature collection 20 High-CPU Medium $882'],
  ['IEEE-SP/2011/design-and-evaluation-of-a-real-time-url-spam-filtering-service', 'Storage 700GB on EBS $70 Total $1,587'],
  ['CCS/2016/online-tracking-a-1-million-site-measurement-and-analysis', 'This virtual machine costs around $300 per month using price estimates from May 2016'],
  ['IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users', 'The crawling service cost us $18.47'],
  ['IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users', 'The costs for the crawling were $6.57'],
  ['IEEE-SP/2010/a-practical-attack-to-de-anonymize-social-network-users', 'we estimate overall costs of about $88 for crawling all 40'],
  ['USENIX/2022/lend-me-your-ear-passive-remote-physical-side-channels-on-pcs', 'at a total computational cost of less than $1.3'],
  ['USENIX/2023/calpric-inclusive-and-fine-grain-labeling-of-privacy-policies-with-crowdsourcing', 'reported a cost of $60 to label each privacy policy. Our system is able to further reduce the average cost of $13.5'],
  ['IEEE-SP/2023/deepfake-text-detection-limitations-and-opportunities', 'We spent $586 to collect the articles from the services'],
  ['IMC/2021/knock-and-talk-investigating-local-network-communications-on-websites', 'Nearly 90% of the failures were due to DNS resolution errors (NAME_NOT_RESOLVED)'],
  ['WWW/2023/online-advertising-in-ukraine-and-russia-during-the-2022-russian-invasion', 'and again on March 20th and April 18th in Ukraine after running for some time'],
  ['CCS/2014/a-nearly-four-year-longitudinal-study-of-search-engine-poisoning', 'complete measurements for 1 004 days'],
  ['CCS/2023/under-the-dark-a-systematical-study-of-stealthy-mining-pools-ab-use-in-the-wild', 'our overall purchase cost is $44.538 for one month'],
  ['PETS/2026/redefining-website-fingerprinting-attacks-with-multi-agent-llms', 'acquiring 1GB of authentic human browsing data costs approximately $35'],
];
 
// Numbers on the page that do not come from the corpus. Each needs a source.
// Numbers the page derives by arithmetic from figures that are verified above.
const ARITHMETIC = [
  ['3,300 machine-hours', '1,000,000 sites / 300 sites per machine-hour = 3,333, rounded to 3,300'],
  ['5 days on 28 machines', '3,300 machine-hours / (24 h * 5 d) = 27.5, rounded to 28'],
  ['a month on four machines', '3,300 / (24 * 30) = 4.6, so four machines is about five weeks; stated as "a month on four"'],
  ['about 10 sites per browser-hour', '300 sites per hour / 30 parallel browsers = 10'],
  ['7,200 sites a day per machine', '300 sites per hour * 24 hours'],
  ['USD 2,700 for one pass', '3,300 machine-hours * USD 0.8148/hour (c7i.4xlarge) = 2,689, rounded to 2,700'],
  ['480 concurrent (Bijmans et al.)', '60 servers * 8 Docker instances each'],
  ['84.6% (Musch and Johns)', 'about 846k sites of the Tranco top 1M = 84.6%. The paper states the numerator and names the list; the percentage is ours. It is also the sum of its own failure classes: ~8% network/DNS + 4% HTTP error + 3% timeout = 15%.'],
  ['the 85-94% range', 'the low end is Musch and Johns at 84.6%, rounded to 85; the high end is Kumar et al. at 944,000/1,000,000 = 94.4%, rounded to 94. Both endpoints are in NEEDLES.'],
  ['the 90-99% range', 'the low end is Kuchhal and Li at ~90%; the high end is Demir et al. at 99%. Both in NEEDLES.'],
  ['$88 to crawl all 40 million profiles', "Wondracek et al.'s own linear-cost estimate from their $6.57 LinkedIn crawl, quoted from the paper. It is NOT derived from the $18.47 Facebook-groups figure; a reviewer caught the page implying it was."],
  ['1,004 of the 1,254 days', 'the page writes Leontiadis et al.\'s figures with thousands commas; the paper writes "1 004" and "1 254" with thin spaces'],
];
 
const EXTERNAL = [
  ['431 MB', 'Content-Length of the eu-central-1 EC2 offer file on 2026-08-28: 431,220,398 bytes'],
  ['0.4074', 'AWS EC2 on-demand USD/hour, c7i.2xlarge, eu-central-1, Linux, shared tenancy, bulk price list version 20260828175757, fetched 2026-08-28 by scripts/ec2_price.py'],
  ['0.8148', 'same source, c7i.4xlarge'],
  ['7', 'venue count of this corpus'],
  ['64', 'Docker default /dev/shm in MiB, documented on programming:docker'],
];
 
const NOT_FIGURES = [
  ['1.62', 'Playwright image tag on programming:docker, not a figure here'],
  ['300', 'sites per hour, a corpus figure (Bouhoula et al.), listed in NEEDLES'],
  ['300.0', 'crawl_queue.py default lease_seconds, a code literal'],
  ['404', 'HTTP status in the error-class names http_404 / TERMINAL_ERRORS'],
  ['410', 'HTTP status in TERMINAL_ERRORS'],
  ['403', 'HTTP status; the Forbidden figure itself is in NEEDLES'],
  ['200', 'crawl_queue.py demo fixture size, printed in the quoted demo output'],
  ['87', 'crawl_queue.py demo crash point, printed in the quoted demo output'],
  ['256', 'sha256 in the seed fingerprint'],
  ['9', 'the c5.9xlarge instance name in the Genkin et al. row'],
  ['1.3', 'Genkin et al.: USD, in NEEDLES'],
  ['13.5', 'Qiu et al.: USD per policy, in NEEDLES'],
  ['586', 'Pu et al.: USD, in NEEDLES'],
  ['6.57', 'Wondracek et al.: USD, in NEEDLES'],
  ['20', 'crawl_queue.py demo ordering fixture size, printed in the quoted demo output'],
  ['301', 'crawl_queue.py demo clock advance in seconds, printed in the quoted demo output'],
  ['301.0', 'the same value as a float literal in the demo source'],
  ['1000000', 'crawl_queue.py demo fake clock origin'],
];
 
const root = dataRoot();
let exact = 0, missing = 0, weak = 0;
const fails = [];
for (const [k, needle] of NEEDLES) {
  const [venue, year, slug] = k.split('/');
  const f = path.join(root, 'fulltext', year, venue, slug, 'paper.cols.txt');
  if (!fs.existsSync(f)) { fails.push(`${k}: NO paper.cols.txt`); missing += 1; continue; }
  const text = fs.readFileSync(f, 'latin1').replace(/\s+/g, ' ');
  const flatNeedle = needle.replace(/\s+/g, ' ');
  if (flatNeedle.length < 20) { console.log(`WEAK NEEDLE (${flatNeedle.length} chars): ${k} :: ${flatNeedle}`); weak += 1; }
  if (text.includes(flatNeedle)) { exact += 1; console.log(`ok   ${k}\n     ${flatNeedle}`); }
  else { fails.push(`${k}\n      NOT FOUND: ${flatNeedle}`); missing += 1; }
}
console.log(`\n${NEEDLES.length} needles: ${exact} found verbatim (whitespace collapsed), ${missing} not found, ${weak} flagged weak.`);
if (fails.length) { console.log('\n--- failures ---'); fails.forEach((f) => console.log(f)); }
 
console.log('\n--- PDF ROUTE (run against paper.pdf with pypdf, not paper.cols.txt) ---');
const PDF_NEEDLES = [
  ['USENIX/2022/leaky-forms-a-study-of-email-and-password-exfiltration-before-form-submission',
   'increased the successfully visited websites from 94,427 (EU pilot crawl) to 99,380 (EU final crawl)'],
];
for (const [k, needle] of PDF_NEEDLES) {
  const [venue, year, slug] = k.split('/');
  const pdf = path.join(root, 'fulltext', year, venue, slug, 'paper.pdf');
  const py = [
    'import sys, re, pypdf',
    'r = pypdf.PdfReader(sys.argv[1])',
    "t = ' '.join(p.extract_text() or '' for p in r.pages)",
    "for a, b in (('\\ufb00','ff'),('\\ufb01','fi'),('\\ufb02','fl'),('\\ufb03','ffi'),('\\ufb04','ffl')): t = t.replace(a, b)",
    "t = re.sub(r'\\s+', ' ', t)",
    'print("FOUND" if sys.argv[2] in t else "NOT FOUND")',
  ].join('\n');
  const res = spawnSync('python3', ['-c', py, pdf, needle], { encoding: 'utf8' });
  const verdict = res.status === 0 ? res.stdout.trim() : `ERROR ${(res.stderr || '').trim().split('\n').pop()}`;
  console.log(`${verdict}  ${k}\n     ${needle}`);
  console.log('     paper.cols.txt renders these as "94, 427" and "99, 380"; the PDF has no space.');
  console.log('     The PDF sets "final" with an fi ligature; the check de-ligatures before matching,');
  console.log('     and the page quotes the de-ligatured form.');
  if (verdict !== 'FOUND') { missing += 1; fails.push(`${k}: PDF route ${verdict}`); }
}
 
console.log('\n--- ARITHMETIC ---');
for (const [n, how] of ARITHMETIC) console.log(`${n}  ${how}`);
 
console.log('\n--- EXTERNAL FIGURES (not from the corpus) ---');
for (const [n, src] of EXTERNAL) console.log(`${n}  ${src}`);
console.log('\n--- NOT FIGURES ---');
for (const [n, why] of NOT_FIGURES) console.log(`${n}  ${why}`);
 
process.exit(missing === 0 ? 0 : 1);

ec2_price.py

#!/usr/bin/env python3
"""Pull a few EC2 on-demand Linux rates out of the AWS bulk price list.
 
The eu-central-1 EC2 offer file is ~431 MB, so this streams it and never holds
it in memory. Two passes: one to find the SKUs whose product attributes match
the instance types we want, one to read those SKUs' OnDemand USD/hour.
 
    python3 scripts/ec2_price.py c7i.2xlarge c7i.4xlarge m7i.2xlarge
"""
import json, re, sys, urllib.request
 
REGION = "eu-central-1"
IDX = "https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/region_index.json"
 
def offer_url():
    with urllib.request.urlopen(IDX, timeout=60) as r:
        d = json.load(r)
    return "https://pricing.us-east-1.amazonaws.com" + d["regions"][REGION]["currentVersionUrl"]
 
def main():
    wanted = set(sys.argv[1:]) or {"c7i.2xlarge"}
    url = offer_url()
    print(f"offer file: {url}")
    prod_re = re.compile(rb'"sku"\s*:\s*"([A-Z0-9]+)"')
    # pass 1: products
    skus = {}
    buf = b""
    with urllib.request.urlopen(url, timeout=900) as r:
        # products come first; stop once we reach "terms"
        while True:
            chunk = r.read(1 << 20)
            if not chunk:
                break
            buf += chunk
            # split on product object boundaries
            while True:
                i = buf.find(b'},\n    "')
                if i < 0:
                    break
                obj, buf = buf[:i + 1], buf[i + 2:]
                j = obj.rfind(b'{"product')
                m = re.search(rb'"instanceType"\s*:\s*"([a-z0-9.]+)"', obj)
                if not m:
                    continue
                it = m.group(1).decode()
                if it not in wanted:
                    continue
                if b'"tenancy" : "Shared"' not in obj and b'"tenancy":"Shared"' not in obj:
                    continue
                if b'"operatingSystem" : "Linux"' not in obj and b'"operatingSystem":"Linux"' not in obj:
                    continue
                if b'"preInstalledSw" : "NA"' not in obj and b'"preInstalledSw":"NA"' not in obj:
                    continue
                if b'"capacitystatus" : "Used"' not in obj and b'"capacitystatus":"Used"' not in obj:
                    continue
                s = prod_re.search(obj)
                if s:
                    skus[s.group(1).decode()] = it
            if b'"terms"' in buf and skus:
                break
            if len(buf) > (8 << 20):
                buf = buf[-(1 << 20):]
        print(f"matched SKUs: {skus}")
        # pass 2: keep streaming, look for OnDemand price dims for those SKUs
        found = {}
        tail = buf
        pat = {sku: re.compile((sku + r'\.[A-Z0-9]+\.[A-Z0-9]+"[\s\S]{0,900}?"USD"\s*:\s*"([0-9.]+)"').encode())
               for sku in skus}
        while skus and len(found) < len(skus):
            chunk = r.read(1 << 20)
            if not chunk:
                break
            tail += chunk
            for sku, p in pat.items():
                if sku in found:
                    continue
                m = p.search(tail)
                if m:
                    found[sku] = float(m.group(1))
            if len(tail) > (4 << 20):
                tail = tail[-(1 << 20):]
    for sku, it in skus.items():
        print(f"{it:16s} {REGION} Linux on-demand shared: USD {found.get(sku, 'NOT FOUND')}/hour")
 
if __name__ == "__main__":
    main()

21 entries, appended before the closing </bibtex> of a fresh export of bibliography (never the workdir copy, which goes stale) — 18 with the first save and 3 more when review round 1 turned the cost table from five rows into eight. Checked before each save: no key collision with the existing entries, no DOI collision, no title collision. The page ended with 679 entries.

agten2015_seven
bijmans2019_inadvertently
chatterjee2018_spyware
dahlberg2023_timeless
genkin2022_lend
konoth2018_minesweeper
koop2020_redirect
kuchhal2021_knock
lee2021_practice
leontiadis2014_nearly
maggi2013_years
miramirkhani2017_dial
murley2021_websocket
pu2023_deepfake
qiu2023_calpric
rueth2018_digging
sabottke2015_vulnerability
thomas2011_design
wang2016_anatomy
wondracek2010_practical
zhang2023_under

Authors for six entries are not in the venue index (PoPETs and USENIX records carry no authors) and were fetched from the landing pages with curl and a browser User-Agent:

sabottke2015_vulnerability   Carl Sabottke, Octavian Suciu, and Tudor Dumitras, University of Maryland
bijmans2019_inadvertently    Hugo L.J. Bijmans, Tim M. Booij, and Christian Doerr, Delft University of Technology
dahlberg2023_timeless        Rasmus Dahlberg and Tobias Pulls, Karlstad University
koop2020_redirect            Martin Koop (Universitat Passau), Erik Tews (University of Twente), Stefan Katzenbeisser (Universitat Passau)
genkin2022_lend              Daniel Genkin, Georgia Tech; Noam Nissan, Tel Aviv University; Roei Schuster, Tel Aviv University and Cornell Tech; Eran Tromer, Tel Aviv University and Columbia University
qiu2023_calpric              Wenjun Qiu, David Lie, and Lisa Austin, University of Toronto

Three keys bibgen.mjs produced were wrong and were corrected by hand: rth2018_digging (diacritic stripping ate “Rüth” down to “rth”) became rueth2018_digging, and the Under the Dark and Timeless Timing entries were initially keyed on the wrong author — the CCS 2023 paper's first author is Zhang, not Duan, and the USENIX 2023 paper's is Dahlberg. The page's attributions were changed to match. bibgen.mjs also emitted Piessensand, Frank for the NDSS 2015 entry, a venue-page scraping artefact for “Piessens and”; corrected to Piessens. The same entry's title lost its possessive apostrophe — Seven Months' Worth of Mistakes — because bibgen.mjs emitted a typographic apostrophe and it was stripped rather than replaced; a reviewer caught it against the live NDSS programme page and a straight apostrophe was restored.

Four passes. The three focused ones ran in parallel against a frozen snapshot (out/dep/frozen_page.txt, the page as saved at rev 1787948833) and each was handed the page text, the report script, dep_fold.mjs, the verifier, the unedited combined output and the published Python. All four were told explicitly that the author's context might not be exhaustive. Findings were applied only after the focused three returned; the generic pass then read the corrected page.

Eight corrections made by re-reading the frozen text against this site's own rule that a probe-scoped negative must stay scoped in the prominent places. All eight were places where a careful sentence in one section had lost its qualifier in the opening box or a section lead.

Everything else on the external pass checked out and is recorded here because the checks are the value:

• AWS bulk price list re-fetched independently: offer version 20260828175757, c7i.2xlarge USD 0.4074/hour at 8 vCPU, c7i.4xlarge USD 0.8148/hour at 16 vCPU, Content-Length: 431220398. vCPU counts cross-checked against aws.amazon.com/ec2/instance-types/c7i/. All four page figures confirmed.
• All 13 internal link targets fetched and confirmed by content, not status code — ?do=export_raw returns HTTP 200 and a “create this page” body for a missing page. provenance:programming:deployment was the only miss, as expected.
• Every named tool in the probe lists checked for death or rename. None is dead. Alexa appears exactly once, correctly framed as what a 2017 paper used; Tranco is still publishing (list of 23 August 2026 live) and OpenWPM's repository is not archived. Two licence changes the page does not depend on and does not claim: Redis relicensed in 2024 and again in 2025, Nomad moved to BUSL in 2023.
• pages/crawl_queue.py runs on this host's Python 3.11.2 and its output matches the published block byte for byte. isolation_level and check_same_thread are passed as keyword arguments, so the Python 3.13 deprecation of their positional forms does not affect it; the code is unchanged-safe from 3.11 to the current 3.14.

Clean on this pass: all 35 citekeys resolved; no duplicate key definitions; none of the 18 new papers already present under another key by DOI or by fuzzy title; every DOI resolved through Crossref to the matching title; the four hand-scraped author lists confirmed against the live landing pages; every numerator/denominator pair in the three tables traced to the same experiment in its source.

Clean on this pass: sections B, C, D, D2, E and F re-derived from a live re-run; all 44 needles at the time; every arithmetic derivation except F5.

Reviewer calibration for round 1. The three focused passes returned 14 items: E1, C1–C7, F1–F6. Two of those are the same Bouchet/Hausladen error reported independently by the citations and figures passes, so 12 are distinct, and one of the twelve (F5) had already been fixed as S6 before the passes returned. All 12 accepted, none rejected. Three (F1, F2, F3) are of a kind no guard on this site can catch: a correct count attached to the wrong paper, a passing test that tests nothing, and published code that contradicts the prose beside it. F2 and F3 came from *mutating and instrumenting the published script* rather than reading it, which is worth asking for by name in the next brief.

Ran against the page as corrected in round 1 (rev 1787949615) and against this log. Fifteen findings, all accepted. Three of them would have hurt the reader the page is written for.

Reviewer calibration across the whole run. 29 reported items across four passes — 14 in round 1 and 15 in round 2 — of which 27 are distinct and all 27 were accepted; none was rejected. Round 0's eight self-review corrections are not counted here; they are not reviewer findings. The three focused passes found 12 distinct, of which two (F2, F3) came from mutating and instrumenting the published script rather than reading it. The generic pass then found 15 more on the already-corrected page, including the three worst defects in the run — a double-claim race in code the page tells you to run at 100-way concurrency, an operational threshold contradicted by the table beneath it, and a probe inflated 2x by a term the log had congratulated itself for catching elsewhere. That is the same pattern the traffic-files run recorded: the focused passes look comprehensive and the unbriefed one finds the things that are only wrong where two sections meet. Do not skip it, and ask the next focused pass to mutate the published code rather than read it.

• The page this log is for
• Corpus-level provenance — venue scope, selection funnel, field stability
• interaction and sampling — neighbouring logs

[1]

Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)

[2]

Acar, Gunes; Eubank, Christian; Englehardt, Steven; Juarez, Marc; Narayanan, Arvind; Díaz, Claudia (2014): "The Web Never Forgets: Persistent Tracking Mechanisms in the Wild", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)

[3]

Senol, Asuman; Acar, Gunes; Humbert, Mathias; Zuiderveen Borgesius, Frederik (2022): "Leaky Forms: A Study of Email and Password Exfiltration Before Form Submission", in: 31st USENIX Security Symposium (USENIX Security 22). (Link)

[4]

Annamalai, Meenatchi Sundaram Muthu Selva; Bilogrevic, Igor; Cristofaro, Emiliano De (2024): "FP-Fed: Privacy-Preserving Federated Detection of Browser Fingerprinting", in: Proceedings of the Network and Distributed System Security Symposium. (Link)

[5]

Demir, Nurullah; Urban, Tobias; Pohlmann, Norbert; Wressnegger, Christian (2024): "A Large-Scale Study of Cookie Banner Interaction Tools and their Impact on Users' Privacy", in: Proceedings on Privacy Enhancing Technologies, pp. 5-20. (DOI)

[6]

Bui, Duc; Tang, Brian; Shin, Kang G. (2023): "Detection of Inconsistencies in Privacy Practices of Browser Extensions", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[7]

Bouhoula, Ahmed; Kubicek, Karel; Zac, Amit; Cotrini, Carlos; Basin, David (2024): "Automated Large-Scale Analysis of Cookie Notice Compliance", in: Proceedings of the USENIX Security Symposium. (Link)

[8]

Hausladen, Katherine; Wang, Oliver; Eng, Sophie; Wang, Jocelyn; Wijaya, Francisca; May, Matthew; Zimmeck, Sebastian (2025): "Websites' Global Privacy Control Compliance at Scale and over Time", in: Proceedings of the USENIX Security Symposium. (Link)

[9]

Genkin, Daniel; Nissan, Noam; Schuster, Roei; Tromer, Eran (2022): "Lend Me Your Ear: Passive Remote Physical Side Channels on PCs", in: Proceedings of the USENIX Security Symposium. (Link)

[10]

Wondracek, Gilbert; Holz, Thorsten; Kirda, Engin; Kruegel, Christopher (2010): "A Practical Attack to De-anonymize Social Network Users", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)