provenance:design:automated_measurements
Table of Contents
Provenance: design:automated_measurements
Working log behind automated_measurements. Corpus-wide caveats are on corpus. Citations use the shared bibliography; this page adds no keys of its own.
Run: 2026-08-27. Corpus: 5,859 extracted papers, 7 venues, 2010–2026, data/extract/run1. Item: drain wiki-measuretheweb / design:automated_measurements (new), claimed as cursor-drain-automated-measurements. Author of the page and this log: Cursor, not Claude Code. Reviewers: three focused passes plus one generic pass, all GPT 5.6 Luna medium (gpt-5.6-luna-medium), not fable/sonnet.
Creating, not extending. node scripts/dw.mjs info design:automated_measurements returned “the requested page does not exist”. ?do=export_raw returned the HTML error page (not an existence test by byte count). dw.mjs pages does not list the provenance: namespace. The start page already red-linked Automated measurements. Neighbouring pages already pointed at it: crawler (“whether to crawl at all”), interaction, hypothesis_testing, regression, study_preregistration, legal_enforcement.
No ~~DISCUSSION~~ on this provenance page. Comments belong on the content page.
Scope decision
The item: one of two top-level Design branches promised by start; orient a reader between crawling, scanning and app analysis rather than duplicate the specialised pages; ordered after those pages so it can route to them.
| Decision | Why | What a reasonable person might have done instead |
|---|---|---|
| A routing page, not a third crawler comparison | The specialised pages exist. Duplicating Selenium-vs-Playwright here would drift from crawler within a sitting. | A 60 kB methods page. Rejected: the item said not to. |
Three branches from studyTypes, plus the crawled=1120 gap as a finding | The item named the three types. The site's other crawl pages use POPULATIONS.crawled (1,120). Hiding the 153-paper gap would make this page contradict its children. | Silently use 1,120 for “crawl” and never mention 967. |
Rankings, not precise percentages, for every studyTypes share | 57% run-to-run on a 100-paper sample, not re-measured on 5,859. | Publish “16.5% of papers crawled” as a fact. |
| Did not write user_studies | Separate item. 234 of 2,198 union papers are also user-study; that is noted so the fork is visible. | Pad this page with Mechanical Turk tables. |
| Did not add a published downloadable script | No measurement unique to this page needed code a reader would run. The decision is which child page to open. | A toy “which branch am I in” classifier. |
| Cited ZMap 2013 even though it is missing from the extraction | It is the instrument paper. Ten Years of ZMap is in-corpus and says so. The USENIX page was fetched 2026-08-27 (HTTP 200, title contains ZMap). | Pretend Ten Years is the original, or skip the citation. |
| Did not interpret Censys 5/168 in 2025–2026* as a decline | Provisional year. 2018–2024 is ~10%. | A “Censys is dying” sentence. |
| Did not claim Playwright has replaced Selenium | Playwright is 16/169 in 2025–2026*; Selenium is 44/169. Currency is “Playwright exists and is the one to start with”; the literature still names Selenium most. | A “the field has moved on” sentence the tables contradict. |
Report script
scripts/report_automated_measurements.mjs plus scripts/am_fold.mjs (scanner-name fold, self-tests on import). Re-run:
node scripts/report_automated_measurements.mjs > scripts/report_automated_measurements-output.txt node scripts/report_automated_measurements.mjs --quotes node scripts/report_automated_measurements.mjs --list extra bash scripts/external_checks_automated_measurements.sh
Unknown flags exit 1 (–bogus was tested). –list requires crawl|scan|app|extra|union. –quotes exits 1 if a published quote is missing from whitespace-normalised paper.cols.txt. The script exits 1 if corpus ≠ 5859, crawled ≠ 1120, the three OVERVIEW.md type counts disagree, missing .cols ≠ 4, or the Venn arithmetic does not recover the union.
SafeAccess on this run, both confirmed before they were written:
p.artifacts === nullon 320 / 5,859 — extraction omits the record. 2026-08-27.p.ethics === nullon 894 / 5,859 — same. 2026-08-27.
vantage is never null (empty array). Direct access everywhere else.
Queries, with their denominators
| # | Query | Population / denominator | Result | On the page? |
|---|---|---|---|---|
| Q1 | Papers in the extraction | all | 5,859 | yes |
| Q2 | studyTypes includes automated-web-crawl | 5,859 | 967 | yes |
| Q3 | studyTypes includes network-scan-or-probe | 5,859 | 930 | yes |
| Q4 | studyTypes includes mobile-app-analysis | 5,859 | 529 | yes |
| Q5 | POPULATIONS.crawled (crawlConfig !== null OR Q2) | 5,859 | 1,120 | yes |
| Q6 | Q5 minus Q2 | 1,120 | 153, all have crawlConfig, 0 of Q2 outside Q5 | yes |
| Q7 | Union Q2 ∪ Q3 ∪ Q4, exclusive Venn | 5,859 | 792 + 780 + 402 + 97 + 74 + 49 + 4 = 2,198 | yes |
| Q8 | Pairwise ∩ including all-three | 5,859 | crawl∩scan 101, crawl∩app 78, scan∩app 53, all three 4 | yes |
| Q9 | Union ∩ user-study | 2,198 | 234 (10.6%) | yes |
| Q10 | Q3 ∩ existing-dataset-analysis | 930 | 429 (46.1%) | yes |
| Q11 | Q2 ∩ existing-dataset-analysis | 967 | 475 (49.1%) | yes |
| Q12 | Empty studyTypes | 5,859 | 14 | yes, count only; keys in the report |
| Q13 | Venue share of venue output | per venue | IMC scan 46.4% (296/638); TheWebConf crawl 26.5% (223/843); PETS crawl 22.0%; IMC crawl 19.4% | yes |
| Q14 | Year buckets (last starred) | all | table | yes |
| Q15 | Vantage location / tool version / own artifact / robots.txt | Q2, Q3, Q4, Q5 | scan location 52.4% vs crawl 28.0%; robots.txt crawl 5.0%, crawled-pop 4.7% | yes |
| Q16 | Scanner fold on Q3, category network-scanner, used+produced | 930 | ZMap 93, nmap 40, Censys-as-category 38; residue 205 strings / 167 papers | ranking yes; residue list on this page via the report file |
| Q17 | Censys name /censys/i used+produced, any category | 5,859 | 72 corpus, 63 in Q3, 43 of those also existing-dataset-analysis | yes |
| Q18 | Crawl tool families (tool_fold.mjs) on Q2 (967), rather than on Q5 (1,120) | 967 | Selenium 214, Puppeteer 71, OpenWPM 57, Playwright 31, PhantomJS-family 27; named any automation 640 | yes |
| Q19 | Playwright / Selenium / PhantomJS by year bucket on Q2 | Q2 in bucket | Playwright 0/0/0/15 (5.1%)/16 (9.5%); PhantomJS 0/294 in 2022–2024 | yes |
| Q20 | Posters | each | corpus 138; scan 34 of 930 | yes |
| Q21 | Q6 studyTypes mix | 153 | system-or-defence 125, mobile-app-analysis 44, network-scan 30; platform web 65 | yes |
Folding, and its residue
Scanner names (scripts/am_fold.mjs). Ordered regexes, first match wins, kind tagged (active-scan / grabber / dataset / atlas / traceroute / dns / alias / ids / routing / custom / censorship / vuln-scanner / passive). Self-tests run on import. Censys is /censys/i because a first draft used ^censys$ and missed Censys.io / Censys Search API. Custom names matching ^custom\b go to the custom family rather than residue, so “custom scanner” is a row (21 papers) and not 21 unique strings.
Residue after that: 205 distinct unmapped strings on 167 papers, printed in full in the report file on this page. They are unmapped scanner/tool strings — research tools, generic utilities, datasets and aliases (LZR is now mapped; Ally, Augur, Goscanner, Hyperquack, Scapy, … are not). Same shape as the home-grown crawler residue on crawler. Do not treat 167 as an error rate — the fold is a ranking aid.
Crawler names reuse tool_fold.mjs on Q2 (967), rather than on Q5 (1,120), so this page cannot silently disagree with a fold change on the child; it can disagree on the population, which it states.
No subject-fold on studyTypes. The values are a closed enum. The instability is labelling, not spelling.
Quotes spot-checked
–quotes checks these against whitespace-normalised paper.cols.txt. All six OK on 2026-08-27.
| Paper | Quote used on the page | Verdict |
|---|---|---|
| WWW/2020/…representativeness… | the number of third parties reported by the crawler is well into the right tail | exact in .cols |
| WWW/2020/apophanies-… | we have a variation of over 16% in the number of successful page loads | exact |
| IMC/2024/ten-years-of-zmap | more than 300 research papers have used ZMap | exact |
| IMC/2024/ten-years-of-zmap | over 33% of all Internet-wide IPv4 scan traffic can be fingerprinted as coming from ZMap | exact |
| CCS/2015/a-search-engine-backed-by-internet-wide-scanning | we introduce Censys, a public search engine and data processing facility backed by data collected from ongoing Internet-wide scans | exact |
| CCS/2016/online-tracking-a-1-million-site-measurement-and-analysis | based on a crawl of the top 1 million websites | exact |
Rejected quote: Zeber's “Crawls may fail to capture the diversity of user environments, including operating systems, geolocation, and profile history.” is in the PDF and is column-spliced in .cols (the two halves land in different column-repair fragments). Not published. The tail-of-the-distribution sentence is the one that survives .cols.
External sources
scripts/external_checks_automated_measurements.sh, 2026-08-27. Honours GH_TOKEN. Prints FAILED per check; exit 1 if any fail.
| Claim | Check | Result |
|---|---|---|
| ZMap still maintained | GitHub zmap/zmap archived and pushed_at | not archived, pushed 2026-08-26T15:13:42Z |
| ZGrab2 still maintained | GitHub zmap/zgrab2 | not archived, pushed 2026-07-27T09:25:44Z |
| Playwright current | npm playwright/latest | 1.62.1 |
| Censys the company | censys.com HTTP | 200 |
| Censys search UI | search.censys.io HTTP, browser UA | 403 (bot wall; same with and without UA). Recorded, not treated as an outage. |
| ZMap 2013 paper exists | USENIX landing page | HTTP 200, body contains ZMap |
Rejected as a source: SEO listicles about “best web crawlers 2026”. Not fetched.
Bibliography keys added
Appended to a fresh export of live literature:bibliography (rev recorded at publish), collision-checked against that export, not against pages/literature_bibliography.txt.
durumeric2013_zmap— USENIX Security 2013, authors from the landing page (Zakir Durumeric, Eric Wustrow, J. Alex Halderman). Not in the extraction. URL citation, no DOI in USENIX metadata.durumeric2014_view— USENIX Security 2014, authors viafetch_authors.py(Zakir Durumeric; Michael Bailey; J. Alex Halderman), thenbibgen.mjs –key. URL citation.
Already live, reused: zeber2020representativeness, ahmad2020_apophanies, durumeric2015_search, durumeric2024_years, pradeep2022_comparative, englehardt2016online, jueckstock2021_realistic, demir2022_reproducibility.
What could not be established
- Whether Censys-the-product in 2026 is still the CCS 2015 search engine or a renamed commercial platform.
censys.comis live; the search UI bot-walls this client. The page dates the HTTP facts and does not claim an API tutorial. - The original ZMap paper's figures (scan rate, hit rate). Not in the extraction; not copied from memory.
studyTypesstability on the 5,859-paper run. The 57% is the 100-paper sample.- Why 14 papers have empty
studyTypes. Listed in the report; not hand-read this sitting.
Review log
Freeze of local drafts: out/freeze_am/, 2026-08-27, before the three focused reviewers. That snapshot was not edited while they ran. Focused model: gpt-5.6-luna-medium. Generic model: the same, no checklist, on the post-focused page.
Focused review
R1 figures vs script ([figures vs script review](c018e0bd-99d4-4436-825b-fa89914ddf2a)). Live stdout matched the freeze; six quotes OK; Venn, 967-vs-1120, and venue ranking intact.
| # | Finding | Verdict |
|---|---|---|
| 1 | PUBLISHED_* percentage markers were string literals (37.5%, 28.0%, venue shares). A later corpus refresh could print a new table next to a stale marker and still exit 0. | Accepted. Markers now derive from the same computed values as the tables. |
R2 citations and quotes ([citations and quotes review](1f1a1aa2-b304-4e3e-b782-38cddbab9a2a)). All six quotes OK; citekeys unique; ZMap 2013-not-in-corpus disclosure OK.
| # | Finding | Verdict |
|---|---|---|
| 1 | Read-first table said “Pradeep et al., USENIX Security 2022”. The BibTeX entry and paper.cols.txt say IMC 2022. check_attributions.mjs does not compare venue to booktitle, only surname to author, so it passed. | Accepted. Row is now IMC 2022. |
R3 external currency ([external currency review](b29b71f6-65a8-4a5b-bfcb-fe68558152bb)). Playwright 1.62.1, both ZMap repos unarchived, USENIX 2013 page 200, censys.com 200, search.censys.io 403 with and without UA. Known-written child wikilinks exist.
| # | Finding | Verdict |
|---|---|---|
| 1 | “ZMap is still the active IPv4 scanner” overclaims what a GitHub push date can prove. | Accepted. Now “an actively maintained IPv4 scanner”, with the 33% traffic figure named as the usage evidence and scoped to that paper. |
| 2 | External-check script only requested search.censys.io with a browser UA while the page claimed 403 both with and without. | Accepted. Script now requests both and asserts 403/403. |
Generic review
No checklist. Same model, on the post-focused local pages ([generic review](3d319a97-1973-4fe7-b8e6-e1cccc145df0)).
| # | Finding | Verdict |
|---|---|---|
| 1 | “The three branches are not growing as a share” is false for app analysis (5.9% → 7.6%). | Accepted. Sentence now splits crawl/scan decline from the app series. |
| 2 | Reporting-rate table called a “ceiling” because posters raise silence. Silence up means reporting rates down, so the table understates full-paper reporting. | Accepted. Reversed. |
| 3 | “Treat every share as a ranking” applied the 57% studyTypes caveat to venue and reporting rates. | Accepted. Caveat scoped to studyTypes shares. |
| 4 | “A crawl cannot see server-side tracking” is too absolute. | Accepted. “cannot directly observe tracking that remains entirely server-side”. |
| 5 | robots.txt is not a comparable app-analysis question; the methods checklist names fields the table does not measure. | Accepted in part. Table kept (the 0.6% would otherwise look like a reporting gap). Lead now says the column is crawl/scan ethics and the checklist lives on child pages. Did not split into three tables. |
| 6 | “Censys is usually a dataset” over-reads the co-tag. | Accepted. “often”; 43 of 63; tag does not prove query-only. |
| 7 | “do not look for this literature there” (IMC, 32 app papers). | Accepted. “less common … not absent”. |
| 8 | Provenance “on Q2 not Q5” reads as an empty set. | Accepted. “on Q2 (967), rather than on Q5 (1,120)”. |
| 9 | Residue called “one-off research scanners”; it includes Scapy, dig, OpenVAS. | Accepted. “unmapped scanner/tool strings”. |
Unedited report output
Stdout of node scripts/report_automated_measurements.mjs on 2026-08-27, copied without edits.
- report_automated_measurements-output.txt
====================================================================== 1. CORPUS ====================================================================== corpus papers 5859 crawled (crawlConfig OR automated-web-crawl) 1120 missing paper.cols.txt 4 empty studyTypes 14 posters (slug poster- OR title Poster:) 138 records with pages <= 4 251 PUBLISHED_CORPUS 5859 PUBLISHED_CRAWLED 1120 PUBLISHED_STUDYTYPES_STABILITY 57 studyTypes run-to-run agreement is 57% on a 100-paper sample (extract/README.md); that sample has not been re-measured on the 5,859-paper run. Rankings, not precise percentages. ====================================================================== 2. STUDYTYPES RANKING (papers, not tuples; multi-valued so rows do not sum to corpus) ====================================================================== studyTypes value Papers Share of 5859 -------------------------- ------ ------------- system-or-defence-proposal 3967 67.7% existing-dataset-analysis 2615 44.6% manual-audit 1826 31.2% code-or-binary-analysis 1484 25.3% user-study 1149 19.6% automated-web-crawl 967 16.5% network-scan-or-probe 930 15.9% interview-or-survey 755 12.9% mobile-app-analysis 529 9.0% simulation-or-theory-only 335 5.7% PUBLISHED_SYSTEM_OR_DEFENCE 3967 PUBLISHED_EXISTING_DATASET 2615 PUBLISHED_USER_STUDY 1149 ====================================================================== 3. THREE BRANCHES AND OVERLAPS ====================================================================== automated-web-crawl 967 16.5% of corpus network-scan-or-probe 930 15.9% mobile-app-analysis 529 9.0% union of the three 2198 37.5% PUBLISHED_CRAWL 967 PUBLISHED_SCAN 930 PUBLISHED_APP 529 PUBLISHED_UNION 2198 PUBLISHED_UNION_PCT 37.5% Venn (a paper is in exactly one row): Slice Papers ------------------------- ------ only crawl 792 only scan 780 only app 402 crawl AND scan, not app 97 crawl AND app, not scan 74 scan AND app, not crawl 49 all three 4 union (sum of rows above) 2198 pairwise crawl ∩ scan (includes all-three) 101 pairwise crawl ∩ app 78 pairwise scan ∩ app 53 PUBLISHED_CRAWL_SCAN 101 PUBLISHED_CRAWL_APP 78 PUBLISHED_SCAN_APP 53 PUBLISHED_ALL_THREE 4 PUBLISHED_ONLY_CRAWL 792 PUBLISHED_ONLY_SCAN 780 PUBLISHED_ONLY_APP 402 union ∩ user-study 234 10.6% of union PUBLISHED_UNION_USER 234 scan ∩ existing-dataset-analysis 429 46.1% of scan crawl ∩ existing-dataset-analysis 475 49.1% of crawl PUBLISHED_SCAN_DATASET 429 PUBLISHED_CRAWL_DATASET 475 ====================================================================== 4. CRAWLED POPULATION vs STUDYTYPES CRAWL ====================================================================== crawled population 1120 of which studyTypes has automated-web-crawl 967 crawled minus that tag (have crawlConfig only) 153 studyTypes crawl minus crawled 0 PUBLISHED_EXTRA 153 PUBLISHED_EXTRA_ALL_HAVE_CRAWLCONFIG 153 studyTypes of the 153 (multi-valued; do not sum to 153): studyTypes Papers of 153 -------------------------- ------------- system-or-defence-proposal 125 manual-audit 77 existing-dataset-analysis 73 code-or-binary-analysis 62 mobile-app-analysis 44 network-scan-or-probe 30 user-study 18 interview-or-survey 13 simulation-or-theory-only 3 platforms of the 153: platform Papers of 153 -------------------- ------------- web 65 other-online-service 54 mobile 47 offline 39 iot 6 not-applicable 1 PUBLISHED_EXTRA_WEB 65 PUBLISHED_EXTRA_SYSTEM 125 PUBLISHED_EXTRA_APP 44 PUBLISHED_EXTRA_SCAN 30 ====================================================================== 5. VENUE (share of that venue's own output — not share of the branch) ====================================================================== Venue Papers crawl crawl % of venue scan scan % of venue app app % of venue --------------- ------ ----- ---------------- ---- --------------- --- -------------- CCS 990 137 13.8% 132 13.3% 106 10.7% IEEE S&P 767 89 11.6% 87 11.3% 63 8.2% IMC 638 124 19.4% 296 46.4% 32 5.0% NDSS 701 106 15.1% 117 16.7% 82 11.7% PETS 510 112 22.0% 49 9.6% 66 12.9% USENIX Security 1410 176 12.5% 199 14.1% 143 10.1% TheWebConf 843 223 26.5% 50 5.9% 37 4.4% PUBLISHED_IMC_SCAN_SHARE 46.4% PUBLISHED_IMC_SCAN_N 296 PUBLISHED_WWW_CRAWL_SHARE 26.5% PUBLISHED_PETS_CRAWL_SHARE 22.0% PUBLISHED_IMC_CRAWL_SHARE 19.4% PUBLISHED_IMC_APP_SHARE 5.0% PUBLISHED_WWW_SCAN_SHARE 5.9% IMC n=638 scan=296 WWW n=843 crawl=223 ====================================================================== 6. YEAR BUCKETS (2025–2026 starred: provisional) ====================================================================== Window Papers crawl scan app crawl share scan share app share ---------- ------ ----- ---- --- ----------- ---------- --------- 2010–2013 511 93 89 30 18.2% 17.4% 5.9% 2014–2017 769 142 173 113 18.5% 22.5% 14.7% 2018–2021 1439 269 255 146 18.7% 17.7% 10.1% 2022–2024 1955 294 245 150 15.0% 12.5% 7.7% 2025–2026* 1185 169 168 90 14.3% 14.2% 7.6% Do not read the last column as a completed year. CCS/IMC 2026 have not been held; IEEE S&P and WWW 2026 abstracts are under-selected by construction. ====================================================================== 7. PLATFORMS of each branch (multi-valued; do not sum to N) ====================================================================== -- crawl N=967 -- platform Papers Share of 967 -------------------- ------ ------------ web 792 81.9% other-online-service 423 43.7% mobile 123 12.7% offline 105 10.9% iot 32 3.3% -- scan N=930 -- platform Papers Share of 930 -------------------- ------ ------------ other-online-service 745 80.1% web 239 25.7% offline 139 14.9% mobile 128 13.8% iot 93 10.0% -- app N=529 -- platform Papers Share of 529 -------------------- ------ ------------ mobile 525 99.2% other-online-service 98 18.5% web 78 14.7% offline 36 6.8% iot 36 6.8% ====================================================================== 8. REPORTING RATES across branches (same question, three denominators) ====================================================================== Population N vantage location stated any used/produced tool version authors-own artifact URL robots.txt stated ------------------------- ---- ----------------------- ------------------------------ ------------------------ ----------------- automated-web-crawl 967 271 (28.0%) 446 (46.1%) 581 (60.1%) 48 (5.0%) network-scan-or-probe 930 487 (52.4%) 401 (43.1%) 514 (55.3%) 22 (2.4%) mobile-app-analysis 529 117 (22.1%) 207 (39.1%) 334 (63.1%) 3 (0.6%) crawled population (1120) 1120 301 (26.9%) 506 (45.2%) 684 (61.1%) 53 (4.7%) PUBLISHED_CRAWL_LOC 28.0% PUBLISHED_SCAN_LOC 52.4% PUBLISHED_APP_LOC 22.1% PUBLISHED_CRAWL_ROBOTS 5.0% PUBLISHED_SCAN_ROBOTS 2.4% PUBLISHED_CRAWLED_ROBOTS 4.7% PUBLISHED_CRAWL_ART 60.1% PUBLISHED_SCAN_ART 55.3% PUBLISHED_APP_ART 63.1% crawl loc n=271 scan loc n=487 ====================================================================== 9. SCANNER TOOLS folded (population: network-scan-or-probe, category=network-scanner, used+produced) ====================================================================== Family kind Papers Share of 930 ------------------- ------------ ------ ------------ ZMap active-scan 93 10.0% nmap active-scan 40 4.3% Censys dataset 38 4.1% traceroute traceroute 31 3.3% ZGrab grabber 31 3.3% RIPE Atlas atlas 25 2.7% custom scanner custom 21 2.3% scamper traceroute 21 2.3% Shodan dataset 19 2.0% XMap active-scan 15 1.6% ping active-scan 13 1.4% ZDNS dns 11 1.2% MIDAR alias 8 0.9% TLS banner grabbers grabber 7 0.8% bdrmap alias 6 0.6% Paris traceroute traceroute 6 0.6% Zeek/Bro ids 6 0.6% fpdns dns 5 0.5% Masscan active-scan 5 0.5% massdns dns 5 0.5% Yarrp traceroute 5 0.5% BGPStream routing 4 0.4% mtr traceroute 4 0.4% Snort ids 4 0.4% Nessus vuln-scanner 3 0.3% OONI censorship 3 0.3% p0f passive 3 0.3% RouteViews routing 3 0.3% Suricata ids 3 0.3% traIXroute traceroute 3 0.3% Acunetix vuln-scanner 2 0.2% LZR grabber 2 0.2% OpenINTEL dns 2 0.2% RIPE RIS routing 2 0.2% Censored Planet censorship 1 0.1% kind union (a paper can match several kinds): kind Papers Share of 930 ------------ ------ ------------ active-scan 152 16.3% traceroute 67 7.2% dataset 49 5.3% grabber 39 4.2% atlas 25 2.7% dns 23 2.5% custom 21 2.3% alias 12 1.3% ids 10 1.1% routing 5 0.5% censorship 4 0.4% vuln-scanner 4 0.4% passive 3 0.3% scan papers naming a network-scanner used/produced 396 corpus papers naming a network-scanner used/produced 445 of those, tagged network-scan-or-probe 396 PUBLISHED_NS_IN_SCAN 396 PUBLISHED_NS_CORPUS 445 PUBLISHED_ZMAP 93 PUBLISHED_CENSYS_SCANNER_CAT 38 PUBLISHED_NMAP 40 PUBLISHED_RIPE 25 residue distinct strings: 205; residue papers: 167 PUBLISHED_SCANNER_RESIDUE_STRINGS 205 PUBLISHED_SCANNER_RESIDUE_PAPERS 167 Unmapped name Papers in scan with this exact string ------------------------------------------------------- ------------------------------------- Ally 2 Augur 2 GeoInspector 2 Goscanner 2 Hyperquack 2 Karma 2 Mercator 2 mrinfo 2 QScanner 2 Quack 2 Satellite 2 Scapy 2 Verfploeter 2 0trace 1 5GSniffer 1 6Gen 1 6Sense 1 active DNS crawler 1 active scanning tool 1 active-geolocator 1 AdaptOver 1 addr6 1 Airmon-ng 1 amass 1 AmpMap 1 Anaximander 1 Aquantone 1 arpscan 1 Bitnodes 1 BitTorrent crawler 1 BreadCrumb 1 Btsniffer 1 CAIDA AS Relationships Dataset 1 Cenzic HailStorm Pro 1 Chaos 1 CheckHost 1 cloud_enum 1 CnCHunter 1 COVScan 1 Critical.IO 1 ct-sans 1 danglingcname 1 Diamond Miner 1 dig 1 Dig 1 dMAP (DPI Mapper) 1 DMiner 1 DNS crawler 1 DNS X-Ray Tool 1 dnspython 1 DoE-Hunter 1 drill 1 DTLS-Fuzzer 1 DTLS-MitM-Scanner (DMS) 1 EAST 1 efficient scanning software 1 ERIS 1 exitmap 1 extended Yarrpbox 1 FastMDA 1 FlashRoute 1 Fofa 1 FOFA 1 forwarders-based scanner 1 gateway checker 1 Geneva 1 gQUIC-module 1 Greynoise 1 Hitlist-Downloader 1 HP WebInspect 1 HTTP/OneProbe 1 IBM Rational AppScan 1 ICMP 1 ICMP ping 1 ICMP_Sonar 1 iffinder 1 iGreedy 1 ike-scan 1 Internet Census 1 iperf 1 iPerf3 1 IPFS Crawler 1 IPv6 Hitlist Service 1 IRBlock 1 Iris 1 IRLscanner 1 IRTT 1 Iterative Network Tracing (INT) 1 James 1 JARM 1 LACeS 1 LanDscAPe 1 LeakIX 1 LFP 1 LTE PROBE 1 Lumscan 1 MacStealer 1 MAnycast anycast census 1 MAnycast2 1 MAP-IT 1 MarcoPolo 1 McAfee SECURE 1 Mint 1 modified ntpdedup 1 Monocle 1 mrinfo-rec 1 N-Stalker QA Edition 1 Namehelp 1 nc 1 ndisc6 1 Netcat 1 NetfilterQueue 1 netperf 1 Network Analyzer 1 nmap plugin for Special DROWN 1 Nmap Security Scanner 1 Nmap service detection probe list 1 Nmap service probes 1 node probing program 1 NodeFinder 1 NR-Scope 1 ns-3 1 nsupdate 1 Nuclei 1 nuttcp 1 OneProbe 1 onesixtyone 1 onion-grab 1 Open Resolver Project 1 OpenVAS 1 OSAVRoute 1 p0f3 1 PciCheckerLite 1 perfSONAR 1 Periscope 1 PingTools 1 PLCScan 1 Prefixscan 1 public looking glasses 1 Python script 1 PyTNT 1 QualysGuard PCI 1 quicgrabber 1 RAINBOW 1 Rapid7 Labs Open Data 1 Rapid7 NeXpose 1 Rapid7 Sonar 1 RAT-Scan 1 RDScan 1 revtr 2.0 1 RMap 1 Route Views 1 SAD DNS 1 SadDNS 1 scamper Python module 1 scanner application 1 SecSpider 1 Shadow 1 ShaperProbe 1 smbmap 1 SNMPv3 1 Speedtrap 1 Spoki 1 Spoofer project 1 sqlmap 1 SSFNet 1 stateless scanner (custom) 1 systematic scanning tool 1 TCP pings 1 tcp_probe 1 ThunderPing 1 Time Series Latency Probing (TSLP) 1 TLS-Attacker 1 TLS-Crawler 1 TMC 1 Tnt 1 Tor DNSEL 1 TorFlow 1 tracebox 1 tracefilter 1 tracenet 1 Trinocular 1 Trufflehunter 1 Tshark 1 Tsunami 1 Ubertooth One 1 unbound 1 Unicorn 1 University of Michigan Daily Full IPv4 HTTPS Handshakes 1 URHunter 1 v6disc.sh 1 Verloc 1 WeHeY 1 WhatWeb 1 whois 1 WiFi Inspector 1 Xmap (extended) 1 Xray 1 Yarrp6 1 Yarrpbox 1 YoDNS 1 Zed Attack Proxy (ZAP) 1 Zoomeye 1 ZoomEye 1 ZreFinder 1 ====================================================================== 10. CENSYS / SHODAN as a named tool on any category (used+produced) ====================================================================== Name corpus used/produced in scan branch in scan AND existing-dataset-analysis ------ -------------------- -------------- ------------------------------------- Censys 72 63 43 Shodan 37 34 18 PUBLISHED_CENSYS_ALL 72 PUBLISHED_CENSYS_SCAN 63 PUBLISHED_CENSYS_SCAN_DS 43 PUBLISHED_SHODAN_ALL 37 Censys by year bucket, of the scan branch: Window scan papers named Censys used/produced share of scan in window ---------- ----------- -------------------------- ----------------------- 2010–2013 89 0 0.0% 2014–2017 173 8 4.6% 2018–2021 255 26 10.2% 2022–2024 245 24 9.8% 2025–2026* 168 5 3.0% ====================================================================== 11. CRAWL TOOL FAMILIES on the studyTypes crawl population (not a duplicate of programming:crawler; that page uses crawled=1120) ====================================================================== Family Papers of 967 Share of 967 ---------------------------------------------- ------------- ------------ Selenium 214 22.1% Puppeteer 71 7.3% Playwright 31 3.2% OpenWPM 57 5.9% PhantomJS and other legacy scriptable browsers 27 2.8% crawl papers naming any crawler-framework/browser-automation used+produced 640 66.2% PUBLISHED_CRAWL_NAMED_TOOL 640 PUBLISHED_SELENIUM 214 PUBLISHED_PUPPETEER 71 PUBLISHED_PLAYWRIGHT 31 PUBLISHED_OPENWPM 57 PUBLISHED_PHANTOM 27 those families by year bucket, share of crawl papers in the window: Family 2010–2013 2014–2017 2018–2021 2022–2024 2025–2026* ---------------------------------------------- --------- ---------- ---------- ---------- ---------- Selenium 3 (3.2%) 30 (21.1%) 66 (24.5%) 71 (24.1%) 44 (26.0%) Puppeteer 0 (0.0%) 0 (0.0%) 20 (7.4%) 37 (12.6%) 14 (8.3%) Playwright 0 (0.0%) 0 (0.0%) 0 (0.0%) 15 (5.1%) 16 (9.5%) OpenWPM 0 (0.0%) 6 (4.2%) 18 (6.7%) 26 (8.8%) 7 (4.1%) PhantomJS and other legacy scriptable browsers 3 (3.2%) 13 (9.2%) 10 (3.7%) 0 (0.0%) 1 (0.6%) residue-only-or-also papers on studyTypes crawl: 161 (printed by report_crawler.mjs on the 1120; not re-listed here) ====================================================================== 12. POSTER / SHORT-RECORD SENSITIVITY ====================================================================== Population N posters poster share pages<=4 short share ---------- ---- ------- ------------ -------- ----------- corpus 5859 138 2.4% 251 4.3% crawl 967 13 1.3% 23 2.4% scan 930 34 3.7% 46 4.9% app 529 11 2.1% 16 3.0% PUBLISHED_POSTERS 138 PUBLISHED_SCAN_POSTERS 34 ====================================================================== 13. QUOTE NEEDLES (must also pass --quotes) ====================================================================== Zeber et al. WWW 2020: the number of third parties reported by the crawler is well into the right tail Zeber splice note: the sentence "Crawls may fail to capture the diversity of user environments, including operating systems, geolocation, and profile history." is in the PDF but column-spliced in paper.cols.txt and is therefore not used as a published quote. Ahmad et al. WWW 2020: we have a variation of over 16% in the number of successful page loads Durumeric et al. IMC 2024: more than 300 research papers have used ZMap Durumeric et al. IMC 2024: over 33% of all Internet-wide IPv4 scan traffic can be fingerprinted as coming from ZMap Durumeric et al. CCS 2015: we introduce Censys, a public search engine and data processing facility backed by data collected from ongoing Internet-wide scans Englehardt and Narayanan CCS 2016: based on a crawl of the top 1 million websites PUBLISHED_ZMAP_PAPERS 300 PUBLISHED_ZMAP_TRAFFIC 33% PUBLISHED_AHMAD_LOAD 16% ====================================================================== Y. ARITHMETIC-DERIVED FIGURES ====================================================================== union check 792+780+402+97+74+49+4 = 2198 crawled - crawl tag = 1120 - 967 = 153 crawlScanOnly = 101 - 4 = 97 ====================================================================== Z. EXTERNAL / NON-CORPUS FIGURES (must appear here if they appear on the page) ====================================================================== 57% studyTypes run-to-run agreement: publications_dataset/data/extract/README.md, 100-paper sample, not re-measured on 5859. 20% free-text name agreement: same README, order of magnitude. ZMap original paper (USENIX Security 2013) is NOT in corpus2/.meta/USENIX-2013.json and NOT in data/fulltext/2013/USENIX/. Cited externally. Playwright npm latest 1.62.1 on 2026-08-27, scripts/external_checks_automated_measurements.sh. search.censys.io HTTP 403 to automated clients (browser UA too) on 2026-08-27; censys.com HTTP 200. zmap/zmap not archived, pushed_at 2026-08-26; zmap/zgrab2 pushed_at 2026-07-27. USENIX Security 2013 ZMap landing page HTTP 200 on 2026-08-27. 138 posters / 251 short records: this report, same definition as literature:corpus and report_artifacts.mjs.
provenance/design/automated_measurements.txt · Last modified: by karel.kubicek.claude