| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:design:dns [2026/08/27 23:21] – Add review pass 5 (generic). Purge the retracted pre-fix run, which had survived in four places in this log after being fixed on the page — the section now states no verdict counts of its own and records instead what three runs showed: identical 46 and no karel.kubicek.claude | provenance:design:dns [2026/09/17 09:37] (current) – Record RFC prose follow-up and source checks; Authored by Claude karel.kubicek.claude |
|---|
| | Code-block builder | ''scripts/build_dns_codeblock.mjs'' | | | Code-block builder | ''scripts/build_dns_codeblock.mjs'' | |
| | Author | Claude (Opus 5), single sitting, unsupervised | | | Author | Claude (Opus 5), single sitting, unsupervised | |
| | | Follow-up run | **2026-09-17**, Claude (Opus 5): reconciled the ZMap/XMap counts with [[programming:internet_scanning]]. Corpus unchanged at 5,859. See [[#Follow-up, 2026-09-17: reconciled with programming:internet_scanning]]. | |
| | Reviewers | see [[#Review log]] | | | Reviewers | see [[#Review log]] | |
| |
| |
| ^ Excluded ^ Papers (corpus-wide) ^ Why not membership ^ | ^ Excluded ^ Papers (corpus-wide) ^ Why not membership ^ |
| | ZMap / XMap / ZGrab / Scapy (''scanner-generic'') | 156 | General-purpose scanners. ZMap finds open resolvers, but a ZMap paper is not a DNS paper. Reported in a separate row on the page. | | | ZMap / XMap / ZGrab / Scapy (''scanner-generic'') | 155 | General-purpose scanners. ZMap finds open resolvers, but a ZMap paper is not a DNS paper. Reported in a separate row on the page. | |
| | RIPE Atlas (''vantage-platform'') | 77 | A vantage network with many measurement types; most of the 77 use it for traceroute. Named on the page as the standard multi-vantage route, not counted as membership. | | | RIPE Atlas (''vantage-platform'') | 77 | A vantage network with many measurement types; most of the 77 use it for traceroute. Named on the page as the standard multi-vantage route, not counted as membership. | |
| | WHOIS / RDAP / PSL / tldextract (''adjacent:registry-metadata'') | 113 + 52 + 4 | Registry questions ("who owns this name", "where does the registrable domain end"), not resolution questions. Already covered by [[design:website_classification]] and [[design:ip_classification]]. | | | WHOIS / RDAP / PSL / tldextract (''adjacent:registry-metadata'') | 113 + 52 + 4 | Registry questions ("who owns this name", "where does the registrable domain end"), not resolution questions. Already covered by [[design:website_classification]] and [[design:ip_classification]]. | |
| 34 papers 34 spellings home-grown DNS code | 34 papers 34 spellings home-grown DNS code |
| |
| [scanner-generic] union 156 papers corpus-wide, 38 of them in DNSPOP — reported only, NOT in the population rule | [scanner-generic] union 154 papers corpus-wide, 37 of them in DNSPOP — reported only, NOT in the population rule |
| 112 papers 13 spellings ZMap / XMap | 110 papers 13 spellings ZMap / XMap |
| 33 papers 4 spellings Scapy | 33 papers 4 spellings Scapy |
| 31 papers 6 spellings ZGrab | 32 papers 7 spellings ZGrab |
| |
| [vantage-platform] union 77 papers corpus-wide, 24 of them in DNSPOP — reported only, NOT in the population rule | [vantage-platform] union 77 papers corpus-wide, 24 of them in DNSPOP — reported only, NOT in the population rule |
| |
| --- F. FOLD RESIDUE AND WHAT WAS DELIBERATELY EXCLUDED --- | --- F. FOLD RESIDUE AND WHAT WAS DELIBERATELY EXCLUDED --- |
| candidate tool tuples matched by the loose sweep: 976 | candidate tool tuples matched by the loose sweep: 974 |
| unmapped residue: 2 distinct strings, 2 tuples | unmapped residue: 2 distinct strings, 2 tuples |
| 1 "IEEE Organizationally Unique Identifier registry" | 1 "IEEE Organizationally Unique Identifier registry" |
| mapped but excluded from the population rule (papers): | mapped but excluded from the population rule (papers): |
| 113 adjacent:registry-metadata — WHOIS / RDAP | 113 adjacent:registry-metadata — WHOIS / RDAP |
| 112 scanner-generic — ZMap / XMap | 110 scanner-generic — ZMap / XMap |
| 75 vantage-platform — RIPE Atlas | 75 vantage-platform — RIPE Atlas |
| 52 adjacent:registry-metadata — Public Suffix List / TLD lists | 52 adjacent:registry-metadata — Public Suffix List / TLD lists |
| 46 offtopic:hardware — lab hardware matching /dot|host|dig/ | 46 offtopic:hardware — lab hardware matching /dot|host|dig/ |
| 33 scanner-generic — Scapy | 33 scanner-generic — Scapy |
| 31 scanner-generic — ZGrab | 32 scanner-generic — ZGrab |
| 21 offtopic:annotation — ATLAS.ti | 21 offtopic:annotation — ATLAS.ti |
| 18 offtopic:blocklist — domain blocklists | 18 offtopic:blocklist — domain blocklists |
| </code> | </code> |
| |
| **976 candidate tuples. 2 unmapped, both off-topic:** ''"IEEE Organizationally Unique Identifier registry"'' (1 tuple) and ''"Chrome DevTools Protocol Network domain"'' (1 tuple). Neither is a DNS instrument and neither is in the population. | **975 candidate tuples. 2 unmapped, both off-topic:** ''"IEEE Organizationally Unique Identifier registry"'' (1 tuple) and ''"Chrome DevTools Protocol Network domain"'' (1 tuple). Neither is a DNS instrument and neither is in the population. |
| |
| Section F of the report prints the residue and the full table of mapped-but-excluded families with their paper counts. | Section F of the report prints the residue and the full table of mapped-but-excluded families with their paper counts. |
| |
| **One quote is normalised and says so on the page.** Ager et al. (IMC 2010) is rendered "2 , 000" and "10, 000" with a stray space and "dif- fer" hyphenated across a line break — in ''paper.cols.txt'' **and** in a ''pypdf'' re-extraction of ''paper.pdf'', so it is the source's own typesetting, not a column-repair artefact. The page carries a footnote saying so. | **One quote is normalised and says so on the page.** Ager et al. (IMC 2010) is rendered "2 , 000" and "10, 000" with a stray space and "dif- fer" hyphenated across a line break — in ''paper.cols.txt'' **and** in a ''pypdf'' re-extraction of ''paper.pdf'', so it is the source's own typesetting, not a column-repair artefact. The page carries a footnote saying so. |
| | |
| | ===== RFC prose follow-up (2026-09-17) ===== |
| | |
| | The external-currency pass in [[#Review log]] intentionally left RFC 9460, RFC 9567, RFC 9520 and RFC 9606 out of the page: four bare RFC numbers would be a reading list, not measurement guidance. This follow-up adds one measurement-facing paragraph for each. It adds no corpus figure and no bibliography entry. The four paragraphs were checked against the [[https://www.rfc-editor.org/rfc-index/|RFC Editor index]] and the individual RFC records on 2026-09-17. |
| | |
| | <code> |
| | Command run: |
| | bash scripts/external_checks_dns.sh | awk '/^== RFCs the page names/{show=1} /^== GitHub repositories/{show=0} show {print}' |
| | |
| | Relevant checker output on 2026-09-17: |
| | RFC9460 November 2023 PROPOSED STANDARD obs_by=- upd_by=- Service Binding and Parameter Specification via the DNS |
| | RFC9520 December 2023 PROPOSED STANDARD obs_by=- upd_by=- Negative Caching of DNS Resolution Failures |
| | RFC9567 April 2024 PROPOSED STANDARD obs_by=- upd_by=- DNS Error Reporting |
| | RFC9606 June 2024 PROPOSED STANDARD obs_by=- upd_by=- DNS Resolver Information |
| | |
| | Source records read: |
| | https://www.rfc-editor.org/info/rfc9460/ |
| | https://www.rfc-editor.org/info/rfc9567/ |
| | https://www.rfc-editor.org/info/rfc9520/ |
| | https://www.rfc-editor.org/info/rfc9606/ |
| | </code> |
| | |
| | The required delegated model 'sonnet' external-currency pass was attempted for this prose change, but the available delegation service rejected model ''sonnet'' (only GPT models were exposed). No alternate model was substituted. I manually re-read the four fetched RFC Editor records and found no correction to the four paragraphs; this environment limitation is recorded rather than mislabeled as a Sonnet result. |
| |
| ===== External checks ===== | ===== External checks ===== |
| | %%/\bdot\b/%% as a DoT alternative | Matched ''Droidot'', ''Godot'', ''OleViewDot-Net'' and every "dot" in prose | Never used. The DoT row uses the spelled-out phrase only, which is why its counts are conservative. | | | %%/\bdot\b/%% as a DoT alternative | Matched ''Droidot'', ''Godot'', ''OleViewDot-Net'' and every "dot" in prose | Never used. The DoT row uses the spelled-out phrase only, which is why its counts are conservative. | |
| | A "which resolver did the paper use" schema query | No such field exists | The schema has no place for it, and a full-text probe for resolver IP literals (69 papers naming ''8.8.8.8'' or similar) is an upper bound on mentions, not on use. **No reporting-rate claim is made about resolver identification.** This is the single biggest thing the corpus cannot answer for this page. | | | A "which resolver did the paper use" schema query | No such field exists | The schema has no place for it, and a full-text probe for resolver IP literals (69 papers naming ''8.8.8.8'' or similar) is an upper bound on mentions, not on use. **No reporting-rate claim is made about resolver identification.** This is the single biggest thing the corpus cannot answer for this page. | |
| | |
| | ===== Follow-up, 2026-09-17: reconciled with programming:internet_scanning ===== |
| | |
| | [[programming:internet_scanning]] was published on 2026-09-10 with **ZMap 95 + XMap 16 = 111** papers over its own fold (''scripts/scan_fold.mjs''), against this page's **112** for the single ''scanner-generic'' family ''ZMap / XMap''. Its generic review pass raised the discrepancy and could not resolve it. It is resolved here, and on the other page's provenance, by ''scripts/reconcile_zmap.mjs'', which builds both paper sets from the same extraction and prints the symmetric difference. Its unedited output is published in full on [[provenance:programming:internet_scanning]]. |
| | |
| | **It was not an overlap artefact.** The suspected cause was a paper naming both ZMap and XMap — counted once as one family here, twice as two families there. **No paper in the corpus names both.** The symmetric difference is exactly **one** paper. |
| | |
| | That paper is **CCS 2016, //A Surfeit of SSH Cipher Suites//**. Its ''tools[]'' entry is one compound string, ''%%ZGrab/ZMap%%'', with ''evidence.quote'' //"We ran two IPv4-address-space-wide scans for SSH servers using ZGrab/ZMap [13]."// — matched verbatim in ''data/fulltext/2016/CCS/a-surfeit-of-ssh-cipher-suites/paper.cols.txt'' on 2026-09-17. The paper used both tools. Both folds are ordered and first-match-wins, so a compound string can only land in one family, and the two test in opposite orders: ''dns_fold.mjs'' has ''ZMap / XMap'' before ''ZGrab'', ''scan_fold.mjs'' has ''ZGrab'' before ''ZMap''. Same string, two families. |
| | |
| | Both folds now carry an identical hand-read ''COMPOUND_SPLIT'' map for that one string. On this page's side: |
| | |
| | ^ Family ^ Was ^ Now ^ Why ^ |
| | | ''ZGrab'' | 31 papers, 6 spellings | **32**, 7 spellings | the compound string now also counts as ZGrab, which is what the paper says | |
| | | ''ZMap / XMap'' | 112 papers | **110** | unchanged by the split; **−2** from the tuple suppressions below | |
| | | ''scanner-generic'' union | 156 corpus-wide, 38 in DNSPOP | **154**, **37** | same suppressions | |
| | | candidate tuples in the loose sweep | 976 | **974** | same suppressions | |
| | |
| | **The tuple suppressions.** Two papers, both found while hand-auditing the other fold, and each costing this page one ''ZMap / XMap'' paper. They live in ''scripts/tool_use_overrides.mjs'', which //both// report scripts import, because "this tuple is not an instrument the paper used" is a fact about the tuple and not about either page. |
| | |
| | - **IMC 2024 //Have you SYN me?//** is a pure network-telescope study that never ran the scanners its ''tools[]'' names — the extraction's own ''purpose'' fields read //"Fingerprint ZMap-generated scanning traffic"//, and the paper says //"we find no evidence of Unicorn being used for Internet-wide scanning"//. Four tuples suppressed, one of them ZMap. |
| | - **USENIX 2022 //Spoki//** did run ZMap, but only inside its own machine: //"A local pair of virtual Ethernet interfaces route packets between the packet source and Spoki. We use ZMap to generate TCP SYN packets, passing the MAC address of the destination interface along with a probing rate and a range of IP addresses to use as a source."// The address range is a synthetic //source// prefix; nothing is probed. Found by a review pass on 2026-09-17, **after** the first version of this section had already been published saying the count was 111. |
| | |
| | Neither paper's other DNS tuples are suppressed, and **DNSPOP is unchanged at 244**. |
| | |
| | After all of it the two pages agree exactly: ZMap ∪ XMap = **110** on both sides (94 + 16 there), ZGrab = **32** on both, symmetric difference **0**. ''reconcile_zmap.mjs'' exits 1 if they ever diverge again, and also if a new compound ''tools[]'' name appears whose lost part is a ''DNS_KIND'' here or a membership kind there. It was mutation-tested on 2026-09-17: removing the ''%%zgrab/zmap%%'' entry produced three failures. |
| | |
| | **Compound-name recall.** A blanket split would be wrong — ''miekg/dns'' is a GitHub path this fold depends on reading whole. The sweep is therefore a guard, not a rule: it looks at every used-or-produced ''tools[]'' name containing ''/'', '' + '', '' and '', '' & '' or a comma (**263 tuples**) and prints each one whose parts fold to a family the whole string does not. There are **four**. Only ''%%ZGrab/ZMap%%'' touches a figure on either page; ''%%HTTP/OneProbe%%'' is one product name, and ''%%tcpdump/libpcap%%'' and ''%%WireShark/tshark%%'' fold to kinds neither page counts. All four are printed in the reconciler's output whether or not they were split. |
| | |
| | **Guards re-run on 2026-09-17** against the current corpus: ''report_dns.mjs'' exit 0, ''verify_dns_figures.mjs'' exit 0, ''reconcile_zmap.mjs'' exit 0, ''check_page_numbers.mjs'' on [[design:dns]] against report+verify+external+resolver+reconcile — //"OK — every figure in the page traces to"//, ''check_wrap.mjs'' OK, rendered DOM 96 ''bibtex_citekey'' spans / 28 headings / 11 tables / **0** red links. No other figure on [[design:dns]] moved. |
| |
| ===== Judgement calls ===== | ===== Judgement calls ===== |
| | [[:design]] | DNS row added to the namespace table and the page count corrected | | | [[:design]] | DNS row added to the namespace table and the page count corrected | |
| | [[design:automated_measurements]] | the scan row of the routing table now points at [[design:dns]] for DNS rather than at the TLS and IP pages | | | [[design:automated_measurements]] | the scan row of the routing table now points at [[design:dns]] for DNS rather than at the TLS and IP pages | |
| | | [[design:dns]] //(2026-09-17)// | rev 1787872903 → 1789612622, saved twice: the generic-scanner sentence now gives **110**/32/37 and names the agreement with [[programming:internet_scanning]]; the fold-residue sentence 976 → **974**. The first save said 111 and 975; a review pass then found the Spoki tuple. | |
| |
| ===== Review log ===== | ===== Review log ===== |
