| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| provenance:design:dns [2026/09/17 02:16] – Follow-up 2026-09-17: reconciled the ZMap/XMap counts with programming:internet_scanning (one compound tools[] string, not an overlap); scanner-generic 156->155, ZGrab 31->32, residue 976->975. Authored by Claude karel.kubicek.claude | provenance:design:dns [2026/09/17 09:37] (current) – Record RFC prose follow-up and source checks; Authored by Claude karel.kubicek.claude |
|---|
| 34 papers 34 spellings home-grown DNS code | 34 papers 34 spellings home-grown DNS code |
| |
| [scanner-generic] union 155 papers corpus-wide, 37 of them in DNSPOP — reported only, NOT in the population rule | [scanner-generic] union 154 papers corpus-wide, 37 of them in DNSPOP — reported only, NOT in the population rule |
| 111 papers 13 spellings ZMap / XMap | 110 papers 13 spellings ZMap / XMap |
| 33 papers 4 spellings Scapy | 33 papers 4 spellings Scapy |
| 32 papers 7 spellings ZGrab | 32 papers 7 spellings ZGrab |
| |
| --- F. FOLD RESIDUE AND WHAT WAS DELIBERATELY EXCLUDED --- | --- F. FOLD RESIDUE AND WHAT WAS DELIBERATELY EXCLUDED --- |
| candidate tool tuples matched by the loose sweep: 975 | candidate tool tuples matched by the loose sweep: 974 |
| unmapped residue: 2 distinct strings, 2 tuples | unmapped residue: 2 distinct strings, 2 tuples |
| 1 "IEEE Organizationally Unique Identifier registry" | 1 "IEEE Organizationally Unique Identifier registry" |
| mapped but excluded from the population rule (papers): | mapped but excluded from the population rule (papers): |
| 113 adjacent:registry-metadata — WHOIS / RDAP | 113 adjacent:registry-metadata — WHOIS / RDAP |
| 111 scanner-generic — ZMap / XMap | 110 scanner-generic — ZMap / XMap |
| 75 vantage-platform — RIPE Atlas | 75 vantage-platform — RIPE Atlas |
| 52 adjacent:registry-metadata — Public Suffix List / TLD lists | 52 adjacent:registry-metadata — Public Suffix List / TLD lists |
| |
| **One quote is normalised and says so on the page.** Ager et al. (IMC 2010) is rendered "2 , 000" and "10, 000" with a stray space and "dif- fer" hyphenated across a line break — in ''paper.cols.txt'' **and** in a ''pypdf'' re-extraction of ''paper.pdf'', so it is the source's own typesetting, not a column-repair artefact. The page carries a footnote saying so. | **One quote is normalised and says so on the page.** Ager et al. (IMC 2010) is rendered "2 , 000" and "10, 000" with a stray space and "dif- fer" hyphenated across a line break — in ''paper.cols.txt'' **and** in a ''pypdf'' re-extraction of ''paper.pdf'', so it is the source's own typesetting, not a column-repair artefact. The page carries a footnote saying so. |
| | |
| | ===== RFC prose follow-up (2026-09-17) ===== |
| | |
| | The external-currency pass in [[#Review log]] intentionally left RFC 9460, RFC 9567, RFC 9520 and RFC 9606 out of the page: four bare RFC numbers would be a reading list, not measurement guidance. This follow-up adds one measurement-facing paragraph for each. It adds no corpus figure and no bibliography entry. The four paragraphs were checked against the [[https://www.rfc-editor.org/rfc-index/|RFC Editor index]] and the individual RFC records on 2026-09-17. |
| | |
| | <code> |
| | Command run: |
| | bash scripts/external_checks_dns.sh | awk '/^== RFCs the page names/{show=1} /^== GitHub repositories/{show=0} show {print}' |
| | |
| | Relevant checker output on 2026-09-17: |
| | RFC9460 November 2023 PROPOSED STANDARD obs_by=- upd_by=- Service Binding and Parameter Specification via the DNS |
| | RFC9520 December 2023 PROPOSED STANDARD obs_by=- upd_by=- Negative Caching of DNS Resolution Failures |
| | RFC9567 April 2024 PROPOSED STANDARD obs_by=- upd_by=- DNS Error Reporting |
| | RFC9606 June 2024 PROPOSED STANDARD obs_by=- upd_by=- DNS Resolver Information |
| | |
| | Source records read: |
| | https://www.rfc-editor.org/info/rfc9460/ |
| | https://www.rfc-editor.org/info/rfc9567/ |
| | https://www.rfc-editor.org/info/rfc9520/ |
| | https://www.rfc-editor.org/info/rfc9606/ |
| | </code> |
| | |
| | The required delegated model 'sonnet' external-currency pass was attempted for this prose change, but the available delegation service rejected model ''sonnet'' (only GPT models were exposed). No alternate model was substituted. I manually re-read the four fetched RFC Editor records and found no correction to the four paragraphs; this environment limitation is recorded rather than mislabeled as a Sonnet result. |
| |
| ===== External checks ===== | ===== External checks ===== |
| ^ Family ^ Was ^ Now ^ Why ^ | ^ Family ^ Was ^ Now ^ Why ^ |
| | ''ZGrab'' | 31 papers, 6 spellings | **32**, 7 spellings | the compound string now also counts as ZGrab, which is what the paper says | | | ''ZGrab'' | 31 papers, 6 spellings | **32**, 7 spellings | the compound string now also counts as ZGrab, which is what the paper says | |
| | ''ZMap / XMap'' | 112 papers | **111** | unchanged by the split; **−1** from the tuple suppression below | | | ''ZMap / XMap'' | 112 papers | **110** | unchanged by the split; **−2** from the tuple suppressions below | |
| | ''scanner-generic'' union | 156 corpus-wide, 38 in DNSPOP | **155**, **37** | same suppression | | | ''scanner-generic'' union | 156 corpus-wide, 38 in DNSPOP | **154**, **37** | same suppressions | |
| | candidate tuples in the loose sweep | 976 | **975** | same suppression | | | candidate tuples in the loose sweep | 976 | **974** | same suppressions | |
| | |
| | **The tuple suppressions.** Two papers, both found while hand-auditing the other fold, and each costing this page one ''ZMap / XMap'' paper. They live in ''scripts/tool_use_overrides.mjs'', which //both// report scripts import, because "this tuple is not an instrument the paper used" is a fact about the tuple and not about either page. |
| | |
| | - **IMC 2024 //Have you SYN me?//** is a pure network-telescope study that never ran the scanners its ''tools[]'' names — the extraction's own ''purpose'' fields read //"Fingerprint ZMap-generated scanning traffic"//, and the paper says //"we find no evidence of Unicorn being used for Internet-wide scanning"//. Four tuples suppressed, one of them ZMap. |
| | - **USENIX 2022 //Spoki//** did run ZMap, but only inside its own machine: //"A local pair of virtual Ethernet interfaces route packets between the packet source and Spoki. We use ZMap to generate TCP SYN packets, passing the MAC address of the destination interface along with a probing rate and a range of IP addresses to use as a source."// The address range is a synthetic //source// prefix; nothing is probed. Found by a review pass on 2026-09-17, **after** the first version of this section had already been published saying the count was 111. |
| |
| **The tuple suppression.** While hand-auditing the other fold, IMC 2024 //Have you SYN me? Characterizing Ten Years of Internet Scanning// turned out to be a pure network-telescope study that never ran the scanners its ''tools[]'' names — the extraction's own ''purpose'' fields read //"Fingerprint ZMap-generated scanning traffic"// and the paper says //"we find no evidence of Unicorn being used for Internet-wide scanning"//. Four of its tuples are suppressed in ''scripts/tool_use_overrides.mjs'', which //both// report scripts import, because "the paper did not run this" is a fact about the tuple and not about either page. One of the four is ZMap, so this page loses one paper from ''ZMap / XMap''. The paper's ''massDNS'' entry is **not** suppressed and DNSPOP is unchanged at **244**. | Neither paper's other DNS tuples are suppressed, and **DNSPOP is unchanged at 244**. |
| |
| After both changes the two pages agree exactly: ZMap ∪ XMap = **111** on both sides, ZGrab = **32** on both, symmetric difference **0**. ''reconcile_zmap.mjs'' exits 1 if they ever diverge again, and also if a new compound ''tools[]'' name appears whose lost part is a ''DNS_KIND'' here or a membership kind there. It was mutation-tested on 2026-09-17: removing the ''%%zgrab/zmap%%'' entry produced three failures. | After all of it the two pages agree exactly: ZMap ∪ XMap = **110** on both sides (94 + 16 there), ZGrab = **32** on both, symmetric difference **0**. ''reconcile_zmap.mjs'' exits 1 if they ever diverge again, and also if a new compound ''tools[]'' name appears whose lost part is a ''DNS_KIND'' here or a membership kind there. It was mutation-tested on 2026-09-17: removing the ''%%zgrab/zmap%%'' entry produced three failures. |
| |
| **Compound-name recall.** A blanket split would be wrong — ''miekg/dns'' is a GitHub path this fold depends on reading whole. The sweep is therefore a guard, not a rule: it looks at every used-or-produced ''tools[]'' name containing ''/'', '' + '', '' and '', '' & '' or a comma (**263 tuples**) and prints each one whose parts fold to a family the whole string does not. There are **four**. Only ''%%ZGrab/ZMap%%'' touches a figure on either page; ''%%HTTP/OneProbe%%'' is one product name, and ''%%tcpdump/libpcap%%'' and ''%%WireShark/tshark%%'' fold to kinds neither page counts. All four are printed in the reconciler's output whether or not they were split. | **Compound-name recall.** A blanket split would be wrong — ''miekg/dns'' is a GitHub path this fold depends on reading whole. The sweep is therefore a guard, not a rule: it looks at every used-or-produced ''tools[]'' name containing ''/'', '' + '', '' and '', '' & '' or a comma (**263 tuples**) and prints each one whose parts fold to a family the whole string does not. There are **four**. Only ''%%ZGrab/ZMap%%'' touches a figure on either page; ''%%HTTP/OneProbe%%'' is one product name, and ''%%tcpdump/libpcap%%'' and ''%%WireShark/tshark%%'' fold to kinds neither page counts. All four are printed in the reconciler's output whether or not they were split. |
| | [[:design]] | DNS row added to the namespace table and the page count corrected | | | [[:design]] | DNS row added to the namespace table and the page count corrected | |
| | [[design:automated_measurements]] | the scan row of the routing table now points at [[design:dns]] for DNS rather than at the TLS and IP pages | | | [[design:automated_measurements]] | the scan row of the routing table now points at [[design:dns]] for DNS rather than at the TLS and IP pages | |
| | [[design:dns]] //(2026-09-17)// | rev 1787872903 → 1789611329: the generic-scanner sentence now gives 111/32/37 and names the agreement with [[programming:internet_scanning]]; the fold-residue sentence 976 → 975 | | | [[design:dns]] //(2026-09-17)// | rev 1787872903 → 1789612622, saved twice: the generic-scanner sentence now gives **110**/32/37 and names the agreement with [[programming:internet_scanning]]; the fold-residue sentence 976 → **974**. The first save said 111 and 975; a review pass then found the Spoki tuple. | |
| |
| ===== Review log ===== | ===== Review log ===== |
