| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| design:automated_measurements [2026/08/28 19:04] – Route the 'existing-dataset analysis' bullet to the new design:existing_datasets page. Authored by Claude karel.kubicek.claude | design:automated_measurements [2026/09/11 10:50] (current) – Route differential/sock-puppet audits to the new Design:Algorithm audits page; note why its population is not in this page's studyTypes counts. Authored by Claude karel.kubicek.claude |
|---|
| ^ If you need to observe… ^ The instrument is ^ Then read ^ | ^ If you need to observe… ^ The instrument is ^ Then read ^ |
| | a rendered page, cookies, JavaScript, banners, third-party requests | a **crawl** | [[Programming:Crawler]], [[Programming:Stateful stateless]], [[Programming:Interaction]], [[Design:Crawling location]], [[Privacy:Consent]] | | | a rendered page, cookies, JavaScript, banners, third-party requests | a **crawl** | [[Programming:Crawler]], [[Programming:Stateful stateless]], [[Programming:Interaction]], [[Design:Crawling location]], [[Privacy:Consent]] | |
| | open ports, certificates, protocol banners, IPv4/IPv6 hosts | a **scan** (or a search engine over someone else's scans) | [[Security:TLS certificates]], [[Design:IP classification]], [[Practices:Ethics]] (scanning checklist), [[Practices:Notifying websites]] | | | open ports, certificates, protocol banners, IPv4/IPv6 hosts | a **scan** (or a search engine over someone else's scans) | [[Programming:Internet scanning]], [[Security:TLS certificates]], [[Design:IP classification]], [[Practices:Ethics]] (scanning checklist), [[Practices:Notifying websites]] | |
| | which resolver answered a name, whether the answer was manipulated, encrypted DNS | a **DNS measurement**, which is a scan with its own failure modes | [[Design:DNS]] | | | which resolver answered a name, whether the answer was manipulated, encrypted DNS | a **DNS measurement**, which is a scan with its own failure modes | [[Design:DNS]] | |
| | an APK or IPA, a store listing, an app's runtime | **app analysis** | [[Design:Mobile and app measurement]] | | | an APK or IPA, a store listing, an app's runtime | **app analysis** | [[Design:Mobile and app measurement]] | |
| | | whether a platform treats **two constructed identities** differently — personalised results, targeted ads, a feed, a quoted price | a **differential audit**, which is a crawl whose population is the arms rather than the sites | [[Design:Algorithm audits]], [[Programming:Stateful stateless]], [[Statistics:Hypothesis testing]] | |
| |
| <WRAP important> | <WRAP important> |
| * **Censys is often a dataset, not a scan you ran.** Durumeric et al. {[durumeric2015_search]} introduced it as //"a public search engine and data processing facility backed by data collected from ongoing Internet-wide scans"//. **72** papers in the corpus name Censys as a used or produced tool; **63** of those are in the scan branch; **43** of those 63 are also tagged ''existing-dataset-analysis''. That tag does not prove they only queried Censys — a paper can scan and re-analyse — but it is why you must not count the name as "they ran ZMap". Of scan papers, Censys is named by about 10% in 2018–2024 (26 of 255, then 24 of 245). The 2025–2026* cell is 5 of 168 (3.0%) and is too thin to read as a decline. ''censys.com'' served HTTP 200 on 2026-08-27; ''search.censys.io'' returns **403** to automated clients, including with a browser User-Agent — a bot wall, not an outage. | * **Censys is often a dataset, not a scan you ran.** Durumeric et al. {[durumeric2015_search]} introduced it as //"a public search engine and data processing facility backed by data collected from ongoing Internet-wide scans"//. **72** papers in the corpus name Censys as a used or produced tool; **63** of those are in the scan branch; **43** of those 63 are also tagged ''existing-dataset-analysis''. That tag does not prove they only queried Censys — a paper can scan and re-analyse — but it is why you must not count the name as "they ran ZMap". Of scan papers, Censys is named by about 10% in 2018–2024 (26 of 255, then 24 of 245). The 2025–2026* cell is 5 of 168 (3.0%) and is too thin to read as a decline. ''censys.com'' served HTTP 200 on 2026-08-27; ''search.censys.io'' returns **403** to automated clients, including with a browser User-Agent — a bot wall, not an outage. |
| * **nmap, traceroute, RIPE Atlas** are the other named families (40, 31, 25 papers in the scan branch). They answer different questions than ZMap does (host discovery vs path vs volunteer vantage points). **167** scan papers name a ''network-scanner'' the fold did not map — one-off research scanners, listed on the provenance page. That residue is the scanning analogue of the home-grown crawler row on [[Programming:Crawler]]. | * **nmap, traceroute, RIPE Atlas** are the other named families (40, 31, 25 papers in the scan branch). They answer different questions than ZMap does (host discovery vs path vs volunteer vantage points). **167** scan papers name a ''network-scanner'' the fold did not map — one-off research scanners, listed on the provenance page. That residue is the scanning analogue of the home-grown crawler row on [[Programming:Crawler]]. |
| | |
| | **The instrument page for this branch is [[Programming:Internet scanning]]**, added on 2026-09-10 — the ZMap/ZGrab invocation, the probe rate and its wall-clock cost, the exclusion file, source-address hygiene, and the IPv6 hitlist problem. It defines its own population — the papers that name an active-scan instrument, a smaller and differently derived set — rather than reusing the 930, and its scanner fold is a separate, stricter one; where the two pages disagree on a count, that page carries the number and says why. |
| |
| Scanning ethics is not this page. Durumeric, Bailey and Halderman {[durumeric2014_view]} measured who was already scanning the Internet; the operational checklist (identify yourself, publish an opt-out, rate-limit) lives on [[Practices:Ethics]], and telling the operator lives on [[Practices:Notifying websites]]. A scan from cloud address space is also a [[Design:Crawling location|vantage-point]] decision. | Scanning ethics is not this page. Durumeric, Bailey and Halderman {[durumeric2014_view]} measured who was already scanning the Internet; the operational checklist (identify yourself, publish an opt-out, rate-limit) lives on [[Practices:Ethics]], and telling the operator lives on [[Practices:Notifying websites]]. A scan from cloud address space is also a [[Design:Crawling location|vantage-point]] decision. |
| * **A manual audit, a code analysis, or a system paper.** Those are the three larger ''studyTypes'' (1,826 / 1,484 / 3,967). A system paper that also crawled is in the 153-paper gap above. | * **A manual audit, a code analysis, or a system paper.** Those are the three larger ''studyTypes'' (1,826 / 1,484 / 3,967). A system paper that also crawled is in the 153-paper gap above. |
| * **A tutorial on HTTP, DNS, or TLS.** Those are specs. This page is which instrument the field uses to observe them. | * **A tutorial on HTTP, DNS, or TLS.** Those are specs. This page is which instrument the field uses to observe them. |
| | * **A differential audit.** Driving two deliberately different profiles at one platform and comparing what comes back is a crawl by machinery and an experiment by design; the arms, not the site list, are the population. [[Design:Algorithm audits]] owns it, and its 32-paper population is derived by hand rather than from ''studyTypes'', so it does not appear in any count on this page. |
| |
| ===== What to report ===== | ===== What to report ===== |
| * [[Design:Longitudinal]] — doing it twice. | * [[Design:Longitudinal]] — doing it twice. |
| * [[Design:Mobile and app measurement]] — the app branch in full. | * [[Design:Mobile and app measurement]] — the app branch in full. |
| | * [[Programming:Internet scanning]] — the scan branch's instrument page: invocation, rate, exclusion file, IPv6. |
| * [[Security:TLS certificates]] — the scan branch when the question is the web PKI. | * [[Security:TLS certificates]] — the scan branch when the question is the web PKI. |
| * [[Design:DNS]] — the scan branch when the question is names, not certificates. | * [[Design:DNS]] — the scan branch when the question is names, not certificates. |
