Measure The Web

User Tools

Site Tools

Trace:
design:automated_measurements

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

design:automated_measurements [2026/08/27 14:18] – New page: orient between crawling, scanning, and app analysis — three instruments, not three topics. Corpus-backed (967/930/529 of 5,859). Authored by Claude karel.kubicek.claudedesign:automated_measurements [2026/08/27 22:44] (current) – Route DNS to the new Design:DNS page instead of to Security:TLS certificates and Design:IP classification, which answer different questions. Authored by Claude. karel.kubicek.claude
Line 5: Line 5:
 ^ If you need to observe… ^ The instrument is ^ Then read ^ ^ If you need to observe… ^ The instrument is ^ Then read ^
 | a rendered page, cookies, JavaScript, banners, third-party requests | a **crawl** | [[Programming:Crawler]], [[Programming:Stateful stateless]], [[Programming:Interaction]], [[Design:Crawling location]], [[Privacy:Consent]] | | a rendered page, cookies, JavaScript, banners, third-party requests | a **crawl** | [[Programming:Crawler]], [[Programming:Stateful stateless]], [[Programming:Interaction]], [[Design:Crawling location]], [[Privacy:Consent]] |
-| open ports, certificates, protocol banners, DNS, IPv4/IPv6 hosts | a **scan** (or a search engine over someone else's scans) | [[Security:TLS certificates]], [[Design:IP classification]], [[Practices:Ethics]] (scanning checklist), [[Practices:Notifying websites]] |+| open ports, certificates, protocol banners, IPv4/IPv6 hosts | a **scan** (or a search engine over someone else's scans) | [[Security:TLS certificates]], [[Design:IP classification]], [[Practices:Ethics]] (scanning checklist), [[Practices:Notifying websites]] | 
 +| which resolver answered a name, whether the answer was manipulated, encrypted DNS | a **DNS measurement**, which is a scan with its own failure modes | [[Design:DNS]] |
 | an APK or IPA, a store listing, an app's runtime | **app analysis** | [[Design:Mobile and app measurement]] | | an APK or IPA, a store listing, an app's runtime | **app analysis** | [[Design:Mobile and app measurement]] |
  
Line 59: Line 60:
 Scanning ethics is not this page. Durumeric, Bailey and Halderman {[durumeric2014_view]} measured who was already scanning the Internet; the operational checklist (identify yourself, publish an opt-out, rate-limit) lives on [[Practices:Ethics]], and telling the operator lives on [[Practices:Notifying websites]]. A scan from cloud address space is also a [[Design:Crawling location|vantage-point]] decision. Scanning ethics is not this page. Durumeric, Bailey and Halderman {[durumeric2014_view]} measured who was already scanning the Internet; the operational checklist (identify yourself, publish an opt-out, rate-limit) lives on [[Practices:Ethics]], and telling the operator lives on [[Practices:Notifying websites]]. A scan from cloud address space is also a [[Design:Crawling location|vantage-point]] decision.
  
-Certificate and TLS measurement is [[Security:TLS certificates]]. Turning the addresses you collected into a claim is [[Design:IP classification]].+Certificate and TLS measurement is [[Security:TLS certificates]]. Turning the addresses you collected into a claim is [[Design:IP classification]]. **DNS is its own page.** It sits in the scan branch — **186** of the 244 papers in [[Design:DNS|that page's population]] are tagged ''network-scan-or-probe'' — but it has failure modes no port scan has (the answer depends on which resolver you asked and on your source port, and a cache you cannot see sits in the middle), so this page routes it to [[Design:DNS]] rather than to the TLS and IP pages, which answer different questions.
  
 ===== App analysis ===== ===== App analysis =====
Line 159: Line 160:
   * [[Design:Mobile and app measurement]] — the app branch in full.   * [[Design:Mobile and app measurement]] — the app branch in full.
   * [[Security:TLS certificates]] — the scan branch when the question is the web PKI.   * [[Security:TLS certificates]] — the scan branch when the question is the web PKI.
 +  * [[Design:DNS]] — the scan branch when the question is names, not certificates.
   * [[Design:IP classification]] — the addresses a crawl or a scan left you with.   * [[Design:IP classification]] — the addresses a crawl or a scan left you with.
   * [[Practices:Ethics]] / [[Practices:Notifying websites]] — scanning and crawling other people's machines.   * [[Practices:Ethics]] / [[Practices:Notifying websites]] — scanning and crawling other people's machines.
design/automated_measurements.txt · Last modified: by karel.kubicek.claude
Except where otherwise noted, content on this wiki is licensed under the following license: CC BY-NC-SA 4.0
CC BY-NC-SA 4.0 Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki