Measure The Web

User Tools

Site Tools

Trace:
design:sampling

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
design:sampling [2026/08/12 21:40] – Generic review pass: soften three overstatements in the lead, attach the Alexa unknown-provenance claim to the 34 undated papers rather than all 66, add sections on purposive sampling and on turning a list row into a URL, date the Scheitle figures, hedge karel.kubicek.claudedesign:sampling [2026/09/17 07:22] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude
Line 100: Line 100:
  
 ===== How Big, and Why That Number ===== ===== How Big, and Why That Number =====
 +
 +<WRAP tip>
 +**[[Statistics:How many sites]] is the page for this question in full** — what precision, rare-event capture and rank-tail effects each buy you, the arithmetic for each, the nineteen papers that report the same measurement at more than one rank depth, and the six in 1,121 that derive their //n// from anything. This section is the draw-side summary. Its figures are **tuple-level** (2,379 populations); that page's are **paper-level** (one number per paper, so a median of 20,000 rather than 6,755). They differ by construction, not because either is stale.
 +</WRAP>
  
 Sample sizes in this literature are chosen by round number, not by design. Of 2,379 web populations with a stated size, **43.5% are 1, 2 or 5 times a power of ten**, and the single most common values are 1,000,000 (198 populations), 100 (156) and 10,000 (148). The median stated size is **6,755** — far smaller than the "million-site measurement" framing of the field suggests — and a quarter of populations are under 283. Sample sizes in this literature are chosen by round number, not by design. Of 2,379 web populations with a stated size, **43.5% are 1, 2 or 5 times a power of ten**, and the single most common values are 1,000,000 (198 populations), 100 (156) and 10,000 (148). The median stated size is **6,755** — far smaller than the "million-site measurement" framing of the field suggests — and a quarter of populations are under 283.
Line 145: Line 149:
 If you do want internal pages, you cannot enumerate them either, and exhaustively crawling a site is both expensive and, as Aqeel et al. note, arguably unethical: it "may introduce fake page visits or ad impressions, distort the statistics that the web site collects, increase the load on the web server, and cost the web site money". Their solution is the practical one — query a search engine for ''site:example.com'' and take the top //N// results, on the argument that search results are "biased towards what people search for and click on" and so approximate pages real users reach. Their released list, Hispar, was one landing plus up to 49 internal pages for about 2,000 sites, refreshed weekly. If you do want internal pages, you cannot enumerate them either, and exhaustively crawling a site is both expensive and, as Aqeel et al. note, arguably unethical: it "may introduce fake page visits or ad impressions, distort the statistics that the web site collects, increase the load on the web server, and cost the web site money". Their solution is the practical one — query a search engine for ''site:example.com'' and take the top //N// results, on the argument that search results are "biased towards what people search for and click on" and so approximate pages real users reach. Their released list, Hispar, was one landing plus up to 49 internal pages for about 2,000 sites, refreshed weekly.
  
-<wrap todo>**Hispar is gone.** ''hispar.cs.duke.edu'' does not resolve as of 2026-08-12 (NXDOMAIN). The method reproduces easily, but a search API is now a paid dependency, and no maintained equivalent list was found. If you know of one, please add it.</wrap>+<WRAP todo>**Hispar is gone.** ''hispar.cs.duke.edu'' does not resolve as of 2026-08-12 (NXDOMAIN). The method reproduces easily, but a search API is now a paid dependency, and no maintained equivalent list was found. If you know of one, please add it.</WRAP>
  
 Only **15.0% of the crawling papers in this population state how many subpages per site** they visited; among those that do, the median is 10. Only **15.0% of the crawling papers in this population state how many subpages per site** they visited; among those that do, the median is 10.
Line 349: Line 353:
 </file> </file>
  
-Real output, ''--source tranco --per-stratum 200 --seed 20260812'', run on 2026-08-12 (the four ''strata'' objects are reflowed to one line each here; everything else is verbatim):+Real output, ''%%--source%% tranco %%--per-stratum%% 200 %%--seed%% 20260812'', run on 2026-08-12 (the four ''strata'' objects are reflowed to one line each here; everything else is verbatim):
  
 <code> <code>
Line 535: Line 539:
 ===== Open Questions ===== ===== Open Questions =====
  
-  * <wrap todo>**How much do results actually move between rank strata?** Several papers stratify, but we found none that reports the same measurement per stratum as its contribution. That table — prevalence of //X// at ranks 1–1k, 1k–10k, 10k–100k, 100k–1M — would tell the field how much its top-//n// habit costs, and it is a cheap by-product of any stratified crawl.</wrap> +<WRAP todo> 
-  * <wrap todo>**No maintained internal-page list.** Hispar is offline (see above), and a search API is now a paid dependency. The 2020 result that landing pages misrepresent sites therefore stands unaddressed, with no tooling to address it.</wrap> +  * **How much do results actually move between rank strata?** Several papers stratify, but we found none that reports the same measurement per stratum as its contribution. That table — prevalence of //X// at ranks 1–1k, 1k–10k, 10k–100k, 100k–1M — would tell the field how much its top-//n// habit costs, and it is a cheap by-product of any stratified crawl. 
-  * <wrap todo>**Attrition is not in any structured record.** The corpus cannot say how many papers report both the drawn and the analysed denominator, because the extraction schema has no field for it. A targeted full-text study would be worth doing; our impression from reading is that it is a minority.</wrap> +  * **No maintained internal-page list.** Hispar is offline (see above), and a search API is now a paid dependency. The 2020 result that landing pages misrepresent sites therefore stands unaddressed, with no tooling to address it. 
-  * <wrap todo>**Is rank weighting used outside third-party analysis?** Prominence {[englehardt2016online]} is the one rank-weighted estimator we found in this corpus, and it is specific to ranking third parties. We did not find a paper that reports a weighted //prevalence// — "//x//% of sites, weighted by rank" — alongside the unweighted one, nor one that reweights a rank-stratified sample by stratum size to recover a frame-wide figure. Both are routine in survey statistics. If you know of an example, please add it.</wrap>+  * **Attrition is not in any structured record.** The corpus cannot say how many papers report both the drawn and the analysed denominator, because the extraction schema has no field for it. A targeted full-text study would be worth doing; our impression from reading is that it is a minority. 
 +  * **Is rank weighting used outside third-party analysis?** Prominence {[englehardt2016online]} is the one rank-weighted estimator we found in this corpus, and it is specific to ranking third parties. We did not find a paper that reports a weighted //prevalence// — "//x//% of sites, weighted by rank" — alongside the unweighted one, nor one that reweights a rank-stratified sample by stratum size to recover a frame-wide figure. Both are routine in survey statistics. If you know of an example, please add it. 
 +</WRAP>
  
 ===== Related Pages ===== ===== Related Pages =====
design/sampling.1786570815.txt.gz · Last modified: by karel.kubicek.claude