There is no register of the web. No authority enumerates websites, so no study can draw a random sample of them — every measurement substitutes some list for the population it wants to talk about, and inherits whatever that list covers, over-covers and misses. Sampling on the web is therefore not the textbook problem of drawing from a frame; it is the problem of choosing a frame that does not exist, and then being honest about the gap.

This page is about how you draw: which method, at what size, at what unit, and what you have to record for the draw to be repeatable. Website selection is about which list — the ranking services, their provenance and their biases — and the two pages are meant to be read together. Where a claim here is about a particular list, it links there rather than restating it.

The short version, from 1,153 papers in our corpus of seven security and privacy venues that drew a population of websites, domains or web pages (see Use in Publications):

• 57.6% take a top-n, and the share is still rising. Only 6.0% stratify by rank.
• 99.1% name the list, 97.2% give a size, and 51.7% say which version — so half of the field's samples cannot be redrawn.
• The list itself changed under the field: papers naming Alexa fell from 47.1% of the population in 2021 to 8.3% in 2025, while Tranco rose from 31.0% to 56.4% over the same years. 66 papers published in 2023–2026 still sample from “the Alexa top n”, a list that has not been published since 2022, and 51.5% of them give no version at all.

Almost every methodological argument about a web measurement is really an argument about one of three gaps:

The first gap is the one nobody can close and the one to be explicit about. The last gap is the one that is routinely closed silently, and it is where percentages go wrong.

You draw 10,000 sites; some do not resolve, some return 5xx, some serve a bot wall, some time out, some are parked. What you report a percentage of is now a choice, and readers cannot check it unless you say which one you made.

The reference behaviour is Englehardt and Narayanan's million-site measurement [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)], which states the attrition and the rule in one sentence:

In presenting our results we only consider sites that loaded successfully. For example, for the 1 Million site measurement, we present statistics for 917,261 sites.

That is 8.3% of the drawn sample gone before analysis, and the paper's percentages mean something precise because it says so. A 2025 example at the other scale: [2Annamalai, Meenatchi Sundaram Muthu Selva; De Cristofaro, Emiliano; Bilogrevic, Igor (2025): "Beyond the Crawl: Unmasking Browser Fingerprinting in Real User Interactions", in: Proceedings of the ACM Web Conference. (DOI)]¹⁾ reports that “the automated crawler failed to visit 15 out of the 3K websites (≈ 1%)”, and attributes most of them to bot detection — hedged as “we believe”, on the evidence that 86.7% of those sites returned 4XX errors. Even at 1%, that attrition is itself a finding, because it is not random with respect to what is being measured.

Attrition is not random. Sites that block datacenter IPs, sites behind bot management, and sites in the tail are all more likely to drop out, and all three correlate with the things privacy and security papers measure. Report both denominators.

The extraction behind the figures on this page assigns one of nine sampling methods (plus a not-stated sentinel), and they are not interchangeable. What matters is not the label but what population the result generalises to:

Counts are papers with at least one web population drawn that way, of 1,153; a paper can appear in several rows. The ninth method, snowball, fires on 2 papers and is omitted.

The dominant design in the field is “we crawled the Tranco top 10k”. It has real advantages — cheap, comparable across papers, weighted towards sites that matter to users — and one property that is constantly forgotten: it involves no sampling at all. There is no randomness, so a confidence interval computed on it is answering a question nobody asked. If 6.4% of your top-10k sites do X, that 6.4% is an exact census figure for those 10,000 sites, and the uncertainty that matters is entirely about how unlike the rest of the web those 10,000 sites are.

They are, measurably, very unlike it. Scheitle et al. [3Scheitle, Quirin; Hohlfeld, Oliver; Gamba, Julien; Jelten, Jonas; Zimmermann, Torsten; Strowes, Stephen D.; Vallina-Rodriguez, Narseo (2018): "A Long Way to the Top: Significance, Structure, and Stability of Internet Top Lists", in: Proceedings of the Internet Measurement Conference 2018, pp. 478–493. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] compared list domains against the general com/net/org population and found the head is a technological outlier: HTTP/2 adoption reached 26.6% for the Alexa Top 1M against 7.84% for com/net/org overall, IPv6 enablement 11–13% against about 4%, and CDN use on the Top 1M lists exceeded the general population “by at least a factor of two”. Any measurement of adoption, modernity or professionalism run on a top list is biased upward, and the size of the bias is of the same order as the effect most papers report.

The head is also a small part of the web but most of the browsing. Ruth et al.'s study of Chrome telemetry [4Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)] finds that “the top million sites capture over 95% of all page loads and time spent online, but they do so extremely unequally”: six sites account for 25% of page loads, and one site alone for 17% of desktop page loads globally. The consequence they draw is the one that should change your design — a study “calculated directly from a simple set of the top million sites place[s] disproportionate focus on the long tail of the web”, because treating a million ranks as a million equal observations weights the rank-900,000 site exactly like Google.

So a top-n sample and a uniform sample of a top-1M list are both wrong in a knowable direction, in opposite directions, and the fix is the same in both cases: say which stratum you are talking about.

Only 69 of 1,153 papers (6.0%) stratify. The designs that do are worth copying verbatim, because they make the head/tail trade-off explicit and cost nothing extra:

• “We collect data for 21,965 sites in total, including 11,965 of the Tranco top 15K, 5,000 websites uniformly sampled between the ranks of 15K to 100K, and 5,000 websites between ranks 100K and 1M.” [5Ukani, Alisha; Haddadi, Hamed; Snoeren, Alex C.; Snyder, Peter (2025): "Local Frames: Exploiting Inherited Origins to Bypass Content Blockers", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, pp. 1349-1363. (DOI)]
• “We selected 20K sites based on their ranking in the Tranco dataset: 2K, 3K, 5K, and 10K sites, respectively, from the rank ranges [1, 10K], [10K, 100K], [100K, 500K], and [500K, 1M].” [6Zhu, Jingyuan; Sun, Huanchen; Madhyastha, Harsha V. (2025): "Toward Better Efficiency vs. Fidelity Tradeoffs in Web Archives", in: Proceedings of the ACM Internet Measurement Conference, pp. 1025-1031. (DOI)]
• “For Switzerland, we selected all top 50k websites as well as 20k randomly sampled ones from the 100k-500k and 500k-1M CrUX popularity buckets, respectively.” [7Nenadić, Luka; Rodriguez, David; Calandrino, Joseph A. (2026): "Overcoming Language Barriers: Multilingual Analysis of the 2023 Swiss Privacy Law's Impact", Proceedings on Privacy Enhancing Technologies 2026(4):703-723. (DOI)] — the same design carried across two jurisdictions, with the list version stated (CrUX, August 2023).

Note what each does: an exhaustive head plus a random draw per decade of rank. That lets you report head figures as a census, tail figures with an interval, and the whole-frame figure as a weighted estimate, with the weights being the stratum sizes you already know. It also survives the reviewer question “would this hold outside the top 10k?” — which a top-n design cannot answer at all.

The script below implements exactly this against a live Tranco or CrUX list and writes the manifest a methods section needs; see versioning.

Random from a frame (190 papers) is the only design where the classical machinery applies, and it applies to the frame, not to the web. It is the right choice when the frame is genuinely the target population — all .nl domains, all extensions in a store, all certificates in a CT log — and a poor one when the frame is a top list, because a uniform draw from a top-1M list is a sample of the tail with a rounding error of head sites.

Exhaustive (165 papers, median stated size 708,000) means the frame is the study: measure every registered domain under a TLD, every host in a zone file, every page in a crawl corpus. It is the strongest design available on the web because it removes sampling error entirely, and it is only available where somebody maintains a register — which is why it clusters on DNS, certificates and app stores rather than on websites.

Seed-and-crawl (100 papers) follows links out from a seed set. It is a snowball sample on the link graph, with the biases of one: well-linked pages are over-represented, islands are unreachable, and the sample depends on crawl order and depth. It is the only way to reach pages no list enumerates, and it should never be described as random.

278 papers (24.1%) take a pre-existing dataset. This inherits two things people forget: the original sampling design, which becomes yours and must be described as such, and the original date. A 2026 paper that reuses a 2019 crawl of the Alexa top 10k is measuring the web of 2019 through a list that no longer exists. That can be exactly right — longitudinal and replication work depends on it — but it is a different claim from a fresh measurement, and the year has to be in the sentence.

Sample sizes in this literature are chosen by round number, not by design. Of 2,379 web populations with a stated size, 43.5% are 1, 2 or 5 times a power of ten, and the single most common values are 1,000,000 (198 populations), 100 (156) and 10,000 (148). The median stated size is 6,755 — far smaller than the “million-site measurement” framing of the field suggests — and a quarter of populations are under 283.

That is not necessarily wrong. What size buys you depends on which question you are asking, and the two questions have opposite scaling:

• Prevalence of a common thing (“what share of sites set a third-party cookie before consent?”). Precision improves with the square root of n, so the returns die quickly: going from 1,000 to 10,000 sites narrows a proportion's interval by about a factor of three, and 10,000 to 1,000,000 by another factor of ten at a hundred times the crawl cost. Beyond a few thousand sites, coverage error typically dominates sampling error²⁾, and adding sites from the same list does nothing about coverage. Spend the budget on a second frame instead.
• Discovery of a rare thing (“which sites run this specific fingerprinting script?”). Here n is the whole game, because your expected count is n×p and p may be 10^-4. This is what genuinely justifies a million-site crawl, and it is worth saying so explicitly when it is your reason — the reader cannot tell the two motivations apart from the number alone.

The corpus median size by method reflects this: top-n designs sit at 10,000, stratified at 4,084, random at 2,000, and exhaustive at 708,000. Purposive samples are small by nature (median 102): they are chosen, not drawn.

There is a third option that the size argument usually misses, and the canonical example is in [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)]. Rather than count how many of n sites a third party appears on — a figure that moves whenever you change n, because the sites you add are always less popular than the ones you had — they define prominence: sum 1/rank(s) over the sites s where the third party appears. That is a rank-weighted prevalence, and it is motivated exactly as a sampling fix. Their motivation is a sampling one: a plain prevalence rank is sensitive to how many websites the measurement visited, and so “different studies may rank third parties differently” for no reason but their n. Prominence instead “de-emphasizes obscure sites, and hence can be adequately … approximated by relatively small-scale measurements”.

The general form is the useful part. If your frame is a ranking, you already have a weight for every row, and a weighted estimate is both more comparable across studies and cheaper to compute at a given n than an unweighted one is to stabilise. The weights need a stated model — prominence assumes the power-law visit distribution that [4Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)] later measured directly — and, like every weighting scheme, it has to be reported as one, because a weighted and an unweighted percentage over the same crawl are different numbers.

The unit is a sampling decision, and it is the one most often made by default. In the corpus, 723 papers sample websites, 393 domains and 251 web pages — and among the 492 papers whose unit is websites and which ran a crawl, 24.8% visit only the landing page and just 32.4% of the crawling papers in this population visit more than one page per site.

The strongest result here is Aqeel et al. [8Aqeel, Waqar; Chandrasekaran, Balakrishnan; Feldmann, Anja; Maggs, Bruce M. (2020): "On Landing and Internal Web Pages: The Strange Case of Jekyll and Hyde in Web Performance Measurement", in: Proceedings of the ACM Internet Measurement Conference, pp. 680-695. (DOI)], who characterised landing versus internal pages of 1,000 sites and then went back through the literature to see how much it mattered. Reviewing web-performance papers from five networking venues 2015–2019, they scored the 119 that used a top list: 41 (34.5%) needed no revision, 48 (40.3%) a minor revision, and 30 (25.2%) a major revision for their claims to apply to internal pages. Nearly two-thirds of published claims about “websites” were really claims about one page per website.

Their incidental figure is worth repeating too: of those 119 papers, only 10 (8.4%) used a top list other than Alexa — a monoculture that the corpus data below shows has since broken up, but that shaped a decade of results.

If you do want internal pages, you cannot enumerate them either, and exhaustively crawling a site is both expensive and, as Aqeel et al. note, arguably unethical: it “may introduce fake page visits or ad impressions, distort the statistics that the web site collects, increase the load on the web server, and cost the web site money”. Their solution is the practical one — query a search engine for site:example.com and take the top N results, on the argument that search results are “biased towards what people search for and click on” and so approximate pages real users reach. Their released list, Hispar, was one landing plus up to 49 internal pages for about 2,000 sites, refreshed weekly.

Hispar is gone. hispar.cs.duke.edu does not resolve as of 2026-08-12 (NXDOMAIN). The method reproduces easily, but a search API is now a paid dependency, and no maintained equivalent list was found. If you know of one, please add it.

Only 15.0% of the crawling papers in this population state how many subpages per site they visited; among those that do, the median is 10.

Even a perfect sample of sites is not a sample of visits. Zeber et al. [9Zeber, David; Bird, Sarah; Oliveira, Camila; Rudametkin, Walter; Segall, Ilana; Wolls´en, Fredrik; Lopatka, Martin (2020): "The Representativeness of Automated Web Crawls as a Surrogate for Human Browsing", in: Proceedings of The Web Conference 2020, pp. 167–178. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] ran a crawler and a panel of real users over the same sites and found the crawler sees a systematically busier web: a median of 11.6 third-party domains per site visit for the crawler against 4.5 for humans, and 6.1 tracking domains against 1.9. Users' browsers connected to at most 8 trackers in 99% of visits to list sites; “the crawler may reach 26”.

Ahmad et al. [10Ahmad, Syed Suleman; Dar, Muhammad Daniyal; Zaffar, Muhammad Fareed; Vallina-Rodriguez, Narseo; Nithyanand, Rishab (2020): "Apophanies or Epiphanies? How Crawlers Impact Our Understanding of the Web", in: Proceedings of The Web Conference, pp. 271-280. (DOI)] come at it from the other side, comparing crawlers to each other: over 16% variation in the fraction of pages that load successfully, 2–22% differences in website-fingerprinting classifier accuracy, and third-party prevalence rankings that change with the crawler. Demir et al. [11Demir, Nurullah; Hörnemann, Jan; Große-Kampmann, Matteo; Urban, Tobias; Pohlmann, Norbert; Holz, Thorsten; Wressnegger, Christian (2023): "On the Similarity of Web Measurements Under Different Experimental Setups", in: Proceedings of the ACM Internet Measurement Conference, pp. 356-369. ACM DOI 10.1145/3618257.3624795 is listed by DBLP but was not registered with the DOI resolver as of 2026-08-12; the DOI above resolves to the authors' institutional record of the same paper (DOI)] quantify the same instability within one crawler across configurations: comparing two profiles, “48% of the underlying data varies”, and 42% of observed cookies appear in only one of five profiles.

None of this makes crawling invalid. It means the sample is a sample of crawler visits to sites, and a claim about user exposure needs either a panel, telemetry, or an explicit argument about the direction of the gap.

A sample is reproducible when a reader can reconstruct the frame. That takes a list identity, not a list name — and this is the field's clearest single failure: 51.7% of papers state a version or access date, and among the 764 papers drawing from a popularity ranking, only 48.0% version the ranking.

The mechanics are cheap, and differ per source:

The last row is the general escape hatch, and it is under-used: of the 557 papers in this population that version nothing, only 29.4% release a dataset of their own that could stand in for the version.

Tranco's permanent id is the best case available, and the corpus shows it being used — by a minority. Of the 262 papers that name Tranco, 157 (59.9%) give a version or date and at least 31 (11.8%) cite the list id itself³⁾. What that looks like in a methods section, from papers in the corpus:

• “We used the list generated on 03/26/2019 (ID: W9L9).”
• “Tranco list https://tranco-list.eu/list/4NKX/1000000, generated on 2019.09.20” — in a table alongside the Disconnect list commit, the crawler commit and the crawl date, which is the whole configuration pinned at once.
• “the research-oriented Tranco list … as of November 5, 2019 (ID: GVWK)”, then “the list from January 31, 2021 (ID: WQW9)”, then “the Tranco list from December 22, 2022 (ID: 82V9V)” — a longitudinal study with one id per wave.

That last one is the pattern to copy for repeated measurements: without a per-wave id, a longitudinal comparison cannot separate a change in the web from a change in the list. That paper goes further and publishes a table of how the top domains moved between its three lists, which is the honest way to show the frame shifting under a trend.

This draws a rank-stratified sample from a live Tranco or CrUX list and writes a manifest containing the list id, the download URL, the SHA-256 of the frame bytes, the strata, the seed, and a ready-made sentence for your methods section. Tested against both sources on 2026-08-12.

stratified_sample.py

#!/usr/bin/env python3
"""Draw a rank-stratified sample of websites, and record enough to redraw it.
 
Two things every sampling section needs and half of them lack: the exact list
that was drawn from, and the rule that turned it into a sample. This script
emits both as a manifest next to the sample.
 
    # current Tranco list, 200 sites per decade of rank, seed 20260812
    python3 stratified_sample.py --source tranco --per-stratum 200 --seed 20260812
 
    # a specific, permanently addressable Tranco list
    python3 stratified_sample.py --source tranco --list-id V3JQN --per-stratum 200
 
    # CrUX, whose ranks are magnitude buckets rather than positions
    python3 stratified_sample.py --source crux --per-stratum 200
 
Writes sample.csv (rank,domain,stratum) and sample.manifest.json. Paste the
manifest's `cite` line into your methods section.
"""
import argparse, csv, gzip, hashlib, io, json, random, sys, urllib.request, zipfile
from datetime import datetime, timezone
 
UA = "measuretheweb-stratified-sampler/1.0"
TRANCO_DAILY = "https://tranco-list.eu/top-1m-id"
TRANCO_DOWNLOAD = "https://tranco-list.eu/download/{list_id}/1000000"
TRANCO_PERMALINK = "https://tranco-list.eu/list/{list_id}"
CRUX_CURRENT = "https://raw.githubusercontent.com/zakird/crux-top-lists/main/data/global/current.csv.gz"
 
# Decades of rank. The head is where the popular-site literature lives and the
# tail is where most of the web is, so a sample that is uniform over ranks is
# almost entirely tail and a top-n sample is entirely head. Strata make the
# trade-off explicit instead of implicit.
STRATA = [(1, 1_000), (1_001, 10_000), (10_001, 100_000), (100_001, 1_000_000)]
 
 
def fetch(url):
    req = urllib.request.Request(url, headers={"User-Agent": UA})
    with urllib.request.urlopen(req, timeout=180) as resp:
        return resp.read()
 
 
def load_tranco(list_id):
    if list_id is None:
        list_id = fetch(TRANCO_DAILY).decode().strip()
    raw = fetch(TRANCO_DOWNLOAD.format(list_id=list_id))
    # /download/<id>/<n> serves bare CSV; /top-1m.csv.zip serves a zip. Sniff,
    # so the script keeps working whichever endpoint you point it at.
    if raw[:2] == b"PK":
        with zipfile.ZipFile(io.BytesIO(raw)) as z:
            text = z.read(z.namelist()[0]).decode()
    else:
        text = raw.decode()
    ranked = [(int(r), d) for r, d in csv.reader(io.StringIO(text)) if r]
    return ranked, {
        "source": "tranco",
        "list_id": list_id,
        "download": TRANCO_DOWNLOAD.format(list_id=list_id),
        "permalink": TRANCO_PERMALINK.format(list_id=list_id),
        "bytes_sha256": hashlib.sha256(raw).hexdigest(),
        "rank_semantics": "position, 1..1000000",
    }
 
 
def load_crux():
    raw = fetch(CRUX_CURRENT)
    text = gzip.decompress(raw).decode()
    rows = list(csv.DictReader(io.StringIO(text)))
    # CrUX publishes a rank-magnitude bucket, not a position, and shuffles
    # within the bucket. Treating the file's line order as a rank is the single
    # most common way to misuse this list; we keep the bucket instead.
    ranked = [(int(r["rank"]), r["origin"]) for r in rows]
    return ranked, {
        "source": "crux",
        "list_id": None,
        "download": CRUX_CURRENT,
        "permalink": "https://github.com/zakird/crux-top-lists",
        "bytes_sha256": hashlib.sha256(raw).hexdigest(),
        "rank_semantics": "magnitude bucket (1000/10000/100000/1000000), random order within bucket",
    }
 
 
def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--source", choices=["tranco", "crux"], default="tranco")
    ap.add_argument("--list-id", help="Tranco list id; default is today's list")
    ap.add_argument("--per-stratum", type=int, default=200)
    ap.add_argument("--seed", type=int, default=0)
    ap.add_argument("--out", default="sample")
    args = ap.parse_args()
 
    ranked, prov = load_tranco(args.list_id) if args.source == "tranco" else load_crux()
    by_rank = {}
    for rank, name in ranked:
        by_rank.setdefault(rank, []).append(name)
 
    rng = random.Random(args.seed)
    sample, strata_report = [], []
    for lo, hi in STRATA:
        pool = [(r, n) for r, ns in by_rank.items() if lo <= r <= hi for n in ns]
        pool.sort()
        take = min(args.per_stratum, len(pool))
        drawn = rng.sample(pool, take) if take < len(pool) else pool
        label = f"{lo}-{hi}"
        sample += [(r, n, label) for r, n in sorted(drawn)]
        strata_report.append({"stratum": label, "in_frame": len(pool), "drawn": take})
        if take < args.per_stratum:
            print(f"WARNING: stratum {label} has only {len(pool)} entries", file=sys.stderr)
 
    with open(f"{args.out}.csv", "w", newline="") as fh:
        w = csv.writer(fh)
        w.writerow(["rank", "name", "stratum"])
        w.writerows(sample)
 
    manifest = {
        "drawn_at_utc": datetime.now(timezone.utc).isoformat(timespec="seconds"),
        "frame": prov,
        "frame_entries": len(ranked),
        "design": "rank-stratified, simple random sample without replacement within each stratum",
        "strata": strata_report,
        "seed": args.seed,
        "n": len(sample),
        "sample_sha256": hashlib.sha256(open(f"{args.out}.csv", "rb").read()).hexdigest(),
        "cite": (
            f"We drew {len(sample)} sites from the {prov['source']} list "
            f"{prov['list_id'] or '(dated snapshot, see manifest)'} "
            f"({prov['permalink']}), {args.per_stratum} per rank decade over "
            f"{', '.join(s['stratum'] for s in strata_report)}, "
            f"simple random sampling without replacement within each stratum, seed {args.seed}."
        ),
    }
    with open(f"{args.out}.manifest.json", "w") as fh:
        json.dump(manifest, fh, indent=2)
 
    print(json.dumps(manifest, indent=2))
 
 
if __name__ == "__main__":
    main()

Real output, –source tranco –per-stratum 200 –seed 20260812, run on 2026-08-12 (the four strata blocks are abbreviated to one line each here; everything else is verbatim):

{
  "drawn_at_utc": "2026-08-12T21:04:41+00:00",
  "frame": {
    "source": "tranco",
    "list_id": "645KX",
    "download": "https://tranco-list.eu/download/645KX/1000000",
    "permalink": "https://tranco-list.eu/list/645KX",
    "bytes_sha256": "feab56e5b4f48e3c1393916d74226d1d6234416c621390be29bb72199caf18d7",
    "rank_semantics": "position, 1..1000000"
  },
  "frame_entries": 1000000,
  "design": "rank-stratified, simple random sample without replacement within each stratum",
  "strata": [
    { "stratum": "1-1000",         "in_frame": 1000,   "drawn": 200 },
    { "stratum": "1001-10000",     "in_frame": 9000,   "drawn": 200 },
    { "stratum": "10001-100000",   "in_frame": 90000,  "drawn": 200 },
    { "stratum": "100001-1000000", "in_frame": 900000, "drawn": 200 }
  ],
  "seed": 20260812,
  "n": 800,
  "sample_sha256": "e3c38a539111c9872b62682f8d289f5d573248d95fafe6ce3bf49ea02bf92249",
  "cite": "We drew 800 sites from the tranco list 645KX (https://tranco-list.eu/list/645KX), 200 per rank decade over 1-1000, 1001-10000, 10001-100000, 100001-1000000, simple random sampling without replacement within each stratum, seed 20260812."
}

The in_frame column is the point: the four strata contain 1,000, 9,000, 90,000 and 900,000 sites, so drawing 200 from each over-samples the head by a factor of 900 relative to a uniform draw. That is deliberate and it is recoverable — reweight by in_frame when you want a frame-wide estimate, and report per stratum when you do not.

The figures below come from a structured extraction over 5,859 full-text papers from CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf and IEEE S&P, 2010–2026. The population for this page is the 1,153 papers that drew at least one study population whose unit is websites, domains or web pages — 19.7% of the corpus. Only those web-unit populations are counted, so a paper that sampled websites top-n and participants by convenience contributes only the first. Sentinel values (not-stated) are counted as silence, never as answers, and papers are counted, never tuples, except where a table says otherwise. 2025 and 2026 are provisional venue-years: CCS and IMC 2026 have not been held and two more 2026 venue-years are incompletely indexed, so they are under-represented by construction.

Web-unit populations in scope: 2,525 across the 1,153 papers. By unit: websites 723 (62.7%), domains 393 (34.1%), web pages 251 (21.8%). By venue: USENIX 230, IMC 193, WWW 186, CCS 179, PETS 132, IEEE S&P 117, NDSS 116 — sampling the web is not concentrated in one venue.

Shares exceed 100% because a paper may draw several populations different ways. 98.4% state a sampling method for at least one web population — this is the field's best-reported design decision, and the contrast with the 51.7% that state a list version is the finding.

Top-n has not been superseded; it has consolidated. Stratified sampling grew from 1.0% to a peak of 8.9% in 2022–2024 and did not continue rising. Nothing in the 2025–2026 slice suggests a new sampling method entering the field: searching the page population from 2024 onward for a frame whose name mentions a language model returns three papers, and none of them uses a model to draw a sample of the web — two sample the GPT Store as a platform and one builds a synthetic corpus. Sampling practice has been stable for a decade; what changed underneath it is the list.

Free-text source names were folded into frame families by scripts/sample_fold.mjs; 1,143 papers (99.1%) name a source.

Two thirds of web sampling is done from a popularity ranking, and the frames that would answer “what does the whole web do” — registries, zone files, certificate logs — carry under a tenth of the field between them. 472 distinct source strings across 519 populations did not fold into any family; 115 papers are left with no frame kind at all. The full residue is on the provenance page.

This is a warning about reusing this field, and about every published table of “which list do papers use”:

“Alexa” appears as Alexa top 1M, Alexa Top 1 Million Global Sites, 1M-top Alexa, Alexa's list, Alex Top 1500 websites and 350 other ways. Tranco is the least mangled because its name carries no size, which is a small argument for naming a list without a number.

Papers naming each list, as a share of the page population in each year:

Tranco is current practice; Alexa is residue. The transition tracks the retirement (May 2022) with the two-to-three-year lag you would expect from submission cycles. CrUX is the outlier: Ruth et al. [12Ruth, Kimberly; Kumar, Deepak; Wang, Brandon; Valenta, Luke; Durumeric, Zakir (2022): "Toppling Top Lists: Evaluating the Accuracy of Popular Website Lists", in: Proceedings of the 22nd ACM Internet Measurement Conference, pp. 374–387. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] measured every public list against Cloudflare's own traffic data and found CrUX the most accurate across all metrics. What “inaccurate” costs, in their numbers, is a property of the other lists — of the 1,790 Alexa top-10K domains they could measure, 70% were placed by Cloudflare in a lower rank-magnitude bucket and 27.2% two or more orders of magnitude lower. Four years on, CrUX is still named by under 10% of papers: the most accurate list is not the most used one. Read the 2026 row with care: it rests on 57 papers from incomplete venue-years.

Tuple-level, because a paper may draw several populations: 2,379 of 2,525 web populations (94.2%) state a size, and 1,121 papers (97.2%) state one for at least one population.

Median 6,755; quartiles 283 and 103,541; the 90th percentile is 1,067,968. 43.5% of stated sizes are 1, 2 or 5 times a power of ten, and 198 populations are exactly 1,000,000. Median size by method: top-n 10,000 (879 populations), exhaustive 708,000 (235), seed-and-crawl 47,847 (116), pre-existing-dataset 24,000 (346), stratified 4,084 (76), random 2,000 (237), purposive 102 (406).

Of the 764 papers drawing from a popularity ranking — the case where the version matters most, because the list changes daily — 367 (48.0%) version it and 397 (52.0%) do not.

This is genuine improvement — versioning has more than doubled — and it has stalled around 60%. Releasing the drawn list would close the gap for the rest, and mostly does not happen: 58.0% of this population release an artifact of their own and 33.8% release a dataset, but of the 557 papers that version nothing, only 164 (29.4%) release a dataset that could stand in for the missing version.

Of the 680 papers in this population that also crawled:

Only 32.4% visit more than one page per site, and only 15.0% state how many subpages per site (median 10, where stated). Among the 492 papers whose unit is explicitly websites and which crawled, 24.8% visited the landing page only — which is precisely the population Aqeel et al. [8Aqeel, Waqar; Chandrasekaran, Balakrishnan; Feldmann, Anja; Maggs, Bruce M. (2020): "On Landing and Internal Web Pages: The Strange Case of Jekyll and Hyde in Web Performance Measurement", in: Proceedings of the ACM Internet Measurement Conference, pp. 680-695. (DOI)] showed would need its claims revised.

• How they were produced. One structured record per paper was extracted from full text; each tuple carries a verbatim evidence quote and its section, so any figure here traces to a sentence. The population is defined by the extraction's population[].unit enum, which is a stable field; the free-text sourceList is not, and is folded before counting (see below).
• Folding. sourceList agrees run-to-run on roughly a fifth of exact strings, so it is reported as folded families and as rankings, never as precise percentages. The fold, its rules and its full unmapped residue are on the provenance page. Family shares are of papers naming any source and exceed 100% because a paper may draw from several frames.
• Silence is not absence. “Does not state a version” means the paper did not say, not that the authors did not know. These are reporting figures.
• samplingMethod is an enum assigned by the extractor, not a phrase copied from the paper. A design that a paper calls “a representative sample” may be scored top-n, correctly. Read the enum as the extractor's reading of what was done.
• Venue coverage. Seven venues only. 2025 and 2026 are incomplete for reasons of calendar and indexing, so per-year trends should be read as ending in 2024; the two provisional rows are labelled everywhere they appear.
• Every query behind this section, its report script and its unedited output are on sampling; corpus-level caveats are on corpus.

A sampling paragraph a reader can act on names, in this order:

1. Target population — what the claim is about, in words, not in list names.
2. Frame — the list, with its identity: a Tranco list id, a CrUX month, a crawl id, a zone-file date, or a hash of the file you archived.
3. Design — top-n, random, stratified (with the strata), exhaustive, purposive (with the selection rule), or reuse (with whose sample and from when).
4. Size — drawn, and analysed after attrition, with the reason for the difference.
5. Unit — site, registrable domain, origin, page; and if pages, how they were chosen and how many per site.
6. Seed, if any randomness was involved, so the draw is repeatable.
7. The drawn list itself, as an artifact, whenever licensing allows it. It makes every other item above checkable.

• How much do results actually move between rank strata? Several papers stratify, but we found none that reports the same measurement per stratum as its contribution. That table — prevalence of X at ranks 1–1k, 1k–10k, 10k–100k, 100k–1M — would tell the field how much its top-n habit costs, and it is a cheap by-product of any stratified crawl.
• No maintained internal-page list. Hispar is offline (see above), and a search API is now a paid dependency. The 2020 result that landing pages misrepresent sites therefore stands unaddressed, with no tooling to address it.
• Attrition is not in any structured record. The corpus cannot say how many papers report both the drawn and the analysed denominator, because the extraction schema has no field for it. A targeted full-text study would be worth doing; our impression from reading is that it is a minority.
• Is rank weighting used outside third-party analysis? Prominence [1Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)] is the one rank-weighted estimator we found in this corpus, and it is specific to ranking third parties. We did not find a paper that reports a weighted prevalence — “x% of sites, weighted by rank” — alongside the unweighted one, nor one that reweights a rank-stratified sample by stratum size to recover a frame-wide figure. Both are routine in survey statistics. If you know of an example, please add it.

• Website selection — which list, and what each ranking actually measures. This page assumes you have chosen one.
• Website classification — sampling by category needs a categoriser, and its error rate becomes your stratum error.
• Crawling location — the vantage point is a second sampling decision; geo-targeted sites are a different population from each country.
• Archives — sampling from an archive samples what the archive captured, which is its own frame.
• Crawler — attrition, timeouts and bot walls are properties of the tool as much as of the sample.
• Biases — coverage and selection bias, downstream of everything on this page.
• Hypothesis testing — what a p-value means when the sample is a census of a list.
• Artifacts — publishing the drawn list is the cheapest reproducibility win available here.

[1]

Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)

[2]

Annamalai, Meenatchi Sundaram Muthu Selva; De Cristofaro, Emiliano; Bilogrevic, Igor (2025): "Beyond the Crawl: Unmasking Browser Fingerprinting in Real User Interactions", in: Proceedings of the ACM Web Conference. (DOI)

[3]

Scheitle, Quirin; Hohlfeld, Oliver; Gamba, Julien; Jelten, Jonas; Zimmermann, Torsten; Strowes, Stephen D.; Vallina-Rodriguez, Narseo (2018): "A Long Way to the Top: Significance, Structure, and Stability of Internet Top Lists", in: Proceedings of the Internet Measurement Conference 2018, pp. 478–493. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)

[4]

Ruth, Kimberly; Fass, Aurore; Azose, Jonathan; Pearson, Mark; Thomas, Emma; Sadowski, Caitlin; Durumeric, Zakir (2022): "A World Wide View of Browsing the World Wide Web", in: Proceedings of the ACM Internet Measurement Conference, pp. 317-336. (DOI)

[5]

Ukani, Alisha; Haddadi, Hamed; Snoeren, Alex C.; Snyder, Peter (2025): "Local Frames: Exploiting Inherited Origins to Bypass Content Blockers", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, pp. 1349-1363. (DOI)

[6]

Zhu, Jingyuan; Sun, Huanchen; Madhyastha, Harsha V. (2025): "Toward Better Efficiency vs. Fidelity Tradeoffs in Web Archives", in: Proceedings of the ACM Internet Measurement Conference, pp. 1025-1031. (DOI)

[7]

Nenadić, Luka; Rodriguez, David; Calandrino, Joseph A. (2026): "Overcoming Language Barriers: Multilingual Analysis of the 2023 Swiss Privacy Law's Impact", Proceedings on Privacy Enhancing Technologies 2026(4):703-723. (DOI)

[8]

Aqeel, Waqar; Chandrasekaran, Balakrishnan; Feldmann, Anja; Maggs, Bruce M. (2020): "On Landing and Internal Web Pages: The Strange Case of Jekyll and Hyde in Web Performance Measurement", in: Proceedings of the ACM Internet Measurement Conference, pp. 680-695. (DOI)

[9]

Zeber, David; Bird, Sarah; Oliveira, Camila; Rudametkin, Walter; Segall, Ilana; Wolls´en, Fredrik; Lopatka, Martin (2020): "The Representativeness of Automated Web Crawls as a Surrogate for Human Browsing", in: Proceedings of The Web Conference 2020, pp. 167–178. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)

[10]

Ahmad, Syed Suleman; Dar, Muhammad Daniyal; Zaffar, Muhammad Fareed; Vallina-Rodriguez, Narseo; Nithyanand, Rishab (2020): "Apophanies or Epiphanies? How Crawlers Impact Our Understanding of the Web", in: Proceedings of The Web Conference, pp. 271-280. (DOI)

[11]

Demir, Nurullah; Hörnemann, Jan; Große-Kampmann, Matteo; Urban, Tobias; Pohlmann, Norbert; Holz, Thorsten; Wressnegger, Christian (2023): "On the Similarity of Web Measurements Under Different Experimental Setups", in: Proceedings of the ACM Internet Measurement Conference, pp. 356-369. ACM DOI 10.1145/3618257.3624795 is listed by DBLP but was not registered with the DOI resolver as of 2026-08-12; the DOI above resolves to the authors' institutional record of the same paper (DOI)

[12]

Ruth, Kimberly; Kumar, Deepak; Wang, Brandon; Valenta, Luke; Durumeric, Zakir (2022): "Toppling Top Lists: Evaluating the Accuracy of Popular Website Lists", in: Proceedings of the 22nd ACM Internet Measurement Conference, pp. 374–387. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)