Measure The Web

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Provenance: Programming:Tranco ======

Back to [[programming:tranco|Tranco]]. Corpus-wide selection and extraction notes are on [[literature:corpus]]. This is the page-specific query log.

===== Run record =====

  * Run date: 2026-08-27 (UTC).
  * Authoring agent: Cursor Grok 4.6, executing the drain item ''programming:tranco (new)'' directly rather than via a headless ''claude -p'' session. Review: four focused/generic passes requested on GPT 5.6 Sol medium (see below).
  * Corpus at run time: 5,859 extracted papers, 2010–2026, CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. Read-only inputs under ''/workspace/publications_dataset/data/''.
  * Read first: ''data/extract/OVERVIEW.md'', the task spec, ''design:website_selection'', ''design:sampling'', ''design:longitudinal'', ''literature:corpus'', Tranco's live front page / methodology / API docs, and the 2019 NDSS PDF.
  * Target page had no revision (''core.getPageInfo'' returned "does not exist"; ''?do=export_raw'' returned the HTML error page). This is a creation, not an extension. [[Design:Website selection]] already promised [[Programming:Tranco]] as the API detail page; that is the overlap judgement: create the promised child rather than widen the selection page.
  * No write to the publication mount. Wiki saves through ''scripts/dw.mjs'' with ''--if-rev''.

===== Why this page, not an overlap =====

Three neighbours already talk about Tranco:

^ Neighbour ^ What it already answers ^ What it does not ^
| [[design:website_selection]] | why pick Tranco vs CrUX vs Umbrella vs a dead Alexa list | construction, ids, HTTP API |
| [[design:sampling]] | how to draw and pin a sample; a stratified sampler that already takes a Tranco id | the generator's knobs; the live provider set as an API fact |
| [[design:longitudinal]] | provider-set change as a comparability problem; 266/159/23 | how to fetch and configure a list |

The item asked for list construction, configurable/hardened variants, permanent citable ids, and the API. That is a programming page. I did not republish the stratified sampler. I did reuse the longitudinal id allowlist rather than re-audit 37 papers the same day; the report fails if those keys stop matching.

===== Population and queries =====

All counts are papers unless labelled tuples. Sentinels are not answers.

^ Query ^ Denominator ^ Result ^
| ''population[].sourceList'' matches ''/\btranco\b/i''; distinct ''(venue, year, slug)'' | 5,859 corpus papers | **266** (4.5%) |
| Same, also in ''POPULATIONS.crawled'' | 1,120 crawled papers | 168 (15.0%) |
| Same, with a web-unit Tranco tuple | 266 Tranco users | 262 (98.5%) |
| Tranco tuples before paper-count | 266 papers | 373 tuples |
| Exact string ''"Tranco"'' only | 266 | 180 (32.3% undercount) |
| Distinct matching ''sourceList'' spellings | 266 | 96 |
| ''listVersion'' stated on a Tranco tuple | 266 | 159 (59.8%) |
| Permanent list id, hand-audited allowlist of 23 | 266 | 23 (8.6%) |
| ''tools[].name'' matches ''/tranco/i'' | 5,859 | 27 papers (26 "Tranco", 1 "Tranco 1M", all ''used'') |
| Tranco users with no tool-field hit | 266 | 240 (90.2%) |
| Full-text ''/\btranco\b/i'' | 5,859 (4 missing ''.cols'') | 332 papers |
| Full-text only (not sourceList, not tools[]) | 332 | 58 — citation/filter, not added to 266 |
| ''inclusionDays'' / ''inclusionLists'' / "custom list" over the 266 | 266 | **0** |
| ''/\bdowdall\b/i'' over the 266 | 266 | 1 |
| ''/\bborda\b/i'' over the 266 | 266 | 2 |
| ''/tranco-list\.eu\/list\//i'' over the 266 | 266 | 57 — **not published** as an id-citation rate; includes bibliography URLs. The 23 is the hand audit. |

Year 2019 has **0** Tranco users: the NDSS 2019 paper introduced the list and did not sample from it. First users appear in 2020.

The 23-id allowlist is copied from ''scripts/report_longitudinal.mjs'' (audited 2026-08-27, 37 candidates read). This report prints FAILURE if a key leaves the Tranco-user set; it did not.

===== Folding and residue =====

Membership **is** the regex ''/\btranco\b/i''. There is no second classifier and therefore no unmapped residue. The 96 spellings are the fold, printed in full in the report output below. Exact-string "Tranco" undercounts by 32.3%.

A homograph sweep for other meanings of "Tranco" was not needed: the 58 full-text-only contexts that were sampled are related-work citations or filters that remove popular names from an abuse set, not a different token.

**Rejected probe:** ''/harden(?:ed|ing)\b/i'' over the 266 papers hits **234 (88.0%)**. That is hardened binaries, hardened TLS, ASLR, and so on — not Tranco's "hardened ranking". It is not on the content page. The publishable uptake finding is the 0/1/2 custom-knob counts above.

===== Size table caveat =====

''n'' on a Tranco tuple is the study population, which is sometimes a union with zone files, CT, Common Crawl, or Citizen Lab. The extracted maximum is 700,000,000 (USENIX 2025 ''irblock'', documents, mixed frame). That number is **not** a Tranco prefix and is not on the content page. Round prefixes (1k / 10k / 100k / 1M) appear on 166 sized tuples; 147 of those are ''top-n''. The page no longer calls all 166 a Tranco head.

===== Evidence quotes checked =====

373 Tranco-tuple ''evidence.quote'' values against ''paper.cols.txt'', whitespace- and hyphen-normalised, then 5-word windows at 60%: **204 exact, 111 partial, 58 below threshold, 0 missing quotes, 0 missing full text**.

The first 15 FAILED rows were read. They are column splices, dropped citation markers, or quotes that name the sample without repeating the word "Tranco" next to the numbers — the same class ''quote_check.mjs'' warns about. None looked fabricated. They stay in the report. No FAILED quote was used on the content page.

===== External and industry verification =====

^ Source ^ Load-bearing fact ^ Verification 2026-08-27 ^ Decision ^
| ''GET /api/lists/date/latest'' | id ''46W9X''; providers crux, farsight, majestic, radar, umbrella; Dowdall; 30-day window; filterPLD on; prefix 1M | Fetched; script ''tranco_api_probe.sh'' re-fetches | Used |
| ''GET /top-1m-id'' | same id as latest | matched ''46W9X'' | Used |
| ''GET /latest_list'' | 303 to ''/list/46W9X/1000000'' | headers | Used |
| ''GET /api/lists/id/GVWK'' | 2019 list still available, original four providers | fetched | Used |
| dated API 20191105 / 20220101 / 20220501 / 20230801 | provider-set history table | fetched, probe asserts exact arrays | Used |
| ''GET /download/46W9X/20'' | CSV ''rank,domain'', no header, 20 rows, first row google.com | python csv count (wc -l is wrong: no trailing newline) | Used |
| ''GET /ranks/domain/google.com'' | 38 days, all rank 1; documented 1 qps | fetched | Used |
| unauthenticated create/auth | HTTP 401 | PUT and GET | Used |
| homepage | still says "all four rankings" (1×) and "five providers"; Farsight "only for the default list" | fetched HTML | Used, including the stale phrase as a warning |
| API docs ''Configuration.providers'' | enum has alexa and quantcast, **not farsight** | [[https://tranco-list.eu/api_documentation]] | Used |
| methodology page | says five providers then "all four providers"; bucket geometry; CrUX/Radar notes | [[https://tranco-list.eu/methodology]] | Used for construction; not used as today's provider list |
| PyPI ''tranco'' | 0.8.1, 2024-04-02 | registry JSON | Used |
| ''tranco==0.8.1'' executed | ''.list()'' → ''46W9X'', google.com rank 1, missing → -1 | ''uv run --with tranco==0.8.1'' | Used |
| README configure example | still names alexa | PyPI description | Used as a staleness warning |
| GitHub DistriNet/tranco-list | last push 2020-03-05; ''DEFAULT_TRANCO_CONFIG'' still 2019 providers | API ''pushed_at''; raw ''shared.py'' | Used as "do not read the default from GitHub" |
| GitHub WangYihang/tranco-go-package | unarchived, pushed 2026-07-01 | API | Named as the contributed Go client |
| Python ''pin_tranco.py --prefix 20'' | manifest SHA-256 ''8c2e47ccc986d74471e2244d5449d524cd6f37fdd58ef9b7cad5f2a2a64d7a1c'', 325 bytes, 20 rows | run 2026-08-27T09:58:31Z | Quoted on the page; five ''head'' objects reflowed to one line |

Rejected:

  * GitHub ''DEFAULT_TRANCO_CONFIG'' as a description of today's default — six years stale.
  * Python README's alexa-inclusive ''configure'' snippet as current default advice.
  * SEO "best top-list" roundups. None were consulted.
  * The 88% "hardened" sweep, as a Tranco-variant rate.
  * The 57-paper ''tranco-list.eu/list/'' sweep, as an id-citation rate.
  * ''/download/<id>/full'' as equivalent to the daily 1M — the response was much larger (~104 MB) than a 1M CSV; I did not count rows and did not publish a figure.
  * BigQuery tables — linked from the front page, not queried this sitting; the content page says so.
  * Custom-list generation — no API token in this environment; 401 without it is the verified fact.

===== What the corpus and sources do not establish =====

  * How many of the 266 used a **custom** list vs the daily default. Schema has no field; full-text knobs are silent. The 23 ids are a mixture and were not classified by configuration.
  * A current accuracy comparison of the five-provider default against Cloudflare HTTP-log ground truth. Ruth et al. is February 2022, pre-CrUX-in-Tranco. (Their GT is server-side HTTP request logs, not 1.1.1.1 resolver traffic; corrected 2026-08-27 during the Radar sitting.)
  * Whether ''alexa''/''quantcast'' still produce a list if you pass them to ''PUT /lists/create'' for a 2026 window. Enum membership is not runtime behaviour. Untested (no token).
  * Tomorrow's daily id. ''46W9X'' is 2026-08-26.

===== Judgement calls =====

  * Create ''programming:tranco'' rather than broaden website_selection. The latter already defers API detail here.
  * Reuse the longitudinal id allowlist rather than spend the sitting re-reading 37 papers.
  * Put live API composition on this page even though sampling and longitudinal also have a dated snapshot — this page is where a reader who was sent "see Programming:Tranco" lands, and those snapshots were 2026-08-12 / 2026-08-27 of the same API.
  * Date the default as "today" from the API, and warn that the front page and methodology page are stale in opposite directions (four vs five).
  * Do not explain Dowdall beyond one sentence; it is a scoring rule, not a web-measurement method.

===== Report script output (unedited) =====

The command is ''node scripts/report_tranco.mjs''. Output of the run that the content page was written from:

<file>

======================================================================
1. POPULATION — papers that name Tranco as a sourceList
======================================================================
corpus                                              5859
crawled (crawlConfig or automated-web-crawl)        1120
web-unit population (websites/domains/web-pages)    1153

PAGE POPULATION: any population[].sourceList matches /\btranco\b/i
                                                 266  4.5% of the 5859-paper corpus
  ... of which also ran a crawl                  168  15.0% of 1120 crawled
  ... with a web-unit Tranco tuple               262  98.5% of Tranco users
  ... Tranco tuple is not a web unit             4
Tranco tuples (not papers): 373

This 266 is the same query as report_longitudinal.mjs "Tranco users".
design:sampling reports a smaller number because it restricts to web-unit tuples.

-- unit of the Tranco tuple, papers of 266 --
Unit          Papers  Share of 266
------------  ------  ------------
websites      160     60.2%
domains       110     41.4%
web-pages     6       2.3%
other         4       1.5%
documents     2       0.8%
mobile-apps   2       0.8%
ip-addresses  1       0.4%

-- venue, papers of 266 --
Venue    Tranco papers  Venue corpus  Share of venue
-------  -------------  ------------  --------------
USENIX   56             1410          4.0%
IMC      46             638           7.2%
PETS     44             510           8.6%
CCS      32             990           3.2%
NDSS     32             701           4.6%
WWW      32             843           3.8%
IEEE-SP  24             767           3.1%

-- year (2025–2026 provisional) --
Year  Corpus  Tranco papers  Share of year  Share of Tranco users
----  ------  -------------  -------------  ---------------------
2010  119     0              0.0%           0.0%
2011  116     0              0.0%           0.0%
2012  151     0              0.0%           0.0%
2013  125     0              0.0%           0.0%
2014  166     0              0.0%           0.0%
2015  190     0              0.0%           0.0%
2016  182     0              0.0%           0.0%
2017  231     0              0.0%           0.0%
2018  254     0              0.0%           0.0%
2019  402     0              0.0%           0.0%
2020  404     7              1.7%           2.6%
2021  379     28             7.4%           10.5%
2022  546     29             5.3%           10.9%
2023  719     48             6.7%           18.0%
2024  690     56             8.1%           21.1%
2025  770     76             9.9%           28.6%
2026  415     22             5.3%           8.3%

-- year buckets --
Bucket      Corpus  Tranco  Share of corpus in bucket
----------  ------  ------  -------------------------
2010–2013   511     0       0.0%
2014–2017   769     0       0.0%
2018–2021   1439    35      2.4%
2022–2024   1955    133     6.8%
2025–2026*  1185    98      8.3%

======================================================================
2. FOLDING — sourceList spellings that matched /\btranco\b/i
======================================================================
distinct sourceList spellings: 96
exact string "Tranco" only:    180  67.7% of 266
any spelling (the page population): 266
exact-string undercount: 32.3%

Every matching spelling, papers (not tuples). Residue of the regex is 0 by construction
because membership IS the regex. This table is the fold, not a second classifier.
sourceList spelling                                                                                                                 Papers
----------------------------------------------------------------------------------------------------------------------------------  ------
Tranco                                                                                                                              180
Tranco list                                                                                                                         5
Tranco Top 1M                                                                                                                       5
Tranco top 1M list                                                                                                                  3
Tranco 1M list                                                                                                                      2
Tranco List                                                                                                                         2
Tranco top 1 million list                                                                                                           2
Tranco top 100K list                                                                                                                2
Tranco Top 1K                                                                                                                       2
Tranco Top 1M list                                                                                                                  2
Tranco top-million list                                                                                                             2
ACVL ∪ Tranco top-2K                                                                                                                1
Adobe and Tranco Top 1K domains                                                                                                     1
Alexa Top 1M, Cisco Umbrella, Majestic Million, and the Tranco Research List                                                        1
Alexa Top 1M, Cisco Umbrella, Majestic Million, and Tranco Research List                                                            1
Alexa, Tranco, Cisco Umbrella, and Google's Chrome User Experience Report (CrUX) Top Lists                                          1
Alexa, Umbrella, Majestic, Tranco, Secrank, and Trexa top lists                                                                     1
Alexa/Tranco top 10,000                                                                                                             1
BuiltWith; Tranco                                                                                                                   1
Candidate sites from ACVL and Tranco top-2K                                                                                         1
Cisco Umbrella, Majestic Million, Tranco, SecRank, Alexa Top 1 Million                                                              1
Citizen Lab lists, full Tranco list, and Common Crawl Project                                                                       1
Citizen Lab test list and Tranco top 500                                                                                            1
combined Alexa Top 1M, Tranco 1M, and Cisco Umbrella 1M history                                                                     1
Common Crawl and Tranco                                                                                                             1
custom seed list derived from Tranco                                                                                                1
CZDS, Tranco list, passive DNS data from SIE Europe, .se, .nu, .ch, .li TLD zone files, and Google Certificate Transparency logs    1
D3 dataset derived from Mathur and Tranco websites                                                                                  1
Deepsee.io, an online publisher intelligence company, based on aggregated ad inventory data and the Tranco list for traffic volume  1
May 2025 Tranco rankings                                                                                                            1
Princeton-Leuven Longitudinal Corpus of Privacy Policies; lowest Tranco rankings                                                    1
Princeton-Leuven Longitudinal Corpus of Privacy Policies; Tranco top 300                                                            1
PublicWWW and Tranco                                                                                                                1
same Tranco list                                                                                                                    1
SimilarWeb and Tranco Top-1M Ranking List                                                                                           1
TLD zone files, Citizen Lab test lists, Tranco list, and Common Crawl project                                                       1
Tranco [31] dataset                                                                                                                 1
Tranco 1M                                                                                                                           1
Tranco 1M most accessed domains list                                                                                                1
Tranco 1M top lists                                                                                                                 1
Tranco and CBL lists                                                                                                                1
Tranco and Citizen Lab Global Block List                                                                                            1
Tranco and Cloudflare Radar                                                                                                         1
Tranco and Cloudflare Radar, filtered by valid AAAA records                                                                         1
Tranco and CrUX                                                                                                                     1
Tranco L394                                                                                                                         1
Tranco list (ID: 7XN5X)                                                                                                             1
Tranco list (ID: W88P9)                                                                                                             1
Tranco list 998W2                                                                                                                   1
Tranco list of top sites                                                                                                            1
Tranco list version 82NJV                                                                                                           1
Tranco Million                                                                                                                      1
Tranco top 1 K list                                                                                                                 1
Tranco top 1 M domains                                                                                                              1
Tranco top 1 million                                                                                                                1
Tranco top 1 million domain dataset                                                                                                 1
Tranco Top 1 Million domains                                                                                                        1
Tranco Top 1 Million list                                                                                                           1
Tranco top 10,000                                                                                                                   1
Tranco top 10,000 domains                                                                                                           1
Tranco Top 1000 websites                                                                                                            1
Tranco top 100k                                                                                                                     1
Tranco Top 100K                                                                                                                     1
Tranco top 10k                                                                                                                      1
Tranco Top 10K                                                                                                                      1
Tranco top 10K domains                                                                                                              1
Tranco top 10k list                                                                                                                 1
Tranco top 10K websites                                                                                                             1
Tranco top 1M                                                                                                                       1
Tranco Top 1M (ID 833KV)                                                                                                            1
Tranco Top 1M and subdomains                                                                                                        1
Tranco Top 1M domain list                                                                                                           1
Tranco top 1M domains                                                                                                               1
Tranco top 1M ranking                                                                                                               1
Tranco Top 200 reachable sites                                                                                                      1
Tranco top 50k domains and OpenPhish                                                                                                1
Tranco Top list                                                                                                                     1
Tranco top site list                                                                                                                1
Tranco top sites list                                                                                                               1
Tranco top website list                                                                                                             1
Tranco top-1M                                                                                                                       1
Tranco top-1M domain list                                                                                                           1
Tranco top-list                                                                                                                     1
Tranco top-sites list                                                                                                               1
Tranco Top10K                                                                                                                       1
Tranco-1M domain list                                                                                                               1
Tranco-derived corpus                                                                                                               1
Tranco-derived severity ranking                                                                                                     1
Tranco, Umbrella, and Majestic top-million domain lists                                                                             1
Tranco; Majestic; Cisco Umbrella; Cloudflare Radar                                                                                  1
Tranco's rankings and the most comprehensive data removal list found                                                                1
Tranco's top 100K and the Acceptable Ads Standard's Exception list                                                                  1
Tranco's top list                                                                                                                   1
Tranco's top-500K list                                                                                                              1
Universal Tranco list                                                                                                               1
Zone Data Service, .se/.nu/.ch zone files, Google Certificate Transparency logs, Tranco, and SIE Europe passive DNS                 1

======================================================================
3. VERSIONING AND PERMANENT IDs
======================================================================
states any listVersion on a Tranco tuple (extraction field): 159  59.8%
cites a permanent Tranco list ID (hand-audited 2026-08-27):  23  8.6%
allowlist size: 23
allowlist: every key still matches a Tranco-user paper

by year (Tranco users / of them, citing an ID):
Year  Tranco users  Cite an ID  Share
----  ------------  ----------  -----
2020  7             1           14.3%
2021  28            2           7.1%
2022  29            2           6.9%
2023  48            5           10.4%
2024  56            4           7.1%
2025  76            8           10.5%
2026  22            1           4.5%

The ID count is a lower bound on papers that *could* be recovered from a
permalink. listVersion often holds a date ("March 2024") rather than an id.

======================================================================
4. SAMPLE SIZE — Tranco tuples that state n
======================================================================
papers stating a size for at least one Tranco population: 256  96.2%
Tranco tuples: 373, of which 362 state a size (97.1%)
min 6  p25 1000  median 10000  p75 634810  p90 1000000  max 700000000
Population size      Tuples  Share of 362
-------------------  ------  ------------
1–1,000              94      26.0%
1,001–10,000         94      26.0%
10,001–100,000       74      20.4%
100,001–1,000,000    84      23.2%
more than 1,000,000  16      4.4%

tuples whose n is exactly a round prefix (any samplingMethod):
  total such tuples: 166
  of those, samplingMethod is top-n: 147
  of those, samplingMethod is not top-n: 19
  n=1000   19  5.2% of sized tuples; 16 top-n, 3 other
  n=10000   49  13.5% of sized tuples; 42 top-n, 7 other
  n=100000   28  7.7% of sized tuples; 25 top-n, 3 other
  n=1000000   70  19.3% of sized tuples; 64 top-n, 6 other
  samplingMethod among those tuples:
    top-n: 147
    random: 10
    pre-existing-dataset: 3
    stratified: 3
    purposive: 2
    seed-and-crawl: 1

======================================================================
5. SAMPLING METHOD — of Tranco tuples
======================================================================
states a sampling method on at least one Tranco tuple: 263  98.9%
Method                Papers  Share of 266
--------------------  ------  ------------
top-n                 215     80.8%
random                31      11.7%
stratified            23      8.6%
pre-existing-dataset  18      6.8%
purposive             18      6.8%
exhaustive            7       2.6%
seed-and-crawl        5       1.9%
not-stated            3       1.1%
convenience           1       0.4%

======================================================================
6. TOOLS[] AND OTHER-TOOLS — schema vs sourceList
======================================================================
tools[].name matches /tranco/i:                 27
otherToolsMentioned matches /tranco/i:          9
either tool field:                              36
in sourceList population AND a tool field:      26
tool field but NOT in sourceList population:    10
sourceList population with NO tool-field hit:   240  90.2%

Raw tools[].name spellings (tuples, then papers):
  "Tranco"  papers=26 tuples=26  [used:26]
  "Tranco 1M"  papers=1 tuples=1  [used:1]

Papers in a tool field but not in the sourceList population (read these; they may be citations):
  NDSS/2026/loki-proactively-discovering-online-scams-by-mining-toxic-search-queries  other:Tranco
  NDSS/2026/ctphishcapture-uncovering-credential-theft-based-phishing-scams-targeting-cryptocurrency-wallets  tools:Tranco[used]
  WWW/2026/unveiling-the-resilience-of-llm-enhanced-search-engines-against-black-hat-seo-ma  tools:Tranco[used]
  NDSS/2025/the-guardians-of-name-street-studying-the-defensive-registration-practices-of-the-fortune-500  other:Tranco
  USENIX/2025/are-captchas-still-bot-hard-generalized-visual-captcha-solving-with-agentic-visi  tools:Tranco[used]
  WWW/2025/before-after-the-effect-of-eus-2022-code-of-practice-on-disinformation  tools:Tranco[used]
  PETS/2025/surveillance-disguised-as-protection-a-comparative-analysis-of-sideloaded-and-in  tools:Tranco[used]
  IMC/2024/looking-at-the-blue-skies-of-bluesky  tools:Tranco[used]
  USENIX/2023/hiding-in-plain-sight-an-empirical-study-of-web-application-abuse-in-malware  tools:Tranco 1M[used]
  CCS/2020/towards-attribution-in-mobile-markets-identifying-developer-account-polymorphism  tools:Tranco[used]

======================================================================
7. FULL-TEXT SWEEP /tranco/i — schema coverage and homographs
======================================================================
full-text /\btranco\b/i: 332 papers; missing .cols files: 4
of those, already in sourceList population: 266
of those, in tools[] but not sourceList:     8
full-text only (neither sourceList nor tools[]): 58

The page population stays the sourceList query. Full-text-only hits are listed so
a citation-only or homograph paper is visible rather than silently added. A sample
of contexts follows (first 12 full-text-only papers).
  NDSS/2026/loki-proactively-discovering-online-scams-by-mining-toxic-search-queries
    from popular domain ranking lists such as Tranco or the
  PETS/2026/analyzing-societal-awareness-and-perception-of-digital-fingerprinting-and-finger
    ing can be split into the concepts [39] discovered that 66.6% of the top 10,000 TRANCO websites [37]
  USENIX/2026/sok-philter-uncovering-security-and-functional-gaps-in-ai-based-phishing-website
    low-reputation or long-tail domains (e.g., Alexa/Tranco lowrank sites, personal domains, recently registered domains, or
  NDSS/2026/indicator-of-benignity-an-industry-view-of-false-positive-in-malicious-domain-detection-and-its-mitigation
     as Tranco [62], cannot effectively mitigate FPs. In particular,
  IEEE-SP/2026/behind-the-curtain-how-shared-hosting-providers-respond-to-vulnerability-notific
    development of Tranco [30] as an alternative to Alexa 1
  PETS/2026/the-empire-strikes-back-at-your-privacy-an-archaeology-of-tracking-on-government
    long-term effects of parameters on the characteristics of the tranco top sites
  IEEE-SP/2025/data-to-infinity-and-beyond-examining-data-sharing-and-reuse-practices-in-the-co
    and published, as seen with Tranco [20] and Censored
  IMC/2025/not-all-visitors-are-bilingual-a-measurement-study-of-the-multilingual-web-from
    Tranco [31] focus on popularity but do not provide insights into
  NDSS/2025/ctrlaltdeceive-quantifying-user-exposure-to-online-scams
    First, we remove 67.3K (4%) domains appearing in the Tranco
  IMC/2025/sibling-prefixes-identifying-similarities-in-ipv4-and-ipv6-prefixes
    Tranco DS percentage
  IMC/2025/how-i-learned-to-stop-worrying-and-love-ipv6-measuring-the-internets-readiness-f
    CrUX top list as input over, e.g., the Tranco Top 1M [49] as the though, downstreams still receive a BGP fulltable, i.e., inc
  NDSS/2025/the-guardians-of-name-street-studying-the-defensive-registration-practices-of-the-fortune-500
    the size of the squatting spaces, we normalized the count of Tranco rank [33] for the base domain name during the

======================================================================
8. ALEXA / CRUX / TRANCO over time, web-unit papers (currency, not this page's population)
======================================================================
Denominator: papers with a web-unit population tuple. 2025–2026 provisional.
Year  Web-unit papers  Alexa       Tranco      CrUX
----  ---------------  ----------  ----------  ---------
2019  85               53 (62.4%)  0 (0.0%)    1 (1.2%)
2020  81               46 (56.8%)  7 (8.6%)    0 (0.0%)
2021  87               41 (47.1%)  27 (31.0%)  0 (0.0%)
2022  106              40 (37.7%)  28 (26.4%)  2 (1.9%)
2023  122              30 (24.6%)  47 (38.5%)  5 (4.1%)
2024  122              21 (17.2%)  56 (45.9%)  12 (9.8%)
2025  133              11 (8.3%)   75 (56.4%)  12 (9.0%)
2026  57               4 (7.0%)    22 (38.6%)  4 (7.0%)

======================================================================
9. QUOTE CHECK — population evidence.quote on Tranco tuples
======================================================================
exact: 204  partial(>=60% 5-word windows): 111  FAILED: 58
no quote on tuple: 0  missing .cols: 0
FAILED (first 15) — read by hand; below-threshold is not automatically unsupported:
  IMC/2020/accept-the-risk-and-continue-measuring-the-long-tail-of-government-https-adoptio  6/15  "We compare https in these top government websites ... with [1] 12,000 random, uniformly sampled top million non-government hostnames"
  WWW/2020/understanding-the-performance-costs-and-benefits-of-privacy-focused-browser-exte  11/21  "Specifically, we use (i) the top 1,000 domains ... (ii) the domains 99,001 to including 100,000 ... Correspondingly, our list of domains contains 2,000 domains."
  IMC/2021/knock-and-talk-investigating-local-network-communications-on-websites  2/6  "one using the Tranco snapshot taken on June 3, 2020"
  PETS/2021/ml-cb-machine-learning-canvas-block  11/20  "Initially, we targeted 2,200 websites, using our original dataset, original Alexa Top Rank list, and the Tranco list of the one million top sites"
  PETS/2021/privacy-preference-signals-past-present-and-future  3/6  "we crawled the top 100k entries from the Tranco toplist"
  PETS/2021/the-cname-of-the-game-large-scale-analysis-of-dns-based-tracking-evasion  6/12  "For each website, we visited up to 20 pages ... on the Tranco top 10,000 websites"
  PETS/2021/deja-vu-abusing-browser-cache-headers-to-identify-and-track-online-users  8/15  "we crawled the Top 100K websites from the Tranco list [33] on the 2nd and the 3rd of June"
  USENIX/2021/fingerprinting-in-style-detecting-browser-extensions-via-injected-style-sheets  6/11  "we crawled the homepage of the Tranco top 200 websites with and without our countermeasure."
  USENIX/2021/how-great-is-the-great-firewall-measuring-chinas-dns-censorship  10/21  "We complement our test list by including domains from the Citizen Lab test lists [13], the Tranco list [66], and the Common Crawl project [14]."
  USENIX/2021/the-circle-of-life-a-large-scale-study-of-the-iot-malware-lifecycle  4/9  "We use Tranco's top site ranking [53] to identify and filter benign domains."
  IMC/2022/a-first-look-at-the-name-resolution-latency-on-handshake  0/3  "extracting unique top-100 TLDs from the latter"
  IMC/2022/html-violations-and-where-to-find-them-a-longitudinal-analysis-of-specification  3/7  "The result is an overall top list of 24,915 unique domains."
  IMC/2022/toppling-top-lists-evaluating-the-accuracy-of-popular-website-lists  5/17  "We retrieved daily snapshots of Alexa, Umbrella, and Majestic ... Tranco rankings ... and Secrank rankings ... for February 1-28, 2022."
  PETS/2022/trace-oddity-methodologies-for-data-driven-traffic-analysis-on-tor  9/17  "The first two datasets, SD and MD consist of visits to the 15,000 most popular websites according to the Tranco ranking"
  USENIX/2022/automating-cookie-consent-and-gdpr-violation-detection  8/16  "Our primary source is the Tranco ranking of May 5th, 2021, which lists domains ranked by their estimated worldwide popularity."

======================================================================
10. OVERVIEW.md CROSS-CHECKS
======================================================================
corpus 5859 (OVERVIEW.md 5,859)
crawled 1120 (OVERVIEW.md 1,120)

======================================================================
11. CUSTOM GENERATOR KNOBS — full-text over the 266 Tranco users
======================================================================
denominator: 266 Tranco-user papers (same population as section 1)
  inclusionDays: 0 papers; missing .cols: 0
  inclusionLists: 0 papers; missing .cols: 0
  custom list: 0 papers; missing .cols: 0
  dowdall: 1 papers; missing .cols: 0
    USENIX/2021/how-great-is-the-great-firewall-measuring-chinas-dns-censorship
  borda: 2 papers; missing .cols: 0
    USENIX/2021/how-great-is-the-great-firewall-measuring-chinas-dns-censorship
    WWW/2024/fingerprinting-the-shadows-unmasking-malicious-servers-with-machine-learning-pow
union inclusionDays | inclusionLists | "custom list": 0
REJECTED probe /harden(?:ed|ing)\b/i: 234 papers (88.0%) — homograph (binaries/TLS/ASLR), not a Tranco-variant rate

======================================================================
Z. EXTERNAL FIGURES this page may quote (live values belong in tranco_api_probe.sh)
======================================================================
These are NOT corpus figures. The probe script re-fetches them.
  latest daily list id (2026-08-27 run): 46W9X
  latest subdomains list id: 567QN
  providers on 46W9X: crux, farsight, majestic, radar, umbrella
  combinationMethod: dowdall; window 2026-07-28..2026-08-26; filterPLD on; listPrefix 1000000
  GVWK (2019-11-05) still available; providers alexa, umbrella, majestic, quantcast
  python package tranco 0.8.1 uploaded 2024-04-02
  DistriNet/tranco-list last push 2020-03-05; DEFAULT_TRANCO_CONFIG still alexa+quantcast
  ranks/domain/google.com returned 38 daily ranks, all 1, on 2026-08-27
  /api/auth/test unauthenticated → 401
  /api/lists/create unauthenticated PUT → 401; docs also name HTTP 429 for concurrent generation
  /api/lists/date/20260827 → 404 (list generated ~22:00 UTC the previous day)
  2023-08-01 daily list id 25299 (CrUX+Radar replace Alexa)
  download format: CSV rank,domain no header; Content-Disposition tranco_{id}.csv
  Farsight is not in the documented Configuration.providers enum for custom lists
  homepage still says "all four rankings" (1 occurrence) and also "five providers"
  /download/<id>/full was ~104 MB; not treated as equal to the daily 1M; row count not published

</file>

===== External probe (unedited) =====

''bash scripts/tranco_api_probe.sh'':

<file>
=== live daily list ===
  list_id=46W9X available=True providers=crux,farsight,majestic,radar,umbrella method=dowdall pld=on prefix=1000000 2026-07-28..2026-08-26
OK    latest available
OK    combinationMethod=dowdall
OK    filterPLD=on
OK    listPrefix=1000000
OK    five current providers
=== subdomains daily list ===
OK    subdomains id differs from PLD id (567QN)
OK    subdomains config omits filterPLD
OK    top-1m-id matches latest (46W9X)
=== GVWK 2019 still resolves ===
OK    GVWK available
OK    GVWK original providers
=== dated provider-set changes ===
OK    date/20191105 providers=alexa,umbrella,majestic,quantcast
OK    date/20220101 providers=alexa,umbrella,majestic
OK    date/20220501 providers=alexa,umbrella,majestic,farsight
OK    date/20230801 providers=crux,farsight,majestic,radar,umbrella
=== download format ===
  first=1,google.com rows=20
OK    row 1 contains google.com
OK    prefix 20 has 20 rows
OK    content-type csv
OK    content-disposition uses list id
=== ranks endpoint ===
OK    google.com has 38 daily ranks
OK    google.com latest rank is 1
=== auth gates ===
OK    auth/test unauthenticated 401
OK    lists/create unauthenticated 401
=== homepage stale phrasing ===
OK    homepage still says 'all four rankings' (stale; page must warn)
OK    homepage names five providers
OK    homepage names Farsight as default-list-only
=== python package ===
OK    PyPI tranco==0.8.1
=== github ===
OK    tranco-list last push 2020-03-05
OK    python package last push 2024-04-02
OK    Go package unarchived
=== latest_list URL ===
  /latest_list HTTP 303 Location=/list/46W9X/1000000
OK    /latest_list HTTP 303
OK    /latest_list redirects to /list/46W9X/1000000

all checks passed
</file>

===== Review log =====

Four passes on **GPT 5.6 Sol medium**, as directed for this sitting (the task spec's sonnet/fable split was overridden). The three focused passes ran in parallel on the frozen first draft. Their findings were applied, then this page and the content page were frozen again for the generic pass. Every reviewer was told the author's context may not be exhaustive, and handed the page text, the report script, its unedited output, the probe transcript, and these notes.

==== GPT 5.6 Sol medium — figures versus script ====

^ Finding ^ Disposition ^
| Corpus sweep figures 0 / 1 / 2 for inclusionDays / inclusionLists / "custom list" / Dowdall / Borda are not computed by ''report_tranco.mjs''; the script only prints ''dowdall'' as a hard-coded external fact. | **Accepted.** Added section 11: full-text sweep over the 266, which reproduces 0 / 0 / 0 / 1 / 2 and names the matching papers. The rejected hardened-homograph probe (234, 88.0%) is now printed there too. |
| Round-size tuples are labelled as Tranco heads. The script counts every Tranco tuple with n ∈ {1k, 10k, 100k, 1M}, regardless of sampling method. Of 166 such tuples, only 147 are ''top-n''; 19 are random, stratified, purposive, a pre-existing dataset, or seed-and-crawl. | **Accepted.** The script now splits those 166 by ''samplingMethod''. The content page says 147 of 166 are a Tranco head. |
| Undercount direction reversed in the methodology section: the script shows exact-string matching finds 180 vs 266, so the exact string undercounts the regex population by 32.3%; the sentence attributed the undercount to the regex. | **Accepted.** Now: "The exact string "Tranco" undercounts that population by 32.3%." The Use-in-publications paragraph was already the right way around. |
| All other checked figures, the quoted pin JSON, SHA-256, byte count, and live probe facts matched. | Accepted, no change. |

==== GPT 5.6 Sol medium — citations and quotes ====

^ Finding ^ Disposition ^
| "Use our default list" and "generate a list whose filters match the study" do not appear in the cited NDSS paper after whitespace normalisation. The paper supports the meaning; they should not be quotation-marked as verbatim. | **Accepted.** Rewritten as paraphrase, citation kept. |
| Live bucket-normalisation (geometric mean of bounds) was attached to {[LePochat2019_tranco]}. The NDSS paper supports Dowdall, aggregation, truncation, and filters, not geometric-mean conversion of CrUX/Radar buckets. | **Accepted.** Geometric mean is now footnoted to the live methodology page (fetched 2026-08-27). Dowdall / Zipf / Borda stay on the 2019 paper. |
| "Caching on disk is required by the API" is unsupported. PyPI says caching is required by the Python package; the HTTP API documentation imposes no disk-cache requirement. | **Accepted.** Now: "The package requires on-disk caching; the HTTP API does not." |
| All three citekeys resolve uniquely, including the new CSET key. Ruth et al. supports the February 2022 and CrUX claims. CSET 2019 is correctly marked outside the seven-venue corpus. | Accepted, no change. |

==== GPT 5.6 Sol medium — external currency ====

^ Finding ^ Disposition ^
| No defects. Provider composition, API behaviour, Farsight limitation, package version, repository push dates, BigQuery references, and downloads match the page's dated claims. The homepage still says "all four rankings"; the page correctly warns. | Accepted, no change. |

==== GPT 5.6 Sol medium — generic ====

Ran after the three focused findings were applied. One execution failure, four overclaims.

^ Finding ^ Disposition ^
| The advertised ''--subdomains'' workflow crashes: live list ''567QN'' omits ''filterPLD'', and ''pin_tranco.py'' indexed ''cfg["filterPLD"]''. Confirmed 2026-08-27: that configuration's keys are combinationMethod, endDate, filterTLD, listPrefix, providers, startDate. | **Accepted.** Direct access stays for keys the daily PLD object always has. ''filterPLD'' uses ''.get'' with a SafeAccess comment for this KeyError. Cite line now prints ''filterPLD=None'' on the subdomains list. Probe asserts the omission. |
| "The rows are registrable domains" is only true of the PLD-filtered daily list, and contradicts the subdomains paragraph above it. | **Accepted.** Now scoped to the pay-level-domain daily list; subdomains rows can be hostnames. |
| Calling a custom-list id "the whole methods section" overstates what it records. The id stores construction configuration, not the downloaded prefix, hash, or subsequent draw. The page's own "What to report" list requires those. | **Accepted.** Rewritten: cite the id for the configuration; still report prefix, hash, and draw. |
| ''report_tranco.mjs'' said it "fails loudly" on allowlist drift but only printed FAILURE and exited 0. | **Accepted.** It now ''process.exit(1)'' after printing. |
| Probe header "Re-fetch every external fact" exceeds the script: licences, bucket geometry, the providers enum, and several dated claims are not in it. | **Accepted.** Header now lists what the probe actually checks. Added the subdomains-id / omitted-''filterPLD'' check and an exact ''/latest_list'' 303 Location assertion. |

<bibtex bibliography></bibtex>