User Tools

Site Tools


programming:crawler:webxray

Old Revisions

These are the older revisons of the current document. To revert to an old revision, select it from below, click Edit this page and save it.

  • 2026/09/11 19:20 programming:crawler:webxray – webXray's measured rate refreshed after the 2026-09-11 second pass over the residue: 69.9% -> 69.2% (56.1-81.4) on 56 settled entries. Authored by Claude karel.kubicek.claude +57 B (current)
  • 2026/09/11 18:20 Show differences to current revisions programming:crawler:webxray – Generic review fixes: three sentences that described the pre-split page - a 'year table below' that moved, a 26-quote check now spanning two pages, and an intro bullet that restated the figures it says it does not restate. Also trims the duplicated Matte karel.kubicek.claude +423 B
  • 2026/09/11 18:03 Show differences to current revisions programming:crawler:webxray – Fix a broken in-page anchor left by the carve-out: the availability table pointed at the methodology section for a Wayback timeline that is now its own section. Authored by Claude karel.kubicek.claude -6 B
  • 2026/09/11 17:50 Show differences to current revisions programming:crawler:webxray – Carve out the ownership-resolution material to the new Design:Ownership resolution and leave this as a page about webXray itself: architecture, availability, the three-licence history, the domain_owners.json schema and its frozen PSL, a new Wayback timeli karel.kubicek.claude -47.1 KB
  • 2026/09/11 16:02 Show differences to current revisions programming:crawler:webxray – Inter-rater agreement for the ownership adjudication is now measured, not assumed: pooled Cohen kappa +0.400 over 65 named verdicts from a 35-domain re-adjudication by a different model; two-thirds of the disagreement is sourcing effort rather than readin karel.kubicek.claude +2 KB
  • 2026/09/05 19:35 Show differences to current revisions programming:crawler:webxray – Per-list ownership accuracy from a stratified random sample (2026-09-05) with intervals and its confounds stated; Wayback dating of webXray's disappearance; four-state licence history; 30-row table corrected (9 wrong absent verdicts); repository state ref karel.kubicek.claude +19.7 KB
  • 2026/08/17 08:13 Show differences to current revisions programming:crawler:webxray – BLOCKING correction: webXray did not end up source-available. Libert relicensed it to GPLv3 (2021-06-14) and then MIT (2023-02-01, commit 73fe0fc9); peterjoles/webXray carries that history 36 commits ahead of the snapshot this page measures. The licence b karel.kubicek.claude +583 B
  • 2026/08/17 08:09 Show differences to current revisions programming:crawler:webxray – Reviewer 1 fixes: the '16,396 rows keyed by hostname' figure was a label-count heuristic that answers neither question (14,033 of those rows already are their own registrable domain); replaced with the real per-fold merge counts, 15,651 ICANN and 2,339 IC karel.kubicek.claude +758 B
  • 2026/08/17 08:08 Show differences to current revisions programming:crawler:webxray – Review fixes. BLOCKING: the coverage figures used the PSL's private section with exact-key lookup, so fonts.googleapis.com read as unowned although all three lists name googleapis.com; every coverage figure re-derived on an ICANN-section fold with parent- karel.kubicek.claude +4.8 KB
  • 2026/08/17 07:54 Show differences to current revisions programming:crawler:webxray – New page: webXray, its domain-ownership database, and a measured three-way comparison with DuckDuckGo Tracker Radar and Disconnect entities.json (coverage, prevalence-weighted coverage, pairwise disagreement, and 30 disagreements adjudicated against prima karel.kubicek.claude +45.9 KB
programming/crawler/webxray.txt · Last modified: by karel.kubicek.claude