Differences

This shows you the differences between two versions of the page.

--- programming:crawler:webxray [2026/09/11 18:20] – Generic review fixes: three sentences that described the pre-split page - a 'year table below' that moved, a 26-quote check now spanning two pages, and an intro bullet that restated the figures it says it does not restate. Also trims the duplicated Matte karel.kubicek.claude
+++ programming:crawler:webxray [2026/09/11 19:20] (current) – webXray's measured rate refreshed after the 2026-09-11 second pass over the residue: 69.9% -> 69.2% (56.1-81.4) on 56 settled entries. Authored by Claude karel.kubicek.claude
@@ Line 12: / Line 12: @@
   * **The tool is gone from where every paper points, but it is not lost and it did not end up proprietary.** ''github.com/timlib/webXray'' returns HTTP 404 and there has never been a PyPI package — but Libert relicensed webXray twice after its most widely mirrored snapshot, **back** to GPLv3 in 2021 — the licence its own website claimed in 2015 — and then to **MIT in 2023**, and that history survives in a fork. The author now runs a commercial product under the same name at ''webxray.ai''.
-  * **The database survives and it is frozen at 2021-03-04.** How stale that makes it has been measured: on a stratified random sample of its own coverage, drawn and adjudicated on 2026-09-05, it names today's owner for **69.9%** of the entries that could be settled (95% CI 55.5–83.1%). — a rate over the fifth of its entries a 2026 crawl still meets, not over the file, and one that sits beside a **coverage** figure of **2.1%** of the frame. The adjudicators were language models, the comparison has two snapshot frames, and there are several reasons not to read 69.9% as a rehabilitation. All of that is on [[Design:Ownership resolution]], and this page deliberately does not restate it: take the two figures from there, with their caveats, rather than from this bullet.
+  * **The database survives and it is frozen at 2021-03-04.** How stale that makes it has been measured: on a stratified random sample of its own coverage, drawn 2026-09-05 and re-adjudicated on 2026-09-11 where the first pass could not settle a row, it names today's owner for **69.2%** of the entries that could be settled (95% CI 56.1–81.4%) — a rate over the fifth of its entries a 2026 crawl still meets, not over the file, and one that sits beside a **coverage** figure of **2.1%** of the frame. The adjudicators were language models, the comparison has two snapshot frames, and there are several reasons not to read 69.2% as a rehabilitation. All of that is on [[Design:Ownership resolution]], and this page deliberately does not restate it: take the two figures from there, with their caveats, rather than from this bullet.
   * **The list and the tool came apart in the literature years ago.** Of the 15 corpus papers that name webXray, **7 used only the ownership list** and exactly **one ran the crawler** — and that one is Libert's own paper.