webXray [1Libert, Timothy (2015): "Exposing the Invisible Web: An Analysis of Third-Party HTTP Requests on 1 Million Websites", International Journal of Communication 9. (Link)] is a third-party request and privacy-policy measurement tool by Timothy Libert, and it was one of the three hand-curated lists the pre-2022 literature used to put a company name next to a third-party domain — the other two being Disconnect and WhoTracks.me, which two corpus papers merge with it explicitly [2Sánchez-Rola, Iskander; Dell'Amico, Matteo; Balzarotti, Davide; Vervier, Pierre-Antoine; Bilge, Leyla (2021): "Journey to the Center of the Cookie Ecosystem: Unraveling Actors' Roles and Relationships", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] [3Dambra, Savino; Sanchez-Rola, Iskander; Bilge, Leyla; Balzarotti, Davide (2022): "When Sally Met Trackers: Web Tracking From the Users' Perspective", in: Proceedings of the USENIX Security Symposium. (Link)]. Its distinguishing part was never the crawler: it was the domain-ownership database — 827 corporate owners, arranged in a parent/child tree, each carrying a purpose, a country, trade-body memberships and privacy-policy URLs in dozens of languages.

Three things a new measurement needs to know before citing webXray:

• The tool is gone from where every paper points, but it is not lost and it did not end up proprietary. github.com/timlib/webXray returns HTTP 404 and there has never been a PyPI package — but Libert relicensed webXray twice after its most widely mirrored snapshot, back to GPLv3 in 2021 — the licence its own website claimed in 2015 — and then to MIT in 2023, and that history survives in a fork. The author now runs a commercial product under the same name at webxray.ai.
• The database survives and it is frozen at 2021-03-04. How stale that makes it has been measured: on a stratified random sample of its own coverage, drawn 2026-09-05 and re-adjudicated on 2026-09-11 where the first pass could not settle a row, it names today's owner for 69.2% of the entries that could be settled (95% CI 56.1–81.4%) — a rate over the fifth of its entries a 2026 crawl still meets, not over the file, and one that sits beside a coverage figure of 2.1% of the frame. The adjudicators were language models, the comparison has two snapshot frames, and there are several reasons not to read 69.2% as a rehabilitation. All of that is on Ownership resolution, and this page deliberately does not restate it: take the two figures from there, with their caveats, rather than from this bullet.
• The list and the tool came apart in the literature years ago. Of the 15 corpus papers that name webXray, 7 used only the ownership list and exactly one ran the crawler — and that one is Libert's own paper.

The last public version (webXray 3.x, “forensic edition”) is not the PhantomJS tool the 2016 OpenWPM paper criticised [5Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)]. It drives consumer Chrome over the DevTools protocol directly — websocket-client, no Selenium and no Puppeteer — and has a client/server split for distributed crawling.¹⁾

Availability, every line checked on 2026-08-17:

domain_owners.json is a flat JSON array of owner records with an explicit tree. It is worth understanding in detail, because it is the most ambitious schema any of the three databases uses, and because most of that ambition is unfilled.

Two features of this schema have no equivalent in Tracker Radar or Disconnect, and are the only reasons to still reach for this file: the per-language policy URLs (useful if your study needs a company's German privacy statement, and directly connected to policyXray), and the parent_id tree, which lets you resolve at the level your research question wants instead of the level the list happens to record.

That tree is not cosmetic. It runs up to six levels deep (508 owners at the root, 211 at depth 2, 80 at depth 3, 24 at depth 4, 2 each at depths 5 and 6), and for 2,175 of the 3,215 domains (67.7%) the immediate owner differs from the root of its tree. doubleclick.net is owned by “DoubleClick”, whose parent chain ends at “Alphabet”. Tracker Radar says “Google LLC” and Disconnect says “Google”. None of the three is wrong; they are answers to three different questions, and Design:Ownership resolution measures what happens when you forget that — resolving webXray to the root of its tree makes agreement with both live lists worse, not better, because its roots are historical holding companies.

The Wayback Machine can date webXray's disappearance, and the dates are bounds, not events, because the Archive has no capture inside either window. Every check below was redone on 2026-09-05 after web.archive.org returned 502/503 all day on 2026-08-17; the CDX queries and the raw id_ captures are on the provenance page.

webxray.org itself went through four states:

• 2010-11-08 to at least 2011-09-27 — something else entirely, before Libert: a page titled “Web Xray - Free Website Data Mashup Tool” carrying a “beta 0.5” badge, a public directory of scraped sites with Google AdSense units. (Two separate elements, the <title> and a badge beside the logo, not one sentence. The domain's first archived capture, 2010-10-08, is a DreamHost parking page.)
• 2022-01-23 to 2024-03-28 — a live demo search engine over “scans of 100,000 sites”, with a company drop-down drawn from domain_owners.json. This is the closest thing to a public interface the ownership list ever had, and it is gone.
• 2024-05-24 to 2026-02-05 — 401 Authorization Required, then from 2024-07-24 a 301 redirect to webxray.ai.
• by 2026-08-17 — the present placeholder. The Archive has no capture of it, so when the domain came back off webxray.ai can only be bounded to between 2026-02-05 and 2026-08-17.

Figures below are from the seven venues on corpus (CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P, 2010–2026), not from the web-measurement literature as a whole.

15 papers name webXray anywhere in their full text — 0.3% of the 5,859-paper corpus. 13 of those 15 are among the 1,120 papers that ran a crawl, which is 1.2% of that population.⁸⁾ The extraction's tool field fires on only 7 of them, because most of the rest cite it in a related-work sentence or a reference list; both signals are reported rather than merged. Every verdict below was read off a quoted sentence.

Three findings from that table matter more than the counts.

The tool and the database came apart, and then the database was superseded. The last corpus paper to use either was published in 2022. Ownership use of Tracker Radar begins in 2021 and rises through the corpus edge; the year-by-year crossover, and what it does and does not license, is on Design:Ownership resolution.

Nobody says which snapshot they used. Of the 8 papers that used the crawler or the list, 2 say anything at all about which version, and 1 names a commit — [10Matte, Célestin; Bielova, Nataliia; Santos, Cristiana Teixeira (2020): "Do Cookie Banners Respect my Choice? Measuring Legal Compliance of Banners from IAB Europe's Transparency and Consent Framework", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)], which pins “WebXRay commit 04c3c8e8 (2019-06-18)” beside the Disconnect list, the Chromium build, the kernel, the user agent, the vantage point and the Tranco list id. The file has no version field and no release tags, so a commit or an archive URL is the only thing that makes a figure derived from it reproducible. Design:Ownership resolution treats that reproducibility table as the standard to meet and says what else belongs in it.

The corpus cannot see most of webXray's impact. The seven venues omit EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS, and webXray's own author published most of its results outside those seven — his homepage lists The BMJ, JAMA, Communications of the ACM and New Media and Society⁹⁾. “15 papers” is a statement about seven security and networking venues, not a measure of how much webXray was used.

The wider question — how many corpus papers do domain-to-company attribution at all, with what, and whether it is growing — is on Design:Ownership resolution: 136 papers name an ownership resource, 107 of them among the 1,120 that ran a crawl.

• The corpus audit script is scripts/report_webxray.mjs, published in full with its unedited output on the provenance page along with every query, every fold residue, the sources rejected and the figures deliberately not published. Corpus-wide selection and extraction caveats are on corpus. The ownership-database comparison, the coverage census and the accuracy rates moved to Ownership resolution on 2026-09-11; their scripts (owner_dbs.py, owner_adjudication.py, owner_sample.py, owner_random_sample.py, owner_irr_kappa.py) and outputs stay on this page's provenance page and its code appendix, whose ids are unchanged so the published record still resolves. ownership_resolution records the split and carries the new page's own queries.
• The webXray population is a full-text sweep, not the extraction's tool field, because the tool field finds 7 papers where the sweep finds 15 and the difference is exactly the citation-only cases this page needs to separate. All 15 carry a hand verdict with a quoted sentence; the residue between sweep and hand map is zero, and the script fails loudly if that changes.
• report_webxray.mjs section G checks 26 literal per-paper figures and quotes, 26 of 26 located against each paper's paper.cols.txt rendering. Since the 2026-09-11 split those 26 are divided between this page and Ownership resolution — the script still checks both sets, and the count has not been re-partitioned. The 16 extraction evidence quotes behind the tool and classification tuples were checked the same way: 4 exact, 11 partial, 1 below threshold; the below-threshold quote is present in the source and mangled by a column splice, not unsupported.
• One figure was not published: [12Steffens, Marius; Musch, Marius; Johns, Martin; Stock, Ben (2021): "Who’s Hosting the Block Party? Studying Third-Party Blockage of CSP and SRI", in: Proceedings of the Network and Distributed System Security Symposium. (Link)] contains the sentence “webXray's list does account for 1,096 of our 1,146 found connections meaning that it alone does not suffice for our purposes”, whose two halves contradict each other. The published PDF reads the same way, so no coverage percentage is taken from it.
• Every repository, licence and URL claim on this page was checked on 2026-08-17 and the Wayback bounds on 2026-09-05. Repository state changes; re-check before citing. The licence finding in particular was wrong in this page's first version, which asserted from one snapshot that webXray “is not open source and redistributing it is prohibited” — true of that snapshot and false of the project's final state.
• The 1.2% of the 1,120 crawling papers above was published as 1.3% until 2026-09-11, when the split found that report_webxray.mjs was dividing a corpus-wide numerator (15) by the crawled denominator (1,120) instead of counting the 13 sweep hits that are actually in that population. The same bug on a larger figure read 12.1% where the comparable number is 9.6%; both are corrected and the script now computes the intersection.

• Ownership resolution — the question this tool's database answered: which source to use now, how much of the web each one covers, and how often each is right.
• Crawler — generic automation libraries and the other specialised crawlers.
• Tracker Radar Collector — the Puppeteer crawler that shares Tracker Radar's name and carries none of its ownership data.
• OpenWPM — the instrument webXray was compared against in 2016, and the one still maintained.
• Policies — what policyXray did, and what has replaced it.
• Traffic files — capturing the requests webXray captured.

[1]

Libert, Timothy (2015): "Exposing the Invisible Web: An Analysis of Third-Party HTTP Requests on 1 Million Websites", International Journal of Communication 9. (Link)

[2]

Sánchez-Rola, Iskander; Dell'Amico, Matteo; Balzarotti, Davide; Vervier, Pierre-Antoine; Bilge, Leyla (2021): "Journey to the Center of the Cookie Ecosystem: Unraveling Actors' Roles and Relationships", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[3]

Dambra, Savino; Sanchez-Rola, Iskander; Bilge, Leyla; Balzarotti, Davide (2022): "When Sally Met Trackers: Web Tracking From the Users' Perspective", in: Proceedings of the USENIX Security Symposium. (Link)

[4]

Libert, Timothy (2018): "An Automated Approach to Auditing Disclosure of Third-Party Data Collection in Website Privacy Policies", in: Proceedings of the ACM Web Conference. (DOI)

[5]

Englehardt, Steven; Narayanan, Arvind (2016): "Online Tracking: A 1-million-site Measurement and Analysis", in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 1388–1401. Association for Computing Machinery, New York, NY, USA. (DOI) (Link)

[6]

Cui, Hao; Trimananda, Rahmadi; Markopoulou, Athina; Jordan, Scott (2023): "PoliGraph: Automated Privacy Policy Analysis using Knowledge Graphs", in: Proceedings of the USENIX Security Symposium. (Link)

[7]

Kashaf, Aqsa; Sekar, Vyas; Agarwal, Yuvraj (2020): "Analyzing Third Party Service Dependencies in Modern Web Services: Have We Learned from the Mirai-Dyn Incident?", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[8]

McQuistin, Stephen; Snyder, Peter; Perkins, Colin; Haddadi, Hamed; Tyson, Gareth (2023): "A First Look at the Privacy Harms of the Public Suffix List", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[9]

Sørensen, Jannick Kirk; Kosta, Sokol (2019): "Before and After GDPR: The Changes in Third Party Presence at Public and Private European Websites", in: Proceedings of the ACM Web Conference. (DOI)

[10]

Matte, Célestin; Bielova, Nataliia; Santos, Cristiana Teixeira (2020): "Do Cookie Banners Respect my Choice? Measuring Legal Compliance of Banners from IAB Europe's Transparency and Consent Framework", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[11]

Yang, Zhiju; Yue, Chuan (2020): "A Comparative Measurement Study of Web Tracking on Mobile and Desktop Environments", in: Proceedings on Privacy Enhancing Technologies. (DOI)

[12]

Steffens, Marius; Musch, Marius; Johns, Martin; Stock, Ben (2021): "Who’s Hosting the Block Party? Studying Third-Party Blockage of CSP and SRI", in: Proceedings of the Network and Distributed System Security Symposium. (Link)

[13]

Musa, Maaz Bin; Nithyanand, Rishab (2022): "ATOM: Ad-network Tomography", in: Proceedings on Privacy Enhancing Technologies. (DOI)

[14]

Cassel, Darion; Lin, Su-Chin; Buraggina, Alessio; Wang, William; Zhang, Andrew; Bauer, Lujo; Hsiao, Hsu-Chun; Jia, Limin; Libert, Timothy (2022): "OmniCrawl: Comprehensive Measurement of Web Tracking With Real Desktop and Mobile Browsers", in: Proceedings on Privacy Enhancing Technologies. (DOI)

[15]

Kats, Daniel; Silva, David Luz; Roturier, Johann (2022): "Who Knows I Like Jelly Beans? An Investigation Into Search Privacy", in: Proceedings on Privacy Enhancing Technologies. (DOI)

[16]

Jakaria, Md; Huang, Danny Yuxing; Das, Anupam (2024): "Connecting the Dots: Tracing Data Endpoints in IoT Devices", in: Proceedings on Privacy Enhancing Technologies. (DOI)

[17]

Wu, Xiaoyuan; Hu, Lydia; Zeng, Eric; Habib, Hana; Bauer, Lujo (2025): "Transparency or Information Overload? Evaluating Users’ Comprehension and Perceptions of the iOS App Privacy Report", in: Proceedings of the Network and Distributed System Security Symposium. (Link)