User Tools

Site Tools


security:phishing

Phishing websites

You are about to crawl phishing sites, or to treat a feed as ground truth for “this URL is a phish”. The instrument is the feed plus whatever your crawler actually fetched. Those two things disagree more often than a methods section usually admits, and they disagree for reasons that are structural: the site is often gone, the feed is a dated snapshot of whatever that provider chose to publish, and the page a researcher-looking crawler gets is not always the page a victim got. PhishTank adds a further twist: a “verified” bit is a community vote, not a crawl.

This page is about measuring phishing websites — live-site crawls, feed ground truth, cloaking, feed rot. It is not a tutorial on writing a phishing detector, not a user-study of who clicks, and not SMS/email phishing unless the landing page is what you measured. Training and susceptibility live on User studies (and are mostly out of scope for this site). VirusTotal as a maliciousness oracle is VirusTotal. Website topic labels that happen to include a “phishing” category are Website classification.

A feed hit is not a live site, and a live fetch is not a victim fetch. Three facts, all measured, that a methods sentence has to survive:

  • The median phishing site is gone in hours, not days. Lee et al. [1Lee, Kiho; Lim, Kyungchan; Kim, Hyoungshick; Kwon, Yonghwi; Kim, Doowon (2025): "7 Days Later: Analyzing Phishing-Site Lifespan After Detected", in: Proceedings of the ACM Web Conference. (DOI)] watched 286,237 detected URLs: mean lifespan 54.04 hours, median 5.46 hours. DNS resolution failure was 67.23% of the takedown errors. Oest et al. [2Oest, Adam; Zhang, Penghui; Wardman, Brad; Nunes, Eric; Burgis, Jakub; Zand, Ali; Thomas, Kurt; Doupé, Adam; Ahn, Gail-Joon (2020): "Sunrise to Sunset: Analyzing the End-to-end Life Cycle and Effectiveness of Phishing Attacks at Scale", in: Proceedings of the USENIX Security Symposium. (Link)] measured the victim-side clock: the average campaign from first to last victim took 21 hours.
  • Cloaking is aimed at anti-phishing crawlers, not at “researchers” as a species. Zhang et al. [3Zhang, Penghui; Oest, Adam; Cho, Haehyun; Sun, Zhibo; Johnson, RC; Wardman, Brad; Sarker, Shaown; Kapravelos, Alexandros; Bao, Tiffany; Wang, Ruoyu; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2021): "CrawlPhish: Large-scale Analysis of Client-side Cloaking Techniques in Phishing", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] found client-side cloaking on 35,067 of 112,005 phishing websites (31.31%). None of their 150 artificial sites were blacklisted in any browser over seven days, except 21 of 50 Click-Through sites in Edge. CAPTCHA cloaking is the 2024 version of the same trick: Teoh et al. [4Teoh, Xiwen; Lin, Yun; Liu, Ruofan; Huang, Zhiyong; Dong, Jin Song (2024): "PhishDecloaker: Detecting CAPTCHA-cloaked Phishing Websites via Hybrid Vision-based Interactive Models", in: Proceedings of the USENIX Security Symposium. (Link)] submitted 100 URLs per cloaking type to VirusTotal, Google Safe Browsing and SmartScreen — 0/100 blacklisted in 24 hours on every service, and still 0 at seven days, against a 100% baseline for uncloaked twins.
  • A vendor label is a snapshot of a pipeline, not a property of the URL. Peng et al. [5Peng, Peng; Yang, Limin; Song, Linhai; Wang, Gang (2019): "Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] submitted 36 simple phishing sites to VirusTotal: 15 of 68 vendors detected at least one. After the pages were replaced with benign content, four vendors flipped some labels, and only in week three, and only for PayPal.

Date the feed, date the crawl, and say which User-Agent and IP pool you used. A sentence that reads “we used PhishTank” is citing a brand, not a method.

What to read first

Paper Why now
Zhang et al., IEEE S&P 2021 [3Zhang, Penghui; Oest, Adam; Cho, Haehyun; Sun, Zhibo; Johnson, RC; Wardman, Brad; Sarker, Shaown; Kapravelos, Alexandros; Bao, Tiffany; Wang, Ruoyu; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2021): "CrawlPhish: Large-scale Analysis of Client-side Cloaking Techniques in Phishing", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] CrawlPhish. Client-side cloaking against anti-phishing crawlers. The 31.31% figure, the forced-execution detector, and the seven-day blacklist experiment. Read it before you write a crawler that looks like a crawler.
Peng et al., IMC 2019 [5Peng, Peng; Yang, Limin; Song, Linhai; Wang, Gang (2019): "Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] VirusTotal's phishing engines as a panel, not as an oracle. The paper to cite when a reviewer asks what a “detected” bit means. The maliciousness-oracle use of the same API is on VirusTotal.
Oest et al., IEEE S&P 2019 [6Oest, Adam; Safaei, Yeganeh; Doupé, Adam; Ahn, Gail-Joon; Wardman, Brad; Tyers, Kevin (2019): "PhishFarm: A Scalable Framework for Measuring the Effectiveness of Evasion Techniques against Browser Phishing Blacklists", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] PhishFarm. You stand up phishing sites, report them, and watch browsers. Cloaking cut blacklisting from 49.4% to 23.0% and slowed mean time-to-blacklist from 126 to 238 minutes.
Oest et al., USENIX Security 2020 [2Oest, Adam; Zhang, Penghui; Wardman, Brad; Nunes, Eric; Burgis, Jakub; Zand, Ali; Thomas, Kurt; Doupé, Adam; Ahn, Gail-Joon (2020): "Sunrise to Sunset: Analyzing the End-to-end Life Cycle and Effectiveness of Phishing Attacks at Scale", in: Proceedings of the USENIX Security Symposium. (Link)] End-to-end life cycle with visibility into victims, not just feeds. 4.8 million victim visits in a year; 7.42% of known visitors later suffered a fraudulent transaction; browser warnings cut the compromised-visitor ratio to 71.51% after one hour and 43.55% after two.
Lee et al., TheWebConf 2025 [1Lee, Kiho; Lim, Kyungchan; Kim, Hyoungshick; Kwon, Yonghwi; Kim, Doowon (2025): "7 Days Later: Analyzing Phishing-Site Lifespan After Detected", in: Proceedings of the ACM Web Conference. (DOI)] What “takedown” actually is, on 286,237 URLs, with a median of 5.46 hours. Current practice for lifespan, not the eight-day kit lifetime from 2016.
Alam et al., USENIX Security 2026 [7Alam, Mahbub; Rahman, Muhammad Lutfor; Paul, Sonjoy Kumar; Hays, Amy W.; Hussain, Aftab; Huq, Md Imanul; Saxena, Nitesh (2026): "SoK: PHILTER: Uncovering Security and Functional Gaps in AI-based Phishing Website Detection Literature via an LLM-based Reasoning Framework", in: Proceedings of the USENIX Security Symposium. (Link)] SoK of AI phishing-website detectors. Outside the measurement-methods question this page is about, but it is the current map of the detector literature; the writing-side reading is on Literature review.

The feeds, today

Checked 2026-08-27. Training data about these APIs is stale by construction; the probe that re-fetches them is scripts/phishing_probe.sh.

PhishTank

Still up, still operated by Cisco Talos Intelligence Group, still offering an HTTP POST lookup and an hourly dump of currently-online verified URLs. 1) Wikipedia's “new user registration closed” claim is a 2020 FAQ note that this sitting could not re-find on the live FAQ; register.php returned HTTP 200. Do not cite Wikipedia for the current registration policy.

The dump that papers actually train on is http://data.phishtank.com/data/online-valid.csv.gz. Fetched without an API key on 2026-08-27 it returned HTTP 200, 2,549,032 gzip bytes, 73,660 rows, every row online=yes and verified=yes. Submission times run from 2011-02-18 to 2026-08-27. 65,457 of 73,660 (88.9%) have target Other — the brand field is not a taxonomy you can publish a breakdown from. The developer page still says that without a key you are limited to a few downloads per day; the unauthenticated fetch worked on this host, once. Identify your User-Agent as phishtank/<name> or you will be rate-limited harder. 2)

stats.php did not answer from this host within 20 seconds. A live count of submissions-per-day is not on this page.

PhishTank is a community vote. A “verified” bit is other users agreeing with a submission, not a crawl of the page. Treating it as ground truth for a detector is a choice; treating it as a census of phishing on the web is a category error. Tian et al. [8Tian, Ke; Jan, Steve T. K.; Hu, Hang; Yao, Danfeng; Wang, Gang (2018): "Needle in a Haystack: Tracking Down Elite Phishing Domains in the Wild", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] confirmed 1,175 squatting phishing domains and then asked PhishTank, VirusTotal and eCrimeX about them: 91.5% were still undetected a month later.

OpenPhish

A commercial feed with a community tier. The public file is https://openphish.com/feed.txt, a plain-text URL list, refresh claimed as 12 hours, “Limited”, free, with a terms-of-use gate. 3) Premium is every 5 minutes, with brand, IP, ASN, geo, and a 30-day archive.

The academic programme is 60 days of the real-time feed plus a 30-day archive, by email from an institutional address to support@openphish.com. Extensions are discretionary. Commercial use and third-party sharing are forbidden. 4)

The community feed on 2026-08-27 contained 300 URLs on 269 hosts, 201 https and 99 http. That is not a sample of “phishing on the web”. It is a 12-hour window of whatever OpenPhish chose to publish for free. Papers that need coverage (Phishpedia's 350k URLs over six months [9Lin, Yun; Liu, Ruofan; Divakaran, Dinil Mon; Ng, Jun Yang; Chan, Qing Zhou; Lu, Yiwen; Si, Yuxuan; Zhang, Fan; Dong, Jin Song (2021): "Phishpedia: A Hybrid Deep Learning Based Approach to Visually Identify Phishing Webpages", in: Proceedings of the USENIX Security Symposium. (Link)], PhishInPatterns' 56,027 [10Subramani, Karthika; Melicher, William; Starov, Oleksii; Vadrevu, Phani; Perdisci, Roberto (2022): "PhishInPatterns: measuring elicited user interactions at scale on phishing websites", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]) subscribed to Premium. Say which tier.

The script below fetches that community file and prints its shape. It does not visit the listed sites.

openphish_community_feed.py
#!/usr/bin/env python3
"""Fetch the OpenPhish community feed and print its shape.
 
This is the script embedded on security:phishing. It downloads the feed file
(the list of URLs OpenPhish currently publishes for free). It does **not**
visit those URLs.
 
    python3 openphish_community_feed.py
 
Exits 1 if the feed is empty or is not a list of http(s) URLs. Prints the
full counts, not a sample of victim sites.
"""
from __future__ import annotations
 
import sys
import urllib.request
from collections import Counter
from urllib.parse import urlparse
 
FEED = "https://openphish.com/feed.txt"
UA = "measuretheweb-openphish-community-feed/1.0"
 
 
def main() -> None:
    req = urllib.request.Request(FEED, headers={"User-Agent": UA})
    with urllib.request.urlopen(req, timeout=30) as resp:
        status = resp.status
        raw = resp.read()
        ctype = resp.headers.get_content_type()
    if status != 200:
        raise SystemExit(f"FAILURE: {FEED} returned HTTP {status}")
    text = raw.decode("utf-8")
    lines = [ln.strip() for ln in text.splitlines() if ln.strip()]
    if len(lines) == 0:
        raise SystemExit("FAILURE: community feed is empty")
    bad = [u for u in lines if urlparse(u).scheme not in ("http", "https")]
    if bad:
        raise SystemExit(f"FAILURE: {len(bad)} rows are not http(s) URLs")
    hosts = [urlparse(u).hostname for u in lines]
    if any(h is None for h in hosts):
        raise SystemExit("FAILURE: a feed URL has no hostname")
    schemes = Counter(urlparse(u).scheme for u in lines)
    print(f"url\t{FEED}")
    print(f"http_status\t{status}")
    print(f"content_type\t{ctype}")
    print(f"bytes\t{len(raw)}")
    print(f"n_urls\t{len(lines)}")
    print(f"n_hosts\t{len(set(hosts))}")
    print(f"https\t{schemes['https']}")
    print(f"http\t{schemes['http']}")
    print("note\tthis script does not visit the listed sites")
 
 
if __name__ == "__main__":
    main()

Real output, python3 openphish_community_feed.py, 2026-08-27. The URL list itself is not reprinted.

url	https://openphish.com/feed.txt
http_status	200
content_type	text/plain
bytes	15020
n_urls	300
n_hosts	269
https	201
http	99
note	this script does not visit the listed sites

Google Safe Browsing

Two different instruments that papers collapse into one name.

  • The browser warning. What Chrome, Firefox, Safari, Edge show a user. Oest et al. [6Oest, Adam; Safaei, Yeganeh; Doupé, Adam; Ahn, Gail-Joon; Wardman, Brad; Tyers, Kevin (2019): "PhishFarm: A Scalable Framework for Measuring the Effectiveness of Evasion Techniques against Browser Phishing Blacklists", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] measured it by screenshot. Mobile Chrome, Safari and Firefox showed no blacklist warnings in their 2017–2018 window. That is a user-protection measurement, and it is dated.
  • The Lookup / Update / hashes.search API. What a research crawler queries. v4 Lookup is POST …/v4/threatMatches:find?key=. v5 is hash-prefix search plus a local database. Both return HTTP 403 without an API key (fetched 2026-08-27). The APIs are non-commercial; commercial use is Web Risk. 5)

v4 is deprecated. Google's own overview says so and points at Migration From V4. Google has told API-key holders that v4 support ends 31 March 2027; that date is in the notification mail (Brave's public copy is brave-browser#56023), not on the docs page this sitting fetched. A 2026 crawl that still cites “we queried Safe Browsing v4” is citing a deprecated API with a published end date.

The migration guide is the paper you want for why v5 exists: “as of 2021, 60% of sites that deliver attacks live less than 10 minutes”; “around 25–30% of missing phishing protection is due to such data staleness.” That is Google measuring the same feed-rot this page is about, on their own list.

Test URLs for the browser instrument still live at testsafebrowsing.appspot.com (“Should show a phishing warning”). They are not a feed.

GSB as ground truth is a look-up against a closed list you cannot download as URLs. GSB as an API is a hash-prefix protocol that will not give you a URL list to crawl. If your study needs URLs, GSB is the wrong instrument. If your study needs “would Chrome warn”, it is the right one — queried the way PhishFarm queried it, by driving the browser.

APWG eCrime Exchange

Member feed. The public page is apwg.org/ecx (HTTP 200, 2026-08-27). Papers that used it had membership (Spartacus [11Zhang, Penghui; Sun, Zhibo; Kyung, Sukwha; Behrens, Hans Walter; Basque, Zion Leonahenahe; Cho, Haehyun; Oest, Adam; Wang, Ruoyu; Bao, Tiffany; Shoshitaishvili, Yan; Ahn, Gail-Joon; Doupé, Adam (2022): "I'm SPARTACUS, No, I'm SPARTACUS: Proactively Protecting Users from Phishing by Intentionally Triggering Cloaking Behavior", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)] crawled 160,728 live APWG URLs; PhishTime [12Oest, Adam; Safaei, Yeganeh; Zhang, Penghui; Wardman, Brad; Tyers, Kevin; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2020): "PhishTime: Continuous Longitudinal Measurement of the Effectiveness of Anti-phishing Blacklists", in: Proceedings of the USENIX Security Symposium. (Link)] mixed eCX with PayPal-reported mail). There is no public dump to probe from this host. If you use it, cite the membership and the date range, not “APWG”.

Cloaking

The student mistake is to crawl a phishing URL with Playwright's default Chrome, from a cloud IP, and to treat a parked page or a CAPTCHA as “not phishing”.

Client-side cloaking is common and it is aimed at crawlers. CrawlPhish [3Zhang, Penghui; Oest, Adam; Cho, Haehyun; Sun, Zhibo; Johnson, RC; Wardman, Brad; Sarker, Shaown; Kapravelos, Alexandros; Bao, Tiffany; Wang, Ruoyu; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2021): "CrawlPhish: Large-scale Analysis of Client-side Cloaking Techniques in Phishing", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] : 31.31% of 112,005 sites (23.32% in 2018, 33.70% in 2019). Spartacus [11Zhang, Penghui; Sun, Zhibo; Kyung, Sukwha; Behrens, Hans Walter; Basque, Zion Leonahenahe; Cho, Haehyun; Oest, Adam; Wang, Ruoyu; Bao, Tiffany; Shoshitaishvili, Yan; Ahn, Gail-Joon; Doupé, Adam (2022): "I'm SPARTACUS, No, I'm SPARTACUS: Proactively Protecting Users from Phishing by Intentionally Triggering Cloaking Behavior", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)] : fingerprinting-based cloaking in 96.52% (2,831 / 2,933) of kits; appending the word “bot” to the User-Agent evaded 99.31% of cloaked sites; a proxy IP evaded 88.98% (7,540 / 8,474). PhishPrint [13Acharya, Bhupendra; Vadrevu, Phani (2021): "PhishPrint: Evading Phishing Detection Crawlers by Prior Profiling", in: Proceedings of the USENIX Security Symposium. (Link)] : 18 of 20 cloaked sites they stood up stayed up for a month. PhishDecloaker [4Teoh, Xiwen; Lin, Yun; Liu, Ruofan; Huang, Zhiyong; Dong, Jin Song (2024): "PhishDecloaker: Detecting CAPTCHA-cloaked Phishing Websites via Hybrid Vision-based Interactive Models", in: Proceedings of the USENIX Security Symposium. (Link)] : 7.6% (66 / 869) of captured sites were CAPTCHA-cloaked, all discovered solely by PhishDecloaker; a separate controlled experiment (100 URLs per type) was 0/100 on VT, GSB and SmartScreen for seven days.

What to do, given that. The papers that measured cloaking did not invent a new crawler stack. They changed the request:

  • Serve a consumer User-Agent, not HeadlessChrome / bot.
  • Come from residential or at least non-cloud IPs if you want to be in the same broad class as a home user. Spartacus [11Zhang, Penghui; Sun, Zhibo; Kyung, Sukwha; Behrens, Hans Walter; Basque, Zion Leonahenahe; Cho, Haehyun; Oest, Adam; Wang, Ruoyu; Bao, Tiffany; Shoshitaishvili, Yan; Ahn, Gail-Joon; Doupé, Adam (2022): "I'm SPARTACUS, No, I'm SPARTACUS: Proactively Protecting Users from Phishing by Intentionally Triggering Cloaking Behavior", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)] found a proxy IP evaded 88.98% (7,540 / 8,474) of cloaked sites — cloud ranges are a filter kits apply. That is a different vantage, not a proof you saw what a given victim saw.
  • Execute JavaScript. A curl of the landing HTML is the thing cloaking is designed to feed you. CrawlPhish's detector is forced execution plus screenshot diff.
  • Interact if the kit asks. PhishInPatterns [10Subramani, Karthika; Melicher, William; Starov, Oleksii; Vadrevu, Phani; Perdisci, Roberto (2022): "PhishInPatterns: measuring elicited user interactions at scale on phishing websites", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] : 45% of 51,859 crawled OpenPhish URLs required the crawler to input data before the terminal page. A landing-only crawl of a phishing URL is the same mistake Interaction already documents for ordinary sites, with a worse bias: the credential page is behind the form.
  • Record the cloaking verdict per URL (cloaked / not / unknown), not a footnote.

PhishFarm [6Oest, Adam; Safaei, Yeganeh; Doupé, Adam; Ahn, Gail-Joon; Wardman, Brad; Tyers, Kevin (2019): "PhishFarm: A Scalable Framework for Measuring the Effectiveness of Evasion Techniques against Browser Phishing Blacklists", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] is the experiment to copy if you want to measure the blacklists rather than the phish: you control the site, you report it, you watch the browsers. Ethics of standing up a phishing site are Ethics; telling the operator of a compromised host is Notifying websites.

Fingerprinting used as cloaking — 73.98%–91.36% of phishing sites with JS traces collected fingerprints [14Lin, Xu; Ilia, Panagiotis; Solanki, Saumya; Polakis, Jason (2022): "Phish in Sheep's Clothing: Exploring the Authentication Pitfalls of Browser Fingerprinting", in: Proceedings of the USENIX Security Symposium. (Link)]; 96.52% of kits in Spartacus — is a phishing-measurement fact. Fingerprinting as tracking is Fingerprinting.

Lifespan, takedown, and what "offline" means

Historical figure, still cited: PhishEye [15Han, Xiao; Kheir, Nizar; Balzarotti, Davide (2016): "PhishEye: Live Monitoring of Sandboxed Phishing Kits", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)] estimated kit lifetime at eight days (5%–95% victim-connection quantiles), with 98% of their honeypot pages blacklisted and a 12-day mean detection latency; 62% of kits were blacklisted only after 75% of victims had already connected.

Current figure, different unit: Lee et al. [1Lee, Kiho; Lim, Kyungchan; Kim, Hyoungshick; Kwon, Yonghwi; Kim, Doowon (2025): "7 Days Later: Analyzing Phishing-Site Lifespan After Detected", in: Proceedings of the ACM Web Conference. (DOI)] on 286,237 detected URLs, mean 54.04 hours, median 5.46 hours. 75.84% changed DNS between discovery and takedown. 99.80% of phishing-related IPs in that dataset sat on CDNs, so IP blocking was the wrong lever there. Bijmans et al. [16Bijmans, Hugo; Booij, Tim; Schwedersky, Anneke; Nedgabat, Aria; Wegberg, Rolf van (2021): "Catching Phishers By Their Bait: Investigating the Dutch Phishing Landscape through Phishing Kit Detection", in: Proceedings of the USENIX Security Symposium. (Link)] on the Dutch landscape: mean uptime 45 hours, median 24 hours.

“Offline” in a feed is not one event. Lee et al. split takedown errors: DNS resolution failure 67.23%. A URL that NXDOMAINs, a URL that 200s a registrar parking page, and a URL that still 200s the phish behind a CAPTCHA are three different instrument readings. Report which one you counted.

Moura et al. [17Moura, Giovane C. M.; Daniels, Thomas; Bosteels, Maarten; Castro, Sebastian; Müller, Moritz; Wabeke, Thymen; Hout, Thijs van Den; Korczynski, Maciej; Smaragdakis, Georgios (2024): "Characterizing and Mitigating Phishing Attacks at ccTLD Scale", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)] is the operator-side counterpart: 28,754 phishing domains across .nl / .ie / .be, 80% of .nl and roughly 70% of .ie mitigated 24 hours after notification. If your question is “how fast does a ccTLD act”, that is the paper; if your question is “how long can a victim still reach it”, Lee et al. is the paper. They are not interchangeable.

What a blocklist hit proves

Claim you want to make Instrument that supports it Instrument that does not
You would like a list of URLs that someone, at a dated moment, labelled phishing PhishTank dump, OpenPhish feed, APWG eCX GSB API (no URL list), a VirusTotal file-hash lookup
Chrome would have warned a user at time T Drive the browser, as PhishFarm did A GSB Lookup bit, which is not the same pipeline as the warning UI
This URL is a phish right now Your crawl, with cloaking addressed, plus a human look at a sample A dump whose online=yes filter is current but whose submission_time can be years old (PhishTank dump min submission 2011-02-18)
This URL is not a phish Nothing a feed can tell you. Feeds have no negative class. Tian et al. [8Tian, Ke; Jan, Steve T. K.; Hu, Hang; Yao, Danfeng; Wang, Gang (2018): "Needle in a Haystack: Tracking Down Elite Phishing Domains in the Wild", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]: 91.5% of confirmed squatting phish were still absent from the lists they queried Using “not in PhishTank” as benign ground truth
How many phishing sites exist Not a feed, and not this corpus. Feeds are whatever was submitted and still up Publishing a feed's cardinality as prevalence

VirusTotal as a panel of engines is Peng et al. [5Peng, Peng; Yang, Limin; Song, Linhai; Wang, Gang (2019): "Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] and VirusTotal. VirusTotal as a topic classifier is Website classification. A paper that says “we labelled domains with VirusTotal” has not yet said which of those three jobs it did.

Use in publications

Everything below is a claim about seven venues — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P, 2010–2026, 5,859 extracted papers. EuroS&P, ACSAC, RAID, AsiaCCS, CHI and SOUPS are absent, so a usable-privacy phishing-training paper can sit outside these counts by construction. See Corpus. 2025–2026 venue-years are provisional.

A full-text match for phish / phishing / phishtank hits 1,098 of the 5,855 papers that have paper.cols.txt. That is not this page's N. These are security venues; the word is everywhere.

The extractor's schema union — a detection phenomenon or technique, a classification resource or targetDetail, a population.sourceList, or the slug, matching /phish/ — is 139 papers (92 web, 64 crawled). That union is still not a population: it mixes live-site crawls, detectors, feed-as-ground-truth for a different question, user studies, email/SMS, on-chain “phishing”, and mobile UI attacks.

Each of those 139 was read for a primary role. The page's population is the 52 papers whose object of study is a phishing website or kit. Detector papers are in that 52 because a student evaluating a feed is usually about to train or test a website classifier on it — the crawl and the detector share the rotten labels. They are not a licence to turn this page into a detector survey; that map is [7Alam, Mahbub; Rahman, Muhammad Lutfor; Paul, Sonjoy Kumar; Hays, Amy W.; Hussain, Aftab; Huq, Md Imanul; Saxena, Nitesh (2026): "SoK: PHILTER: Uncovering Security and Functional Gaps in AI-based Phishing Website Detection Literature via an LLM-based Reasoning Framework", in: Proceedings of the USENIX Security Symposium. (Link)].

Role Papers of 139 In the page's N? What it is
live-site 29 yes crawled or otherwise observed live phishing websites
detector 13 yes built or evaluated a phishing-website detector
cloaking 8 yes measured anti-crawler or anti-blocklist evasion on live sites
kit 2 yes measured phishing-kit artefacts (PhishEye; the credential-theft kit pipeline)
feed-gt 19 no a phishing feed used as a label source for a different question
user-study 29 no humans judging phishing — training, fMRI, screenshots, simulations
email-sms 9 no email / SMS / voice / spearphishing; landing page not the object
onchain 6 no blockchain address or transaction “phishing”
mobile-ui 6 no OS / app / WebView / password-manager UI
mention 18 no incidental

Single-label: the 139 roles sum to 139. PAGE_N is 52 (37.4% of the union). Dropping the one poster (CCS 2014, PhishTrack) moves 52 → 51. Of the 52, 49 (94.2%) include the web platform and 40 (76.9%) ran a crawl.

USENIX Security holds 17 of the 52 (32.7%); TheWebConf 11; IMC 9; CCS 8; NDSS 4; IEEE S&P 3; PETS 0. The two PETS papers in the union are a 2026 email-language user study and a privacy-training user study — both user-study, neither a website crawl. A “PETS does not do phishing websites” claim over two papers is a small-N observation, not a finding about the venue.

By year-bucket, of the 52: 2010–2013 1; 2014–2017 7; 2018–2021 13; 2022–2024 18; 2025–2026* 13. The last bucket is starred because it is provisional. Detector papers (Phishpedia 2021, PhishIntention 2022, DynaPhish 2023, KnowPhish / PhishLLM 2024, PhishLang 2026, PHILTER 2026) are the 2020s method; lifespan and cloaking measurements are what a crawler still has to get right regardless of the classifier.

Feeds named in the schema (sourceList, resourceName, or tools.name), papers of the 139, after a regex family fold:

Family Papers of 139 Of PAGE_N (52)
VirusTotal (as phishing oracle) 32 16
PhishTank 30 11
Google Safe Browsing 27 13
APWG / eCX 13 12
OpenPhish 10 8
CertStream (discovery) 9 9
Microsoft SmartScreen 3 3
phishunt.io 3 3

The namespace page's “PhishTank is the named source in 21” counted sourceList/resourceName inside the union only. Adding tools[].name and a fold that catches “Google Safe Browsing API, Malware Patrol, PhishTank, …” moves PhishTank 21 → 30. Both numbers are in the report; this page uses the fold. Residue of the fold (custom kit names, “phishing emails”, GPPF, …) is 118 strings, printed on the provenance page — they are not a missing family, they are papers that built their own thing.

Full-text probes on the 52, upper bounds until read: “cloak” 34/52, “takedown” 31/52, “lifespan” 26/52, “PhishTank” 32/52, “OpenPhish” 20/52, “Safe Browsing” 35/52, “CAPTCHA” 21/52. A probe hit is not a paper about that thing; the role table is.

One corpus defect, recorded rather than patched: IEEE S&P 2024 from-chatbots-to-phishbots-… is in the 139 because the slug matches /phish/. The stored PDF and paper.cols.txt are Nanayakkara et al. on differential privacy (DOI 10.1109/SP54263.2024.00182 on the PDF; OpenAlex attached that DOI to the phishing paper's slug). Role: mention, quote taken from the bibliographic title. Do not cite the stored PDF for that slug.

What to report so a reviewer can repeat it

  • Which feed, which tier, which file, which date range. “PhishTank” is not enough: dump vs API vs website scrape, verified-only or not, the dump's online filter. “OpenPhish” is not enough: community feed.txt (300 URLs / 12 h, today) vs Premium.
  • The list id of anything you sampled from, and a hash of the bytes you actually used. Feeds move under you. The PhishTank dump's newest row on 2026-08-27 was submitted that morning; its oldest is 2011.
  • When you crawled relative to when the URL entered the feed. Lee et al.'s median 5.46 hours is the clock you are racing.
  • User-Agent, IP pool, JS execution, interaction depth. Those four decide whether cloaking ate the crawl. See Interaction for the depth field; do not treat a phishing URL as single-target-page and then call it a site crawl.
  • What “offline” meant — NXDOMAIN, timeout, parking, CAPTCHA, 200-with-wrong-content — with a count per bin.
  • A hand-checked sample of the feed labels, with the disagreement rate. Peng et al. is the existence proof that engines disagree; you still have to measure your feed on your sample.
  • Cloaking as a first-class verdict, not a limitation paragraph.

If you take one thing off this page into a crawl: log, per URL, the feed row, the fetch timestamp, the final URL after redirects, a screenshot after JS, and whether a form was submitted. That is the instrumentation this page recommends. PhishFarm, CrawlPhish, Spartacus, PhishInPatterns and 7 Days Later each recorded some form of it. A CSV of feed URLs with a status column of “ok/dead” does not.

  • Security — namespace outline; this is the phishing-feeds child. Google Safe Browsing is here, not a sixth page.
  • VirusTotal — the other use of the same vendor panel.
  • Website classification — do not read a phishing verdict as a topic label.
  • Crawler — the crawler stack; cloaking is a detection-surface problem for the stack you picked.
  • Interaction — “landing page” in the phishing literature means the redirect target, not the site root.
  • Fingerprinting — fingerprinting-as-cloaking vs fingerprinting-as-tracking.
  • Ethics — standing up a phishing site, or scanning one, is a live-site ethics problem.
  • Notifying websites — the ccTLD notification numbers belong next to that page's response rates.

Methodology and limitations of these figures

Every corpus number is a count of papers, from the 5,859-paper extraction, with the denominator named in the same sentence. The 52-paper N is a hand map over the 139-paper schema union; the report exits 1 if the union and the map diverge. Full-text probes read paper.cols.txt (4 of 5,859 have none). 2025–2026 are provisional. The queries, the role list, the feed-fold residue, the quote check, and the unedited report output are on phishing. Corpus-wide caveats: corpus.

[1]
Lee, Kiho; Lim, Kyungchan; Kim, Hyoungshick; Kwon, Yonghwi; Kim, Doowon (2025): "7 Days Later: Analyzing Phishing-Site Lifespan After Detected", in: Proceedings of the ACM Web Conference. (DOI)
[2]
Oest, Adam; Zhang, Penghui; Wardman, Brad; Nunes, Eric; Burgis, Jakub; Zand, Ali; Thomas, Kurt; Doupé, Adam; Ahn, Gail-Joon (2020): "Sunrise to Sunset: Analyzing the End-to-end Life Cycle and Effectiveness of Phishing Attacks at Scale", in: Proceedings of the USENIX Security Symposium. (Link)
[3]
Zhang, Penghui; Oest, Adam; Cho, Haehyun; Sun, Zhibo; Johnson, RC; Wardman, Brad; Sarker, Shaown; Kapravelos, Alexandros; Bao, Tiffany; Wang, Ruoyu; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2021): "CrawlPhish: Large-scale Analysis of Client-side Cloaking Techniques in Phishing", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
[4]
Teoh, Xiwen; Lin, Yun; Liu, Ruofan; Huang, Zhiyong; Dong, Jin Song (2024): "PhishDecloaker: Detecting CAPTCHA-cloaked Phishing Websites via Hybrid Vision-based Interactive Models", in: Proceedings of the USENIX Security Symposium. (Link)
[5]
Peng, Peng; Yang, Limin; Song, Linhai; Wang, Gang (2019): "Opening the Blackbox of VirusTotal: Analyzing Online Phishing Scan Engines", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
[6]
Oest, Adam; Safaei, Yeganeh; Doupé, Adam; Ahn, Gail-Joon; Wardman, Brad; Tyers, Kevin (2019): "PhishFarm: A Scalable Framework for Measuring the Effectiveness of Evasion Techniques against Browser Phishing Blacklists", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)
[7]
Alam, Mahbub; Rahman, Muhammad Lutfor; Paul, Sonjoy Kumar; Hays, Amy W.; Hussain, Aftab; Huq, Md Imanul; Saxena, Nitesh (2026): "SoK: PHILTER: Uncovering Security and Functional Gaps in AI-based Phishing Website Detection Literature via an LLM-based Reasoning Framework", in: Proceedings of the USENIX Security Symposium. (Link)
[8]
Tian, Ke; Jan, Steve T. K.; Hu, Hang; Yao, Danfeng; Wang, Gang (2018): "Needle in a Haystack: Tracking Down Elite Phishing Domains in the Wild", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
[9]
Lin, Yun; Liu, Ruofan; Divakaran, Dinil Mon; Ng, Jun Yang; Chan, Qing Zhou; Lu, Yiwen; Si, Yuxuan; Zhang, Fan; Dong, Jin Song (2021): "Phishpedia: A Hybrid Deep Learning Based Approach to Visually Identify Phishing Webpages", in: Proceedings of the USENIX Security Symposium. (Link)
[10]
Subramani, Karthika; Melicher, William; Starov, Oleksii; Vadrevu, Phani; Perdisci, Roberto (2022): "PhishInPatterns: measuring elicited user interactions at scale on phishing websites", in: Proceedings of the ACM Internet Measurement Conference. (DOI)
[11]
Zhang, Penghui; Sun, Zhibo; Kyung, Sukwha; Behrens, Hans Walter; Basque, Zion Leonahenahe; Cho, Haehyun; Oest, Adam; Wang, Ruoyu; Bao, Tiffany; Shoshitaishvili, Yan; Ahn, Gail-Joon; Doupé, Adam (2022): "I'm SPARTACUS, No, I'm SPARTACUS: Proactively Protecting Users from Phishing by Intentionally Triggering Cloaking Behavior", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)
[12]
Oest, Adam; Safaei, Yeganeh; Zhang, Penghui; Wardman, Brad; Tyers, Kevin; Shoshitaishvili, Yan; Doupé, Adam; Ahn, Gail-Joon (2020): "PhishTime: Continuous Longitudinal Measurement of the Effectiveness of Anti-phishing Blacklists", in: Proceedings of the USENIX Security Symposium. (Link)
[13]
Acharya, Bhupendra; Vadrevu, Phani (2021): "PhishPrint: Evading Phishing Detection Crawlers by Prior Profiling", in: Proceedings of the USENIX Security Symposium. (Link)
[14]
Lin, Xu; Ilia, Panagiotis; Solanki, Saumya; Polakis, Jason (2022): "Phish in Sheep's Clothing: Exploring the Authentication Pitfalls of Browser Fingerprinting", in: Proceedings of the USENIX Security Symposium. (Link)
[15]
Han, Xiao; Kheir, Nizar; Balzarotti, Davide (2016): "PhishEye: Live Monitoring of Sandboxed Phishing Kits", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)
[16]
Bijmans, Hugo; Booij, Tim; Schwedersky, Anneke; Nedgabat, Aria; Wegberg, Rolf van (2021): "Catching Phishers By Their Bait: Investigating the Dutch Phishing Landscape through Phishing Kit Detection", in: Proceedings of the USENIX Security Symposium. (Link)
[17]
Moura, Giovane C. M.; Daniels, Thomas; Bosteels, Maarten; Castro, Sebastian; Müller, Moritz; Wabeke, Thymen; Hout, Thijs van Den; Korczynski, Maciej; Smaragdakis, Georgios (2024): "Characterizing and Mitigating Phishing Attacks at ccTLD Scale", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)
1)
Home page, API page, developer page and FAQ, all fetched 2026-08-27. The operator string is “PhishTank is operated by Cisco Talos Intelligence Group”.
2)
Developer Information, fetched 2026-08-27: “If your User Agent is blank or generic, you may recieve an increased number of rate limited requests”. Their spelling.
3)
phishing_feeds.html, fetched 2026-08-27. Community row: 12 hours, Limited, Text File, Free.
4)
academic_use.html, fetched 2026-08-27.
5)
Safe Browsing APIs (v4) overview, fetched 2026-08-27: “These APIs are for non-commercial use only; commercial users should refer to the Web Risk API. Warning: The Safe Browsing APIs (v4) are deprecated.”
You could leave a comment if you were logged in.
security/phishing.txt · Last modified: by karel.kubicek.claude

Except where otherwise noted, content on this wiki is licensed under the following license: CC BY-NC-SA 4.0
CC BY-NC-SA 4.0 Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki