Logged-out is the default crawl in this literature, and it is a design, not an accident of tooling. Of the 857 web crawls in this corpus, 553 (64.5%) say they did not authenticate, 204 (23.8%) do not say, and the schema marks 81 (9.5%) as some kind of login or registration. That 81 is a mixed bag: lab scanners of applications the authors control, one social-network account, marketplace cookies, and eleven papers the schema simply mislabelled. The figure this page publishes is the 17 papers (2.0% of 857) that actually created accounts or logged in as an instrument on the sites they were measuring.

Getting past a login is a different job from clicking a subpage. Depth, typing-versus-fill(), and the half of the web you do not measure live on Interaction. Consent banners are Consent. Whether the profile survives between visits is Stateful Stateless. This page is the remaining four jobs: log in with credentials you already have, create accounts at scale, subscribe to a newsletter, and fill a form without submitting it.

1. Login with existing credentials. Shepherd [5Jonker, Hugo; Karsch, Stefan; Krumnow, Benjamin; Sleegers, Marc (2020): "Shepherd: a Generic Approach to Automating Website Login", in: Proceedings of the Workshop on Measurements, Attacks, and Defenses for the Web. (DOI)] is the method paper. Calzavara et al. [7Calzavara, Stefano; Tolomei, Gabriele; Bugliesi, Michele; Orlando, Salvatore (2014): "Quite a mess in my cookie jar!: leveraging machine learning to protect web authentication", in: Proceedings of the ACM Web Conference. (DOI)] is the cookie-identification paper (70 sites). Kaizer et al. [3Kaizer, Andrew J.; Gupta, Minaxi (2016): "Characterizing Website Behaviors Across Logged-in and Not-logged-in Users", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] and Rautenstrauch et al. [2Rautenstrauch, Jannis; Mitkov, Metodi; Helbrecht, Thomas; Hetterich, Lorenz; Stock, Ben (2024): "To Auth or Not To Auth? A Comparative Analysis of the Pre- and Post-Login Security Landscape", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] are the measurement papers that then crawl while logged in.
2. Create accounts at scale. Cookie Hunter [1Drakonakis, Kostas; Ioannidis, Sotiris; Polakis, Jason (2020): "The Cookie Hunter: Automated Black-box Auditing for Web Authentication and Authorization Flaws", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)], the login-policies crawl [8Al Roomi, Suood; Li, Frank (2023): "A Large-Scale Measurement of Website Login Policies", in: Proceedings of the USENIX Security Symposium. (Link)], the password-policies crawl [9Alroomi, Suood; Li, Frank (2023): "Measuring Website Password Creation Policies At Scale", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)], Tripwire's honey accounts [10DeBlasio, Joe; Savage, Stefan; Voelker, Geoffrey M.; Snoeren, Alex C. (2017): "Tripwire: inferring internet site compromise", in: Proceedings of the ACM Internet Measurement Conference. (DOI)], and Kubicek et al. [4Kubicek, Karel; Merane, Jakob; Bouhoula, Ahmed; Basin, David A. (2024): "Automating Website Registration for Studying GDPR Compliance", in: Proceedings of the ACM Web Conference. (DOI)].
3. Newsletter / mailing-list signup. Lighter than a passworded account. Englehardt et al. [6Englehardt, Steven; Han, Jeffrey; Narayanan, Arvind (2018): "I never signed up for this! Privacy implications of email tracking", Proceedings on Privacy Enhancing Technologies 2018(1):109-126. (DOI)] automated it; Kubíček et al. [11Kubíček, Karel; Merane, Jakob; Cotrini, Carlos; Stremitzer, Alexander; Bechtold, Stefan; Basin, David (2022): "Checking Websites' GDPR Consent Compliance for Marketing Emails", Proceedings on Privacy Enhancing Technologies 2022(2). (DOI)] registered by hand on 666 of 1,000 sites. Mathur et al. [12Mathur, Arunesh; Wang, Angelina; Schwemmer, Carsten; Hamin, Maia; Stewart, Brandon M.; Narayanan, Arvind (2023): "Manipulative tactics are the norm in political emails: Evidence from 300K emails from the 2020 US election cycle", Big Data & Society . (DOI)] is the political-email follow-up, outside this corpus.
4. Fill without submitting. Senol et al. [13Senol, Asuman; Acar, Gunes; Humbert, Mathias; Zuiderveen Borgesius, Frederik (2022): "Leaky Forms: A Study of Email and Password Exfiltration Before Form Submission", in: 31st USENIX Security Symposium (USENIX Security 22). (Link)] filled email on 52,055 of 99,380 loaded Tranco 100k sites and left. Chatzimpyrros et al. [14Chatzimpyrros, Manolis; Solomos, Konstantinos; Ioannidis, Sotiris (2020): "You Shall Not Register! Detecting Privacy Leaks Across Registration Forms", in: Computer Security, pp. 91-104. (DOI)] did the registration-form version on 200,000 sites, outside this corpus. Typing versus setting value is on Interaction — do not duplicate it here.

Population: 857 web crawls (crawled and platforms includes web). The 1,120-crawled row that includes mobile and network scans is the OVERVIEW.md number (authentication stated by 779/1,120 = 69.6%); it is not this page's denominator.

The 42 + 24 + 15 = 81. Hand-read, they split like this. A paper can do more than one of these; the label is the job that decided whether it belongs on this page.

Four papers the schema labelled none are load-bearing on this page anyway, and are not unioned into the 81: Englehardt et al. [6Englehardt, Steven; Han, Jeffrey; Narayanan, Arvind (2018): "I never signed up for this! Privacy implications of email tracking", Proceedings on Privacy Enhancing Technologies 2018(1):109-126. (DOI)] (newsletter bot), Ghasemisharif et al. [15Ghasemisharif, Mohammad; Ramesh, Amrutha; Checkoway, Stephen; Kanich, Chris; Polakis, Jason (2018): "O Single Sign-Off, Where Art Thou? An Empirical Analysis of Single Sign-On Account Hijacking and Session Management on the Web", in: Proceedings of the USENIX Security Symposium. (Link)] (SSO census), Senol et al. [13Senol, Asuman; Acar, Gunes; Humbert, Mathias; Zuiderveen Borgesius, Frederik (2022): "Leaky Forms: A Study of Email and Password Exfiltration Before Form Submission", in: 31st USENIX Security Symposium (USENIX Security 22). (Link)] (fill without submit), Dimova et al. [16Dimova, Yana; Van Goethem, Tom; Joosen, Wouter (2023): "Everybody's Looking for SSOmething: A large-scale evaluation on the privacy of OAuth authentication on the web", Proceedings on Privacy Enhancing Technologies 2023(4). (DOI)] (OAuth buttons, no RP login at scale).

Kubicek et al. [4Kubicek, Karel; Merane, Jakob; Bouhoula, Ahmed; Basin, David A. (2024): "Automating Website Registration for Studying GDPR Compliance", in: Proceedings of the ACM Web Conference. (DOI)] is in corpus2/.meta (DOI 10.1145/3589334.3645709) and not in extractions.jsonl. There is no paper.cols.txt. Every figure from it below is from the author PDF at people.inf.ethz.ch/basin/pubs/www24.pdf, read 2026-08-27.

The 17 instrument papers, in year order: SSOScan [17Zhou, Yuchen; Evans, David (2014): "SSOScan: Automated Testing of Web Applications for Single Sign-On Vulnerabilities", in: Proceedings of the USENIX Security Symposium. (Link)]; Calzavara et al. [7Calzavara, Stefano; Tolomei, Gabriele; Bugliesi, Michele; Orlando, Salvatore (2014): "Quite a mess in my cookie jar!: leveraging machine learning to protect web authentication", in: Proceedings of the ACM Web Conference. (DOI)]; Tripwire [10DeBlasio, Joe; Savage, Stefan; Voelker, Geoffrey M.; Snoeren, Alex C. (2017): "Tripwire: inferring internet site compromise", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]; Cookie Hunter [1Drakonakis, Kostas; Ioannidis, Sotiris; Polakis, Jason (2020): "The Cookie Hunter: Automated Black-box Auditing for Web Authentication and Authorization Flaws", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)]; SAAT [18Ghasemisharif, Mohammad; Kanich, Chris; Polakis, Jason (2022): "Towards Automated Auditing for Account and Session Management Flaws in Single Sign-On Deployments", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)]; password policies [9Alroomi, Suood; Li, Frank (2023): "Measuring Website Password Creation Policies At Scale", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)]; login policies [8Al Roomi, Suood; Li, Frank (2023): "A Large-Scale Measurement of Website Login Policies", in: Proceedings of the USENIX Security Symposium. (Link)]; To Auth [2Rautenstrauch, Jannis; Mitkov, Metodi; Helbrecht, Thomas; Hetterich, Lorenz; Stock, Ben (2024): "To Auth or Not To Auth? A Comparative Analysis of the Pre- and Post-Login Security Landscape", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)]; plus nine scale-login crawls (IMC 2016 [3Kaizer, Andrew J.; Gupta, Minaxi (2016): "Characterizing Website Behaviors Across Logged-in and Not-logged-in Users", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] among them). The list is in the report script.

Year shape of the 17, of web crawls in that bucket. 2025–2026 is provisional:

The method that is still the one to copy is Shepherd [5Jonker, Hugo; Karsch, Stefan; Krumnow, Benjamin; Sleegers, Marc (2020): "Shepherd: a Generic Approach to Automating Website Login", in: Proceedings of the Workshop on Measurements, Attacks, and Defenses for the Web. (DOI)]. Out of this corpus (MADWeb 2020, an NDSS workshop). Four steps: find a login start, submit, check the response, verify. The last step is the one people skip. Shepherd runs a check twice — once with the candidate session's cookies, once without — and claims a login only when the check succeeds with cookies and fails without them. Three checks: a logout button or user identifier after submit; the same on the landing page; an attempt to re-open the login area (which a well-designed site should refuse). Verification accuracy on a 100-site sample was ≥97%.

On BugMeNot credentials for 49,846 sites, 23,088 were rejected as invalid, 26,758 remained, and Shepherd verified 7,113 logins (26.6% of the remainder; a lower bound). Facebook SSO, separately, verified 383. Finding the login area on a 5,000-site Alexa sample: landing page 225, URLs with login terms 1,119, clickables 1,149, standard URLs 1,366, search engines 1,948, combined 2,759. The methods are disjoint enough that combining them is the result; landing pages almost never contain the form, but they have to run first because everything else needs the post-redirect domain. BugMeNot is crowd-sourced credentials with site-owner removal and a ban on banks and age-restricted sites; that is an ethics fact about this instrument, not a recommendation to use it.

Calzavara et al. [7Calzavara, Stefano; Tolomei, Gabriele; Bugliesi, Michele; Orlando, Salvatore (2014): "Quite a mess in my cookie jar!: leveraging machine learning to protect web authentication", in: Proceedings of the ACM Web Conference. (DOI)] is the other half of the login problem: once you are in, which cookies are the session. They built a gold set from 70 Alexa sites; a naïve script was wrong on 2 of 70. That is identification, not login.

Kaizer et al. [3Kaizer, Andrew J.; Gupta, Minaxi (2016): "Characterizing Website Behaviors Across Logged-in and Not-logged-in Users", in: Proceedings of the ACM Internet Measurement Conference. (DOI)] created accounts manually on 345 sites in 14 Alexa categories, then used Selenium to log in. Logged-in users saw more ads and more privacy-concerning behaviour. If your question is “what does the logged-in web look like”, this is still the paper that posed it cleanly.

Rautenstrauch et al. [2Rautenstrauch, Jannis; Mitkov, Metodi; Helbrecht, Thomas; Hetterich, Lorenz; Stock, Ben (2024): "To Auth or Not To Auth? A Comparative Analysis of the Pre- and Post-Login Security Landscape", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)] is the 2024 version of that question for security headers and XSS. Semi-automatic account framework; more than 400 accounts by hand; then 200 sites where automated login kept succeeding. Playwright 1.33, Gmail for email verification. The rest of the list died of captcha loops / timeouts / blocks (177), payment or phone (170), or SSO-only (88). Those three numbers are why “we logged in where we could” is a biased sample: the survivors are the sites that let a research crawler in.

The script below is Shepherd's last step on two HTML fixtures. It does not find the form and it does not talk to the network. It does refuse a substring match of “session” against “sessionStorage”, and it does refuse English prose that says “log out”.

login_diff.py

#!/usr/bin/env python3
"""Differential login-success check (Shepherd's last step).
 
A 200 and a page that still has a login form is not a login. Jonker, Karsch,
Krumnow and Sleegers (MADWeb 2020) verify a candidate login by running the
same check twice: once with the cookies from the candidate session, once
without. A login is claimed only when the check succeeds with cookies and
fails without them.
 
This script is that comparison on two HTML snapshots. It does not find the
login form, submit credentials, or talk to the network. Those are earlier
Shepherd steps; this is the one that is easy to get wrong by substring.
 
    python3 login_diff.py                  # self-tests, then the documented run
    python3 login_diff.py --logged-out out.html --logged-in in.html
 
Exit 0 if the logged-in snapshot has at least one login marker that the
logged-out snapshot does not. Exit 1 otherwise, or if a self-test fails.
"""
from __future__ import annotations
 
import argparse
import re
import sys
 
# Markers a logged-in page typically grows and a logged-out page must not
# already contain. Exact token match, not a substring of the HTML: "session"
# inside "sessionStorage" is not a logout button.
LOGOUT_RE = re.compile(
    r"(?:id|name|aria-label|href|data-testid)\s*=\s*['\"]([^'\"]*(?:log[\s_-]*out|sign[\s_-]*out)[^'\"]*)['\"]",
    re.IGNORECASE,
)
USER_RE = re.compile(
    r"(?:id|name|aria-label|data-testid)\s*=\s*['\"]([^'\"]*(?:account|avatar|user-menu|logged-in)[^'\"]*)['\"]",
    re.IGNORECASE,
)
LOGIN_AREA_RE = re.compile(
    r"(?:id|name|aria-label|href|type)\s*=\s*['\"]([^'\"]*(?:log[\s_-]*in|sign[\s_-]*in|password)[^'\"]*)['\"]",
    re.IGNORECASE,
)
 
 
LOGGED_OUT_HTML = """<!doctype html>
<html><body>
  <a href="/login" id="login-link">Log in</a>
  <form id="login-form">
    <input type="password" name="password">
    <button type="submit">Sign in</button>
  </form>
  <script>sessionStorage.setItem("anon", "1")</script>
</body></html>
"""
 
LOGGED_IN_HTML = """<!doctype html>
<html><body>
  <a href="/account" id="user-menu">alice@example.com</a>
  <a href="/logout" id="logout-link">Log out</a>
  <div data-testid="logged-in-badge">signed in</div>
</body></html>
"""
 
# Trap: the word "session" appears, and so does "logout" as English prose,
# but no attribute token is a login marker. A substring test would claim login.
TRAP_HTML = """<!doctype html>
<html><body>
  <p>Your session expired. Please log out of other devices, then log in.</p>
  <script>sessionStorage.clear()</script>
  <a href="/login" id="login-link">Log in</a>
</body></html>
"""
 
 
def markers(html: str) -> dict[str, set[str]]:
    return {
        "logout": {m.group(1).lower() for m in LOGOUT_RE.finditer(html)},
        "user": {m.group(1).lower() for m in USER_RE.finditer(html)},
        "login_area": {m.group(1).lower() for m in LOGIN_AREA_RE.finditer(html)},
    }
 
 
def verdict(logged_out: str, logged_in: str) -> tuple[bool, list[str]]:
    """Shepherd: a method must succeed with cookies and fail without them."""
    out_m = markers(logged_out)
    in_m = markers(logged_in)
    reasons: list[str] = []
 
    logout_only_in = in_m["logout"] - out_m["logout"]
    if logout_only_in:
        reasons.append("logout marker present only when logged in: " + ", ".join(sorted(logout_only_in)))
 
    user_only_in = in_m["user"] - out_m["user"]
    if user_only_in:
        reasons.append("user marker present only when logged in: " + ", ".join(sorted(user_only_in)))
 
    login_only_out = out_m["login_area"] - in_m["login_area"]
    if out_m["login_area"] and not in_m["login_area"] and login_only_out:
        reasons.append(
            "login area present logged-out and absent logged-in: " + ", ".join(sorted(login_only_out))
        )
 
    return bool(reasons), reasons
 
 
def self_test() -> None:
    failed = 0
 
    ok, why = verdict(LOGGED_OUT_HTML, LOGGED_IN_HTML)
    if not ok:
        print("SELFTEST FAIL: documented pair should verify")
        failed += 1
    else:
        print("SELFTEST ok   documented pair verifies:")
        for line in why:
            print("         ", line)
 
    ok, why = verdict(LOGGED_OUT_HTML, LOGGED_OUT_HTML)
    if ok:
        print("SELFTEST FAIL: two logged-out snapshots must not verify:", why)
        failed += 1
    else:
        print("SELFTEST ok   two logged-out snapshots do not verify")
 
    ok, why = verdict(TRAP_HTML, TRAP_HTML)
    if ok:
        print("SELFTEST FAIL: prose 'log out' must not count as a marker:", why)
        failed += 1
    else:
        print("SELFTEST ok   prose 'log out' / sessionStorage is not a marker")
 
    # Equality, not substring: a logged-in page whose only new token is
    # "sessionid" must not be claimed by looking for the string "session".
    session_in = '<a href="/x" id="sessionid-link">account</a>'
    session_out = '<a href="/x" id="login-link">Log in</a>'
    in_m = markers(session_in)
    if any("session" == tok for toks in in_m.values() for tok in toks):
        print("SELFTEST FAIL: token equality collapsed to substring 'session'")
        failed += 1
    else:
        print("SELFTEST ok   'sessionid-link' is not the token 'session'")
 
    if failed:
        raise SystemExit(f"login_diff.py: {failed} self-test(s) failed")
 
 
def main() -> int:
    ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
    ap.add_argument("--logged-out", help="HTML snapshot without the candidate session")
    ap.add_argument("--logged-in", help="HTML snapshot with the candidate session")
    ap.add_argument("--skip-self-test", action="store_true")
    args = ap.parse_args()
 
    if not args.skip_self_test:
        self_test()
 
    if args.logged_out is None and args.logged_in is None:
        logged_out = LOGGED_OUT_HTML
        logged_in = LOGGED_IN_HTML
        print("\nDocumented run (embedded fixtures):")
    else:
        if args.logged_out is None or args.logged_in is None:
            ap.error("give both --logged-out and --logged-in, or neither")
        logged_out = open(args.logged_out, encoding="utf-8").read()
        logged_in = open(args.logged_in, encoding="utf-8").read()
        print("\nRun on supplied snapshots:")
 
    ok, why = verdict(logged_out, logged_in)
    if ok:
        print("VERIFIED login")
        for line in why:
            print("  ", line)
        return 0
    print("NOT VERIFIED: no marker present only in the logged-in snapshot")
    print("  logged-out markers:", markers(logged_out))
    print("  logged-in markers: ", markers(logged_in))
    return 1
 
 
if __name__ == "__main__":
    sys.exit(main())

Documented run, 2026-08-27, four self-tests then the fixtures:

SELFTEST ok   documented pair verifies:
          logout marker present only when logged in: /logout, logout-link
          user marker present only when logged in: logged-in-badge, user-menu
          login area present logged-out and absent logged-in: /login, login-form, login-link, password
SELFTEST ok   two logged-out snapshots do not verify
SELFTEST ok   prose 'log out' / sessionStorage is not a marker
SELFTEST ok   'sessionid-link' is not the token 'session'

Documented run (embedded fixtures):
VERIFIED login
   logout marker present only when logged in: /logout, logout-link
   user marker present only when logged in: logged-in-badge, user-menu
   login area present logged-out and absent logged-in: /login, login-form, login-link, password

These are different papers. Using Facebook as a door into a ranked list is scale-login / scale-register. Counting OAuth buttons is an SSO study, and several of those studies never complete the relying-party login.

• SSOScan, Zhou and Evans, USENIX Security 2014 [17Zhou, Yuchen; Evans, David (2014): "SSOScan: Automated Testing of Web Applications for Single Sign-On Vulnerabilities", in: Proceedings of the USENIX Security Symposium. (Link)]. Quantcast, 17,913 sites after filtering; Facebook SSO on 1,660 (9.3%). The enrolment process succeeds for 80% of Facebook-SSO sites in the top 10k. About 44% of sites still require a traditional registration after SSO. Top-10k automation failures included 34 CAPTCHAs. This is the 2014 “Zhou” the stub named; it is not old in the sense of a dead method, it is the first automated SSO enrolment crawl in this corpus.
• O Single Sign-Off, Ghasemisharif et al., USENIX Security 2018 [15Ghasemisharif, Mohammad; Ramesh, Amrutha; Checkoway, Stephen; Kanich, Chris; Polakis, Jason (2018): "O Single Sign-Off, Where Art Thou? An Empirical Analysis of Single Sign-On Account Hijacking and Session Management on the Web", in: Proceedings of the USENIX Security Symposium. (Link)]. Puppeteer over Alexa 1M; 912,206 processed; SSO on 57,555 (6.30%); Facebook 4.62% (42,232), Google 2.75% (25,142). Schema: none. The RP takeover was manual.
• SSOmething, Dimova et al., PoPETs 2023 [16Dimova, Yana; Van Goethem, Tom; Joosen, Wouter (2023): "Everybody's Looking for SSOmething: A large-scale evaluation on the privacy of OAuth authentication on the web", Proceedings on Privacy Enhancing Technologies 2023(4). (DOI)]. CDP crawl of CrUX 100k; OAuth buttons on 6,211 (7.23%); 18.53% requested non-minimal scopes. Schema: none. They did not complete RP login at scale.
• SAAT, Ghasemisharif, Kanich and Polakis, IEEE S&P 2022 [18Ghasemisharif, Mohammad; Kanich, Chris; Polakis, Jason (2022): "Towards Automated Auditing for Account and Session Management Flaws in Single Sign-On Deployments", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)]. Facebook SSO login on 1,900 RPs; traditional-credential registration on 243. ReCAPTCHA v2 audio via Wit.ai; Gmail API plus Twilio SMS; puppeteer-extra-plugin-stealth. They wrote down the ToS caveat: “While this could potentially violate RPs' terms of service, automation is a widely established common practice in web security research”.

A crawl that only detects an SSO button has not logged in. Dimova et al. are explicit about that. Do not cite 7.23% as a login rate.

Five numbers people confuse. They do not share a denominator.

Kubicek et al. compare themselves to Cookie Hunter's 1.6% of Alexa 1M, not to 13.7% of signup domains. Use the same comparison they used.

Cookie Hunter. XDriver on Selenium. Email verification by visiting links in mail. SMS / SSN blocked leftover of the top 1K, completed by hand; the ~25K that make the evaluation “did not require any manual intervention”. reCAPTCHA on 19,491 (~13.8%) of signup domains. They tried an audio-reCAPTCHA solver; Google was not serving captchas to WebDriver. They did not pay a farm:

funding human captcha-solving services to create accounts presents an ethical dilemma, we opted to not handle such cases.

Login policies (Al Roomi and Li) [8Al Roomi, Suood; Li, Frank (2023): "A Large-Scale Measurement of Website Login Policies", in: Proceedings of the USENIX Security Symposium. (Link)]. CrUX 1M (they started ground-truth on Tranco and switched because CrUX is websites). Signup page on 258,200 domains; 45.0K initial accounts; a second account on only 37,300. AZcaptcha. In the ground-truth accounts they created by hand, 39% of domains sent a verification email — that 39% is not of the 45.0K. They did not complete phone verification.

Password policies (Alroomi and Li) [9Alroomi, Suood; Li, Frank (2023): "Measuring Website Password Creation Policies At Scale", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)]. Tranco 1M; 20,119 domains fully evaluated. CAPTCHAs on at least 49% of signup forms; AZcaptcha solved 94%. They avoided human-driven solvers “due to ethical issues identified with such services”, and they “are unable to explicitly check” terms of service at this scale.

Tripwire [10DeBlasio, Joe; Savage, Stefan; Voelker, Geoffrey M.; Snoeren, Alex C. (2017): "Tripwire: inferring internet site compromise", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]. Honey accounts to infer site compromise from password reuse. 65,413 registration attempts across 33,634 sites; 3,664 accounts on around 2,302 sites. Third-party CAPTCHA solver (DeCaptcher). “While we make no attempt to explicitly check the terms of service”.

Kubicek et al. [4Kubicek, Karel; Merane, Jakob; Bouhoula, Ahmed; Basin, David A. (2024): "Automating Website Registration for Studying GDPR Compliance", in: Proceedings of the ACM Web Conference. (DOI)], author PDF. Tranco list 82Q3V, June 2022. A processing error sampled one million domains with replacement, so the results are 660,202 unique domains. Loaded 504,509; form on 25.7% (169,765); form-submit estimated success 30.2% (51,290), fail 38.4%, unknown 31.4%. Headline 5.9% register-or-newsletter. CrUX overlap 51.9%; load 90.6% vs 65.3% non-CrUX; “successful registration” 11.7% CrUX vs 3.9% non-CrUX. Selenium + Chrome, 60 parallel Docker containers, 12 German Research Network IPs, unique email per site at sybilmail.de. CAPTCHA on one-third; of those, 75% reCAPTCHA v2, 20% v3, 2% hCaptcha, 3% image. They used a human CAPTCHA farm, discussed it with their legal department, and later switched to research assistants. The crawler is not public (bomb-attack risk); access is by request at https://karelkubicek.github.io/post/reg-www. 37.2% (12,605) marketing without proper consent; 1.8% email shared with undeclared third parties; 59.8% fail double opt-in. They do not do SSO registration.

A mailing-list form is not an account. Coverage is higher, legal exposure is different, and you still have to say which one you did.

Englehardt, Han and Narayanan [6Englehardt, Steven; Han, Jeffrey; Narayanan, Arvind (2018): "I never signed up for this! Privacy implications of email tracking", Proceedings on Privacy Enhancing Technologies 2018(1):109-126. (DOI)]: OpenWPM; 15,700 sites visited; forms attempted on 3,335; of a sample of 252, 38% successful submit and 16% failed for a CAPTCHA; 12,618 emails from 902 senders. Schema: none.

Kubíček et al. [11Kubíček, Karel; Merane, Jakob; Cotrini, Carlos; Stremitzer, Alexander; Bechtold, Stefan; Basin, David (2022): "Checking Websites' GDPR Consent Compliance for Marketing Emails", Proceedings on Privacy Enhancing Technologies 2022(2). (DOI)]: human annotators; 666 of 1,000 registered; more than 5,000 emails. Pair these two sentences from the same paragraph, and do not paste the abstract's 59% onto the 666:

Only 59% of websites that sent us at least one email first sent us a double opt-in email. Moreover, 5.5% of services sent us an unsolicited marketing email without any confirmation or double opt-in email.

The 59% is of websites that sent mail. The 5.5% is of services that sent unsolicited marketing with no confirmation. Neither is “59% of 666”.

Mathur et al. [12Mathur, Arunesh; Wang, Angelina; Schwemmer, Carsten; Hamin, Maia; Stewart, Brandon M.; Narayanan, Arvind (2023): "Manipulative tactics are the norm in political emails: Evidence from 300K emails from the 2020 US election cycle", Big Data & Society . (DOI)]: Big Data & Society 2023, outside this corpus. 300K political emails from the 2020 US cycle, built on Englehardt's OpenWPM signup bot. Kubicek et al. cite the 2020 project page; the journal version is the citable record.

Senol et al. [13Senol, Asuman; Acar, Gunes; Humbert, Mathias; Zuiderveen Borgesius, Frederik (2022): "Leaky Forms: A Study of Email and Password Exfiltration Before Form Submission", in: 31st USENIX Security Symposium (USENIX Security 22). (Link)] filled email on 52,055 of 99,380 loaded Tranco 100k sites (EU, no-action) and left. Schema: none. How they typed — randomised key intervals, not input.value — is on Interaction. Playwright's locator.pressSequentially() versus fill() is there too.

Chatzimpyrros, Solomos and Ioannidis [14Chatzimpyrros, Manolis; Solomos, Konstantinos; Ioannidis, Sotiris (2020): "You Shall Not Register! Detecting Privacy Leaks Across Registration Forms", in: Computer Security, pp. 91-104. (DOI)]: ESORICS workshop, LNCS 2020, outside this corpus. ~200,000 sites; about 5% leaked PII from registration forms without submit.

CAPTCHA is non-random loss. Cookie Hunter's 22 unsolved anti-bot cases, quoted on Interaction, are the same observation: the sites that block you are not a random sample of sites.

Do not take a share from Similarweb, wmtips, or any SEO page. The 75/20/2/3 mix is Kubicek et al.'s 2022 crawl. Google's reCAPTCHA documentation still serves (HTTP 200 on 2026-08-27); so does hCaptcha's. Turnstile is the one that changed the “show a puzzle” default.

Email verification is the common path (login-policies ground-truth 39%; Cookie Hunter visited the link). SMS is the expensive path: SAAT used Twilio; Cookie Hunter treated phone/SSN as a leftover for the top 1K; login-policies did not do phone. Plan for email. Budget SMS only if the research question lives behind it.

Human CAPTCHA farms: name who used them, and when. Cookie Hunter no; Kubicek et al. yes then research assistants; Tripwire yes; password-policies AZcaptcha, not human, on purpose; Teoh et al. infiltrated 2Captcha in 2025 as an experiment, not as a measurement instrument they recommend. Do not present a farm as current best practice.

The ethics page already ranks what crawling papers say they did. Of 697 papers that describe a mitigation, 92 (13.2%) mention test accounts or synthetic identities, and 17 (2.4%) mention robots.txt, terms of service or acceptable-use policies. A full-text sweep on the same 1,120 crawling papers finds 129 (11.5%) that mention terms of service at all — without separating “we complied” from “the site's ToS prohibits X”. This page does not re-derive those figures. See Ethics.

What the registration papers themselves wrote:

• Tripwire [10DeBlasio, Joe; Savage, Stefan; Voelker, Geoffrey M.; Snoeren, Alex C. (2017): "Tripwire: inferring internet site compromise", in: Proceedings of the ACM Internet Measurement Conference. (DOI)]: “While we make no attempt to explicitly check the terms of service”.
• Password policies [9Alroomi, Suood; Li, Frank (2023): "Measuring Website Password Creation Policies At Scale", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)]: consulted general counsel; “unable to explicitly check” ToS at this scale; avoided human CAPTCHA solvers.
• SAAT [18Ghasemisharif, Mohammad; Kanich, Chris; Polakis, Jason (2022): "Towards Automated Auditing for Account and Session Management Flaws in Single Sign-On Deployments", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)]: named the ToS tension and did it anyway, as “widely established common practice”.
• Cookie Hunter [1Drakonakis, Kostas; Ioannidis, Sotiris; Polakis, Jason (2020): "The Cookie Hunter: Automated Black-box Auditing for Web Authentication and Authorization Flaws", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)]: the farm was the ethical line they would not cross; they still created 25,242 accounts.
• Kubicek et al. [4Kubicek, Karel; Merane, Jakob; Bouhoula, Ahmed; Basin, David A. (2024): "Automating Website Registration for Studying GDPR Compliance", in: Proceedings of the ACM Web Conference. (DOI)]: legal review, farm conditions discussed with the university legal department, crawler withheld because it is a bomb-attack tool.

There is no consensus in this literature that creating test accounts is permitted, forbidden, or ToS-exempt. There is a consensus that almost nobody reads the ToS of 45.0K sites. Write the sentence: synthetic identities, unique emails, whether you solved CAPTCHAs and how, whether you completed email/SMS, and that you did not check ToS at scale (if you did not). Ethics is the page for robots.txt and for the network acceptable-use policy that will actually stop you.

A methods paragraph that answers these is enough. None of them takes more than a clause.

1. Which of the four jobs you did (login / register / newsletter / fill-without-submit).
2. How many accounts, of how many sites that offered the form, of how many sites you tried. Three numbers, not one rate.
3. How you verified login — differential (Shepherd), identifier in the page, session cookie still set on a second visit. “HTTP 200 after POST” is not a verification.
4. CAPTCHA: which providers you saw, whether you solved, automated API or human or VLM or not at all, and that this censors the sample.
5. Email and SMS: unique addresses? visited the link? phone numbers? Twilio?
6. ToS / ethics: test accounts, whether you checked ToS (usually: you did not, at this scale), IRB/legal if you have it.
7. SSO: did you complete the RP login, or only count buttons?

Corpus: 5,859 extracted papers, 2010–2026, CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. Population for every share on this page is the 857 web crawls unless a sentence names another. 2025–2026 venue-years are incomplete by construction; tables that include them are starred.

The 81 is a schema enum; the 17 is a hand map (scripts/registration_fold.mjs) checked both ways at run time by scripts/report_registration.mjs. A missing ROLE or a stale ROLE key prints FAILURE and exits 1. Loose full-text sweeps for “register” and “shepherd” are homograph-dominated and were not used as counts.

Out of corpus, and labelled so wherever they appear: Shepherd (MADWeb 2020), Chatzimpyrros et al. (ESORICS workshop), Mathur et al. (Big Data & Society). In the bibliographic index but not extracted: Kubicek et al. WWW 2024.

The query log, the ROLE list, the rejected sweeps, and the review log are on registration. Corpus-level caveats are on corpus.

• Interaction — depth, typing versus fill(), the logged-out default as a coverage problem
• Stateful Stateless — the profile that login writes into
• Consent — the banner in front of the form
• Ethics — robots.txt, test accounts, ToS as a harm-mitigation family
• Crawler — the library that clicks the button
• Tranco — pinning the list id (Kubicek et al. used 82Q3V)

[1]

Drakonakis, Kostas; Ioannidis, Sotiris; Polakis, Jason (2020): "The Cookie Hunter: Automated Black-box Auditing for Web Authentication and Authorization Flaws", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)

[2]

Rautenstrauch, Jannis; Mitkov, Metodi; Helbrecht, Thomas; Hetterich, Lorenz; Stock, Ben (2024): "To Auth or Not To Auth? A Comparative Analysis of the Pre- and Post-Login Security Landscape", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[3]

Kaizer, Andrew J.; Gupta, Minaxi (2016): "Characterizing Website Behaviors Across Logged-in and Not-logged-in Users", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[4]

Kubicek, Karel; Merane, Jakob; Bouhoula, Ahmed; Basin, David A. (2024): "Automating Website Registration for Studying GDPR Compliance", in: Proceedings of the ACM Web Conference. (DOI)

[5]

Jonker, Hugo; Karsch, Stefan; Krumnow, Benjamin; Sleegers, Marc (2020): "Shepherd: a Generic Approach to Automating Website Login", in: Proceedings of the Workshop on Measurements, Attacks, and Defenses for the Web. (DOI)

[6]

Englehardt, Steven; Han, Jeffrey; Narayanan, Arvind (2018): "I never signed up for this! Privacy implications of email tracking", Proceedings on Privacy Enhancing Technologies 2018(1):109-126. (DOI)

[7]

Calzavara, Stefano; Tolomei, Gabriele; Bugliesi, Michele; Orlando, Salvatore (2014): "Quite a mess in my cookie jar!: leveraging machine learning to protect web authentication", in: Proceedings of the ACM Web Conference. (DOI)

[8]

Al Roomi, Suood; Li, Frank (2023): "A Large-Scale Measurement of Website Login Policies", in: Proceedings of the USENIX Security Symposium. (Link)

[9]

Alroomi, Suood; Li, Frank (2023): "Measuring Website Password Creation Policies At Scale", in: Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. (DOI)

[10]

DeBlasio, Joe; Savage, Stefan; Voelker, Geoffrey M.; Snoeren, Alex C. (2017): "Tripwire: inferring internet site compromise", in: Proceedings of the ACM Internet Measurement Conference. (DOI)

[11]

Kubíček, Karel; Merane, Jakob; Cotrini, Carlos; Stremitzer, Alexander; Bechtold, Stefan; Basin, David (2022): "Checking Websites' GDPR Consent Compliance for Marketing Emails", Proceedings on Privacy Enhancing Technologies 2022(2). (DOI)

[12]

Mathur, Arunesh; Wang, Angelina; Schwemmer, Carsten; Hamin, Maia; Stewart, Brandon M.; Narayanan, Arvind (2023): "Manipulative tactics are the norm in political emails: Evidence from 300K emails from the 2020 US election cycle", Big Data & Society . (DOI)

[13]

Senol, Asuman; Acar, Gunes; Humbert, Mathias; Zuiderveen Borgesius, Frederik (2022): "Leaky Forms: A Study of Email and Password Exfiltration Before Form Submission", in: 31st USENIX Security Symposium (USENIX Security 22). (Link)

[14]

Chatzimpyrros, Manolis; Solomos, Konstantinos; Ioannidis, Sotiris (2020): "You Shall Not Register! Detecting Privacy Leaks Across Registration Forms", in: Computer Security, pp. 91-104. (DOI)

[15]

Ghasemisharif, Mohammad; Ramesh, Amrutha; Checkoway, Stephen; Kanich, Chris; Polakis, Jason (2018): "O Single Sign-Off, Where Art Thou? An Empirical Analysis of Single Sign-On Account Hijacking and Session Management on the Web", in: Proceedings of the USENIX Security Symposium. (Link)

[16]

Dimova, Yana; Van Goethem, Tom; Joosen, Wouter (2023): "Everybody's Looking for SSOmething: A large-scale evaluation on the privacy of OAuth authentication on the web", Proceedings on Privacy Enhancing Technologies 2023(4). (DOI)

[17]

Zhou, Yuchen; Evans, David (2014): "SSOScan: Automated Testing of Web Applications for Single Sign-On Vulnerabilities", in: Proceedings of the USENIX Security Symposium. (Link)

[18]

Ghasemisharif, Mohammad; Kanich, Chris; Polakis, Jason (2022): "Towards Automated Auditing for Account and Session Management Flaws in Single Sign-On Deployments", in: Proceedings of the IEEE Symposium on Security and Privacy. (DOI)

[19]

Searles, Andrew; Nakatsuka, Yoshimichi; Ozturk, Ercan; Paverd, Andrew; Tsudik, Gene; Enkoji, Ai (2023): "An Empirical Study & Evaluation of Modern CAPTCHAs", in: Proceedings of the USENIX Security Symposium. (Link)

[20]

Teoh, Xiwen; Lin, Yun; Li, Siqi; Liu, Ruofan; Sollomoni, Avi; Harel, Yaniv; Dong, Jin Song (2025): "Are CAPTCHAs Still Bot-hard? Generalized Visual CAPTCHA Solving with Agentic Vision Language Model", in: Proceedings of the USENIX Security Symposium. (Link)