User Tools

Site Tools


programming:registration

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
programming:registration [2026/08/27 12:46] – Replace 2.5 KB stub with corpus-backed login/registration page: 17/857 instrument figure, ROLE split of the schema 81, Shepherd demo. Authored by Claude. karel.kubicek.claudeprogramming:registration [2026/08/27 13:08] (current) – Apply focused+generic review: split Cookie Hunter 13.7% vs 25,242; Kubicek 25.7% of unique domains; login_diff segment match and aria-label; recaptcha deprecation; no ToS-consensus claim. Authored by Claude karel.kubicek.claude
Line 1: Line 1:
 ====== Automating Login and Registration to Websites ====== ====== Automating Login and Registration to Websites ======
  
-Logged-out is the default crawl in this literature, and it is a **design**, not an accident of tooling. Of the 857 web crawls in this corpus, **553 (64.5%)** say they did not authenticate, **204 (23.8%)** do not say, and the schema marks **81 (9.5%)** as some kind of login or registration. That 81 is a mixed bag: lab scanners of applications the authors control, one social-network account, marketplace cookies, and eleven papers the schema simply mislabelled. The figure this page publishes is the **17 papers (2.0% of 857)** that actually created accounts or logged in as an instrument on the sites they were measuring.+Logged-out is the default crawl in this literature. Of the 857 web crawls in this corpus, **553 (64.5%)** are labelled ''none'' (did not authenticate), **204 (23.8%)** do not say, and the schema marks **81 (9.5%)** as some kind of login or registration. That 81 is a mixed bag: lab scanners of applications the authors control, one social-network account, marketplace cookies, and eleven papers the schema simply mislabelled. The figure this page publishes is the **17 papers (2.0% of 857)** that, among those 81, created accounts or logged in as an instrument on the sites they were measuring — a hand split of the schema hits, not a full-text census of the 776 papers the schema did not mark as logged in.
  
 <WRAP important> <WRAP important>
-Do not read the 81, or the 42 + 24 + 15 split next to it, as "papers that logged in." ''crawlConfig.authentication'' has **one evidence quote for the whole object**, so the label cannot be checked from the quote. A neighbouring page published 81/857 = 9.5% without a role split; this page keeps that schema count visible and replaces it, for advice, with the 17. SSO as a schema value is **0** among web crawls — the one corpus ''sso'' label is an Android subscription-app study. Papers that //measured// SSO still exist; they are labelled ''none'' or something else.+Do not read the 81, or the 42 + 24 + 15 split next to it, as "papers that logged in." ''crawlConfig.authentication'' has **one evidence quote for the whole object**, so the label cannot be checked from the quote. A neighbouring page published 81/857 = 9.5% without a role split; this page keeps that schema count visible and replaces it, for advice, with the 17. The 17 is a hand split of those 81 (plus a closed SCHEMA_MISS list that is not unioned in). It is not a census of all 857. SSO as a schema value is **0** among web crawls — the one corpus ''sso'' label is an Android subscription-app study. Papers that //measured// SSO still exist; they are labelled ''none'' or something else.
 </WRAP> </WRAP>
  
Line 12: Line 12:
  
 ^ Paper ^ Why ^ ^ Paper ^ Why ^
-| Drakonakis et al., CCS 2020, //Cookie Hunter// {[drakonakis2020_cookie]} | The scale-registration baseline: 25,242 accounts on 168,594 signup domains (13.7%), and they **refused** a human CAPTCHA farm |+| Drakonakis et al., CCS 2020, //Cookie Hunter// {[drakonakis2020_cookie]} | The scale-registration baseline: **13.7%** registered and logged in of 168,594 signup domains; **25,242** accounts created (not the same figure). They **refused** a human CAPTCHA farm |
 | Rautenstrauch et al., IEEE S&P 2024, //To Auth or Not To Auth// {[rautenstrauch2024_auth]} | Why you bother: 200 sites where automated login then kept working; more than 400 accounts made by hand; the failure modes that ate the rest of the list | | Rautenstrauch et al., IEEE S&P 2024, //To Auth or Not To Auth// {[rautenstrauch2024_auth]} | Why you bother: 200 sites where automated login then kept working; more than 400 accounts made by hand; the failure modes that ate the rest of the list |
 | Kaizer et al., IMC 2016 {[kaizer2016_characterizing]} | The existence proof that logged-in and logged-out are different websites: 345 sites, 14 Alexa categories, accounts created by hand, Selenium for the login | | Kaizer et al., IMC 2016 {[kaizer2016_characterizing]} | The existence proof that logged-in and logged-out are different websites: 345 sites, 14 Alexa categories, accounts created by hand, Selenium for the login |
Line 109: Line 109:
 import sys import sys
  
-Markers logged-in page typically grows and a logged-out page must not +Attribute values are split into path segments. A segment matches keyword 
-already contain. Exact token matchnot a substring of the HTML: "session" +only if it equals the keywordor the keyword plus a button/link suffix
-# inside "sessionStorage" is not logout button. +# So "logoutand "logout-linkhit; "logout-policy", "accounting""gravatar
-LOGOUT_RE = re.compile( +# do not. Substring search of the HTML is how those three false-positived
-    r"(?:id|name|aria-label|href|data-testid)\s*=\s*['\"]([^'\"]*(?:log[\s_-]*out|sign[\s_-]*out)[^'\"]*)['\"]", +ATTR_RE = re.compile( 
-    re.IGNORECASE, +    r"(?:id|name|aria-label|href|data-testid|type)\s*=\s*['\"]([^'\"]+)['\"]",
-+
-USER_RE = re.compile( +
-    r"(?:id|name|aria-label|data-testid)\s*=\s*['\"]([^'\"]*(?:account|avatar|user-menu|logged-in)[^'\"]*)['\"]", +
-    re.IGNORECASE, +
-) +
-LOGIN_AREA_RE = re.compile( +
-    r"(?:id|name|aria-label|href|type)\s*=\s*['\"]([^'\"]*(?:log[\s_-]*in|sign[\s_-]*in|password)[^'\"]*)['\"]",+
     re.IGNORECASE,     re.IGNORECASE,
 ) )
 +SUFFIXES = ("", "-link", "-btn", "-button", "_link")
 +LOGOUT_KW = ("logout", "log-out", "signout", "sign-out", "log_out", "sign_out")
 +USER_KW = ("account", "avatar", "user-menu", "logged-in", "logged_in")
 +LOGIN_KW = ("login", "log-in", "signin", "sign-in", "log_in", "sign_in", "password")
 +
 +
 +def segments(value: str) -> set[str]:
 +    parts = re.split(r"[/?#&=]+", value.lower())
 +    out = {p.strip() for p in parts if p.strip()}
 +    # aria-label="Log out" is one attribute, not a path. Collapse space/underscore
 +    # to hyphen so it matches the keyword "log-out". Do not split on spaces: that
 +    # would turn the label into the tokens "log" and "out", which are not keywords.
 +    out.add(value.lower().replace("_", "-").replace(" ", "-"))
 +    return out
 +
 +
 +def hits(html: str, keywords: tuple[str, ...]) -> set[str]:
 +    found: set[str] = set()
 +    allowed = {kw + suf for kw in keywords for suf in SUFFIXES}
 +    for m in ATTR_RE.finditer(html):
 +        for seg in segments(m.group(1)):
 +            if seg in allowed:
 +                found.add(seg)
 +    return found
 +
 +
 +def markers(html: str) -> dict[str, set[str]]:
 +    return {
 +        "logout": hits(html, LOGOUT_KW),
 +        "user": hits(html, USER_KW),
 +        "login_area": hits(html, LOGIN_KW),
 +    }
  
  
Line 141: Line 166:
   <a href="/account" id="user-menu">alice@example.com</a>   <a href="/account" id="user-menu">alice@example.com</a>
   <a href="/logout" id="logout-link">Log out</a>   <a href="/logout" id="logout-link">Log out</a>
-  <div data-testid="logged-in-badge">signed in</div>+  <div data-testid="logged-in">signed in</div>
 </body></html> </body></html>
 """ """
  
-# Trap: the word "session" appears, and so does "logout" as English prose, 
-# but no attribute token is a login marker. A substring test would claim login. 
 TRAP_HTML = """<!doctype html> TRAP_HTML = """<!doctype html>
 <html><body> <html><body>
Line 154: Line 177:
 </body></html> </body></html>
 """ """
- 
- 
-def markers(html: str) -> dict[str, set[str]]: 
-    return { 
-        "logout": {m.group(1).lower() for m in LOGOUT_RE.finditer(html)}, 
-        "user": {m.group(1).lower() for m in USER_RE.finditer(html)}, 
-        "login_area": {m.group(1).lower() for m in LOGIN_AREA_RE.finditer(html)}, 
-    } 
  
  
Line 199: Line 214:
             print("         ", line)             print("         ", line)
  
-    ok, why = verdict(LOGGED_OUT_HTML, LOGGED_OUT_HTML)+    ok, _why = verdict(LOGGED_OUT_HTML, LOGGED_OUT_HTML)
     if ok:     if ok:
-        print("SELFTEST FAIL: two logged-out snapshots must not verify:", why)+        print("SELFTEST FAIL: two logged-out snapshots must not verify")
         failed += 1         failed += 1
     else:     else:
         print("SELFTEST ok   two logged-out snapshots do not verify")         print("SELFTEST ok   two logged-out snapshots do not verify")
  
-    ok, why = verdict(TRAP_HTML, TRAP_HTML)+    ok, _why = verdict(TRAP_HTML, TRAP_HTML)
     if ok:     if ok:
-        print("SELFTEST FAIL: prose 'log out' must not count as a marker:", why)+        print("SELFTEST FAIL: prose 'log out' must not count as a marker")
         failed += 1         failed += 1
     else:     else:
         print("SELFTEST ok   prose 'log out' / sessionStorage is not a marker")         print("SELFTEST ok   prose 'log out' / sessionStorage is not a marker")
  
-    # Equality, not substring: a logged-in page whose only new token is +    # Substring traps the first regex version accepted. Both snapshots keep the 
-    # "sessionid" must not be claimed by looking for the string "session"+    # login form; the candidate only adds the tempting attribute
-    session_in = '<a href="/x" id="sessionid-link">account</a>' +    for name, extra in ( 
-    session_out = '<a href="/x" id="login-link">Log in</a>' +        ("accounting", '<id="accounting" href="/x">ledger</a>'), 
-    in_m = markers(session_in) +        ("gravatar", '<img data-testid="gravatar" src="g.png">'), 
-    if any("session" == tok for toks in in_m.values() for tok in toks):+        ("logout-policy", '<a href="/logout-policy">policy</a>'), 
 +    ): 
 +        candidate = LOGGED_OUT_HTML.replace("</body>", extra + "</body>"
 +        ok, why = verdict(LOGGED_OUT_HTML, candidate) 
 +        if ok: 
 +            print(f"SELFTEST FAIL: {name} must not verify:", why) 
 +            failed += 1 
 +        else: 
 +            print(f"SELFTEST ok   {name} is not a login marker"
 + 
 +    # Equality, not substring: a segment "sessionid-link" is not the token "session". 
 +    session_html = '<a href="/x" id="sessionid-link">account</a>' 
 +    if "session" in hits(session_html, ("session",)):
         print("SELFTEST FAIL: token equality collapsed to substring 'session'")         print("SELFTEST FAIL: token equality collapsed to substring 'session'")
         failed += 1         failed += 1
     else:     else:
         print("SELFTEST ok   'sessionid-link' is not the token 'session'")         print("SELFTEST ok   'sessionid-link' is not the token 'session'")
 +
 +    aria = LOGGED_OUT_HTML.replace(
 +        "</body>", '<a aria-label="Log out" href="/x">leave</a></body>'
 +    )
 +    ok, why = verdict(LOGGED_OUT_HTML, aria)
 +    if not ok:
 +        print("SELFTEST FAIL: aria-label='Log out' must verify:", why)
 +        failed += 1
 +    else:
 +        print("SELFTEST ok   aria-label='Log out' is a logout marker")
  
     if failed:     if failed:
Line 265: Line 302:
 </file> </file>
  
-Documented run, 2026-08-27, four self-tests then the fixtures:+Documented run, 2026-08-27, eight self-tests then the fixtures (path-segment equality, not substring; ''aria-label="Log out"'' counts):
  
 <code> <code>
 SELFTEST ok   documented pair verifies: SELFTEST ok   documented pair verifies:
-          logout marker present only when logged in: /logout, logout-link +          logout marker present only when logged in: logout, logout-link 
-          user marker present only when logged in: logged-in-badge, user-menu +          user marker present only when logged in: account, logged-in, user-menu 
-          login area present logged-out and absent logged-in: /login, login-form, login-link, password+          login area present logged-out and absent logged-in: login, login-link, password
 SELFTEST ok   two logged-out snapshots do not verify SELFTEST ok   two logged-out snapshots do not verify
 SELFTEST ok   prose 'log out' / sessionStorage is not a marker SELFTEST ok   prose 'log out' / sessionStorage is not a marker
 +SELFTEST ok   accounting is not a login marker
 +SELFTEST ok   gravatar is not a login marker
 +SELFTEST ok   logout-policy is not a login marker
 SELFTEST ok   'sessionid-link' is not the token 'session' SELFTEST ok   'sessionid-link' is not the token 'session'
 +SELFTEST ok   aria-label='Log out' is a logout marker
  
 Documented run (embedded fixtures): Documented run (embedded fixtures):
 VERIFIED login VERIFIED login
-   logout marker present only when logged in: /logout, logout-link +   logout marker present only when logged in: logout, logout-link 
-   user marker present only when logged in: logged-in-badge, user-menu +   user marker present only when logged in: account, logged-in, user-menu 
-   login area present logged-out and absent logged-in: /login, login-form, login-link, password+   login area present logged-out and absent logged-in: login, login-link, password
 </code> </code>
  
Line 299: Line 340:
  
 ^ Paper ^ What was created ^ Of what ^ Rate ^ ^ Paper ^ What was created ^ Of what ^ Rate ^
-| Drakonakis et al., CCS 2020 {[drakonakis2020_cookie]} | 25,242 accounts | 168,594 domains with a signup option | **13.7%** | +| Drakonakis et al. registered and logged in {[drakonakis2020_cookie]} | count not given | 168,594 domains with a signup option | **13.7%** | 
-| Drakonakis et al., CCS 2020, of the full crawl {[drakonakis2020_cookie]} | 25,242 accounts | 1,585,964 unique domains crawled | **~1.6%** |+| Drakonakis et al. accounts created {[drakonakis2020_cookie]} | 25,242 accounts | 168,594 signup domains | they write "almost 12%"; 25,242/168,594 is **15.0%** | 
 +| Drakonakis et al., of the full crawl {[drakonakis2020_cookie]} | 25,242 accounts | 1,585,964 unique domains crawled | **~1.6%** |
 | Al Roomi et al., USENIX Security 2023 {[alroomi2023_login]} | 45.0K initial test accounts | Google CrUX top 1M | **4.5%** | | Al Roomi et al., USENIX Security 2023 {[alroomi2023_login]} | 45.0K initial test accounts | Google CrUX top 1M | **4.5%** |
 | Alroomi et al., CCS 2023 {[alroomi2023_password]} | 20,119 domains fully evaluated | Tranco 1M after filtering | a completed-evaluation count, not a success rate | | Alroomi et al., CCS 2023 {[alroomi2023_password]} | 20,119 domains fully evaluated | Tranco 1M after filtering | a completed-evaluation count, not a success rate |
 | Kubicek et al., TheWebConf 2024 {[kubicek2024_register]} | register-or-newsletter | 660,202 unique Tranco domains | **5.9%** | | Kubicek et al., TheWebConf 2024 {[kubicek2024_register]} | register-or-newsletter | 660,202 unique Tranco domains | **5.9%** |
  
-Kubicek et al. compare themselves to Cookie Hunter's **1.6% of Alexa 1M**, not to 13.7% of signup domains. Use the same comparison they used.+Kubicek et al. compare themselves to Cookie Hunter's **1.6% of Alexa 1M** (the 25,242 count over the crawl), not to 13.7% of signup domains. Use the same comparison they used. Do not write "25,242 (13.7%)": 13.7% is registered-and-logged-in of signup domains; 25,242 is a separate "accounts created" count.
  
-**Cookie Hunter.** XDriver on Selenium. Email verification by visiting links in mail. SMS / SSN blocked leftover of the top 1K, completed by hand; the ~25K that make the evaluation "did not require any manual intervention". reCAPTCHA on 19,491 (~13.8%) of signup domains. They tried an audio-reCAPTCHA solver; Google was not serving captchas to WebDriver. They **did not** pay a farm:+**Cookie Hunter.** XDriver on Selenium. Email verification by visiting links in mail. SMS / SSN blocked leftover of the top 1K, completed by hand; the ~25K that make the evaluation "did not require any manual intervention". reCAPTCHA on 19,491 (~13.8%) of signup domains. They tried an audio-reCAPTCHA solver; Google was not serving captchas to WebDriver. They **did not** pay a farm {[drakonakis2020_cookie]}:
  
 > funding human captcha-solving services to create accounts presents an ethical dilemma, we opted to not handle such cases. > funding human captcha-solving services to create accounts presents an ethical dilemma, we opted to not handle such cases.
Line 317: Line 359:
 **Tripwire** {[deblasio2017_tripwire]}. Honey accounts to infer site compromise from password reuse. **65,413** registration attempts across **33,634** sites; **3,664** accounts on around **2,302** sites. Third-party CAPTCHA solver (DeCaptcher). "While we make no attempt to explicitly check the terms of service". **Tripwire** {[deblasio2017_tripwire]}. Honey accounts to infer site compromise from password reuse. **65,413** registration attempts across **33,634** sites; **3,664** accounts on around **2,302** sites. Third-party CAPTCHA solver (DeCaptcher). "While we make no attempt to explicitly check the terms of service".
  
-**Kubicek et al.** {[kubicek2024_register]}, author PDF. Tranco list ''82Q3V'', June 2022. A processing error sampled one million domains **with replacement**, so the results are **660,202 unique** domains. Loaded 504,509; form on **25.7% (169,765)**; form-submit estimated success **30.2% (51,290)**, fail 38.4%, unknown 31.4%. Headline **5.9%** register-or-newsletter. CrUX overlap 51.9%; load 90.6% vs 65.3% non-CrUX; "successful registration" 11.7% CrUX vs 3.9% non-CrUX. Selenium + Chrome, 60 parallel Docker containers, 12 German Research Network IPs, unique email per site at ''sybilmail.de''. CAPTCHA on **one-third**; of those, **75% reCAPTCHA v2, 20% v3, 2% hCaptcha, 3% image**. They used a **human CAPTCHA farm**, discussed it with their legal department, and later switched to research assistants. The crawler is **not public** (bomb-attack risk); access is by request at ''https://karelkubicek.github.io/post/reg-www''. 37.2% (12,605) marketing without proper consent; 1.8% email shared with undeclared third parties; 59.8% fail double opt-in. They do not do SSO registration.+**Kubicek et al.** {[kubicek2024_register]}, author PDF. Tranco list ''82Q3V'', June 2022. A processing error sampled one million domains **with replacement**, so the results are **660,202 unique** domains. Loaded 504,509 of those. Form on **25.7% (169,765)** of the **660,202** unique domains — the PDF writes this "among the loaded websites"169,765 of 504,509 loaded is 33.7%, so the printed 25.7% is the unique-domain rate. Form-submit estimated success **30.2% (51,290)**, fail 38.4%, unknown 31.4%. Headline **5.9%** register-or-newsletter. CrUX overlap 51.9%; load 90.6% vs 65.3% non-CrUX; "successful registration" 11.7% CrUX vs 3.9% non-CrUX. Selenium + Chrome, 60 parallel Docker containers, 12 German Research Network IPs, unique email per site at ''sybilmail.de''. CAPTCHA on **one-third**; of those, **75% reCAPTCHA v2, 20% v3, 2% hCaptcha, 3% image**. They used a **human CAPTCHA farm**, discussed it with their legal department, and later switched to research assistants. The crawler is **not public** (bomb-attack risk); access is by request at ''https://karelkubicek.github.io/post/reg-www''. 37.2% (12,605) marketing without proper consent; 1.8% email shared with undeclared third parties; 59.8% fail double opt-in. They do not do SSO registration.
  
 ===== Newsletter ===== ===== Newsletter =====
Line 347: Line 389:
 | 2020 (Cookie Hunter) | no farm; WebDriver already made Google not serve reCAPTCHA | The ethical refusal is still the argument to cite. The detection of WebDriver is not the 2026 bottleneck | | 2020 (Cookie Hunter) | no farm; WebDriver already made Google not serve reCAPTCHA | The ethical refusal is still the argument to cite. The detection of WebDriver is not the 2026 bottleneck |
 | 2022 crawl (Kubicek et al., published 2024) | human farm; mix **75 / 20 / 2 / 3** v2 / v3 / hCaptcha / image; later, research assistants | Dated snapshot of **what was on the web in 2022**. Not a 2026 vendor share | | 2022 crawl (Kubicek et al., published 2024) | human farm; mix **75 / 20 / 2 / 3** v2 / v3 / hCaptcha / image; later, research assistants | Dated snapshot of **what was on the web in 2022**. Not a 2026 vendor share |
-| 2023 (Alroomi / Al Roomi) | AZcaptcha, advertised as OCR not humans; 94% solve rate on the password-policy crawl | Current-enough //automated// farm-API. They avoided human solvers on purpose |+| 2023 (Alroomi / Al Roomi) | AZcaptcha, described in those papers as OCR not humans; 94% solve rate on the password-policy crawl | **Paper-era** automated farm-API. They avoided human solvers on purpose. Live AZcaptcha docs (fetched 2026-08-27) mention a workers pool — do not treat the 2023 OCR claim as a 2026 service description |
 | 2023 (Searles et al.) {[searles2023_captcha]} | **manual** user study, 1,400 people, 14,000 CAPTCHAs; 185 of ~200 Alexa sites had account creation, **142** succeeded | Not a crawler. Useful for "will a human complete signup" | | 2023 (Searles et al.) {[searles2023_captcha]} | **manual** user study, 1,400 people, 14,000 CAPTCHAs; 185 of ~200 Alexa sites had account creation, **142** succeeded | Not a crawler. Useful for "will a human complete signup" |
 | 2025 (Teoh et al., Halligan) {[teoh2025_captcha]} | agentic VLM **60.7%** of 2,600 challenges; infiltrated **2Captcha** at **70.6%** | **Current.** Farms still exist in 2025. VLMs change the "unsolvable CAPTCHA" story; they do not end it | | 2025 (Teoh et al., Halligan) {[teoh2025_captcha]} | agentic VLM **60.7%** of 2,600 challenges; infiltrated **2Captcha** at **70.6%** | **Current.** Farms still exist in 2025. VLMs change the "unsolvable CAPTCHA" story; they do not end it |
 | 2026 (Turnstile) | Cloudflare's CAPTCHA-free widget: Managed / Non-interactive / Invisible; hosted at ''challenges.cloudflare.com''; WCAG 2.2 AA | **Current, and almost absent from this corpus.** Docs last updated 2026-08-14 (overview) and 2026-04-16 (widget types), fetched 2026-08-27. A 2022 crawl's 75% v2 mix will not describe a 2026 sample | | 2026 (Turnstile) | Cloudflare's CAPTCHA-free widget: Managed / Non-interactive / Invisible; hosted at ''challenges.cloudflare.com''; WCAG 2.2 AA | **Current, and almost absent from this corpus.** Docs last updated 2026-08-14 (overview) and 2026-04-16 (widget types), fetched 2026-08-27. A 2022 crawl's 75% v2 mix will not describe a 2026 sample |
  
-Do not take a share from Similarweb, wmtips, or any SEO page. The 75/20/2/3 mix is Kubicek et al.'s 2022 crawl. Google's reCAPTCHA documentation still serves (HTTP 200 on 2026-08-27); so does hCaptcha's. Turnstile is the one that changed the "show a puzzle" default.+Do not take a share from Similarweb, wmtips, or any SEO page. The 75/20/2/3 mix is Kubicek et al.'s 2022 crawl. Google's reCAPTCHA landing page (''https://developers.google.com/recaptcha'') still returns HTTP 200 (2026-08-27) but is **deprecated**the replacement is Google Cloud Fraud Defense (''https://cloud.google.com/recaptcha/docs/overview'', HTTP 200: "reCAPTCHA has become a part of Google Cloud Fraud Defense"). hCaptcha'docs also serve. Turnstile is the one that changed the "show a puzzle" default.
  
 Email verification is the common path (login-policies ground-truth 39%; Cookie Hunter visited the link). SMS is the expensive path: SAAT used Twilio; Cookie Hunter treated phone/SSN as a leftover for the top 1K; login-policies did not do phone. Plan for email. Budget SMS only if the research question lives behind it. Email verification is the common path (login-policies ground-truth 39%; Cookie Hunter visited the link). SMS is the expensive path: SAAT used Twilio; Cookie Hunter treated phone/SSN as a leftover for the top 1K; login-policies did not do phone. Plan for email. Budget SMS only if the research question lives behind it.
  
-Human CAPTCHA farms: **name who used them, and when**. Cookie Hunter no; Kubicek et al. yes then research assistants; Tripwire yes; password-policies AZcaptcha, not human, on purpose; Teoh et al. infiltrated 2Captcha in 2025 as an experiment, not as a measurement instrument they recommend. Do not present a farm as current best practice.+Human CAPTCHA farms: **name who used them, and when**. Cookie Hunter no; Kubicek et al. yes then research assistants; Tripwire used DeCaptcher (a third-party solver; the paper does not establish that the solvers were human); password-policies AZcaptcha, described there as not human, on purpose; Teoh et al. infiltrated 2Captcha in 2025 as an experiment, not as a measurement instrument they recommend. Do not present a farm as current best practice.
  
 ===== Terms of service and ethics ===== ===== Terms of service and ethics =====
Line 370: Line 412:
   * Kubicek et al. {[kubicek2024_register]}: legal review, farm conditions discussed with the university legal department, crawler withheld because it is a bomb-attack tool.   * Kubicek et al. {[kubicek2024_register]}: legal review, farm conditions discussed with the university legal department, crawler withheld because it is a bomb-attack tool.
  
-There is no consensus in this literature that creating test accounts is permitted, forbidden, or ToS-exempt. There is a consensus that **almost nobody reads the ToS of 45.0K sites**. Write the sentence: synthetic identities, unique emails, whether you solved CAPTCHAs and how, whether you completed email/SMS, and that you did not check ToS at scale (if you did not). [[Practices:Ethics]] is the page for robots.txt and for the network acceptable-use policy that will actually stop you.+There is no consensus in this literature that creating test accounts is permitted, forbidden, or ToS-exempt. The papers that created accounts at this scale wrote that they did **not** check ToS (Tripwire, password-policies) or named the tension and did it anyway (SAAT). That is five papers, not a vote of 45.0K. Write the sentence: synthetic identities, unique emails, whether you solved CAPTCHAs and how, whether you completed email/SMS, and that you did not check ToS at scale (if you did not). [[Practices:Ethics]] is the page for robots.txt and for the network acceptable-use policy that will actually stop you.
  
 ===== Which methods are current ===== ===== Which methods are current =====
  
 ^ Method ^ Status ^ ^ Method ^ Status ^
-| Logged-out crawl of a ranked list | **Still the default, and still defensible** for questions that do not live behind an account. 553/857 said they did this |+| Logged-out crawl of a ranked list | **Still the default, and still defensible** for questions that do not live behind an account. 553/857 are labelled ''none'' |
 | Manual accounts, then scripted login (Kaizer 2016, To Auth 2024) | **Current** when the list is hundreds, not hundreds of thousands | | Manual accounts, then scripted login (Kaizer 2016, To Auth 2024) | **Current** when the list is hundreds, not hundreds of thousands |
 | Shepherd-style differential verification | **Current, and underused.** The 2020 workshop paper is out of corpus; the method is not obsolete | | Shepherd-style differential verification | **Current, and underused.** The 2020 workshop paper is out of corpus; the method is not obsolete |
-| Cookie Hunter-style full registration on a million-site list | **Done, expensive, and CAPTCHA-censored.** The 13.7% is of signup domains in 2020, under a no-farm constraint | +| Cookie Hunter-style full registration on a million-site list | **Done, expensive, and CAPTCHA-censored.** The 13.7% is registered-and-logged-in of signup domains in 2020, under a no-farm constraint; the 25,242 is a separate accounts-created count 
-| Human CAPTCHA farm as a measurement instrument | **Used (Kubicek 2022 crawl; Tripwire 2015). Not best practice.** Password-policies 2023 and Cookie Hunter 2020 both refused it. Teoh 2025 shows the farm is still there to infiltrate | +| Human CAPTCHA farm as a measurement instrument | **Used (Kubicek 2022 crawl). Not best practice.** Tripwire 2015 used a third-party solver, not established as a human farm. Password-policies 2023 and Cookie Hunter 2020 both refused a human farm. Teoh 2025 shows the farm is still there to infiltrate | 
-| AZcaptcha / automated solver APIs | **Current practice** in the 2023 policy crawls |+| AZcaptcha / automated solver APIs | **Used in the 2023 policy crawls.** Date the OCR/not-humans claim to those papers; live AZcaptcha docs (fetched 2026-08-27) mention a workers pool |
 | Agentic VLM solving (Halligan 2025) | **New.** 60.7% is a paper about whether the technique works, which is the stage it is at | | Agentic VLM solving (Halligan 2025) | **New.** 60.7% is a paper about whether the technique works, which is the stage it is at |
 | Turnstile / CAPTCHA-free widgets | **Current on the web, almost unmeasured in this corpus.** Date any CAPTCHA-mix figure | | Turnstile / CAPTCHA-free widgets | **Current on the web, almost unmeasured in this corpus.** Date any CAPTCHA-mix figure |
Line 403: Line 445:
 Corpus: 5,859 extracted papers, 2010–2026, CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. Population for every share on this page is the 857 web crawls unless a sentence names another. 2025–2026 venue-years are incomplete by construction; tables that include them are starred. Corpus: 5,859 extracted papers, 2010–2026, CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. Population for every share on this page is the 857 web crawls unless a sentence names another. 2025–2026 venue-years are incomplete by construction; tables that include them are starred.
  
-The 81 is a schema enum; the 17 is a hand map (''scripts/registration_fold.mjs'') checked both ways at run time by ''scripts/report_registration.mjs''. A missing ROLE or a stale ROLE key prints ''FAILURE'' and exits 1. Loose full-text sweeps for "register" and "shepherd" are homograph-dominated and were not used as counts.+The 81 is a schema enum; the 17 is a hand map over those 81 (''scripts/registration_fold.mjs'') checked both ways at run time by ''scripts/report_registration.mjs''. It is not a full-text census of the 776 papers the schema did not mark as logged in. A missing ROLE or a stale ROLE key prints ''FAILURE'' and exits 1. Loose full-text sweeps for "register" and "shepherd" are homograph-dominated and were not used as counts.
  
 Out of corpus, and labelled so wherever they appear: Shepherd (MADWeb 2020), Chatzimpyrros et al. (ESORICS workshop), Mathur et al. (Big Data & Society). In the bibliographic index but not extracted: Kubicek et al. WWW 2024. Out of corpus, and labelled so wherever they appear: Shepherd (MADWeb 2020), Chatzimpyrros et al. (ESORICS workshop), Mathur et al. (Big Data & Society). In the bibliographic index but not extracted: Kubicek et al. WWW 2024.
programming/registration.txt · Last modified: by karel.kubicek.claude

Except where otherwise noted, content on this wiki is licensed under the following license: CC BY-NC-SA 4.0
CC BY-NC-SA 4.0 Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki