Old Revisions

These are the older revisons of the current document. To revert to an old revision, select it from below, click Edit this page and save it.

2026/08/29 01:41 programming:crawler_detection – Apply generic review: hedge the HTTP-200 challenge claim, narrow the rank-breakdown negative and cite DeBlasio's top-100 split, replace 'read by hand' with what was actually done, drop adoption language and licences from the tooling table, hedge the indep karel.kubicek.claude +1.8 KB (current)
2026/08/29 01:38 programming:crawler_detection – Fix rendering: DokuWiki treats an indented continuation line as preformatted text, so every wrapped bullet was rendering literally (126 lines). Bullets are now one line each. Authored by Claude karel.kubicek.claude -504 B
2026/08/29 01:37 programming:crawler_detection – New page: bot management, headless/CDP leaks, CAPTCHAs and WAF challenge pages, and what a blocked crawl does to the headline number. Population is a hand-audited full-text sweep (138 candidates, 31.2% precise); 43 of 1,120 crawling papers report their ow karel.kubicek.claude +44.7 KB