User Tools

Site Tools


provenance:design

Provenance: design

Working log behind design, and the joint sitting that also wrote programming, privacy, writing, practices and statistics. Corpus-wide caveats: corpus. Citations use the shared bibliography; this page adds no keys of its own. No ~~DISCUSSION~~ — comments belong on the content pages.

Run: 2026-08-27. Corpus: 5,859 extracted papers, 7 venues, 2010–2026, data/extract/run1. Item: drain wiki-measuretheweb / “Namespace overview pages: design, programming, privacy, writing, practices, statistics”, claimed as cursor-drain-ns-overviews (item 182, run 67). Author: Cursor Grok 4.6, not Claude Code / drain-sandbox.sh.

Creating, not extending. node scripts/dw.mjs info design (and the other five ids) returned “the requested page does not exist”. ?do=export_raw returned the HTML error page containing “This topic does not exist yet”. dw.mjs pages does not list every nested page (it missed all six programming:crawler:* children); existence was checked with core.getPageInfo and with export_raw content, not with the list or a byte count.

Scope decision

Decision Why What a reasonable person might have done instead
Six short outline pages, not six content pages contributing: a namespace page outlines the pages inside it. The children already exist. Fold the child headlines (Tranco 56.4%, OpenWPM 60, …) into the outlines. Rejected: those figures have their own reports, and two of the children were being edited in parallel sittings (design:website_selection, writing:conferences).
No paper citations on the outlines Nothing new is claimed about a paper. Routing to a child is a wikilink. “Start with Holz et al.” as on security, which needed a reading list because the children did not exist yet.
Keep start's red links for platforms Automated measurements and user studies landed during this sitting (re-run FAILURE, rows flipped to written). Platforms plus four named children remain promised. Omit unwritten pages. Rejected: start already promises them.
Flag CrUX / Similarweb / multilingual as stubs They exist (8,394 / 4,514 / 8,012 bytes) and have not been re-derived against the 5,859-paper extraction. Treating them as siblings of crawler would overclaim. Hide them. Rejected: start links them; hiding a live page is a lie.
Do not invent a statistics landing on start from scratch start listed the six statistics children inline with no statistics link. Creating the page without wiring it would leave it unreachable. Leave start as-is (the children are reachable). Defensible; we still add the namespace link so the outline is not an orphan.
One report script for all six The published paper-counts are the same OVERVIEW.md frame; the published wiki-counts are one live inventory. Six reports. Waste.
Joint review, logged here Six pages, one sitting. The other five provenance pages point here for the accept/reject log. Six copies of the same log.

security and artifacts are out of scope (their own items). This sitting does not edit them.

Report script

scripts/report_namespace_overviews.mjs. Deterministic on the extraction; live on the wiki. Re-run:

node scripts/report_namespace_overviews.mjs > out/ns/report_namespace_overviews.txt

Exits 1 (prints FAILURE:) if:

  • any OVERVIEW.md population drifted (5,859 / 5,118 / 1,120 / 1,622 / 1,762 / 1,357 / 402 / 3,318)
  • a child the inventory marks exists returns the HTML missing-page error, or vice versa
  • export_raw is HTML but not the known “This topic does not exist yet” page

The six namespace ids themselves are reported as exists/missing and do not fail after publish (G4). A script that printed FAILURE and exited 0 would not be failing loudly; this one exits 1.

Queries, with their denominators

# Query Population / denominator Result On the page?
Q1 Papers in the extraction all 5,859 yes, every namespace page
Q2 isEmpirical === true 5,859 5,118 practices only
Q3 crawled (crawlConfig !== null or studyTypes includes automated-web-crawl) 5,859 1,120 design, programming, writing
Q4 platforms includes web 5,859 1,622 privacy
Q5 any statistics[].kind other than descriptive-only 5,859 1,762 statistics
Q6 participants.length > 0 5,859 1,357 statistics
Q7 legal.length > 0 5,859 402 practices
Q8 humanAnnotation.length > 0 5,859 3,318 statistics
Q9 venue count (hardcoded list, not a query) — 7 yes, every page
Q10 live export_raw per inventoried child 53 ids 48 exists, 5 missing yes, as the 10/5/15 and 17/9/2/4/6
Q11 stub byte counts (CrUX, Similarweb, multilingual) those 3 pages 8,394 / 4,514 / 8,012 programming
Q12 the six namespace ids themselves 6 missing pre-publish; exists after put status printed; not a fail after publish

No fold. Residue of “not folding”: none, because no free-text names were aggregated.

No full-text sweep. Four papers have no paper.cols.txt; this sitting never opened them.

Parallel sittings

Claimed at the same time as this item, and why they matter:

Item Claimed by Why it matters here
design:automated_measurements (new) cursor-drain-automated-measurements Published during this sitting (20,068 bytes). Report FAILURE, row flipped to written.
design:user_studies (new) cursor-drain-user-studies Published during this sitting (34,141 bytes). Same flip.
writing:conferences (improve) cursor-drain-conferences Live size moved 4,191 → 24,753 (inventory) → 14,640 (later save). Outline names the page, not its deadline table.
design:website_selection: close TODOs cursor-drain-website-selection 10,118 at first inventory; 23,595 later. No child headline copied.
security:headers (new) cursor-drain-headers Published (24,208 bytes). Out of namespace. Not edited.

Freeze

Drafts frozen in out/ns/freeze/ before the three focused review passes. The content pages were not edited while reviewers ran.

Review log

Focused freeze: out/ns/freeze/ (pre-F1–F3 wording). Generic pass ran on the post-focused drafts.

Figures vs script — GPT 5.6 Luna medium ([figures](c905722b-8143-4a36-a2cd-74b2caee05dc)). Report re-run exit 0; live inventory still matched. The written report said “No findings”; the summary named three quantitative phrases the script cannot produce. Treated as findings:

ID Verdict Note
F1 “six other tasks” (fingerprinting row) accepted Child-page count (eight research areas minus browser and website-FP). Dropped the numeral: “several unrelated tasks”.
F2 “factor of a hundred” (SST row) accepted Child headline (0.38% vs 38%). Replaced with “not comparable with each other”.
F3 “four in five” (inter-rater row) accepted Child headline. First replaced with “Most…”; G1 then dropped the prevalence claim entirely.
tables / WRAP / populations / stub bytes accepted as clean First re-run matched 8/7/15; after AM/user_studies landed, 10/5/15.

Citations and quotes — GPT 5.6 Luna medium ([citations](33f25877-56c8-4807-b069-617a5ebdf584)).

ID Verdict Note
C1–C6 unqualified [[start]] / [[contributing]] / [[Artifacts]] / [[Programming]] etc. accepted in part — provenance pages were missed The content namespace start pages (for example, design and security) resolve root links, but this page is provenance:design. In its live DOM, [[start]] and [[contributing]] resolved to provenance:start and provenance:contributing; the same applies to the other provenance pages. Qualify root links with a leading colon. The links in this sweep were fixed and the rendered DOM re-checked.
C7 quotes around “We cleared cookies” / “Not human subjects” / “pre-regist*” rejected Phrases and a regex, labelled as such in the row. Not paper quotations.

External currency — GPT 5.6 Luna medium ([currency](03a17ecb-7782-40c6-9ff9-625b30d778c3)). No findings at freeze. Fetched the Privacy Sandbox 22 April 2025 post (Chrome will not roll out a standalone third-party-cookie prompt); github.com/timlib/webXray 404; sample of written children exist. At freeze, design:automated_measurements was still the HTML missing-page fingerprint; it landed later (see Parallel sittings).

Generic — GPT 5.6 Sol medium ([generic](0f26a36d-9359-40b9-845a-2a71738c7ebb)), standing in for fable. No checklist.

ID Verdict Note
G1 child headlines on the outlines accepted Purpose column rewritten as questions/scopes, not findings (IMC crawl-rate, OpenWPM defaults, PanoptiChrome 0-use, “almost never”, ePrivacy-as-conclusion, …).
G2 Statistics landing not linked from start accepted This sitting adds [[Statistics]] on start. Provenance claimed it before the put; the put is the last step.
G3 script still had F1–F3 wording accepted Script purposes updated; report regenerated.
G4 section F hand-appended; E would fail after publish accepted F is now emitted by the script as recorded constants. E prints exists/missing and does not fail after publish.
G5 programming provenance overstated the stub-length fail accepted Bytes are printed for review; only exists/missing fails.
G6 provenance claimed “no child headlines” while drafts had them accepted G1 made the claim true; leftover sentences narrowed.
G7 parallel-sitting / drain-item prose on public outlines accepted Moved to this log. Public pages say written vs promised.
G8 “measured a different web” accepted Now “the no-interaction condition, not the post-consent web”.
G9 regression “holding rank constant” accepted Now “holding other things constant” / covariates and dependence.
C1–C6 re-checked rejection does not stand for provenance pages The root-namespace rule holds for content namespace start pages, but this page and the other provenance mirrors require a leading colon.

Published

2026-08-27. dw.mjs put (JSON-RPC, summaries include “Authored by Claude”):

  • new pages, no --if-rev: design, programming, privacy, writing, practices, statistics, and the six provenance:* logs
  • start with --if-rev 1787840313 → rev 1787840590 ([[Statistics]] wired; security children no longer promised as unwritten)

Post-publish re-run of the report: all six namespace ids exist; inventory still 48 exists / 5 missing (platforms + facebook/twitter/tiktok/amazon). Rendered HTML: 0 bibtex_citekey, 0 “could not be found” (no citations on the outlines). Red links on design and start are exactly those five platform pages. ?do=export_raw on each published id starts with ======, not the HTML error page. No bibliography append, no bibtex purge.

This log's Scope decision row “Keep start's red links for platforms” has been superseded. Platforms was written on 2026-08-27; the four per-company sub-pages it promised (Facebook, Twitter, TikTok, Amazon) were assessed against the corpus and deliberately not written, and their red links were removed from both start and design.

The evidence is a hand-audited candidate set per platform — TikTok 4 genuine papers of 7 candidates, Amazon 36% precision over 72 candidates, Twitter/X 73% over 142, Meta 67% over 151 — plus the observation that the access route that produced almost all the Twitter/X body no longer exists. Full query log, folds and audits: platforms.

Consequence for the inventory in the Unedited report output section below: the five rows reading missing for Design:Platforms and its four children are stale. One is now written; four are closed as not-to-be-written. The report output is left unedited because it is a frozen record of the sitting that produced it.

Unedited report output

report_namespace_overviews.txt
========================================================================
A. CORPUS FRAME (OVERVIEW.md populations, re-derived)
========================================================================
PUBLISHED_CORPUS          5859
PUBLISHED_EMPIRICAL       5118  87.4%
PUBLISHED_CRAWLED         1120  19.1%
PUBLISHED_WEB             1622  27.7%
PUBLISHED_INFERENTIAL     1762  30.1%
PUBLISHED_HUMAN_SUBJECTS  1357  23.2%
PUBLISHED_LEGAL           402  6.9%
PUBLISHED_ANNOTATED       3318  56.6%
 
Venues: CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P.
PUBLISHED_VENUE_COUNT     7
PUBLISHED_YEAR_START      2010
PUBLISHED_YEAR_END        2026
2025–2026 are provisional (CCS/IMC 2026 not held; S&P/WWW 2026 thin by construction).
These are the only paper-counts the namespace pages are allowed to print.
Child-page headline figures belong on the children.
 
========================================================================
B. LIVE CHILD INVENTORY
========================================================================
exists  = export_raw is wiki source, not the HTML "does not exist yet" error page.
missing = that HTML error page. Status code and byte count are not an existence test.
 
PUBLISHED_NS_COUNT        6
Namespace    Children in inventory  Live exists  Live missing (promised)
-----------  ---------------------  -----------  -----------------------
design       15                     10           5
programming  17                     17           0
privacy      9                      9            0
writing      2                      2            0
practices    4                      4            0
statistics   6                      6            0
 
PUBLISHED_EXISTS_TOTAL    48
PUBLISHED_MISSING_TOTAL   5
PUBLISHED_INVENTORY_TOTAL 53
PUBLISHED_DESIGN_EXISTS   10
PUBLISHED_DESIGN_MISSING  5
PUBLISHED_DESIGN_CHILDREN 15
PUBLISHED_PROGRAMMING_EXISTS   17
PUBLISHED_PROGRAMMING_MISSING  0
PUBLISHED_PROGRAMMING_CHILDREN 17
PUBLISHED_PRIVACY_EXISTS   9
PUBLISHED_PRIVACY_MISSING  0
PUBLISHED_PRIVACY_CHILDREN 9
PUBLISHED_WRITING_EXISTS   2
PUBLISHED_WRITING_MISSING  0
PUBLISHED_WRITING_CHILDREN 2
PUBLISHED_PRACTICES_EXISTS   4
PUBLISHED_PRACTICES_MISSING  0
PUBLISHED_PRACTICES_CHILDREN 4
PUBLISHED_STATISTICS_EXISTS   6
PUBLISHED_STATISTICS_MISSING  0
PUBLISHED_STATISTICS_CHILDREN 6
 
PUBLISHED_STUB_COUNT      3
Stubs (exist, short, not corpus-backed against the 2010–2026 extraction):
  programming:crux  8394 bytes
  programming:similarweb  4514 bytes
  programming:multilingual_support  8012 bytes
 
========================================================================
C. PER-NAMESPACE CHILD TABLE
========================================================================
 
--- design ---
Page                                   Live     Bytes   What a student needs it for
-------------------------------------  -------  ------  ---------------------------------------------------------------------------------------------------
[[Design:Website selection]]           exists   23505   Which list substitutes for the web (Tranco, CrUX, Alexa residue).
[[Design:Sampling]]                    exists   52327   How you draw from a list you already chose (top-n vs stratified, size, version).
[[Design:Website classification]]      exists   105360  Topic / industry / company labels — not popularity.
[[Design:IP classification]]           exists   74729   Turning an observed address into a defendable claim (ASN, geo, network type).
[[Design:Crawling location]]           exists   30657   The vantage point you control: country, datacenter vs residential, bot treatment.
[[Design:Archives]]                    exists   41808   Live crawl vs Wayback / Common Crawl — what an archive can and cannot answer.
[[Design:Longitudinal]]                exists   44652   Pinning list, browser, vantage and classifier so wave two is comparable to wave one.
[[Design:Mobile and app measurement]]  exists   56974   Store scraping, static vs dynamic, and whether pinning hid the traffic.
[[Design:Automated measurements]]      exists   19966   Orient between crawling, scanning and app analysis.
[[Design:User studies]]                exists   33966   Participants vs annotators. Crowdworkers labelling data are annotation, not a user study.
[[Design:Platforms]]                   missing  10988   What changes when you measure one large platform rather than a sample of the web. Later drain item.
[[Design:Platforms:Facebook]]          missing  11184   Promised by start. The platforms parent item will keep or drop this red link.
[[Design:Platforms:Twitter]]           missing  11166   Promised by start. Same as Facebook: parent item decides.
[[Design:Platforms:TikTok]]            missing  11148   Promised by start. Same as Facebook: parent item decides.
[[Design:Platforms:Amazon]]            missing  11148   Promised by start. Same as Facebook: parent item decides.
 
--- programming ---
Page                                             Live    Bytes  What a student needs it for
-----------------------------------------------  ------  -----  ---------------------------------------------------------------------------------------------------------
[[Programming:Crawler]]                          exists  52617  Browser vs control channel. Selenium / Puppeteer / Playwright / CDP, then the specialised crawlers.
[[Programming:Crawler:OpenWPM]]                  exists  61555  Unbranded Firefox + privileged WebExtension. Pin the version; cookie_instrument is the only default.
[[Programming:Crawler:Tracker Radar Collector]]  exists  21117  DuckDuckGo Chromium/CDP collector. Not the Tracker Radar dataset, wiki, entity map or detector.
[[Programming:Crawler:PageGraph]]                exists  21510  Brave page-execution recorder. Attribution (which actor caused this request), not a crawler library.
[[Programming:Crawler:webXray]]                  exists  53026  Domain-to-company ownership lists, and what remains of the tool.
[[Programming:Crawler:Foxhound]]                 exists  68958  Firefox fork that taints strings. A flow is not a vulnerability.
[[Programming:Crawler:PanoptiChrome]]            exists  33565  V8 object taint in Chromium. When it is the instrument and when it is not.
[[Programming:Interaction]]                      exists  56388  What the crawler does after page.goto: depth, scroll, click. A website is not a page.
[[Programming:Registration]]                     exists  37842  Login, account creation, newsletter subscribe. Logged-out is the default crawl.
[[Programming:Stateful stateless]]               exists  60504  Whether the profile survives between visits. "We cleared cookies" is not a reset.
[[Programming:Traffic files]]                    exists  45656  HAR vs pcap vs proxy vs CDP. The three taps are not nested.
[[Programming:Docker]]                           exists  24157  Pin the image digest and size /dev/shm. A tag is not a pin.
[[Programming:Tranco]]                           exists  27540  Tranco API and construction. Cite the list id, not "the Tranco top 1M".
[[Programming:Cloudflare Radar]]                 exists  23850  Radar Domain Rankings. A bucket is not a rank; only the top 100 is ordered.
[[Programming:CrUX]]                             exists  8394   Chrome UX Report access (BigQuery). Short instrument note, not corpus-backed.
[[Programming:Similarweb]]                       exists  4514   Similarweb extension/API note. Short stub, not corpus-backed.
[[Programming:Multilingual support]]             exists  8012   Language detection and translation in a crawl. Short, not yet refreshed against the 2010–2026 extraction.
 
--- privacy ---
Page                              Live    Bytes  What a student needs it for
--------------------------------  ------  -----  -------------------------------------------------------------------------------------------------------------------
[[Privacy:Requests]]              exists  78819  Classifying requests (filter lists vs learned). The list is usually both instrument and ground truth.
[[Privacy:Cookies]]               exists  19875  Classifying cookies. First-party is not always benign; third-party cookies were not discontinued.
[[Privacy:Fingerprinting]]        exists  45596  Measuring browser fingerprinting. The word also names unrelated tasks (website / traffic fingerprinting, …).
[[Privacy:JavaScript]]            exists  58197  Observing and labelling scripts. The unit of analysis (domain vs URL vs content) is the load-bearing choice.
[[Privacy:Server side tracking]]  exists  46745  Measuring tracking that never sends a request to the tracker.
[[Privacy:Cookie syncing]]        exists  49815  Detecting ID sync from a crawl. The identifier heuristic is the measurement.
[[Privacy:Consent]]               exists  67169  What the crawler does with the banner. That choice is the treatment, not a config detail.
[[Privacy:TCF consent strings]]   exists  55858  TC string and Google Additional Consent as artefacts. A decode is not a user choice.
[[Privacy:Darkpatterns]]          exists  15075  Deceptive patterns as the thing you measure, and as a confounder of the crawler. HCI venues are not in this corpus.
 
--- writing ---
Page                           Live    Bytes  What a student needs it for
-----------------------------  ------  -----  ---------------------------------------------------------------------------------------------------
[[Writing:Conferences]]        exists  14552  Where to send a web-measurement paper, among these seven venues and the ones the corpus is missing.
[[Writing:Literature review]]  exists  27896  The keyword-derived denominator trap, measured on the SoKs in this corpus.
 
--- practices ---
Page                              Live    Bytes  What a student needs it for
--------------------------------  ------  -----  -------------------------------------------------------------------------------------------------
[[Practices:Ethics]]              exists  71251  "Not human subjects" is a paperwork finding, not a harm finding. What the field actually reports.
[[Practices:Notifying websites]]  exists  58860  Telling operators at crawl scale. Hold back a control arm.
[[Practices:Legal enforcement]]   exists  51641  Which authority is competent. A cookie finding is ePrivacy, not the GDPR one-stop-shop.
[[Practices:Public relations]]    exists  55718  The number and its denominator have to travel in the same sentence.
 
--- statistics ---
Page                                  Live    Bytes  What a student needs it for
------------------------------------  ------  -----  --------------------------------------------------------------------------------------------------------------
[[Statistics:Hypothesis testing]]     exists  63563  Which tests this field uses. Sites are not independent; clustered errors are almost never used.
[[Statistics:Pvalue corrections]]     exists  62373  Multiplicity as a property of the crawl, not a decision. Correction tracks participants, not hypothesis count.
[[Statistics:Regression]]             exists  72483  Counts, clustered rows, and classifiers-that-look-like-regressions in this literature.
[[Statistics:Biases]]                 exists  49101  Selection, survivorship, vantage and denominator bias with the effect sizes this field has measured.
[[Statistics:Interrater agreement]]   exists  40942  Reliability of hand-coded ground truth, and what to report.
[[Statistics:Study preregistration]]  exists  50827  Almost unheard of for crawls. "pre-regist*" is usually a domain, not a study.
 
========================================================================
D. WHAT THIS REPORT DOES NOT PUBLISH
========================================================================
Headline figures from child pages (Tranco 56.4%, OpenWPM 60 papers, …).
Those have their own report scripts. Copying them here would go stale when
a parallel sitting refreshes the child (website_selection and conferences
are claimed in other Cursor sittings as of 2026-08-27).
Full-text sweeps: none. No fold, no residue.
security and artifacts are out of scope (their own drain items).
 
========================================================================
E. LIVE CHECK OF THE SIX NAMESPACE PAGES THEMSELVES
========================================================================
Before this sitting publishes them they are missing. After publish a re-run
will show exists; that is not a FAILURE. Clobbering is prevented by dw.mjs
--if-rev on later edits, not by this check forever requiring absence.
design: exists. export_raw 4375 bytes.
programming: exists. export_raw 4691 bytes.
privacy: exists. export_raw 3997 bytes.
writing: exists. export_raw 1980 bytes.
practices: exists. export_raw 2941 bytes.
statistics: exists. export_raw 3569 bytes.
 
========================================================================
F. SITTING-START OBSERVATIONS (recorded constants, not live-fetched)
========================================================================
These are contemporaneous notes from 2026-08-27, written into the script
so the report file is generated, not hand-appended. They are not claims
about a later wiki state.
PUBLISHED_CONFERENCES_BYTES_AT_SITTING_START  4191
LIVE_CONFERENCES_BYTES_AT_INVENTORY           24753
LIVE_WEBSITE_SELECTION_BYTES_AT_INVENTORY     10118
CHROME_TP_COOKIE_REVERSAL_DAY                 22
CHROME_TP_COOKIE_REVERSAL_YEAR                2025
CHROME_PRIVACY_SANDBOX_POST                   22 April 2025
REFUSED_CHILD_HEADLINE_EXAMPLE                Tranco 56.4%; OpenWPM 60 papers
 
PUBLISHED_SNAPSHOT_DATE   2026-08-27
OK — inventory matches live wiki; corpus frame matches OVERVIEW.md.

Markup sweep, 2026-09-17

Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with check_wrap.mjs and check_typography.mjs. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.

The fresh rendered-DOM sweep found that the earlier C1–C6 rejection was wrong for provenance pages: bare root links resolved inside the provenance: namespace. The log now records that distinction, the affected links are root-anchored, and the post-save DOM was re-checked for red links. No figures or citations changed.

Namespace-table reconciliation, 2026-09-21

Item: wiki-sweep-figures / reconcile the seven namespace tables and their counts. This section is the joint log for all seven; programming, privacy, security, statistics, practices and writing each carry their own row and point here.

What was wrong

Three things, of which only one was a wrong number on a page.

  • programming published two different child counts, and both were wrong. The WRAP box said “All 19 children below are written”; the Methodology sentence three screens below said “The 18 / 2”. The namespace held 22 pages. Crawler detection (56,859 bytes), LLM agents (40,294 bytes) and Deployment (60,280 bytes) were live pages in this namespace with no row in any table on their own namespace page, so the only route to them was a direct link from a sibling or the search box.
  • Three pages leaned on what start promises, which start stopped doing when it was rewritten on 2026-09-07 to route rather than enumerate. design's box counted “All 11 pages start used to promise here”; statistics' box said “start used to list them inline without a namespace landing”; security's said the page “exists so a reader landing from start is not sent into an empty namespace”. None of the three is false, and all three make a namespace page's own inventory depend on a page that no longer holds one.
  • The report script was the defect, not just its output. report_namespace_overviews.mjs carried a hand-maintained CHILDREN array of 43 entries with an expected: exists|missing bit, and FAILed when the live wiki disagreed with the array. That is the wrong direction — it guarded the array, not the page — and it had been failing on its own stale inventory since Platforms was written. No guard on this wiki compared a namespace page's table to its namespace, which is exactly how design carried a 12-row table next to a 13-page namespace in September and how programming carried a 19-row table next to a 22-page one until today.

Two claims in the brief that were already fixed

Rejected as findings, because a cosmetic re-edit is worse than saying so:

  • design: table has 12 rows, namespace has 13, Existing datasets missing entirely. Fixed on 2026-09-11; on 2026-09-21 the table has 18 rows and the namespace 18 pages, and existing_datasets is row 10. The Methodology section already narrated the correction.
  • security: the box says VirusTotal remains a red link. That page was created 2026-08-27 (rev 1787838295) and the box was corrected on 2026-09-09 (rev 1788952164), in the edit that added the sixth child. Verified today: 27,629 bytes, and the rendered DOM of security carries no wikilink2 span. The box's start-dependent framing was still there and was rewritten; the red-link claim was not.

The script, rewritten

scripts/report_namespace_overviews.mjs now has no inventory. It enumerates each namespace from core.listPages at depth 0 (depth 1 silently returns the top two levels — 88 of 160 pages on 2026-09-03), parses each namespace page's own table rows out of its wiki source, and compares the two sets in both directions:

  • a page in the namespace that no table row links to → MISSING_FROM_TABLE
  • a table row whose target is not a page → RED_LINK_IN_TABLE

A one-sided or size-only check passes on a table that lost one row and gained one red link. It also refuses a namespace whose table links the same page twice, and asserts that children + landings + named top-level pages sums to the content-page total, so a namespace this report does not know about cannot hide.

Before the edits (node scripts/report_namespace_overviews.mjs, exit 1):

FAILURES (3):
  programming: MISSING_FROM_TABLE programming:crawler:llm_agents (40294 bytes, live page, no table row)
  programming: MISSING_FROM_TABLE programming:crawler_detection (56859 bytes, live page, no table row)
  programming: MISSING_FROM_TABLE programming:deployment (60280 bytes, live page, no table row)

design, privacy, security, statistics, practices and writing were already equal in both directions before this sitting touched them. Their counts were re-derived, not changed.

Mutation test of the guard

Reading a set comparison is not testing it, and a guard built from one source can compare it to itself. sh scripts/report_namespace_overviews_mutations.sh <dir> breaks the table side four ways against a --pages-from directory of page sources and requires the guard to name each break. M3 is the one that matters: it drops a real row and adds a red link, so the row count is unchanged at 8 vs 8 and only a set comparison can see it.

=== control: unmutated sources
FAILURES (3):
  programming: MISSING_FROM_TABLE programming:crawler:llm_agents (40294 bytes, live page, no table row)
  programming: MISSING_FROM_TABLE programming:crawler_detection (56859 bytes, live page, no table row)
  programming: MISSING_FROM_TABLE programming:deployment (60280 bytes, live page, no table row)

PASS  M1 dropped row  -> guard reported: MISSING_FROM_TABLE design:sampling
PASS  M2 red link  -> guard reported: RED_LINK_IN_TABLE privacy:supercookies
PASS  M3 swap, count unchanged  -> guard reported: MISSING_FROM_TABLE statistics:biases
PASS  M3 swap, count unchanged  -> guard reported: RED_LINK_IN_TABLE statistics:effect_sizes
PASS  M4 duplicate row  -> guard reported: table links writing:conferences twice

all mutations caught

After: the unedited report output

node scripts/report_namespace_overviews.mjs against the live wiki after the seven saves, exit 0:

report_namespace_overviews-output.txt
========================================================================
A. CORPUS FRAME (OVERVIEW.md populations, re-derived)
========================================================================
PUBLISHED_CORPUS          5859
PUBLISHED_EMPIRICAL       5118  87.4%
PUBLISHED_CRAWLED         1120  19.1%
PUBLISHED_WEB             1622  27.7%
PUBLISHED_INFERENTIAL     1762  30.1%
PUBLISHED_HUMAN_SUBJECTS  1357  23.2%
PUBLISHED_LEGAL           402  6.9%
PUBLISHED_ANNOTATED       3318  56.6%
 
Venues: CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P.
PUBLISHED_VENUE_COUNT     7
PUBLISHED_YEAR_START      2010
PUBLISHED_YEAR_END        2026
2025-2026 are provisional (CCS/IMC 2026 not held; S&P/WWW 2026 thin by construction).
These are the only paper-counts the namespace pages are allowed to print.
Child-page headline figures belong on the children.
 
========================================================================
B. LIVE PAGE INDEX
========================================================================
PUBLISHED_WIKI_PAGE_TOTAL 188   (core.listPages, depth 0)
PUBLISHED_CONTENT_PAGES   97   (excluding provenance: mirrors)
PUBLISHED_PROVENANCE_PAGES 91
PUBLISHED_NS_COUNT        7
 
========================================================================
C. NAMESPACE vs ITS OWN PUBLISHED TABLE
========================================================================
children = pages whose id starts "<ns>:" in the live index (provenance: excluded).
rows     = table rows on the namespace page whose first cell links inside <ns>:.
The two must be equal as SETS, not merely in size: a table that dropped one
page and gained one red link has the right count and the wrong content.
 
PUBLISHED_DESIGN_CHILDREN 18
PUBLISHED_DESIGN_TABLE_ROWS 18
PUBLISHED_PROGRAMMING_CHILDREN 22
PUBLISHED_PROGRAMMING_TABLE_ROWS 22
PUBLISHED_PRIVACY_CHILDREN 18
PUBLISHED_PRIVACY_TABLE_ROWS 18
PUBLISHED_SECURITY_CHILDREN 7
PUBLISHED_SECURITY_TABLE_ROWS 7
PUBLISHED_STATISTICS_CHILDREN 8
PUBLISHED_STATISTICS_TABLE_ROWS 8
PUBLISHED_PRACTICES_CHILDREN 4
PUBLISHED_PRACTICES_TABLE_ROWS 4
PUBLISHED_WRITING_CHILDREN 2
PUBLISHED_WRITING_TABLE_ROWS 2
 
Namespace    Pages in namespace  Rows in its table  Sets
-----------  ------------------  -----------------  -----
design       18                  18                 equal
programming  22                  22                 equal
privacy      18                  18                 equal
security     7                   7                  equal
statistics   8                   8                  equal
practices    4                   4                  equal
writing      2                   2                  equal
 
PUBLISHED_NS_CHILD_TOTAL  79   (pages inside the seven namespaces)
PUBLISHED_TOP_LEVEL       11
PUBLISHED_LANDING_PAGES   7
 
========================================================================
D. PER-NAMESPACE DETAIL
========================================================================
 
--- design --- 18 pages, 18 table rows
  design:algorithm_audits                            36033 bytes  in table
  design:archives                                    42001 bytes  in table
  design:automated_measurements                      22225 bytes  in table
  design:blocking_and_geodifference                  52866 bytes  in table
  design:connected_tv                                64226 bytes  in table
  design:crawling_location                           31740 bytes  in table
  design:dns                                         68076 bytes  in table
  design:existing_datasets                           42216 bytes  in table
  design:ip_classification                           78050 bytes  in table
  design:longitudinal                                59486 bytes  in table
  design:mobile_and_app_measurement                  58332 bytes  in table
  design:ownership_resolution                        71898 bytes  in table
  design:platforms                                   54667 bytes  in table
  design:platforms:ad_archives                       70351 bytes  in table
  design:sampling                                    55740 bytes  in table
  design:user_studies                                34922 bytes  in table
  design:website_classification                     125188 bytes  in table
  design:website_selection                           33899 bytes  in table
 
--- programming --- 22 pages, 22 table rows
  programming:cloudflare_radar                       23912 bytes  in table
  programming:crawler                                58342 bytes  in table
  programming:crawler:foxhound                       69398 bytes  in table
  programming:crawler:llm_agents                     40294 bytes  in table
  programming:crawler:openwpm                        68843 bytes  in table
  programming:crawler:pagegraph                      21594 bytes  in table
  programming:crawler:panoptichrome                  33720 bytes  in table
  programming:crawler:tracker_radar_collector        47819 bytes  in table
  programming:crawler:webxray                        27781 bytes  in table
  programming:crawler_detection                      56859 bytes  in table
  programming:crux                                    8394 bytes  in table
  programming:deployment                             60280 bytes  in table
  programming:docker                                 24451 bytes  in table
  programming:filter_lists                           69322 bytes  in table
  programming:interaction                            56951 bytes  in table
  programming:internet_scanning                      50559 bytes  in table
  programming:multilingual_support                   35544 bytes  in table
  programming:registration                           38174 bytes  in table
  programming:similarweb                              4514 bytes  in table
  programming:stateful_stateless                     62619 bytes  in table
  programming:traffic_files                          45948 bytes  in table
  programming:tranco                                 27817 bytes  in table
 
--- privacy --- 18 pages, 18 table rows
  privacy:ads_txt                                    45391 bytes  in table
  privacy:age_assurance                              49599 bytes  in table
  privacy:browser_extensions                         62948 bytes  in table
  privacy:browser_protection                         66557 bytes  in table
  privacy:browser_storage                            59839 bytes  in table
  privacy:consent                                    72901 bytes  in table
  privacy:cookie_syncing                             50098 bytes  in table
  privacy:cookies                                    71663 bytes  in table
  privacy:darkpatterns                               15132 bytes  in table
  privacy:data_subject_rights                        62247 bytes  in table
  privacy:email_tracking                             66624 bytes  in table
  privacy:fingerprinting                             69714 bytes  in table
  privacy:javascript                                 65003 bytes  in table
  privacy:policies                                   45706 bytes  in table
  privacy:privacy_sandbox                            56115 bytes  in table
  privacy:requests                                   55932 bytes  in table
  privacy:server_side_tracking                       46965 bytes  in table
  privacy:tcf_consent_strings                        56068 bytes  in table
 
--- security --- 7 pages, 7 table rows
  security:authentication                            60123 bytes  in table
  security:email_authentication                      57854 bytes  in table
  security:headers                                   24197 bytes  in table
  security:phishing                                  26710 bytes  in table
  security:tls_certificates                          26604 bytes  in table
  security:virustotal                                27629 bytes  in table
  security:web_vulnerabilities                       25369 bytes  in table
 
--- statistics --- 8 pages, 8 table rows
  statistics:annotation                              57987 bytes  in table
  statistics:biases                                  49359 bytes  in table
  statistics:how_many_sites                          55508 bytes  in table
  statistics:hypothesis_testing                      64191 bytes  in table
  statistics:interrater_agreement                    41707 bytes  in table
  statistics:pvalue_corrections                      63056 bytes  in table
  statistics:regression                              73042 bytes  in table
  statistics:study_preregistration                   50988 bytes  in table
 
--- practices --- 4 pages, 4 table rows
  practices:ethics                                   72279 bytes  in table
  practices:legal_enforcement                        52204 bytes  in table
  practices:notifying_websites                       71782 bytes  in table
  practices:public_relations                         55975 bytes  in table
 
--- writing --- 2 pages, 2 table rows
  writing:conferences                                14640 bytes  in table
  writing:literature_review                          28004 bytes  in table
 
========================================================================
E. LITERATURE: NO LANDING PAGE (recorded decision)
========================================================================
PUBLISHED_LITERATURE_PAGES 2
  literature:bibliography  485378 bytes
  literature:corpus  88068 bytes
 
literature: has no landing page, which is why [[:start]] links its two pages
directly rather than through a namespace row. Two pages do not need a
landing: a landing page exists to carry an annotated list too long for the
index, and a list of two is not. Recorded 2026-09-21; revisit if a third
literature: page is written.
 
========================================================================
F. WHAT THIS REPORT DOES NOT PUBLISH
========================================================================
Headline figures from child pages (Tranco 56.4%, OpenWPM 60 papers, ...).
Those have their own report scripts. Copying them here would go stale when
a parallel sitting refreshes the child.
Full-text sweeps: none. No fold, no residue.
Whether a page is a stub: that is a judgement, not a measurement. The byte
counts in section D are the input to that judgement, not the judgement.
 
OK - every namespace table matches its namespace in both directions;
corpus frame matches OVERVIEW.md; content pages sum.

What each count counts

Every one of the seven Methodology and limitations sections now says the same thing in the same words, because the ambiguity is what let design explain a 12-against-13 mismatch as if it were a fact about the wiki: the published number is both the number of rows in the table above and the number of pages whose id begins <namespace>: in the live index, and the script refuses to pass unless those are the same set. They are not the same quantity in general, and a page that states only one of them cannot detect a page that is in neither.

Namespace Pages in namespace Rows in its table Published count, before After
design 18 18 18 (box said 11, meaning start's old promise) 18
programming 22 22 19 in the box, 18 in methodology 22
privacy 18 18 18 18
security 7 7 seven seven
statistics 8 8 8 8
practices 4 4 4 4
writing 2 2 2 (“Both children”) 2

The 2 on programming — CrUX and Similarweb as short instrument notes — is a judgement about depth, not a measurement, and the page now says so. Their live byte counts (8,394 and 4,514) are unchanged from 2026-08-27 and the column header was re-dated to 2026-09-21 on that basis.

The six pages that were reachable only through start

The brief asked these be verified as present in their namespace tables. Section D of the output above shows each as in table; five were named:

Page In its namespace table Bytes
Foxhound yes, Driving the browser 69,398
PanoptiChrome yes, Driving the browser 33,720
webXray yes, Driving the browser 27,781
Stateful stateless yes, After the page loads 62,619
Email tracking yes, The pages 66,624
LLM agents no — added by this sitting 40,294

The sixth was the one that was still missing.

Decision: literature: keeps no landing page

literature: holds exactly two pages — bibliography (485,378 bytes) and corpus (88,068 bytes) — and has no landing page, which is why start links both directly instead of through a namespace row. Recorded decision: it stays that way. A landing page exists to carry an annotated list too long for the index and a boundary statement about where the namespace stops; a list of two needs neither, and adding one would put a third page in the namespace purely to describe the other two. The seven namespaces are seven because literature: is not one of them, and start's “Outside the seven” sentence already says so. Revisit if a third literature: page is written. Section E of the report output prints the enumeration this rests on, so a third page changes the output rather than only the prose.

Checks run

  • node scripts/report_namespace_overviews.mjs — exit 1 before (3 failures), exit 0 after. Output embedded above.
  • sh scripts/report_namespace_overviews_mutations.sh — 5 of 5 mutations caught.
  • check_wrap.mjs and check_typography.mjs on all seven sources (run again by dw.mjs on the save path).
  • Rendered DOM (do=export_xhtml) of all seven after saving: design 1 table / 19 tr, programming 8 / 30, privacy 1 / 19, practices 1 / 5, statistics 1 / 9, security 2 / 12, writing 1 / 3 — and zero wikilink2 spans on any of them.
  • node scripts/sitemap.mjs site-wide after saving: 189 pages, “Promised but missing: 0”, “UNDECLARED (defects): 0”. Positive control: appending [[Writing:Nonexistent page]] to the cached source moved it to 1 undeclared defect, so the zero is a measurement and not a silent parser failure. The sitemap page cache was overwritten with the saved sources first — it never expires on its own.

Reviewers

One model: sonnet figures-vs-script pass, per the item's review layer. No citations pass: no {[citekey]} and no quoted claim was touched on any of the seven pages.

Item: wiki-sweep-figures / provenance:design bare links resolve to provenance: siblings.

What was wrong

The Link-hygiene sweep, 2026-09-17 section above says the affected links were root-anchored. That is true of the links that sweep looked at ([[start]], [[contributing]]) and false of seventeen others on this page. In the live rendered DOM of 2026-09-21, [[design]] resolved to provenance:design — this page linking to itself — and [[statistics]], [[practices]], [[programming]], [[privacy]] and [[writing]] resolved to the sibling logs in provenance: rather than to the content pages the sentences are about.

None of these ever rendered red, because provenance:statistics and the rest exist. That is the whole failure mode: DokuWiki resolves a colon-free target relative to the current page's namespace first, and falls back to the root only when the relative page does not exist. A wrong target is therefore silent in exactly the case where a sibling happens to exist, and visible only where one does not. Checked on the live wiki the same day: [[start]] on provenance:privacy:browser_storage renders data-wiki-id=“start”, because provenance:privacy:start does not exist.

What changed

Seventeen links on this page, decided one at a time. Fifteen were meant to be the content pages and are now root-anchored:

Where Was Now
Intro sentence [[design]] + [[programming]] [[privacy]] [[writing]] [[practices]] [[statistics]] [[:design]] and the five [[:namespace]] equivalents — the sitting wrote the content pages
Scope decision, statistics-landing row [[Statistics]] [[:statistics]]
Queries table, On the page? column (Q2, Q4–Q8, Q11) [[practices]] [[privacy]] [[statistics]] [[programming]] root-anchored; the column names the content page that prints the figure
Addendum 2026-08-27 closing sentence [[design]] [[:design]]

Two were not links at all and are now nowiki, matching how row C1–C6 already writes them: the G2 row and the Published bullet both quote the markup added to start, so [[Statistics]] there is a sample of source, not a route. The explicit [[provenance:programming]]-style links in the 2026-09-21 section and [[provenance:design:platforms]] in the addendum are correct as written and were left alone: those sentences do mean the sibling log.

No figure, citekey or quoted claim was touched.

The sweep that reported zero

scripts/sitemap.mjs ran site-wide on 2026-09-21 and printed “Promised but missing: 0 / UNDECLARED (defects): 0” with all seventeen of these live. It was not wrong; it was blind. Its toId strips a leading colon, so [[design]] and [[:design]] are the same target to it, and both name a page that exists. The script models missing targets, not wrong ones.

It now also collects every colon-free link written on a page that sits inside a namespace, and splits them:

  • resolves to a namespace sibling — the link already points somewhere other than the root page of the same name. A defect; the run exits 1 and names the pages.
  • falls back to root today — reads correctly now, flips the day somebody creates the sibling. Reported, not gated.

The sibling test is strict existence, not the script's resolves() helper: that helper treats x:start as resolving whenever x exists, which is right for “does the wiki have this page” and wrong for relative resolution. Its first draft used resolves() and reported the provenance:privacy:browser_storage link above as a defect when the live DOM says it is not.

Checks run

  • node scripts/sitemap.mjs before the edit: 189 pages, “Promised but missing: 0”, “UNDECLARED (defects): 0”, and the new counter reading 42 bare colon-free links inside a namespace, 38 of them resolving to a sibling across 7 pages — 17 on this one. Exit 1.
  • node scripts/sitemap.mjs after the edit: 0 on this page. The other six provenance: pages still fail and are listed below; they are not this item.
  • Rendered DOM (do=export_xhtml) after saving, counted by data-wiki-id: zero provenance:design self-links, and provenance:statistics / provenance:practices / provenance:programming / provenance:privacy / provenance:writing down to the one explicit link each that the 2026-09-21 section writes out in full.
  • node scripts/report_namespace_overviews.mjs — exit 0, “OK — every namespace table matches its namespace in both directions”. Every figure in the Queries table whose row was edited still matches section A of that output: 5,859 / 5,118 / 1,120 / 1,622 / 1,762 / 1,357 / 402 / 3,318, and the 7 venues and the 2010–2026 window. The stub bytes on the Q11 row read 8,394 / 4,514 for CrUX and Similarweb, which section D confirms; the third, 8,012 for programming:multilingual_support, is a 2026-08-27 figure and that page is 35,544 bytes today. It is left as written because the row records what the 2026-08-27 sitting measured, and the 2026-09-21 section above already restricts its “unchanged” claim to the first two.

Left for a later sitting

The same defect is live on six sibling logs and was found by the same run, but each belongs to that page's own item, not this one: provenance:roadmap (8 self-links written [[roadmap]], plus [[Security]] → provenance:security), provenance:design-style pairs on provenance:practices, provenance:privacy, provenance:programming and provenance:writing (2 each), and provenance:statistics (4). Until they are fixed scripts/sitemap.mjs exits 1 site-wide and names them; that is the gate working, not a regression.

Reviewer finding, accepted: "bytes" in the frozen report output are characters

The figures-vs-script pass raised a figure this item did not touch. The Parallel sittings table says design:automated_measurements was published at 20,068 bytes and design:user_studies at 34,141; the Unedited report output section, frozen the same day, lists the same two pages at 19,966 and 33,966. Nothing on the page reconciled them.

Both are right, in different units, and the revision that settles it is still fetchable:

  • curl “…?id=design:automated_measurements&rev=1787840280&do=export_raw” → 20,068 bytes, 19,966 characters.
  • curl “…?id=design:user_studies&rev=1787840285&do=export_raw” → 34,141 bytes, 33,966 characters.

The prose figures are DokuWiki's own revision sizes (dw.mjs history prints C 20068 and C 34141 for the creating revisions), which are bytes. The 2026-08-27 report measured the export_raw body with JavaScript .length, which counts UTF-16 code units — so every em-dash and curly quote in those pages costs one “byte” less than it should. The gaps, 102 and 175, are the multibyte-character counts of those two revisions, not drift.

Consequence, and why nothing was edited. Every “N bytes” in the Unedited report output section is a character count wearing a byte label. Today's report_namespace_overviews.mjs does not have this defect — it reads size from core.listPages, which is bytes — so the two are not comparable and a difference between them is not evidence of a page changing. The frozen section is left unedited, as it already says it is. The label is wrong; the measurement is not, and re-typing numbers inside a frozen record to fix a unit would destroy the only thing that record is for. The live figure that this does affect is the Q11 stub row: programming:crux 8,394 and programming:similarweb 4,514 are the same in both units and match today's script exactly, so the 2026-09-21 section's “unchanged” claim about those two stands.

Reviewers

One model: sonnet figures-vs-script pass, per the item's review layer. No citations pass: no {[citekey]} and no quoted claim was touched.

provenance/design.txt · Last modified: by karel.kubicek.claude