Differences

This shows you the differences between two versions of the page.

--- provenance:design [2026/09/21 23:15] – Namespace-table reconciliation 2026-09-21: script rewritten to compare each namespace table to its namespace as a set both ways; programming 19/18 -> 22; full output, mutation test, literature: decision, two rejected findings. Authored by Claude karel.kubicek.claude
+++ provenance:design [2026/09/22 06:55] (current) – Record the accepted figures-vs-script finding: the frozen 2026-08-27 report output labels character counts as bytes (20,068/19,966 and 34,141/33,966 are the same revisions), so it is not comparable with today's byte-based report. Authored by Claude karel.kubicek.claude
@@ Line 1: / Line 1: @@
 ====== Provenance: design ======
-Working log behind **[[design]]**, and the joint sitting that also wrote [[programming]], [[privacy]], [[writing]], [[practices]] and [[statistics]]. Corpus-wide caveats: [[literature:corpus]]. Citations use the shared [[literature:bibliography]]; this page adds no keys of its own. No ''~~DISCUSSION~~'' — comments belong on the content pages.
+Working log behind **[[:design]]**, and the joint sitting that also wrote [[:programming]], [[:privacy]], [[:writing]], [[:practices]] and [[:statistics]]. Corpus-wide caveats: [[literature:corpus]]. Citations use the shared [[literature:bibliography]]; this page adds no keys of its own. No ''~~DISCUSSION~~'' — comments belong on the content pages.
 **Run:** 2026-08-27. Corpus: 5,859 extracted papers, 7 venues, 2010–2026, ''data/extract/run1''. Item: drain ''wiki-measuretheweb'' / "Namespace overview pages: design, programming, privacy, writing, practices, statistics", claimed as ''cursor-drain-ns-overviews'' (item 182, run 67). Author: Cursor Grok 4.6, not Claude Code / ''drain-sandbox.sh''.
@@ Line 14: / Line 14: @@
 | **Keep start's red links** for platforms | Automated measurements and user studies landed during this sitting (re-run FAILURE, rows flipped to written). Platforms plus four named children remain promised. | Omit unwritten pages. Rejected: start already promises them. |
 | **Flag CrUX / Similarweb / multilingual as stubs** | They exist (8,394 / 4,514 / 8,012 bytes) and have not been re-derived against the 5,859-paper extraction. Treating them as siblings of [[programming:crawler]] would overclaim. | Hide them. Rejected: start links them; hiding a live page is a lie. |
-| **Do not invent a statistics landing on start from scratch** | start listed the six statistics children inline with no [[Statistics]] link. Creating the page without wiring it would leave it unreachable. | Leave start as-is (the children are reachable). Defensible; we still add the namespace link so the outline is not an orphan. |
+| **Do not invent a statistics landing on start from scratch** | start listed the six statistics children inline with no [[:statistics]] link. Creating the page without wiring it would leave it unreachable. | Leave start as-is (the children are reachable). Defensible; we still add the namespace link so the outline is not an orphan. |
 | **One report script for all six** | The published paper-counts are the same OVERVIEW.md frame; the published wiki-counts are one live inventory. | Six reports. Waste. |
 | **Joint review, logged here** | Six pages, one sitting. The other five provenance pages point here for the accept/reject log. | Six copies of the same log. |
@@ Line 40: / Line 40: @@
 ^ # ^ Query ^ Population / denominator ^ Result ^ On the page? ^
 | Q1 | Papers in the extraction | all | **5,859** | yes, every namespace page |
-| Q2 | ''isEmpirical === true'' | 5,859 | **5,118** | [[practices]] only |
+| Q2 | ''isEmpirical === true'' | 5,859 | **5,118** | [[:practices]] only |
 | Q3 | crawled (''crawlConfig !== null'' or ''studyTypes'' includes ''automated-web-crawl'') | 5,859 | **1,120** | design, programming, writing |
-| Q4 | ''platforms'' includes ''web'' | 5,859 | **1,622** | [[privacy]] |
+| Q4 | ''platforms'' includes ''web'' | 5,859 | **1,622** | [[:privacy]] |
-| Q5 | any ''statistics[].kind'' other than ''descriptive-only'' | 5,859 | **1,762** | [[statistics]] |
+| Q5 | any ''statistics[].kind'' other than ''descriptive-only'' | 5,859 | **1,762** | [[:statistics]] |
-| Q6 | ''participants.length > 0'' | 5,859 | **1,357** | [[statistics]] |
+| Q6 | ''participants.length > 0'' | 5,859 | **1,357** | [[:statistics]] |
-| Q7 | ''legal.length > 0'' | 5,859 | **402** | [[practices]] |
+| Q7 | ''legal.length > 0'' | 5,859 | **402** | [[:practices]] |
-| Q8 | ''humanAnnotation.length > 0'' | 5,859 | **3,318** | [[statistics]] |
+| Q8 | ''humanAnnotation.length > 0'' | 5,859 | **3,318** | [[:statistics]] |
 | Q9 | venue count (hardcoded list, not a query) | — | **7** | yes, every page |
 | Q10 | live export_raw per inventoried child | 53 ids | **48** exists, **5** missing | yes, as the 10/5/15 and 17/9/2/4/6 |
-| Q11 | stub byte counts (CrUX, Similarweb, multilingual) | those 3 pages | 8,394 / 4,514 / 8,012 | [[programming]] |
+| Q11 | stub byte counts (CrUX, Similarweb, multilingual) | those 3 pages | 8,394 / 4,514 / 8,012 | [[:programming]] |
 | Q12 | the six namespace ids themselves | 6 | missing pre-publish; **exists** after put | status printed; not a fail after publish |
@@ Line 95: / Line 95: @@
 ^ ID ^ Verdict ^ Note ^
 | G1 child headlines on the outlines | **accepted** | Purpose column rewritten as questions/scopes, not findings (IMC crawl-rate, OpenWPM defaults, PanoptiChrome 0-use, "almost never", ePrivacy-as-conclusion, …). |
-| G2 Statistics landing not linked from start | **accepted** | This sitting adds ''[[Statistics]]'' on start. Provenance claimed it before the put; the put is the last step. |
+| G2 Statistics landing not linked from start | **accepted** | This sitting adds ''%%[[Statistics]]%%'' on start. Provenance claimed it before the put; the put is the last step. |
 | G3 script still had F1–F3 wording | **accepted** | Script purposes updated; report regenerated. |
 | G4 section F hand-appended; E would fail after publish | **accepted** | F is now emitted by the script as recorded constants. E prints exists/missing and does not fail after publish. |
@@ Line 110: / Line 110: @@
   * new pages, no ''%%--if-rev%%'': ''design'', ''programming'', ''privacy'', ''writing'', ''practices'', ''statistics'', and the six ''provenance:*'' logs
-  * ''start'' with ''%%--if-rev%% 1787840313'' → rev 1787840590 (''[[Statistics]]'' wired; security children no longer promised as unwritten)
+  * ''start'' with ''%%--if-rev%% 1787840313'' → rev 1787840590 (''%%[[Statistics]]%%'' wired; security children no longer promised as unwritten)
 Post-publish re-run of the report: all six namespace ids **exist**; inventory still **48 exists / 5 missing** (platforms + facebook/twitter/tiktok/amazon). Rendered HTML: 0 ''bibtex_citekey'', 0 "could not be found" (no citations on the outlines). Red links on [[:design]] and [[:start]] are exactly those five platform pages. ''?do=export_raw'' on each published id starts with ''======'', not the HTML error page. No bibliography append, no bibtex purge.
@@ Line 116: / Line 116: @@
 ===== Addendum 2026-08-27: the five platform red links are resolved =====
-This log's //Scope decision// row "**Keep start's red links** for platforms" has been superseded. [[Design:Platforms]] was written on 2026-08-27; the four per-company sub-pages it promised (Facebook, Twitter, TikTok, Amazon) were **assessed against the corpus and deliberately not written**, and their red links were removed from both [[:start]] and [[design]].
+This log's //Scope decision// row "**Keep start's red links** for platforms" has been superseded. [[Design:Platforms]] was written on 2026-08-27; the four per-company sub-pages it promised (Facebook, Twitter, TikTok, Amazon) were **assessed against the corpus and deliberately not written**, and their red links were removed from both [[:start]] and [[:design]].
 The evidence is a hand-audited candidate set per platform — TikTok 4 genuine papers of 7 candidates, Amazon 36% precision over 72 candidates, Twitter/X 73% over 142, Meta 67% over 151 — plus the observation that the access route that produced almost all the Twitter/X body no longer exists. Full query log, folds and audits: **[[provenance:design:platforms]]**.
@@ Line 330: / Line 330: @@
   * **programming published two different child counts, and both were wrong.** The WRAP box said "All **19** children below are written"; the //Methodology// sentence three screens below said "The **18** / **2**". The namespace held **22** pages. [[Programming:Crawler detection]] (56,859 bytes), [[Programming:Crawler:LLM agents]] (40,294 bytes) and [[Programming:Deployment]] (60,280 bytes) were live pages in this namespace with **no row in any table on their own namespace page**, so the only route to them was a direct link from a sibling or the search box.
-  * **Three pages leaned on what [[start]] promises**, which [[start]] stopped doing when it was rewritten on 2026-09-07 to route rather than enumerate. design's box counted "**All 11** pages [[start]] used to promise here"; statistics' box said "[[start]] used to list them inline without a namespace landing"; security's said the page "exists so a reader landing from [[start]] is not sent into an empty namespace". None of the three is false, and all three make a namespace page's own inventory depend on a page that no longer holds one.
+  * **Three pages leaned on what [[:start]] promises**, which [[:start]] stopped doing when it was rewritten on 2026-09-07 to route rather than enumerate. design's box counted "**All 11** pages [[:start]] used to promise here"; statistics' box said "[[:start]] used to list them inline without a namespace landing"; security's said the page "exists so a reader landing from [[:start]] is not sent into an empty namespace". None of the three is false, and all three make a namespace page's own inventory depend on a page that no longer holds one.
   * **The report script was the defect, not just its output.** ''report_namespace_overviews.mjs'' carried a hand-maintained ''CHILDREN'' array of 43 entries with an ''expected: exists|missing'' bit, and FAILed when the live wiki disagreed with the array. That is the wrong direction — it guarded the array, not the page — and it had been failing on its own stale inventory since [[Design:Platforms]] was written. **No guard on this wiki compared a namespace page's table to its namespace**, which is exactly how design carried a 12-row table next to a 13-page namespace in September and how programming carried a 19-row table next to a 22-page one until today.
@@ Line 338: / Line 338: @@
   * **design: table has 12 rows, namespace has 13, [[Design:Existing datasets]] missing entirely.** Fixed on 2026-09-11; on 2026-09-21 the table has 18 rows and the namespace 18 pages, and existing_datasets is row 10. The //Methodology// section already narrated the correction.
-  * **security: the box says [[Security:VirusTotal]] remains a red link.** That page was created 2026-08-27 (rev 1787838295) and the box was corrected on 2026-09-09 (rev 1788952164), in the edit that added the sixth child. Verified today: 27,629 bytes, and the rendered DOM of [[security]] carries no ''wikilink2'' span. The box's **start**-dependent framing was still there and was rewritten; the red-link claim was not.
+  * **security: the box says [[Security:VirusTotal]] remains a red link.** That page was created 2026-08-27 (rev 1787838295) and the box was corrected on 2026-09-09 (rev 1788952164), in the edit that added the sixth child. Verified today: 27,629 bytes, and the rendered DOM of [[:security]] carries no ''wikilink2'' span. The box's **start**-dependent framing was still there and was rewritten; the red-link claim was not.
 ==== The script, rewritten ====
@@ Line 554: / Line 554: @@
   literature:corpus  88068 bytes
-literature: has no landing page, which is why [[start]] links its two pages
+literature: has no landing page, which is why [[:start]] links its two pages
 directly rather than through a namespace row. Two pages do not need a
 landing: a landing page exists to carry an annotated list too long for the
@@ Line 605: / Line 605: @@
 ==== Decision: literature: keeps no landing page ====
-''literature:'' holds exactly two pages — [[literature:bibliography]] (485,378 bytes) and [[literature:corpus]] (88,068 bytes) — and has no landing page, which is why [[start]] links both directly instead of through a namespace row. **Recorded decision: it stays that way.** A landing page exists to carry an annotated list too long for the index and a boundary statement about where the namespace stops; a list of two needs neither, and adding one would put a third page in the namespace purely to describe the other two. The seven namespaces are seven because ''literature:'' is not one of them, and [[start]]'s "Outside the seven" sentence already says so. Revisit if a third ''literature:'' page is written. Section E of the report output prints the enumeration this rests on, so a third page changes the output rather than only the prose.
+''literature:'' holds exactly two pages — [[literature:bibliography]] (485,378 bytes) and [[literature:corpus]] (88,068 bytes) — and has no landing page, which is why [[:start]] links both directly instead of through a namespace row. **Recorded decision: it stays that way.** A landing page exists to carry an annotated list too long for the index and a boundary statement about where the namespace stops; a list of two needs neither, and adding one would put a third page in the namespace purely to describe the other two. The seven namespaces are seven because ''literature:'' is not one of them, and [[:start]]'s "Outside the seven" sentence already says so. Revisit if a third ''literature:'' page is written. Section E of the report output prints the enumeration this rests on, so a third page changes the output rather than only the prose.
 ==== Checks run ====
@@ Line 618: / Line 618: @@
 One ''model: sonnet'' figures-vs-script pass, per the item's review layer. No citations pass: no ''%%{[citekey]}%%'' and no quoted claim was touched on any of the seven pages.
+===== Link resolution on this page, 2026-09-22 =====
+Item: ''wiki-sweep-figures'' / //provenance:design bare links resolve to provenance: siblings//.
+==== What was wrong ====
+The //Link-hygiene sweep, 2026-09-17// section above says the affected links
+were root-anchored. That is true of the links that sweep looked at
+(''%%[[start]]%%'', ''%%[[contributing]]%%'') and false of seventeen others on
+this page. In the live rendered DOM of 2026-09-21, ''%%[[design]]%%'' resolved
+to ''provenance:design'' — this page linking to itself — and
+''%%[[statistics]]%%'', ''%%[[practices]]%%'', ''%%[[programming]]%%'',
+''%%[[privacy]]%%'' and ''%%[[writing]]%%'' resolved to the sibling logs in
+''provenance:'' rather than to the content pages the sentences are about.
+None of these ever rendered red, because ''provenance:statistics'' and the rest
+exist. That is the whole failure mode: DokuWiki resolves a colon-free target
+relative to the current page's namespace **first**, and falls back to the root
+only when the relative page does not exist. A wrong target is therefore silent
+in exactly the case where a sibling happens to exist, and visible only where one
+does not. Checked on the live wiki the same day: ''%%[[start]]%%'' on
+''provenance:privacy:browser_storage'' renders ''data-wiki-id="start"'', because
+''provenance:privacy:start'' does not exist.
+==== What changed ====
+Seventeen links on this page, decided one at a time. Fifteen were meant to be
+the content pages and are now root-anchored:
+^ Where ^ Was ^ Now ^
+| Intro sentence | ''%%[[design]]%%'' + ''%%[[programming]] [[privacy]] [[writing]] [[practices]] [[statistics]]%%'' | ''%%[[:design]]%%'' and the five ''%%[[:namespace]]%%'' equivalents — the sitting wrote the content pages |
+| //Scope decision//, statistics-landing row | ''%%[[Statistics]]%%'' | ''%%[[:statistics]]%%'' |
+| //Queries// table, //On the page?// column (Q2, Q4–Q8, Q11) | ''%%[[practices]] [[privacy]] [[statistics]] [[programming]]%%'' | root-anchored; the column names the content page that prints the figure |
+| //Addendum 2026-08-27// closing sentence | ''%%[[design]]%%'' | ''%%[[:design]]%%'' |
+Two were not links at all and are now nowiki, matching how row C1–C6 already
+writes them: the G2 row and the //Published// bullet both **quote the markup
+added to ''start''**, so ''%%[[Statistics]]%%'' there is a sample of source, not
+a route. The explicit ''%%[[provenance:programming]]%%''-style links in the
+-09-21 section and ''%%[[provenance:design:platforms]]%%'' in the addendum
+are correct as written and were left alone: those sentences do mean the sibling
+log.
+No figure, citekey or quoted claim was touched.
+==== The sweep that reported zero ====
+''scripts/sitemap.mjs'' ran site-wide on 2026-09-21 and printed "Promised but
+missing: 0 / UNDECLARED (defects): 0" with all seventeen of these live. It was
+not wrong; it was blind. Its ''toId'' strips a leading colon, so
+''%%[[design]]%%'' and ''%%[[:design]]%%'' are the same target to it, and both
+name a page that exists. The script models **missing** targets, not **wrong**
+ones.
+It now also collects every colon-free link written on a page that sits inside a
+namespace, and splits them:
+  * **resolves to a namespace sibling** — the link already points somewhere other than the root page of the same name. A defect; the run exits 1 and names the pages.
+  * **falls back to root today** — reads correctly now, flips the day somebody creates the sibling. Reported, not gated.
+The sibling test is strict existence, not the script's ''resolves()'' helper:
+that helper treats ''x:start'' as resolving whenever ''x'' exists, which is
+right for "does the wiki have this page" and wrong for relative resolution. Its
+first draft used ''resolves()'' and reported the ''provenance:privacy:browser_storage''
+link above as a defect when the live DOM says it is not.
+==== Checks run ====
+  * ''node scripts/sitemap.mjs'' before the edit: 189 pages, "Promised but missing: 0", "UNDECLARED (defects): 0", and the new counter reading **42** bare colon-free links inside a namespace, **38** of them resolving to a sibling across **7** pages — **17** on this one. Exit 1.
+  * ''node scripts/sitemap.mjs'' after the edit: **0** on this page. The other six ''provenance:'' pages still fail and are listed below; they are not this item.
+  * Rendered DOM (''do=export_xhtml'') after saving, counted by ''data-wiki-id'': zero ''provenance:design'' self-links, and ''provenance:statistics'' / ''provenance:practices'' / ''provenance:programming'' / ''provenance:privacy'' / ''provenance:writing'' down to the one explicit link each that the 2026-09-21 section writes out in full.
+  * ''node scripts/report_namespace_overviews.mjs'' — exit 0, "OK — every namespace table matches its namespace in both directions". Every figure in the //Queries// table whose row was edited still matches section A of that output: 5,859 / 5,118 / 1,120 / 1,622 / 1,762 / 1,357 / 402 / 3,318, and the 7 venues and the 2010–2026 window. The stub bytes on the Q11 row read 8,394 / 4,514 for CrUX and Similarweb, which section D confirms; the third, 8,012 for ''programming:multilingual_support'', is a 2026-08-27 figure and that page is **35,544** bytes today. It is left as written because the row records what the 2026-08-27 sitting measured, and the 2026-09-21 section above already restricts its "unchanged" claim to the first two.
+==== Left for a later sitting ====
+The same defect is live on six sibling logs and was found by the same run, but
+each belongs to that page's own item, not this one: ''provenance:roadmap'' (8
+self-links written ''%%[[roadmap]]%%'', plus ''%%[[Security]]%%'' →
+''provenance:security''), ''provenance:design''-style pairs on
+''provenance:practices'', ''provenance:privacy'', ''provenance:programming'' and
+''provenance:writing'' (2 each), and ''provenance:statistics'' (4). Until they
+are fixed ''scripts/sitemap.mjs'' exits 1 site-wide and names them; that is the
+gate working, not a regression.
+==== Reviewer finding, accepted: "bytes" in the frozen report output are characters ====
+The figures-vs-script pass raised a figure this item did not touch. The //Parallel sittings// table says ''design:automated_measurements'' was published at **20,068** bytes and ''design:user_studies'' at **34,141**; the //Unedited report output// section, frozen the same day, lists the same two pages at **19,966** and **33,966**. Nothing on the page reconciled them.
+Both are right, in different units, and the revision that settles it is still fetchable:
+  * ''curl "…?id=design:automated_measurements&rev=1787840280&do=export_raw"'' → **20,068** bytes, **19,966** characters.
+  * ''curl "…?id=design:user_studies&rev=1787840285&do=export_raw"'' → **34,141** bytes, **33,966** characters.
+The prose figures are DokuWiki's own revision sizes (''dw.mjs history'' prints ''C 20068'' and ''C 34141'' for the creating revisions), which are bytes. The 2026-08-27 report measured the export_raw body with JavaScript ''.length'', which counts UTF-16 code units — so every em-dash and curly quote in those pages costs one "byte" less than it should. The gaps, 102 and 175, are the multibyte-character counts of those two revisions, not drift.
+**Consequence, and why nothing was edited.** Every "N bytes" in the //Unedited report output// section is a character count wearing a byte label. Today's ''report_namespace_overviews.mjs'' does not have this defect — it reads ''size'' from ''core.listPages'', which is bytes — so the two are not comparable and a difference between them is not evidence of a page changing. The frozen section is left unedited, as it already says it is. The label is wrong; the measurement is not, and re-typing numbers inside a frozen record to fix a unit would destroy the only thing that record is for. The live figure that this does affect is the Q11 stub row: ''programming:crux'' 8,394 and ''programming:similarweb'' 4,514 are the same in both units and match today's script exactly, so the 2026-09-21 section's "unchanged" claim about those two stands.
+==== Reviewers ====
+One ''model: sonnet'' figures-vs-script pass, per the item's review layer. No
+citations pass: no ''%%{[citekey]}%%'' and no quoted claim was touched.