Differences

This shows you the differences between two versions of the page.

--- provenance:programming:crawler:pagegraph [2026/08/14 18:16] – Record the full review layer that round 0 skipped: rounds 0-2 logs with accepted and rejected findings, rendering-cache trap, corrected external-source table, honest run record. Authored by Claude karel.kubicek.claude
+++ provenance:programming:crawler:pagegraph [2026/09/22 04:33] (current) – Figures-vs-script finding accepted: report_pagegraph.mjs's hardcoded 'Manual follow-up' string still asserted the retracted column-repair cause; corrected in the script and the embedded block regenerated from the re-run. Authored by Claude karel.kubicek.claude
@@ Line 56: / Line 56: @@
 ===== Evidence quotes checked =====
-The report checks all 9 PageGraph tool evidence quotes against the source file at ''data/fulltext/<year>/<venue>/<slug>/paper.cols.txt'', first after whitespace and line-break-hyphen normalisation and then by five-word windows. Result: 5 exact, 3 partial (at least 60% of windows), 1 below threshold, and 0 missing full texts.
+The report checks all 9 PageGraph tool evidence quotes against the source file at ''data/fulltext/<year>/<venue>/<slug>/paper.cols.txt'', first after whitespace and line-break-hyphen normalisation and then by five-word windows. Anything still below threshold is re-checked against a second rendering of the same ''paper.pdf'' produced by ''pypdf'' (''scripts/pdftext.py''), which is reported as **rescued**. Result: 5 exact, 3 partial (at least 60% of windows), **0 rescued from the PDF**, 1 below threshold **in both renderings**, and 0 missing full texts.
-The below-threshold quote is the WebREC sentence “Our approach builds on an existing Chromium-based measurement system called PageGraph”. It is present in the source paragraph; the repaired ''.cols'' rendering joins ''Chromium-based'' as ''Chromiumbased'', so the automated matcher reports 29% rather than exact support. I checked the surrounding paragraph manually. It is a column-repair mismatch, not an unsupported quote. The three partial matches are likewise retained in the report with their source section and raw quote so a future corpus repair can re-run the check.
+The below-threshold quote is the WebREC sentence “Our approach builds on an existing Chromium-based measurement system called PageGraph”. It is present in the source paragraph; both stored renderings join ''Chromium-based'' as ''Chromiumbased'', so the automated matcher reports 29% rather than exact support. I checked the surrounding paragraph manually. It is a rendering mismatch, not an unsupported quote.
+**The ''pypdf'' fallback added on 2026-09-22 does not rescue it, and the reason corrects this page's own earlier diagnosis.** The re-run scores the quote at the identical 29% — 2 of its 7 five-word windows — against ''paper.cols.txt'' and against ''pypdf'''s rendering of the same ''paper.pdf''. The cause is not the two-column repair. Lines 343–345 of the raw ''paper.txt'' dump, **verbatim and with the two columns still interleaved as that file stores them**, show the word broken across a line in the PDF itself. The right-hand column is the passage; the left-hand column is unrelated text from the facing column:
+<code>
+   And second, .web is general, allowing it to generate            Chromium. Our approach builds on an existing Chromium-
+archives to support reproducible studies beyond what exist-        based measurement system called PageGraph, which has been
+ing common measurement tools and archive formats allow.            used in prior work [9, 58, 59]. PageGraph comprehensively
+</code>
+Read the right-hand column alone: ''%%Chromium-%%'' ends line 343 and ''based'' begins line 344.
+Every renderer strips a hyphen at a line break, because that is what a line-break hyphen usually is — and here it is a **real** hyphen in a real compound word. ''.cols'' and ''pypdf'' therefore produce the same wrong token, and no second rendering of this PDF can recover it. The matcher's own de-hyphenation only fires on ''%%-%%'' followed by whitespace, so it does not fold the needle to match. This is a class of below-threshold quote the PDF fallback is structurally unable to help with, and it is recorded rather than engineered around: widening the fold to strip every intra-word hyphen would make the matcher accept quotes it should not. See //Quote-check refresh, 2026-09-22// at the foot of this page.
+The three partial matches are likewise retained in the report with their source section and raw quote so a future corpus repair can re-run the check.
 ===== External and industry verification =====
@@ Line 66: / Line 80: @@
 ^ Source ^ Load-bearing fact ^ Verification on 2026-08-14 ^ Decision ^
 | [[https://github.com/brave/brave-browser/wiki/PageGraph|Brave PageGraph wiki]] | PageGraph is in Brave from 1.46; Nightly is needed for built-in JS APIs; supported events and limitations; hard-crash-over-uncertainty policy | Read the current wiki page directly; checked the feature, limitation, build, documentation and crash sections | Used |
-| [[https://github.com/brave/pagegraph-crawl|pagegraph-crawl]] | Node CLI, binary path, ''-t'' crawl duration, GraphML output, and warning that Puppeteer automation is not itself tracked | Read the README; GitHub API reported unarchived, MPL-2.0, pushed 2026-08-11. **In review the repo was additionally cloned, built and executed**, which is how the dead ''--debug'' flag was caught | Used, with the README's own example corrected |
+| [[https://github.com/brave/pagegraph-crawl|pagegraph-crawl]] | Node CLI, binary path, ''-t'' crawl duration, GraphML output, and warning that Puppeteer automation is not itself tracked | Read the README; GitHub API reported unarchived, MPL-2.0, pushed 2026-08-11. **In review the repo was additionally cloned, built and executed**, which is how the dead ''%%--debug%%'' flag was caught | Used, with the README's own example corrected |
-| ''pagegraph-crawl'' README crawl example | The ''--debug debug'' invocation | **Rejected.** Running the current CLI gives ''run.js: error: unrecognized arguments: --debug debug''; ''src/run.ts'' at 1.2.13 defines ''--logging {none,info,verbose}'' instead. Upstream documentation is stale | Rejected; page publishes the working command and warns about the README |
+| ''pagegraph-crawl'' README crawl example | The ''%%--debug debug%%'' invocation | **Rejected.** Running the current CLI gives ''%%run.js: error: unrecognized arguments: --debug debug%%''; ''src/run.ts'' at 1.2.13 defines ''%%--logging {none,info,verbose}%%'' instead. Upstream documentation is stale | Rejected; page publishes the working command and warns about the README |
 | [[https://github.com/brave-experiments/pagegraph-query|pagegraph-query]] | Current query subcommands and PageGraph's attributed event model | Read the README; GitHub API reported unarchived, activity in 2026. Subcommand list re-checked in review: **8**, not 7 — ''elm'' was missing. Confirmed Python from ''pyproject.toml''/''run.py'' | Used, corrected |
 | ''pagegraph-query'' README | Originally cited for "format documentation is incomplete" | **Rejected as the source.** Its 74-line README has zero occurrences of "incomplete"; the sentence is on Brave's wiki, describing the archived ''pagegraph-rust'' docs | Rejected; claim re-sourced to the Brave wiki |
@@ Line 105: / Line 119: @@
 | sonnet, citations and quotes | ''{[iqbal2020_adgraph]}'' was cited for "PageGraph is the maintained successor to the AdGraph line". AdGraph (2020) predates PageGraph and never mentions it. The succession is stated in ''siby2022_webgraph'', already cited on the page. | **Accepted.** Reworded so AdGraph is cited for AdGraph and the extension claim is cited to WebGraph. Quote re-verified locally: "Sjösten et al. [54] introduced PageGraph, which extends A D G RAPH's graph representation by improving event attribution and capturing more behaviors." |
 | sonnet, citations and quotes | Everything else clean: 10 keys resolve uniquely; bibliography entries match ''corpus2/.meta''; all load-bearing figures, all 8 table roles and all 9 evidence quotes verified against ''paper.cols.txt'', including the SVG limitation and the WebREC hyphenation artefact. | **Accepted as confirmation.** |
-| sonnet, external currency | **The published crawl command was broken.** ''--debug debug'' does not exist in the current CLI; it exits with ''run.js: error: unrecognized arguments: --debug debug''. The reviewer cloned, built and ran ''pagegraph-crawl'' 1.2.13 to establish this. The upstream README still shows the same stale example, which is how it got onto the page. | **Accepted — most valuable finding of the round.** Replaced with ''--logging verbose'' and added a warning box, after confirming against ''src/run.ts'' that the flag is ''--logging {none,info,verbose}'', default ''info''. |
+| sonnet, external currency | **The published crawl command was broken.** ''%%--debug debug%%'' does not exist in the current CLI; it exits with ''%%run.js: error: unrecognized arguments: --debug debug%%''. The reviewer cloned, built and ran ''pagegraph-crawl'' 1.2.13 to establish this. The upstream README still shows the same stale example, which is how it got onto the page. | **Accepted — most valuable finding of the round.** Replaced with ''%%--logging verbose%%'' and added a warning box, after confirming against ''src/run.ts'' that the flag is ''%%--logging {none,info,verbose}%%'', default ''info''. |
 | sonnet, external currency | ''pagegraph-query'' has **8** subcommands; the page listed 7, omitting ''elm''. | **Accepted.** Added ''elm'', and noted the tool is a Python CLI. Verified against the live README. |
 | sonnet, external currency | The "format documentation is incomplete" sentence was footnoted to ''pagegraph-query'', whose 74-line README contains no such statement (0 hits for "incomplete"). The sentence is on Brave's wiki, describing the docs.rs types page of the **archived** ''pagegraph-rust''. | **Accepted.** Re-sourced to the Brave wiki and made the archived-project dependency explicit. |
@@ Line 117: / Line 131: @@
 ^ Finding ^ Decision ^
-| The round-1 fixes existed only in the local files; the live site still served the broken ''--debug'' command, and the provenance page's "republished both pages" was therefore not yet true. | **Accepted.** Publication had been deliberately batched until all four passes returned, but the provenance wording was written in the past tense before the save, which is exactly the kind of premature claim this page exists to prevent. Both pages were then published and the rendered result verified. Logged rather than edited away. |
+| The round-1 fixes existed only in the local files; the live site still served the broken ''%%--debug%%'' command, and the provenance page's "republished both pages" was therefore not yet true. | **Accepted.** Publication had been deliberately batched until all four passes returned, but the provenance wording was written in the past tense before the save, which is exactly the kind of premature claim this page exists to prevent. Both pages were then published and the rendered result verified. Logged rather than edited away. |
 | **The corrected crawl command was still broken.** The block used ''\\'' line continuations; in a DokuWiki verbatim block that reaches the shell as an escaped backslash, so the command dies after the first line. Inherited from the original page, not introduced by round 1. | **Accepted.** Replaced with single ''\''. The block was then pasted into ''bash -n'' and executed with a stub: it parses and passes the spaced binary path as one argument. |
-| Backticks are Markdown, not DokuWiki: ~40 spans on the two pages rendered as literal backtick characters. House style on neighbouring pages is ''%%''…''%%''. | **Accepted.** 34 spans on the content page and 90 on this one converted, skipping ''<code>'' and ''<file>'' blocks. |
+| Backticks are Markdown, not DokuWiki: ~40 spans on the two pages rendered as literal backtick characters. House style on neighbouring pages is ''%%''…''%%''. | **Accepted.** 34 spans on the content page and 90 on this one converted, skipping ''%%<code>%%'' and ''%%<file>%%'' blocks. |
 | The Run record contradicted itself: a stale first-sitting sentence ("No reviewer finding was integrated") sat in a list that now also described the second sitting; and the rewrite had quietly dropped who directed the unreviewed publication. | **Accepted.** Sentence scoped to the first sitting, and the actor restored as an attributed claim rather than an asserted fact. |
 | "the fix is permanent" (about ''?purge=true'') contradicted the same section's warning two paragraphs later. | **Accepted.** Corrected to per-render. |
@@ Line 142: / Line 156: @@
 This is not specific to this page. **Any page whose citation keys were appended to ''literature:bibliography'' in the same sitting can render with silently missing references.** Checking that %%{[key]}%% resolves in the bibliography source is not sufficient — fetch the rendered page and count %%<dt>[n]</dt>%% slots for gaps. Both pages here were verified this way after the final save.
+==== A second rendering trap: DokuWiki eats command-line flags ====
+Caught only because the rendered page was re-read after publishing. **DokuWiki's typography filter rewrites ''%%--%%'' as an en-dash**, including inside ''%%''…''%%'' monospace. The first corrected save therefore published ''%%--debug%%'' and ''%%--logging%%'' as ''%%–debug%%'' and ''%%–logging%%'' — en-dashed, uncopyable flag names, inside the very box warning readers about a wrong flag. Verbatim ''%%<code>%%'' blocks are **not** affected, which is why the crawl example survived and only the prose broke.
+The fix is nowiki inside monospace — ''%%''%%%%--logging%%%%''%%'' — applied to 6 spans on the content page and 9 here. Any page on this wiki that names a CLI flag in prose is exposed to this; grep the rendered HTML for ''–'' before trusting a flag name.
 ===== Reproduction artifact =====
@@ Line 217: / Line 237: @@
 USENIX/2025/web-execution-bundles-reproducible-accurate-and-archivable-web-measurements | WebREC storage comparison | source match=true | WebREC .web files averaged 10.2 MB, including 1.5 MB behavior records and 0.4 MB screenshots; the proxy HAR baseline averaged 13.6 MB
-=== Evidence quotes checked against paper.cols.txt ===
+=== Evidence quotes checked against paper.cols.txt, with a pypdf fallback ===
 IMC/2020/hiding-in-plain-site-detecting-javascript-obfuscation-through-concealed-browser | PageGraph | methodology | partial (73%) | "PageGraph complements VV8's low-level tracing of all browser API accesses with high level tracking of script provenance, including via complicated DOM interactions and dynamic script injections."
@@ Line 228: / Line 248: @@
 USENIX/2025/web-execution-bundles-reproducible-accurate-and-archivable-web-measurements | PageGraph | implementation | below-threshold (29%) | "Our approach builds on an existing Chromium-based measurement system called PageGraph"
 IEEE-SP/2021/detecting-filter-list-evasion-with-event-loop-turn-granularity-javascript-signat | PageGraph | methodology | partial (72%) | "To solve this problem, we used and extended PageGraph, a system for representing the execution of a page as a directed graph."
-Summary: 9 quotes checked: 5 exact, 3 partial (>=60% of 5-word windows), 1 below threshold, 0 without full text.
+Summary: 9 quotes checked: 5 exact, 3 partial (>=60% of 5-word windows), 0 rescued from the PDF, 1 below threshold in both renderings, 0 without full text.
-Manual follow-up: the below-threshold WebREC quote is present in the source rendering at the PageGraph paragraph; the repaired .cols text joins "Chromium-based" as "Chromiumbased", which defeats the normalised quote matcher. It is a column-repair mismatch, not an unsupported claim.
+Manual follow-up: the below-threshold WebREC quote is present in the paper at the PageGraph paragraph; raw paper.txt breaks "Chromium-" at the end of a line and resumes with "based", so BOTH paper.cols.txt and the pypdf rendering join it as "Chromiumbased" and the normalised quote matcher scores 29% against each. It is a line-break-hyphen mismatch in the PDF text layer, not a column-repair artefact and not an unsupported claim; no second rendering can rescue it.
 </file>
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Quote-check refresh, 2026-09-22 =====
+The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that
+.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in
+an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs''
+and ''scripts/report_webxray.mjs'' were patched to consult that second rendering and
+report a **rescued** verdict. ''scripts/report_pagegraph.mjs'' had no such fallback, so
+its quote figures were not comparable with the rest of the site's. It has now been
+patched the same way and re-run.
+**The figures did not move.** That is the result, and it is recorded here because a
+refresh that changes nothing is still a refresh that had to be done — until it was run,
+nobody could say whether the 1 below-threshold quote was a stored-text defect or a real
+one.
+<code>
+$ node scripts/report_pagegraph.mjs
+Summary: 9 quotes checked: 5 exact, 3 partial (>=60% of 5-word windows), 0 rescued from the PDF, 1 below threshold in both renderings, 0 without full text.
+</code>
+^ Figure ^ Was ^ Is ^ Why ^
+| quotes checked | 9 | 9 | population unchanged — the corpus has not moved |
+| exact | 5 | 5 | unchanged |
+| partial (≥60% of 5-word windows) | 3 | 3 | unchanged |
+| rescued from the PDF | — | **0** | new verdict; the one candidate is not rescuable, for the reason below |
+| below threshold | 1 | **1** (in both renderings) | unchanged |
+**Why the one candidate cannot be rescued, and why that is worth a paragraph.** The
+WebREC quote scores 29% — 2 of 7 five-word windows — against ''paper.cols.txt'' and
+**29% against ''pypdf''** too. Both renderers see ''Chromium-'' at the end of a line and
+''based'' at the start of the next, and both strip the hyphen, because a hyphen at a line
+break is usually a hyphenation artefact. Here it is a real hyphen in a real compound.
+A second rendering of the same PDF cannot recover information the PDF's own text layer
+does not carry, so this quote would stay below threshold under **any** number of
+renderings. The page previously called this "a column-repair mismatch"; that was the
+right verdict on the quote and the wrong attribution of the cause, and the wording above
+has been corrected.
+**A finding recorded and not acted on.** The matcher's de-hyphenation folds ''%%-%%''
+only when it is followed by whitespace. Folding every intra-word hyphen in the needle
+would rescue this quote, and would also let the matcher accept quotes whose hyphenation
+genuinely differs from the paper's. The fold was **not** widened, and this quote stays
+in the below-threshold bucket with its reason written down. Widening it is a change to
+what the check means and belongs in a sitting of its own.
+**Scope of this edit.** The §"Evidence quotes checked" prose, the embedded
+''report_pagegraph-output.txt'' block regenerated in full from the re-run, and the
+corresponding bullet on [[:programming:crawler:pagegraph]]. Diffing the fresh output
+against the published block found **no other line had moved** — the population query,
+the year table, the role fold and the four source facts all reproduce byte-for-byte.
+No figure outside the quote section was touched, and no ''%%{[citekey]}%%'' was added or moved.
+^ Item ^ Value ^
+| Date | 2026-09-22, unsupervised |
+| Command | ''node scripts/report_pagegraph.mjs'' |
+| Artifact | ''out/qc0922/report_pagegraph.txt'' (the full 76-line run, pasted into the block above) |
+| Script change | ''scripts/report_pagegraph.mjs'': imports ''pdfText'' from ''scripts/pdftext_cache.mjs''; a below-threshold verdict is re-scored against the ''pypdf'' rendering and reported as ''rescued'' at ≥60%; the summary line gained a rescued bucket |
+| Positive control | the fallback was confirmed live, not assumed: ''scripts/pdftext.py'' returns 98,886 characters for the WebREC paper and the same 2-of-7 window hits, so the 0 is a measured zero and not a silently-failed subprocess |
+| Reviewers | one ''sonnet'' figures-vs-script pass, one ''sonnet'' citations pass |
+| Reviewer finding accepted | The figures-vs-script pass caught that ''report_pagegraph.mjs'' still carried the **retracted** explanation in a hardcoded string: its "Manual follow-up" line said //"It is a column-repair mismatch"//, so every re-run reproduced a debunked causal claim underneath the prose that had retracted it. The string is now corrected **in the script**, not on the page, and the embedded block regenerated from the re-run. A figure can be corrected in prose and survive inside the published output that is supposed to prove it |
+| Reviewer finding accepted | The citations pass caught that the ''paper.txt'' excerpt first published here was **not verbatim**: it had been de-interleaved by hand, so grepping ''paper.txt'' for those two lines returns nothing at all. The real three lines are now published with both columns intact, read straight out of the file rather than retyped. The //claim// was right — the hyphen is a line break in the PDF's own text layer — but an excerpt presented as a raw dump has to be a raw dump. Fixed the same day |
+| Pages saved | this page and [[:programming:crawler:pagegraph]] |