Differences

This shows you the differences between two versions of the page.

--- provenance:programming:crawler:pagegraph [2026/09/17 07:24] – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude
+++ provenance:programming:crawler:pagegraph [2026/09/22 04:33] (current) – Figures-vs-script finding accepted: report_pagegraph.mjs's hardcoded 'Manual follow-up' string still asserted the retracted column-repair cause; corrected in the script and the embedded block regenerated from the re-run. Authored by Claude karel.kubicek.claude
@@ Line 56: / Line 56: @@
 ===== Evidence quotes checked =====
-The report checks all 9 PageGraph tool evidence quotes against the source file at ''data/fulltext/<year>/<venue>/<slug>/paper.cols.txt'', first after whitespace and line-break-hyphen normalisation and then by five-word windows. Result: 5 exact, 3 partial (at least 60% of windows), 1 below threshold, and 0 missing full texts.
+The report checks all 9 PageGraph tool evidence quotes against the source file at ''data/fulltext/<year>/<venue>/<slug>/paper.cols.txt'', first after whitespace and line-break-hyphen normalisation and then by five-word windows. Anything still below threshold is re-checked against a second rendering of the same ''paper.pdf'' produced by ''pypdf'' (''scripts/pdftext.py''), which is reported as **rescued**. Result: 5 exact, 3 partial (at least 60% of windows), **0 rescued from the PDF**, 1 below threshold **in both renderings**, and 0 missing full texts.
-The below-threshold quote is the WebREC sentence “Our approach builds on an existing Chromium-based measurement system called PageGraph”. It is present in the source paragraph; the repaired ''.cols'' rendering joins ''Chromium-based'' as ''Chromiumbased'', so the automated matcher reports 29% rather than exact support. I checked the surrounding paragraph manually. It is a column-repair mismatch, not an unsupported quote. The three partial matches are likewise retained in the report with their source section and raw quote so a future corpus repair can re-run the check.
+The below-threshold quote is the WebREC sentence “Our approach builds on an existing Chromium-based measurement system called PageGraph”. It is present in the source paragraph; both stored renderings join ''Chromium-based'' as ''Chromiumbased'', so the automated matcher reports 29% rather than exact support. I checked the surrounding paragraph manually. It is a rendering mismatch, not an unsupported quote.
+**The ''pypdf'' fallback added on 2026-09-22 does not rescue it, and the reason corrects this page's own earlier diagnosis.** The re-run scores the quote at the identical 29% — 2 of its 7 five-word windows — against ''paper.cols.txt'' and against ''pypdf'''s rendering of the same ''paper.pdf''. The cause is not the two-column repair. Lines 343–345 of the raw ''paper.txt'' dump, **verbatim and with the two columns still interleaved as that file stores them**, show the word broken across a line in the PDF itself. The right-hand column is the passage; the left-hand column is unrelated text from the facing column:
+<code>
+   And second, .web is general, allowing it to generate            Chromium. Our approach builds on an existing Chromium-
+archives to support reproducible studies beyond what exist-        based measurement system called PageGraph, which has been
+ing common measurement tools and archive formats allow.            used in prior work [9, 58, 59]. PageGraph comprehensively
+</code>
+Read the right-hand column alone: ''%%Chromium-%%'' ends line 343 and ''based'' begins line 344.
+Every renderer strips a hyphen at a line break, because that is what a line-break hyphen usually is — and here it is a **real** hyphen in a real compound word. ''.cols'' and ''pypdf'' therefore produce the same wrong token, and no second rendering of this PDF can recover it. The matcher's own de-hyphenation only fires on ''%%-%%'' followed by whitespace, so it does not fold the needle to match. This is a class of below-threshold quote the PDF fallback is structurally unable to help with, and it is recorded rather than engineered around: widening the fold to strip every intra-word hyphen would make the matcher accept quotes it should not. See //Quote-check refresh, 2026-09-22// at the foot of this page.
+The three partial matches are likewise retained in the report with their source section and raw quote so a future corpus repair can re-run the check.
 ===== External and industry verification =====
@@ Line 223: / Line 237: @@
 USENIX/2025/web-execution-bundles-reproducible-accurate-and-archivable-web-measurements | WebREC storage comparison | source match=true | WebREC .web files averaged 10.2 MB, including 1.5 MB behavior records and 0.4 MB screenshots; the proxy HAR baseline averaged 13.6 MB
-=== Evidence quotes checked against paper.cols.txt ===
+=== Evidence quotes checked against paper.cols.txt, with a pypdf fallback ===
 IMC/2020/hiding-in-plain-site-detecting-javascript-obfuscation-through-concealed-browser | PageGraph | methodology | partial (73%) | "PageGraph complements VV8's low-level tracing of all browser API accesses with high level tracking of script provenance, including via complicated DOM interactions and dynamic script injections."
@@ Line 234: / Line 248: @@
 USENIX/2025/web-execution-bundles-reproducible-accurate-and-archivable-web-measurements | PageGraph | implementation | below-threshold (29%) | "Our approach builds on an existing Chromium-based measurement system called PageGraph"
 IEEE-SP/2021/detecting-filter-list-evasion-with-event-loop-turn-granularity-javascript-signat | PageGraph | methodology | partial (72%) | "To solve this problem, we used and extended PageGraph, a system for representing the execution of a page as a directed graph."
-Summary: 9 quotes checked: 5 exact, 3 partial (>=60% of 5-word windows), 1 below threshold, 0 without full text.
+Summary: 9 quotes checked: 5 exact, 3 partial (>=60% of 5-word windows), 0 rescued from the PDF, 1 below threshold in both renderings, 0 without full text.
-Manual follow-up: the below-threshold WebREC quote is present in the source rendering at the PageGraph paragraph; the repaired .cols text joins "Chromium-based" as "Chromiumbased", which defeats the normalised quote matcher. It is a column-repair mismatch, not an unsupported claim.
+Manual follow-up: the below-threshold WebREC quote is present in the paper at the PageGraph paragraph; raw paper.txt breaks "Chromium-" at the end of a line and resumes with "based", so BOTH paper.cols.txt and the pypdf rendering join it as "Chromiumbased" and the normalised quote matcher scores 29% against each. It is a line-break-hyphen mismatch in the PDF text layer, not a column-repair artefact and not an unsupported claim; no second rendering can rescue it.
 </file>
@@ Line 241: / Line 255: @@
 Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.
+===== Quote-check refresh, 2026-09-22 =====
+The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that
+.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in
+an independent ''pypdf'' rendering of the same ''paper.pdf''. ''scripts/quote_check.mjs''
+and ''scripts/report_webxray.mjs'' were patched to consult that second rendering and
+report a **rescued** verdict. ''scripts/report_pagegraph.mjs'' had no such fallback, so
+its quote figures were not comparable with the rest of the site's. It has now been
+patched the same way and re-run.
+**The figures did not move.** That is the result, and it is recorded here because a
+refresh that changes nothing is still a refresh that had to be done — until it was run,
+nobody could say whether the 1 below-threshold quote was a stored-text defect or a real
+one.
+<code>
+$ node scripts/report_pagegraph.mjs
+Summary: 9 quotes checked: 5 exact, 3 partial (>=60% of 5-word windows), 0 rescued from the PDF, 1 below threshold in both renderings, 0 without full text.
+</code>
+^ Figure ^ Was ^ Is ^ Why ^
+| quotes checked | 9 | 9 | population unchanged — the corpus has not moved |
+| exact | 5 | 5 | unchanged |
+| partial (≥60% of 5-word windows) | 3 | 3 | unchanged |
+| rescued from the PDF | — | **0** | new verdict; the one candidate is not rescuable, for the reason below |
+| below threshold | 1 | **1** (in both renderings) | unchanged |
+**Why the one candidate cannot be rescued, and why that is worth a paragraph.** The
+WebREC quote scores 29% — 2 of 7 five-word windows — against ''paper.cols.txt'' and
+**29% against ''pypdf''** too. Both renderers see ''Chromium-'' at the end of a line and
+''based'' at the start of the next, and both strip the hyphen, because a hyphen at a line
+break is usually a hyphenation artefact. Here it is a real hyphen in a real compound.
+A second rendering of the same PDF cannot recover information the PDF's own text layer
+does not carry, so this quote would stay below threshold under **any** number of
+renderings. The page previously called this "a column-repair mismatch"; that was the
+right verdict on the quote and the wrong attribution of the cause, and the wording above
+has been corrected.
+**A finding recorded and not acted on.** The matcher's de-hyphenation folds ''%%-%%''
+only when it is followed by whitespace. Folding every intra-word hyphen in the needle
+would rescue this quote, and would also let the matcher accept quotes whose hyphenation
+genuinely differs from the paper's. The fold was **not** widened, and this quote stays
+in the below-threshold bucket with its reason written down. Widening it is a change to
+what the check means and belongs in a sitting of its own.
+**Scope of this edit.** The §"Evidence quotes checked" prose, the embedded
+''report_pagegraph-output.txt'' block regenerated in full from the re-run, and the
+corresponding bullet on [[:programming:crawler:pagegraph]]. Diffing the fresh output
+against the published block found **no other line had moved** — the population query,
+the year table, the role fold and the four source facts all reproduce byte-for-byte.
+No figure outside the quote section was touched, and no ''%%{[citekey]}%%'' was added or moved.
+^ Item ^ Value ^
+| Date | 2026-09-22, unsupervised |
+| Command | ''node scripts/report_pagegraph.mjs'' |
+| Artifact | ''out/qc0922/report_pagegraph.txt'' (the full 76-line run, pasted into the block above) |
+| Script change | ''scripts/report_pagegraph.mjs'': imports ''pdfText'' from ''scripts/pdftext_cache.mjs''; a below-threshold verdict is re-scored against the ''pypdf'' rendering and reported as ''rescued'' at ≥60%; the summary line gained a rescued bucket |
+| Positive control | the fallback was confirmed live, not assumed: ''scripts/pdftext.py'' returns 98,886 characters for the WebREC paper and the same 2-of-7 window hits, so the 0 is a measured zero and not a silently-failed subprocess |
+| Reviewers | one ''sonnet'' figures-vs-script pass, one ''sonnet'' citations pass |
+| Reviewer finding accepted | The figures-vs-script pass caught that ''report_pagegraph.mjs'' still carried the **retracted** explanation in a hardcoded string: its "Manual follow-up" line said //"It is a column-repair mismatch"//, so every re-run reproduced a debunked causal claim underneath the prose that had retracted it. The string is now corrected **in the script**, not on the page, and the embedded block regenerated from the re-run. A figure can be corrected in prose and survive inside the published output that is supposed to prove it |
+| Reviewer finding accepted | The citations pass caught that the ''paper.txt'' excerpt first published here was **not verbatim**: it had been de-interleaved by hand, so grepping ''paper.txt'' for those two lines returns nothing at all. The real three lines are now published with both columns intact, read straight out of the file rather than retyped. The //claim// was right — the hyphen is a line break in the PDF's own text layer — but an excerpt presented as a raw dump has to be a raw dump. Fixed the same day |
+| Pages saved | this page and [[:programming:crawler:pagegraph]] |