Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revision | |
| programming:crawler:pagegraph [2026/08/14 18:16] – Review-layer fixes: repair broken crawl command (dead --debug flag, \\ line continuations), add elm subcommand, re-source doc-incompleteness claim, full 9-item limitations list, Shields-state warning, runtime-cost admission, VisibleV8 comparison row, AdGr karel.kubicek.claude | programming:crawler:pagegraph [2026/08/14 18:16] (current) – Escape CLI flags with nowiki: DokuWiki typography was converting -- into an en-dash, printing --debug/--logging as en-dashed flags. Authored by Claude karel.kubicek.claude |
|---|
| |
| <WRAP important> | <WRAP important> |
| Do not copy the ''--debug debug'' example that ''pagegraph-crawl'''s own README still shows. That flag no longer exists: the current argument parser accepts ''--logging {none,info,verbose}'' (default ''info''), and ''--debug debug'' exits with ''run.js: error: unrecognized arguments: --debug debug''. Verified against ''src/run.ts'' at version 1.2.13 on 2026-08-14. It is a good reminder to run any documented command before putting it in a paper's artifact appendix. | Do not copy the ''%%--debug debug%%'' example that ''pagegraph-crawl'''s own README still shows. That flag no longer exists: the current argument parser accepts ''%%--logging {none,info,verbose}%%'' (default ''info''), and ''%%--debug debug%%'' exits with ''%%run.js: error: unrecognized arguments: --debug debug%%''. Verified against ''src/run.ts'' at version 1.2.13 on 2026-08-14. It is a good reminder to run any documented command before putting it in a paper's artifact appendix. |
| </WRAP> | </WRAP> |
| |
| <WRAP important> | <WRAP important> |
| **Record the Shields state, and do not assume it.** PageGraph runs inside Brave, which ships an ad and tracker blocker, so "what the page did" depends on whether that blocker was on. The trap is that the two obvious ways to run it disagree: ''pagegraph-crawl'' defaults to ''-s down'' — Shields **off**, so a default crawl sees an essentially unfiltered web — whereas a Brave you launch yourself has Shields **on** by default. A prevalence figure means something different under each, and ''--shields'' is ignored entirely when ''--existing-user-data-dir'' is used. Verified against ''src/run.ts'' (''defaultShieldsSetting = "down"'') on 2026-08-14. | **Record the Shields state, and do not assume it.** PageGraph runs inside Brave, which ships an ad and tracker blocker, so "what the page did" depends on whether that blocker was on. The trap is that the two obvious ways to run it disagree: ''pagegraph-crawl'' defaults to ''-s down'' — Shields **off**, so a default crawl sees an essentially unfiltered web — whereas a Brave you launch yourself has Shields **on** by default. A prevalence figure means something different under each, and ''%%--shields%%'' is ignored entirely when ''%%--existing-user-data-dir%%'' is used. Verified against ''src/run.ts'' (''defaultShieldsSetting = "down"'') on 2026-08-14. |
| </WRAP> | </WRAP> |
| |
