| |
| provenance:programming:docker [2026/08/27 12:45] – Create provenance:programming:docker query log. Authored by Claude karel.kubicek.claude | provenance:programming:docker [2026/09/17 07:24] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude |
|---|
| * Read first: the task spec (short, practical: reproducible crawler images, pin browser versions, ''/dev/shm'', Xvfb), ''programming:multilingual_support'' (already links here; LibreTranslate compose uses '':latest''), ''programming:crawler'', ''programming:crawler:openwpm'', ''design:longitudinal'' (0 digest mentions, 293 Docker mentions), ''literature:corpus'', Docker Engine run docs, Playwright Docker/CI docs, Chromium ''base/base_switches.h'', OpenWPM README, Hub APIs. | * Read first: the task spec (short, practical: reproducible crawler images, pin browser versions, ''/dev/shm'', Xvfb), ''programming:multilingual_support'' (already links here; LibreTranslate compose uses '':latest''), ''programming:crawler'', ''programming:crawler:openwpm'', ''design:longitudinal'' (0 digest mentions, 293 Docker mentions), ''literature:corpus'', Docker Engine run docs, Playwright Docker/CI docs, Chromium ''base/base_switches.h'', OpenWPM README, Hub APIs. |
| * Target page had no revision (creation). Overlap judgement: create the promised child rather than widen multilingual_support or OpenWPM. Those pages keep their own compose / OpenWPM flags; this page owns digest + shm + Xvfb for a generic crawler image. | * Target page had no revision (creation). Overlap judgement: create the promised child rather than widen multilingual_support or OpenWPM. Those pages keep their own compose / OpenWPM flags; this page owns digest + shm + Xvfb for a generic crawler image. |
| * No write to the publication mount. Wiki saves through ''scripts/dw.mjs'' with ''--if-rev'' on existing pages. New pages omit ''--if-rev''. | * No write to the publication mount. Wiki saves through ''scripts/dw.mjs'' with ''%%--if-rev%%'' on existing pages. New pages omit ''%%--if-rev%%''. |
| * This host: aarch64, no Docker/Podman binary, ''/dev/shm'' 2.0G, Xvfb present, ''xauth'' absent so ''xvfb-run'' exits 3. Digests fetched via registry HTTP, not ''docker pull''. | * This host: aarch64, no Docker/Podman binary, ''/dev/shm'' 2.0G, Xvfb present, ''xauth'' absent so ''xvfb-run'' exits 3. Digests fetched via registry HTTP, not ''docker pull''. |
| |
| ^ Neighbour ^ What it already answers ^ What it does not ^ | ^ Neighbour ^ What it already answers ^ What it does not ^ |
| | [[programming:multilingual_support]] | LibreTranslate compose, ''docker compose up'' | crawler shm, digest pins, Xvfb; the compose uses '':latest'' | | | [[programming:multilingual_support]] | LibreTranslate compose, ''docker compose up'' | crawler shm, digest pins, Xvfb; the compose uses '':latest'' | |
| | [[programming:crawler:openwpm]] | OpenWPM image, ''--shm-size=2g'', ''--init'', ''display_mode'' | Playwright/Chromium compose; corpus Docker-role split | | | [[programming:crawler:openwpm]] | OpenWPM image, ''%%--shm-size%%=2g'', ''%%--init%%'', ''display_mode'' | Playwright/Chromium compose; corpus Docker-role split | |
| | [[design:longitudinal]] | 0 image-digest papers; 293 ''/\bdocker\b/i'' | how to pin, how to size shm, Xvfb | | | [[design:longitudinal]] | 0 image-digest papers; 293 ''/\bdocker\b/i'' | how to pin, how to size shm, Xvfb | |
| | [[programming:crawler]] | which library | wrapping that library in a container | | | [[programming:crawler]] | which library | wrapping that library in a container | |
| ===== Evidence quotes checked ===== | ===== Evidence quotes checked ===== |
| |
| ''node scripts/report_docker.mjs --quotes'' re-checks every ROLE and XVFB_ROLE deciding sentence against ''paper.cols.txt''. After replacing a spliced Cookie-law quote with the contiguous ''.cols'' string ("CookieCheck is a scalable system based on Docker"), the run printed **0 MISS**. Content-page quotes (Retana Playwright image, Borgolte reproducibility, Bui Swarm/resource-constrained) are ROLE sentences and were OK. | ''node scripts/report_docker.mjs %%--quotes%%'' re-checks every ROLE and XVFB_ROLE deciding sentence against ''paper.cols.txt''. After replacing a spliced Cookie-law quote with the contiguous ''.cols'' string ("CookieCheck is a scalable system based on Docker"), the run printed **0 MISS**. Content-page quotes (Retana Playwright image, Borgolte reproducibility, Bui Swarm/resource-constrained) are ROLE sentences and were OK. |
| |
| OpenWPM README and Playwright Docker sentences on the content page are external; they were copied from the 2026-08-27 fetches, not from the corpus. | OpenWPM README and Playwright Docker sentences on the content page are external; they were copied from the 2026-08-27 fetches, not from the corpus. |
| |
| ^ Source ^ Load-bearing fact ^ Verification 2026-08-27 ^ Decision ^ | ^ Source ^ Load-bearing fact ^ Verification 2026-08-27 ^ Decision ^ |
| | docs.docker.com/engine/containers/run/ | omitted ''--shm-size'' default is 64m | ''docker_probe.sh'' grepped 64m and shm-size | Used | | | docs.docker.com/engine/containers/run/ | omitted ''%%--shm-size%%'' default is 64m | ''docker_probe.sh'' grepped 64m and shm-size | Used | |
| | chromium ''base/base_switches.h'' (googlesource ''?format=TEXT'') | ''kDisableDevShmUsage''; Linux-only; temp dir for anonymous shm files; crbug 715363 | header, not ''.cc'' (GitHub chromium/chromium 404s this path) | Used | | | chromium ''base/base_switches.h'' (googlesource ''?format=TEXT'') | ''kDisableDevShmUsage''; Linux-only; temp dir for anonymous shm files; crbug 715363 | header, not ''.cc'' (GitHub chromium/chromium 404s this path) | Used | |
| | OpenWPM README | ''--shm-size=2g'' required; Firefox crashes on 20-30% of sites without it; ''--init'' | raw.githubusercontent.com | Used | | | OpenWPM README | ''%%--shm-size%%=2g'' required; Firefox crashes on 20-30% of sites without it; ''%%--init%%'' | raw.githubusercontent.com | Used | |
| | playwright.dev/docs/docker | tag ''v1.62.0-noble''; ''--ipc=host''; ''--init''; ''pwuser'' + seccomp; untrusted-websites warning | fetched | Used | | | playwright.dev/docs/docker | tag ''v1.62.0-noble''; ''%%--ipc%%=host''; ''%%--init%%''; ''pwuser'' + seccomp; untrusted-websites warning | fetched | Used | |
| | playwright.dev/docs/ci | headed Linux: ''xvfb-run npx playwright test'' | fetched | Used | | | playwright.dev/docs/ci | headed Linux: ''xvfb-run npx playwright test'' | fetched | Used | |
| | MCR registry v2 | index digest ''sha256:baed2032d533817f3dbe6425de795788430ba345e819a1201337009ba17c9d07'', 2 manifests | ''Docker-Content-Digest'' header; MAR HTML is JS-only | Used as the example pin | | | MCR registry v2 | index digest ''sha256:baed2032d533817f3dbe6425de795788430ba345e819a1201337009ba17c9d07'', 2 manifests | ''Docker-Content-Digest'' header; MAR HTML is JS-only | Used as the example pin | |
| ===== What the corpus and sources do not establish ===== | ===== What the corpus and sources do not establish ===== |
| |
| * How many of the 41 actually passed ''--shm-size'' or a digest in unpublished scripts. | * How many of the 41 actually passed ''%%--shm-size%%'' or a digest in unpublished scripts. |
| * Whether Playwright's vendor image is acceptable under a given IRB/ethics review for live-web crawls. The docs sentence is the fact; the policy call is the reader's. | * Whether Playwright's vendor image is acceptable under a given IRB/ethics review for live-web crawls. The docs sentence is the fact; the policy call is the reader's. |
| * Tomorrow's Playwright tag digest. | * Tomorrow's Playwright tag digest. |
| * Keep the page short. Venue/year tables stay in the report, not on the content page. | * Keep the page short. Venue/year tables stay in the report, not on the content page. |
| * Hand-role the 81 rather than publish the sweep as "used Docker for crawling." | * Hand-role the 81 rather than publish the sweep as "used Docker for crawling." |
| * Checker accepts ''shm_size'' **or** ''ipc: host'', not Chromium flags. A methods sentence is required if you only pass ''--disable-dev-shm-usage''. | * Checker accepts ''shm_size'' **or** ''ipc: host'', not Chromium flags. A methods sentence is required if you only pass ''%%--disable-dev-shm-usage%%''. |
| * Do not "fix" multilingual_support's '':latest'' in this sitting; point at it. | * Do not "fix" multilingual_support's '':latest'' in this sitting; point at it. |
| * Reuse ''borgolte2020_performance'' already in the bibliography; add ''retana2025_permissions'' and ''bui2023_detection''. | * Reuse ''borgolte2020_performance'' already in the bibliography; add ''retana2025_permissions'' and ''bui2023_detection''. |
| |
| * ''pages/crawler.compose.yaml'' — example; digest fetched 2026-08-27. | * ''pages/crawler.compose.yaml'' — example; digest fetched 2026-08-27. |
| * ''pages/check_crawler_compose.py'' — stdlib; ''--demo'' self-test (bad fixture exit 1, good exit 0, overall 0); missing digest or shm knob prints ''FAILURE'' and exits 1. Advertised flags were run: compose path, ''--demo'', ''--all''. | * ''pages/check_crawler_compose.py'' — stdlib; ''%%--demo%%'' self-test (bad fixture exit 1, good exit 0, overall 0); missing digest or shm knob prints ''FAILURE'' and exits 1. Advertised flags were run: compose path, ''%%--demo%%'', ''%%--all%%''. |
| * Embed bytes on the wiki page were diffed equal to those files before freeze. | * Embed bytes on the wiki page were diffed equal to those files before freeze. |
| |
| |
| ^ # ^ Finding ^ Decision ^ | ^ # ^ Finding ^ Decision ^ |
| | 1 | Bui quote used “resource-constrained”; ''.cols'' has “resourceconstrained”. | **Accepted.** Page, ROLE map, and report ''--quotes'' now use the ''.cols'' spelling. Citekeys and first authors were OK. | | | 1 | Bui quote used “resource-constrained”; ''.cols'' has “resourceconstrained”. | **Accepted.** Page, ROLE map, and report ''%%--quotes%%'' now use the ''.cols'' spelling. Citekeys and first authors were OK. | |
| |
| ==== Reviewer 3 — external currency (GPT 5.6 Sol medium) ==== | ==== Reviewer 3 — external currency (GPT 5.6 Sol medium) ==== |
| | 3 | Checker accepted ''shm_size: 64m''. | **Accepted.** Values not strictly above 64 MiB now fail. | | | 3 | Checker accepted ''shm_size: 64m''. | **Accepted.** Values not strictly above 64 MiB now fail. | |
| | 4 | Nested ''build.shm_size'' satisfied the runtime check. | **Accepted.** Only four-space service properties count. | | | 4 | Nested ''build.shm_size'' satisfied the runtime check. | **Accepted.** Only four-space service properties count. | |
| | 5 | ''--all'' demanded shm on LibreTranslate. | **Accepted.** ''--all'' digest-pins every service; shm remains crawler-only. | | | 5 | ''%%--all%%'' demanded shm on LibreTranslate. | **Accepted.** ''%%--all%%'' digest-pins every service; shm remains crawler-only. | |
| | 6 | Example omits Playwright seccomp. | **Rejected inlining** a vendor seccomp JSON we do not maintain. **Accepted a label:** digest/shm skeleton, not a hardened sandbox. | | | 6 | Example omits Playwright seccomp. | **Rejected inlining** a vendor seccomp JSON we do not maintain. **Accepted a label:** digest/shm skeleton, not a hardened sandbox. | |
| | 7 | Provenance omitted the 40 non-crawler ROLE keys. | **Accepted in part.** Report section 6b now lists all 81 keys and roles. The deciding sentences for non-crawler roles stay in ''scripts/docker_fold.mjs'' rather than a second 40-quote dump. | | | 7 | Provenance omitted the 40 non-crawler ROLE keys. | **Accepted in part.** Report section 6b now lists all 81 keys and roles. The deciding sentences for non-crawler roles stay in ''scripts/docker_fold.mjs'' rather than a second 40-quote dump. | |
| docker_probe.sh: all checks printed OK | docker_probe.sh: all checks printed OK |
| </file> | </file> |
| | |
| | ===== Markup sweep, 2026-09-17 ===== |
| | |
| | Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving. |
| |
