Table of Contents
Provenance: Programming:Docker
Back to Docker. Corpus-wide selection and extraction notes are on corpus. This is the page-specific query log.
Run record
- Run date: 2026-08-27 (UTC).
- Authoring agent: Cursor Grok 4.6, executing drain item
programming:docker (new)(store id 177, run 54, claimed bycursor-drain-docker) directly rather than via a headlessclaude -p/drain-sandbox.shsession. - Review models requested by the sitting: GPT 5.6 Luna max (
gpt-5.6-luna-max) for the generic pass, luna@medium for the three focused passes. Those slugs are not in this session's Task allowlist. Available:inherit,claude-opus-5-thinking-high,composer-2.5-fast,cursor-grok-4.5-high,cursor-grok-4.6-high,gpt-5.6-sol-medium. Reviews below ran on GPT 5.6 Sol medium (gpt-5.6-sol-medium), the only GPT 5.6 slug offered. This is a documented substitute, not a silent one. - Corpus at run time: 5,859 extracted papers, 2010–2026, CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P. Read-only inputs under
/workspace/publications_dataset/data/. - Read first: the task spec (short, practical: reproducible crawler images, pin browser versions,
/dev/shm, Xvfb),programming:multilingual_support(already links here; LibreTranslate compose uses:latest),programming:crawler,programming:crawler:openwpm,design:longitudinal(0 digest mentions, 293 Docker mentions),literature:corpus, Docker Engine run docs, Playwright Docker/CI docs, Chromiumbase/base_switches.h, OpenWPM README, Hub APIs. - Target page had no revision (creation). Overlap judgement: create the promised child rather than widen multilingual_support or OpenWPM. Those pages keep their own compose / OpenWPM flags; this page owns digest + shm + Xvfb for a generic crawler image.
- No write to the publication mount. Wiki saves through
scripts/dw.mjswith–if-revon existing pages. New pages omit–if-rev. - This host: aarch64, no Docker/Podman binary,
/dev/shm2.0G, Xvfb present,xauthabsent soxvfb-runexits 3. Digests fetched via registry HTTP, notdocker pull.
Why this page, not an overlap
| Neighbour | What it already answers | What it does not |
|---|---|---|
| multilingual_support | LibreTranslate compose, docker compose up | crawler shm, digest pins, Xvfb; the compose uses :latest |
| openwpm | OpenWPM image, –shm-size=2g, –init, display_mode | Playwright/Chromium compose; corpus Docker-role split |
| longitudinal | 0 image-digest papers; 293 /\bdocker\b/i | how to pin, how to size shm, Xvfb |
| crawler | which library | wrapping that library in a container |
Population and queries
All counts are papers unless labelled tuples. Sentinels are not answers. Four corpus papers have no paper.cols.txt; zero crawled papers are missing one.
| Query | Denominator | Result |
|---|---|---|
/\bdocker\b/i over full text | 5,859 corpus | 293 (5.0%), miss=4 |
| Same | 1,120 crawled | 81 (7.2%), miss=0 |
| ROLE hand map over the 81 | 81 | crawler 41, sandbox 20, target 8, artifact 6, citation 6. Residue 0 both directions. |
| PAGE POPULATION: ROLE=crawler | 1,120 crawled | 41 (3.7%) |
tools[].name /docker/i among crawled | 1,120 | 56; version stated 5 (8.9%); 36 of 41 crawler-role also in tools[]; schema misses 5 |
| Exact string “Docker” on that name field | 56 | 53 (undercount) |
SHM_RE /dev/shm OR shm-size OR disable-dev-shm-usage | 1,120 crawled | 0 |
| Same | 5,859 | 0 |
| DIGEST_RE image digest OR sha256: + 12 hex | 1,120 crawled | 0 |
| Same | 5,859 | 1, a homograph (USENIX 2023 perceptual hashing, “illicit image digests”). Published: 0 papers name a container image digest. |
| Dockerfile | 1,120 crawled | 11 (1.0%) |
| docker-compose / compose.yaml | 1,120 crawled | 8 (0.7%) |
| XVFB_RE | 1,120 crawled | 13 (1.2%); used 12, recommend 1 |
| crawler-role ∩ xvfb | 41 | 7 (17.1%) |
| xvfb used, not in docker sweep | 13 | 5 |
crawler-role crawlConfig.headless stated | 41 | 15 (36.6%); crawled overall 140/1120 (12.5%) |
| crawler-role automation-tool version | 41 | 2 (4.9%) |
| playwright…docker window among crawled | 1,120 | 2 (Retana load-bearing; Bui bibliography URL) |
Year 2017 is the first crawler-role paper. 2025–2026 year shares are starred in the report.
Folding and residue
Membership is /\bdocker\b/i over crawled full text. ROLE is a hand classification, not a regex fold: crawler / target / sandbox / artifact / citation, each with the deciding sentence. The report prints FAILURE and exits 1 if the sweep and the map diverge in either direction. Residue: 0.
XVFB_ROLE is the same discipline over the 13 xvfb hits (used / recommend). Residue: 0.
Rejected probe: publishing the raw 293 as “crawler Docker papers”. 60% of crawled Docker mentions are not the crawler image (sandbox/target/artifact/citation). The 41 is the page population.
Rejected probe: the 1 corpus sha256: hit as a digest pin. It is a perceptual-hashing homograph. Published figure is 0.
Rejected probe: schema tools[].name /docker/i as the population. It misses 5 crawler-role papers and includes papers that are not crawler-role.
Evidence quotes checked
node scripts/report_docker.mjs –quotes re-checks every ROLE and XVFB_ROLE deciding sentence against paper.cols.txt. After replacing a spliced Cookie-law quote with the contiguous .cols string (“CookieCheck is a scalable system based on Docker”), the run printed 0 MISS. Content-page quotes (Retana Playwright image, Borgolte reproducibility, Bui Swarm/resource-constrained) are ROLE sentences and were OK.
OpenWPM README and Playwright Docker sentences on the content page are external; they were copied from the 2026-08-27 fetches, not from the corpus.
External and industry verification
| Source | Load-bearing fact | Verification 2026-08-27 | Decision |
|---|---|---|---|
| docs.docker.com/engine/containers/run/ | omitted –shm-size default is 64m | docker_probe.sh grepped 64m and shm-size | Used |
chromium base/base_switches.h (googlesource ?format=TEXT) | kDisableDevShmUsage; Linux-only; temp dir for anonymous shm files; crbug 715363 | header, not .cc (GitHub chromium/chromium 404s this path) | Used |
| OpenWPM README | –shm-size=2g required; Firefox crashes on 20-30% of sites without it; –init | raw.githubusercontent.com | Used |
| playwright.dev/docs/docker | tag v1.62.0-noble; –ipc=host; –init; pwuser + seccomp; untrusted-websites warning | fetched | Used |
| playwright.dev/docs/ci | headed Linux: xvfb-run npx playwright test | fetched | Used |
| MCR registry v2 | index digest sha256:baed2032d533817f3dbe6425de795788430ba345e819a1201337009ba17c9d07, 2 manifests | Docker-Content-Digest header; MAR HTML is JS-only | Used as the example pin |
| Hub libretranslate/libretranslate:latest | last_updated 2026-08-23; amd64 and arm64 digests | Hub API | Used as the sibling :latest lesson |
| Hub openwpm/openwpm | exists; last_updated 2026-08-24; pull_count 32310 | Hub API | Named, not a figure on the content page |
| this host | no docker; Xvfb yes; xauth no; xvfb-run exit 3; raw Xvfb :98 works | executed | Used, including the xauth failure |
Rejected:
- GitHub
chromium/chromiummirror for the shm flag — 404. - MAR tags HTML page — JS-only; used registry API instead.
- Running the example compose — no daemon. Checker is offline.
xvfb-runas a command that succeeded here — it did not.- Claiming 0 digest papers means 0 papers pinned a digest in a repo. Reporting figure only.
- LibreTranslate last_updated as a crawler fact — it is the sibling compose's pin lesson.
What the corpus and sources do not establish
- How many of the 41 actually passed
–shm-sizeor a digest in unpublished scripts. - Whether Playwright's vendor image is acceptable under a given IRB/ethics review for live-web crawls. The docs sentence is the fact; the policy call is the reader's.
- Tomorrow's Playwright tag digest.
- A crash rate for Chromium at 64m
/dev/shmin this corpus (0 papers name the knob). OpenWPM's 20-30% is Firefox, README, not a corpus measurement.
Judgement calls
- Keep the page short. Venue/year tables stay in the report, not on the content page.
- Hand-role the 81 rather than publish the sweep as “used Docker for crawling.”
- Checker accepts
shm_sizeoripc: host, not Chromium flags. A methods sentence is required if you only pass–disable-dev-shm-usage. - Do not “fix” multilingual_support's
:latestin this sitting; point at it. - Reuse
borgolte2020_performancealready in the bibliography; addretana2025_permissionsandbui2023_detection. - Document the review-model substitute (Luna unavailable).
Published code
pages/crawler.compose.yaml— example; digest fetched 2026-08-27.pages/check_crawler_compose.py— stdlib;–demoself-test (bad fixture exit 1, good exit 0, overall 0); missing digest or shm knob printsFAILUREand exits 1. Advertised flags were run: compose path,–demo,–all.- Embed bytes on the wiki page were diffed equal to those files before freeze.
Review log
Drafts frozen under out/freeze_docker/ before the three focused passes. The content page was not edited while those reviewers ran. Requested review models were GPT 5.6 Luna max / luna@medium; this session only offers gpt-5.6-sol-medium among GPT 5.6 slugs, and that is what ran.
Reviewer 1 — figures vs script (GPT 5.6 Sol medium)
| # | Finding | Decision |
|---|---|---|
| 1 | Corpus-wide 293/5,859 and “0 of 5,859 name a digest” overstate coverage: 4 papers have no .cols and the sweep skips them. | Accepted. WRAP now says 5,855 readable (4 of 5,859 missing .cols); corpus paragraph names miss=4. 5.0% remains 293/5859 as the report prints it. |
| 2 | “Checker that fails if you omit them” included headed/Xvfb, which the checker does not validate. | Accepted. Intro bullet now: checker fails on unpin / 64m shm; it does not check headed. |
| 3 | “pwuser plus a seccomp profile. Those are in the example.” Compose has pwuser, no seccomp. | Accepted. (Same as reviewer 3.1.) Example sets init and pwuser; seccomp is a host file, not inlined. |
| 4 | DIGEST_RE used a word boundary, so @sha256:<64 hex>-garbage pinned. | Accepted. Pin regex is now @sha256:[0-9a-f]{64}$ on the whole image token. Malformed Playwright image exits 1, pin=unpinned. |
| 5 | Direct x.name (the draft used a fallback empty string and would swallow a missing tools-name). | Accepted. Direct x.name. Counts unchanged. |
Reviewer 2 — citations and quotes (GPT 5.6 Sol medium)
| # | Finding | Decision |
|---|---|---|
| 1 | Bui quote used “resource-constrained”; .cols has “resourceconstrained”. | Accepted. Page, ROLE map, and report –quotes now use the .cols spelling. Citekeys and first authors were OK. |
Reviewer 3 — external currency (GPT 5.6 Sol medium)
| # | Finding | Decision |
|---|---|---|
| 1 | Seccomp claimed in the example, absent. | Accepted. Same fix as 1.3. |
| 2 | crbug.com/715363 redirects to a Google sign-in wall. | Accepted as a caveat. Dropped the live bugs.chromium.org link; the Chromium source comment still cites 715363 (re-fetched). |
| 3–11 | 64m default, shm flag, OpenWPM README, Playwright docs/tag/digest, LibreTranslate Hub, host Xvfb/xauth, probe script. | Held. Re-fetched; no edit. |
Reviewer 4 — generic, no checklist (GPT 5.6 Sol medium)
Re-run of reviewer 1 after focused fixes: no leftover defects.
| # | Finding | Decision |
|---|---|---|
| 1 | Index digest without platform: is not architecture-reproducible. | Rejected as blocking. An index digest freezes both child manifests; Compose without platform: selects the host arch of that frozen pair. Accepted as a methods bullet: report uname -m. Hard-coding platform: linux/amd64 on this aarch64 host would be a footgun. |
| 2 | npx playwright test plus a bind mount does not pin the Playwright package. | Accepted as a methods warning, not as a lockfile in the 15-line example. Named --offline / --no-install. |
| 3 | Checker accepted shm_size: 64m. | Accepted. Values not strictly above 64 MiB now fail. |
| 4 | Nested build.shm_size satisfied the runtime check. | Accepted. Only four-space service properties count. |
| 5 | –all demanded shm on LibreTranslate. | Accepted. –all digest-pins every service; shm remains crawler-only. |
| 6 | Example omits Playwright seccomp. | Rejected inlining a vendor seccomp JSON we do not maintain. Accepted a label: digest/shm skeleton, not a hardened sandbox. |
| 7 | Provenance omitted the 40 non-crawler ROLE keys. | Accepted in part. Report section 6b now lists all 81 keys and roles. The deciding sentences for non-crawler roles stay in scripts/docker_fold.mjs rather than a second 40-quote dump. |
| 8 | Page is not short enough. | Rejected. The 41/81 split is the load-bearing corpus finding; the checker is the practical artifact. Tranco-length would drop both. |
| 9 | Provenance review-log cell had a DokuWiki pipe in \|\|. | Accepted. Reworded the cell. |
| 10 | Provenance source table is not fully linked. | Rejected this sitting. Primary sources are named and were re-fetched by docker_probe.sh; adding a second URL column is nice-to-have, not a hole in the content page. |
No GENERIC_REVIEW placeholder. The content page was not edited while reviewers 1–3 or 4 ran; fixes landed after each layer returned.
Report script output (unedited)
The command is node scripts/report_docker.mjs. Output of the run that the content page was written from:
======================================================================
0. CORPUS
======================================================================
corpus 5859
crawled (crawlConfig or automated-web-crawl) 1120
missing paper.cols.txt corpus 4
missing paper.cols.txt crawled 0
ROLE map size 81
crawled docker sweep 81 miss=0
ROLE vs sweep OK both directions
======================================================================
1. POPULATION — crawled papers matching /\bdocker\b/i
======================================================================
PAGE SWEEP: crawled ∩ /\bdocker\b/i 81 7.2% of 1120 crawled
same regex over the 5859-paper corpus 293 5.0% miss=4
crawled docker is 27.6% of all docker mentions
ROLE over the crawled docker sweep (papers, not tuples):
Role Papers Share of 81 Share of 1120 crawled
-------- ------ ----------- ---------------------
crawler 41 50.6% 3.7%
target 8 9.9% 0.7%
sandbox 20 24.7% 1.8%
artifact 6 7.4% 0.5%
citation 6 7.4% 0.5%
PAGE POPULATION (role=crawler): 41 3.7% of crawled
... of crawled docker mentions 50.6%
A crawler-role paper ran the measurement browser/crawler inside Docker.
target/sandbox/artifact/citation are printed so they cannot silently vanish.
-- venue of crawler-role papers --
Venue Crawler-docker Crawled at venue Share of venue crawled
------- -------------- ---------------- ----------------------
USENIX 10 221 4.5%
CCS 7 163 4.3%
IMC 7 132 5.3%
PETS 6 123 4.9%
WWW 5 242 2.1%
IEEE-SP 3 110 2.7%
NDSS 3 129 2.3%
-- year (2025–2026 provisional) of crawler-role --
Year Crawled Crawler-docker Share of year crawled
----- ------- -------------- ---------------------
2010 25 0 0.0%
2011 22 0 0.0%
2012 29 0 0.0%
2013 26 0 0.0%
2014 35 0 0.0%
2015 41 0 0.0%
2016 40 0 0.0%
2017 51 1 2.0%
2018 61 2 3.3%
2019 97 5 5.2%
2020 75 5 6.7%
2021 75 5 6.7%
2022 110 4 3.6%
2023 125 5 4.0%
2024 110 4 3.6%
2025* 129 6 4.7%
2026* 69 4 5.8%
-- year buckets --
Bucket Crawled Crawler-docker Share of crawled in bucket
---------- ------- -------------- --------------------------
2010–2013 102 0 0.0%
2014–2017 167 1 0.6%
2018–2021 308 17 5.5%
2022–2024 345 13 3.8%
2025–2026* 198 10 5.1%
======================================================================
2. SCHEMA vs FULL TEXT
======================================================================
crawled papers with tools[].name matching /docker/i 56
... of which used or produced 56
... of which state tools[].version 5 8.9%
crawler-role papers also in tools[] 36 87.8% of 41
crawler-role papers the schema does not see as Docker 5
-- tools[].category for crawled docker name hits (papers) --
Category Papers
-------------- ------
infrastructure 55
http-client 1
other 1
Exact-string "Docker" undercount on tools[].name among crawled:
exact "Docker": 53 papers; any /docker/i: 56
======================================================================
3. THE KNOBS THIS PAGE IS ABOUT — reporting rates
======================================================================
crawled papers matching /dev/shm OR shm-size OR disable-dev-shm-usage: 0
corpus papers matching the same: 0 miss=4
crawled papers matching docker-compose / compose.yaml: 8 0.7%
crawled papers matching Dockerfile: 11 1.0%
crawled papers matching /xvfb(?:-run)?/i: 13 1.2%
corpus papers matching xvfb: 14 miss=4
crawled papers matching image digest / sha256:<12+hex>: 0
corpus papers matching image digest / sha256:<12+hex>: 1 miss=4
The one corpus digest hit is a HOMOGRAPH, not a container pin:
2023/USENIX/squint-hard-enough-attacking-perceptual-hashing-with-adversarial-machine-learnin
| ser). The adversary can cause an illicit-looking image of their choice to be added to the database of illicit image digests (e.g., by submitting it to the database's curator, or planting somewhere it would be found and s
Published figure: 0 of 5,859 papers name a container image digest.
0 of 1,120 crawled papers name /dev/shm, shm-size, or --disable-dev-shm-usage.
-- xvfb role over crawled xvfb sweep --
Xvfb role Papers Share of 13
--------- ------ -----------
used 12 92.3%
recommend 1 7.7%
xvfb used (not merely recommended): 12
crawler-role ∩ xvfb sweep: 7 17.1% of crawler-role
xvfb used but NOT in the crawled docker sweep: 5
======================================================================
4. HEADLESS / VERSION among crawler-role papers
======================================================================
crawler-role papers stating crawlConfig.headless (not sentinel): 15 36.6%
same field over all 1120 crawled: 140 12.5%
headless enum Crawler-docker papers
------------- ---------------------
not-stated 26
headless 7
headful 7
both 1
crawler-role papers stating a version on a used/produced automation tool: 2 4.9%
======================================================================
5. PLAYWRIGHT IMAGE NAMED IN THE CORPUS
======================================================================
crawled papers matching playwright…docker or docker…playwright: 2
2025/IMC/a-permissions-odyssey-a-systematic-study-of-browser-permissions-on-modern-websit
| n only once, so multiple measurements do not apply. A.2.2 Experimental Design. C5 We relied on Playwright v1.45.1, packaged in Microsoft's Playwright Docker image (v1.46.0-jammy). C6 We disabled the AutomationControlled Blink feature (navigator.webdriver) to reduce detection and
2023/IEEE-SP/detection-of-inconsistencies-in-privacy-practices-of-browser-extensions
| mation (PII). Technical report NIST Special Publication (SP) 800-122, National Institute of Standards and Technology, 2010. [48] Microsoft. Docker | playwright python. 2021. URL: https : / / playwright . dev / python / docs / docker/ (visited on 10/05/2021). [49] Microsoft. Micro
| tion (SP) 800-122, National Institute of Standards and Technology, 2010. [48] Microsoft. Docker | playwright python. 2021. URL: https : / / playwright . dev / python / docs / docker/ (visited on 10/05/2021). [49] Microsoft. Microsoft playwright python. 2020. URL: https://github.c
======================================================================
6. CRAWLER-ROLE LIST
======================================================================
2017/CCS/poster-probing-tor-hidden-service-with-dockers have automated our tool with the virtualization software, Docker, to improve the crawling performance
2018/CCS/minesweeper-an-in-depth-look-into-drive-by-cryptocurrency-mining-and-its-defense We deployed our web crawler in Docker containers running on Kubernetes
2018/CCS/mystique-uncovering-information-leakage-from-browser-extensions Each instance of our modified Chromium browser is run in a separate Docker [7] container
2019/IMC/visiblev8-in-browser-monitoring-of-javascript-in-the-wild The collection worker node Docker image contains the VV8 binary itself
2019/IMC/what-you-see-is-not-what-you-get-discovering-and-tracking-social-engineering-att Our browsing modules consist of a Docker container running a headless instrumented version of the Chromium browser
2019/PETS/4-years-of-eu-cookie-law-results-and-lessons-learned CookieCheck is a scalable system based on Docker
2019/USENIX/inadvertently-making-cyber-criminals-rich-a-comprehensive-study-of-cryptojacking We deployed the crawler in Docker containers on 60 servers within the university network, each running 8 Docker instances in parallel
2019/WWW/web-experience-in-mobile-networks-lessons-from-two-million-page-visits We leverage a customizable Docker container called MONROE-browsertime
2020/IMC/when-push-comes-to-ads-measuring-the-rise-of-malicious-push-advertising we leverage Docker containers [34] to launch several parallel instances of our instrumented browser-based crawlers
2020/PETS/in-depth-evaluation-of-redirect-tracking-and-link-usage The nodes take jobs from the queue and execute them in individual Docker containers in parallel
2020/WWW/comparing-the-effects-of-dns-dot-and-doh-on-web-performance Our code is packaged as a Docker image to enable reproducible measurements, and to clear the browser's HTTP cache between page loads
2020/WWW/de-kodi-understanding-the-kodi-ecosystem we leverage Docker [1] to scale our software while isolating crawler instances from potential malware and/or crashes
2020/WWW/understanding-the-performance-costs-and-benefits-of-privacy-focused-browser-exte To ease deployment and reproducibility, we created Docker containers
2021/IMC/polls-clickbait-and-commemorative-2-bills-problematic-political-advertising-on-n we ran separate browser instances inside a new Docker container, so that no tracking cookies or other state persisted across domains
2021/IMC/trackersift-untangling-mixed-tracking-and-functional-web-resources Each node uses a Docker container to crawl a subset of 100K webpages
2021/PETS/website-fingerprinting-in-the-age-of-quic We deployed each instance in a docker container to isolate the state, network, memory, and CPU consumption of each browser instance
2021/USENIX/catching-phishers-by-their-bait-investigating-the-dutch-phishing-landscape-throu The complete measurement system is deployed in Docker containers on a cloud server
2021/WWW/cookie-swap-party-abusing-first-party-cookies-for-web-tracking Each instance of Chromium runs in a separate Docker [5] container
2022/CCS/distinct-identity-theft-using-in-browser-communications-in-dual-window-single-si We designed Distinct as a containerized Docker application that is straightforward to install and operate via a single web interface
2022/IEEE-SP/surakav-generating-realistic-traces-for-a-strong-website-fingerprinting-defense we create in total ten docker containers acting as ten independent clients to visit webpages in parallel
2022/IMC/phishinpatterns-measuring-elicited-user-interactions-at-scale-on-phishing-websit we built a Docker-ized version of our crawler, and used a "clean" container for each phishing website we visit
2022/USENIX/the-dangers-of-human-touch-fingerprinting-browser-extensions-through-user-action we modified our framework and built it into a Docker Container [1]
2023/CCS/read-between-the-lines-detecting-tracking-javascript-with-bytecode-classificatio For WebGraph, we used the Docker image provided by the authors [15] and crawled the same 5K websites
2023/CCS/transformer-based-model-for-multi-tab-website-fingerprinting-attack we created a Docker image integrating the Selenium browser automation tool, the Chrome browser, and the Tor proxy service
2023/IEEE-SP/detection-of-inconsistencies-in-privacy-practices-of-browser-extensions The framework is based on Docker Swarm [39] and the browser is started with arguments to make it run in the resourceconstrained docker environments
2023/USENIX/trident-towards-detecting-and-mitigating-web-based-social-engineering-attacks We deployed the crawlers in 20 docker containers simulating users' interactions with websites
2023/USENIX/autofr-automated-filter-rule-generation-for-adblocking (a) spawns n = 10 docker instances and visits the site until it finishes loading
2024/NDSS/flow-correlation-attacks-on-tor-onion-service-sessions-with-sliding-subset-sum Clients also run on an isolated Docker container that executes a 0.4.7.10 Tor process and issues requests
2024/USENIX/fv8-a-forced-execution-javascript-engine-for-detecting-evasive-techniques we utilize the bash scripting language and employed Docker and Docker Compose to scale up jobs
2024/USENIX/arcanum-detecting-and-evaluating-the-privacy-risks-of-browser-extensions-on-web We run each instance of Arcanum in a Docker container on Ubuntu 18.04
2024/USENIX/sok-state-of-the-krawlers-evaluating-the-effectiveness-of-crawling-algorithms-fo The crawler and web applications modules rely on Docker [32], allowing us to scale experiments by running parallel experiments of the same web application and crawlers concurrently
2025/CCS/automatically-detecting-online-deceptive-patterns We package Lighthouse with our custom audit in a docker container for better usability
2025/IEEE-SP/403-forbidden-ethically-evaluating-broken-access-control-in-the-wild We implement the VSF over multiple inter-connected Docker containers (e.g., a manual visitation container, swapping container, automated visitation coordinator, etc.)
2025/IMC/a-permissions-odyssey-a-systematic-study-of-browser-permissions-on-modern-websit We relied on Playwright v1.45.1, packaged in Microsoft's Playwright Docker image (v1.46.0-jammy)
2025/PETS/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic we simultaneously launched two Docker containers, each running a Puppeteer controlled Chrome browser configured with/without ABP adblocker
2025/USENIX/double-edged-shield-on-the-fingerprintability-of-customized-ad-blockers We perform the measurements on 3K sites from the set using 20 crawlers (Docker containers)
2025/USENIX/navigating-cookie-consent-violations-across-the-globe We conducted experiments in a distributed framework based on Docker Swarm [37] on 4 machines
2026/NDSS/dom-xss-detection-via-webpage-interaction-fuzzing-and-url-component-synthesis The analysis of each page and condition is run on an isolated Docker container restricted to 6GB RAM and 6 cores
2026/NDSS/cross-boundary-mobile-tracking-exploring-java-to-javascript-information-diffusion-in-webviews a Python-based distributor script alongside Docker is used as the orchestration layer to parallelize our crawls across multiple devices
2026/PETS/redefining-website-fingerprinting-attacks-with-multi-agent-llms The Computer-Using Agent (CUA) runs inside a lightweight Docker container using Anthropic's official execution environment
2026/PETS/more-space-less-privacy-measuring-the-effectiveness-of-ip-based-website-fingerpr All crawls were conducted inside isolated Docker containers to eliminate any DNS or browser cache interference between successive website visits
======================================================================
6b. ALL 81 ROLES (key, role) — full map is docker_fold.mjs
======================================================================
2017/CCS/poster-probing-tor-hidden-service-with-dockers crawler
2018/CCS/minesweeper-an-in-depth-look-into-drive-by-cryptocurrency-mining-and-its-defense crawler
2018/CCS/mystique-uncovering-information-leakage-from-browser-extensions crawler
2018/USENIX/understanding-the-reproducibility-of-crowd-reported-security-vulnerabilities artifact
2019/IMC/an-empirical-study-of-the-cost-of-dns-over-https sandbox
2019/IMC/visiblev8-in-browser-monitoring-of-javascript-in-the-wild crawler
2019/IMC/what-you-see-is-not-what-you-get-discovering-and-tracking-social-engineering-att crawler
2019/NDSS/how-bad-can-it-git-characterizing-secret-leakage-in-public-github-repositories citation
2019/PETS/4-years-of-eu-cookie-law-results-and-lessons-learned crawler
2019/PETS/maps-scaling-privacy-compliance-analysis-to-a-million-apps sandbox
2019/USENIX/inadvertently-making-cyber-criminals-rich-a-comprehensive-study-of-cryptojacking crawler
2019/USENIX/less-is-more-quantifying-the-security-benefits-of-debloating-web-applications sandbox
2019/WWW/web-experience-in-mobile-networks-lessons-from-two-million-page-visits crawler
2020/IMC/when-push-comes-to-ads-measuring-the-rise-of-malicious-push-advertising crawler
2020/PETS/in-depth-evaluation-of-redirect-tracking-and-link-usage crawler
2020/WWW/apophanies-or-epiphanies-how-crawlers-impact-our-understanding-of-the-web citation
2020/WWW/comparing-the-effects-of-dns-dot-and-doh-on-web-performance crawler
2020/WWW/de-kodi-understanding-the-kodi-ecosystem crawler
2020/WWW/understanding-the-performance-costs-and-benefits-of-privacy-focused-browser-exte crawler
2021/IEEE-SP/good-bot-bad-bot-characterizing-automated-browsing-activity citation
2021/IMC/polls-clickbait-and-commemorative-2-bills-problematic-political-advertising-on-n crawler
2021/IMC/trackersift-untangling-mixed-tracking-and-functional-web-resources crawler
2021/PETS/website-fingerprinting-in-the-age-of-quic crawler
2021/USENIX/catching-phishers-by-their-bait-investigating-the-dutch-phishing-landscape-throu crawler
2021/WWW/cookie-swap-party-abusing-first-party-cookies-for-web-tracking crawler
2022/CCS/distinct-identity-theft-using-in-browser-communications-in-dual-window-single-si crawler
2022/IEEE-SP/meshup-stateless-cache-side-channel-attack-on-cpu-mesh sandbox
2022/IEEE-SP/robbery-on-devops-understanding-and-mitigating-illicit-cryptomining-on-continuou target
2022/IEEE-SP/surakav-generating-realistic-traces-for-a-strong-website-fingerprinting-defense crawler
2022/IMC/phishinpatterns-measuring-elicited-user-interactions-at-scale-on-phishing-websit crawler
2022/USENIX/exploring-the-unchartered-space-of-container-registry-typosquatting target
2022/USENIX/the-dangers-of-human-touch-fingerprinting-browser-extensions-through-user-action crawler
2023/CCS/read-between-the-lines-detecting-tracking-javascript-with-bytecode-classificatio crawler
2023/CCS/transformer-based-model-for-multi-tab-website-fingerprinting-attack crawler
2023/CCS/under-the-dark-a-systematical-study-of-stealthy-mining-pools-ab-use-in-the-wild citation
2023/CCS/understanding-and-detecting-abused-image-hosting-modules-as-malicious-services sandbox
2023/IEEE-SP/detection-of-inconsistencies-in-privacy-practices-of-browser-extensions crawler
2023/IEEE-SP/investigating-package-related-security-threats-in-software-registries target
2023/IEEE-SP/its-dom-clobbering-time-attack-techniques-prevalence-and-defenses sandbox
2023/IEEE-SP/toss-a-fault-to-your-witcher-applying-grey-box-coverage-guided-mutational-fuzzin sandbox
2023/NDSS/rescan-a-middleware-framework-for-realistic-and-robust-black-box-web-application-scanning sandbox
2023/USENIX/animatedead-debloating-web-applications-using-concolic-execution sandbox
2023/USENIX/autofr-automated-filter-rule-generation-for-adblocking crawler
2023/USENIX/greenhouse-single-service-rehosting-of-linux-based-firmware-binaries-in-user-spa sandbox
2023/USENIX/trident-towards-detecting-and-mitigating-web-based-social-engineering-attacks crawler
2024/CCS/toward-understanding-the-security-of-plugins-in-continuous-integration-services target
2024/IEEE-SP/holistic-concolic-execution-for-dynamic-web-applications-via-symbolic-interprete citation
2024/NDSS/cp-iot-a-cross-platform-monitoring-system-for-smart-home sandbox
2024/NDSS/flow-correlation-attacks-on-tor-onion-service-sessions-with-sliding-subset-sum crawler
2024/NDSS/quack-hindering-deserialization-attacks-via-static-duck-typing artifact
2024/USENIX/arcanum-detecting-and-evaluating-the-privacy-risks-of-browser-extensions-on-web crawler
2024/USENIX/fv8-a-forced-execution-javascript-engine-for-detecting-evasive-techniques crawler
2024/USENIX/smudged-fingerprints-characterizing-and-improving-the-performance-of-web-applica sandbox
2024/USENIX/sok-state-of-the-krawlers-evaluating-the-effectiveness-of-crawling-algorithms-fo crawler
2024/USENIX/trust-me-if-you-can-how-usable-is-trusted-types-in-practice artifact
2024/WWW/satguard-concealing-endless-and-bursty-packet-losses-in-leo-satellite-networks-f sandbox
2025/CCS/automatically-detecting-online-deceptive-patterns crawler
2025/IEEE-SP/403-forbidden-ethically-evaluating-broken-access-control-in-the-wild crawler
2025/IEEE-SP/hey-your-secrets-leaked-detecting-and-characterizing-secret-leakage-in-the-wild sandbox
2025/IEEE-SP/predator-directed-web-application-fuzzing-for-efficient-vulnerability-validation sandbox
2025/IMC/a-permissions-odyssey-a-systematic-study-of-browser-permissions-on-modern-websit crawler
2025/NDSS/duumviri-detecting-trackers-and-mixed-trackers-with-a-breakage-detector artifact
2025/NDSS/yurascanner-leveraging-llms-for-task-driven-web-app-scanning sandbox
2025/PETS/sheeps-clothing-wolfish-intent-automated-detection-and-evaluation-of-problematic crawler
2025/USENIX/beyond-exploit-scanning-a-functional-change-driven-approach-to-remote-software-v sandbox
2025/USENIX/double-edged-shield-on-the-fingerprintability-of-customized-ad-blockers crawler
2025/USENIX/lost-in-the-mists-of-time-expirations-in-dns-footprints-of-mobile-apps sandbox
2025/USENIX/navigating-cookie-consent-violations-across-the-globe crawler
2025/USENIX/the-doom-of-device-drivers-your-android-device-most-likely-has-n-day-kernel-vuln citation
2025/WWW/dr-docker-a-large-scale-security-measurement-of-docker-image-ecosystem target
2026/NDSS/breaking-the-bulkhead-demystifying-cross-namespace-reference-vulnerabilities-in-kubernetes-operators target
2026/NDSS/cellshift-rtt-aware-trace-transduction-for-real-world-website-fingerprinting artifact
2026/NDSS/cross-boundary-mobile-tracking-exploring-java-to-javascript-information-diffusion-in-webviews crawler
2026/NDSS/dom-xss-detection-via-webpage-interaction-fuzzing-and-url-component-synthesis crawler
2026/NDSS/eviction-notice-reviving-and-advancing-page-cache-attacks target
2026/NDSS/losing-the-beat-understanding-and-mitigating-desynchronization-risks-in-container-isolation target
2026/NDSS/sack-systematic-generation-of-function-substitution-attacks-against-control-flow-integrity sandbox
2026/PETS/more-space-less-privacy-measuring-the-effectiveness-of-ip-based-website-fingerpr crawler
2026/PETS/redefining-website-fingerprinting-attacks-with-multi-agent-llms crawler
2026/USENIX/from-mirai-to-gorilla-deep-dive-into-a-long-lasting-ddos-for-hire-botnet sandbox
2026/USENIX/the-state-of-passkeys-studying-the-adoption-and-security-of-passkeys-on-the-web artifact
======================================================================
Z. NON-CORPUS FIGURES (primary sources, re-fetched by docker_probe.sh)
======================================================================
Docker Engine default /dev/shm when --shm-size is omitted: 64m
source: docs.docker.com/engine/containers/run/ (--shm-size default 64m)
Chromium --disable-dev-shm-usage: Linux-only (BUILDFLAG IS_LINUX); anonymous shm files go to a temp dir
source: chromium/src base/base_switches.h kDisableDevShmUsage; comment cites crbug.com/715363
(the flag is declared in the header, not in base_switches.cc)
OpenWPM docker run requires --shm-size=2g; without it Firefox crashes on 20-30% of sites
source: github.com/openwpm/OpenWPM README.md "Docker Deployment for OpenWPM"
OpenWPM docker run also passes --init
OpenWPM Docker Hub openwpm/openwpm pull_count 32310 last_updated 2026-08-24
Playwright Docker image documented tag at fetch: mcr.microsoft.com/playwright:v1.62.0-noble
Playwright recommended docker flags: --ipc=host, --init; crawling should use user pwuser + seccomp
source: playwright.dev/docs/docker
Playwright headed on Linux CI: xvfb-run npx playwright test; image ships Xvfb
source: playwright.dev/docs/ci Running headed
Playwright image is "testing and development purposes only" / not recommended to visit untrusted websites
Playwright live index digest 2026-08-27: sha256:baed2032d533817f3dbe6425de795788430ba345e819a1201337009ba17c9d07
mediaType application/vnd.oci.image.index.v1+json n_manifests 2 (amd64 + arm64)
Retana et al. IMC 2025 named Playwright v1.45.1 inside image v1.46.0-jammy (package/image mismatch)
This host has no Docker binary (which docker → not found); /dev/shm on the host is 2.0G; Xvfb is /usr/bin/Xvfb
xvfb-run on this host exits 3: xauth command not found
raw Xvfb :98 -ac -screen 0 1280x720x24 then xdpyinfo -display :98 succeeds on this host (exit 0)
display :99 already had a leftover Xvfb lock on this host; :98 was started for the check
LibreTranslate compose on programming:multilingual_support uses libretranslate/libretranslate:latest
libretranslate/libretranslate:latest last_updated 2026-08-23; n_images=2
linux/amd64 digest sha256:8402eea7845b6890acdd4b1fb70320c130d2771935d36d697a901e8c149682ca size=208639823
linux/arm64 digest sha256:110905ed2b68530de438c9126c951b0bb1c79669b8cd59d2f3bf2ecf679ea534 size=178654387
check_crawler_compose.py --demo: DEMO_BAD exit 1, DEMO_GOOD exit 0, self-test OK
DEMO_GOOD pin uses placeholder sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
ports 5000:5000 appear only in DEMO_BAD libretranslate fixture
digest pin is sha256 plus 64 hex chars
full-text digest probe is sha256: plus 12 or more hex (homograph filter is hand-read)
DEFAULT_SHM_BYTES = 64 * 1024 * 1024 (checker rejects shm_size at or below Docker 64m)
======================================================================
Y. ARITHMETIC
======================================================================
crawler-role share of crawled: 41/1120 = 3.7%
crawled docker share of crawled: 81/1120 = 7.2%
crawler-role share of crawled docker: 41/81 = 50.6%
non-crawler roles sum: 40
readable full text: 5859 - 4 missing .cols = 5855
Probe output (unedited)
bash scripts/docker_probe.sh:
=== host === OK: docker binary absent (expected on this aarch64 research host) OK: Xvfb at /usr/bin/Xvfb host /dev/shm size: 2.0G === Docker Engine --shm-size default === OK: docs.docker.com names 64m as the omitted --shm-size default he size entirely, the system uses <code>64m</code>.</td></tr></tbody></table></div> OK: docs.docker.com documents --shm-size === Chromium --disable-dev-shm-usage === OK: chromium base/base_switches.h defines disable-dev-shm-usage 153-// Chrome to fail or crash (see http://crbug.com/715363). Use this flag to 154-// work-around this issue (a temporary directory will always be used to create 155-// anonymous shared memory files). 156:inline constexpr char kDisableDevShmUsage[] = "disable-dev-shm-usage"; 157-#endif 158- 159-#if BUILDFLAG(IS_POSIX) OK: comment still cites crbug.com/715363 === OpenWPM README shm-size === OK: OpenWPM README still requires --shm-size=2g OK: OpenWPM README still claims 20-30% crash rate without shm-size OK: OpenWPM docker run still passes --init === Playwright Docker docs === documented noble tag: mcr.microsoft.com/playwright:v1.62.0-noble OK: playwright.dev/docs/docker names a versioned mcr image OK: Playwright still recommends --ipc=host OK: Playwright still recommends --init OK: Playwright still warns the image is not for untrusted websites === Playwright CI headed / xvfb-run === OK: playwright.dev/docs/ci documents xvfb-run for headed Linux === Playwright image digest (registry v2, no docker binary) === OK: MCR tag list includes v1.62.0-noble OK: mcr.microsoft.com/playwright:v1.62.0-noble digest sha256:baed2032d533817f3dbe6425de795788430ba345e819a1201337009ba17c9d07 PIN=mcr.microsoft.com/playwright:v1.62.0-noble@sha256:baed2032d533817f3dbe6425de795788430ba345e819a1201337009ba17c9d07 mediaType application/vnd.oci.image.index.v1+json n_manifests 2 === LibreTranslate :latest (the sibling-page example) === tag=latest last_updated=2026-08-23T22:56:12.456983Z n_images=2 linux/amd64 digest=sha256:8402eea7845b6890acdd4b1fb70320c130d2771935d36d697a901e8c149682ca size=208639823 linux/arm64 digest=sha256:110905ed2b68530de438c9126c951b0bb1c79669b8cd59d2f3bf2ecf679ea534 size=178654387 OK: libretranslate/libretranslate:latest still exists and moves (last_updated above) === OpenWPM Docker Hub === name openwpm last_updated 2026-08-24T23:20:17.579708Z pull_count 32310 OK: openwpm/openwpm Hub repo exists docker_probe.sh: all checks printed OK
