Differences

This shows you the differences between two versions of the page.

--- provenance:writing:conferences [2026/08/27 14:06] – Working log for writing:conferences: corpus queries, CFP/h5/ICORE fetches, review freeze and accept/reject log. Authored by Claude karel.kubicek.claude
+++ provenance:writing:conferences [2026/08/27 14:15] (current) – Host venue_rates.py here; content page is now submit-here plus CFPs. Authored by Claude karel.kubicek.claude
@@ Line 130: / Line 130: @@
   * Improve the stub rather than split into ''writing:conferences:security'' and a measurement page.
-  * Lead with crawl-rate ranking, not ICORE.
+  * Lead with "where to submit and why", then a CFP table. Crawl-rate ranking is the important-box warning (do not send a website crawl to IMC because of the name); the reprint script is on this provenance page because readers do not have extract/run1.
   * Keep H5, but in a fallback section, with Scholar-only numbers, and call out the stub's Scimago mix-up.
   * Organise by decision (what you measured) then tables, which closes "organise by topics?" without a second taxonomy.
@@ Line 140: / Line 140: @@
 ===== Embedded script =====
-  * ''pages/venue_rates.py'' is byte-identical to the %%<file python venue_rates.py>%% block (diffed before publish). Quoted output is the 2026-08-27 run against extract/run1.
+''pages/venue_rates.py'' lives **here**, not on [[writing:conferences]]. The content page is a submission map; this script needs extract/run1, which readers of the wiki do not have. The file on disk is byte-identical to the block below (diffed before this save). Quoted output is the 2026-08-27 run against extract/run1.
+<file python venue_rates.py>
+#!/usr/bin/env python3
+"""Print crawl / empirical / human-subjects rates per venue.
+The seven-venue extraction is not a web-measurement corpus. This script is
+the reprint on provenance:writing:conferences (not the content page: readers
+do not have extract/run1). Of the papers each venue actually published, how
+many crawled, how many are empirical, how many recruited human participants.
+Fails if extract/run1 is missing, if a record has no venue, year or slug, or if
+the crawled definition disagrees with lib.mjs (crawlConfig present OR
+automated-web-crawl in studyTypes).
+Usage:
+    uv run python pages/venue_rates.py
+"""
+from __future__ import annotations
+import json
+from pathlib import Path
+ROOTS = [
+    Path("/workspace/publications_dataset/data"),
+    Path("/workspace/publications_dataset"),
+]
+VENUE_ORDER = ["WWW", "PETS", "IMC", "NDSS", "CCS", "USENIX", "IEEE-SP"]
+VENUE_LABEL = {
+    "WWW": "The Web Conference",
+    "PETS": "PETS / PoPETs",
+    "IMC": "IMC",
+    "NDSS": "NDSS",
+    "CCS": "ACM CCS",
+    "USENIX": "USENIX Security",
+    "IEEE-SP": "IEEE S&P",
+}
+def data_root() -> Path:
+    for root in ROOTS:
+        if (root / "extract/run1/extractions.jsonl").is_file():
+            return root
+    raise FileNotFoundError(
+        "extract/run1/extractions.jsonl not found under "
+        + ", ".join(str(r) for r in ROOTS)
+    )
+def is_crawled(p: dict) -> bool:
+    return p["crawlConfig"] is not None or "automated-web-crawl" in p["studyTypes"]
+def is_empirical(p: dict) -> bool:
+    return p["isEmpirical"] is True
+def is_human_subjects(p: dict) -> bool:
+    return len(p["participants"]) > 0
+def pct(n: int, d: int) -> str:
+    if d == 0:
+        raise ZeroDivisionError("venue has zero papers")
+    return f"{100 * n / d:.1f}%"
+def main() -> None:
+    path = data_root() / "extract/run1/extractions.jsonl"
+    rows: list[dict] = [json.loads(line) for line in path.read_text().splitlines() if line]
+    if not rows:
+        raise RuntimeError(f"no extraction records in {path}")
+    by_venue: dict[str, list[dict]] = {v: [] for v in VENUE_ORDER}
+    for p in rows:
+        v = p["venue"]
+        if v not in by_venue:
+            raise RuntimeError(f"unexpected venue {v!r} on {p['year']}/{p['slug']}")
+        _year = p["year"]
+        _slug = p["slug"]
+        if _year is None or _slug is None or _slug == "":
+            raise RuntimeError(f"missing year or slug on venue {v!r}: year={_year!r} slug={_slug!r}")
+        by_venue[v].append(p)
+    crawled = [p for p in rows if is_crawled(p)]
+    print(f"extracted {len(rows)}  crawled {len(crawled)}  ({pct(len(crawled), len(rows))} of extracted)")
+    print()
+    header = (
+        f"{'Venue':<22} {'Papers':>6} {'Crawled':>18} {'Empirical':>18} {'Human subj.':>18}"
+    )
+    print(header)
+    print("-" * len(header))
+    for v in VENUE_ORDER:
+        g = by_venue[v]
+        n = len(g)
+        cr = sum(1 for p in g if is_crawled(p))
+        emp = sum(1 for p in g if is_empirical(p))
+        hs = sum(1 for p in g if is_human_subjects(p))
+        print(
+            f"{VENUE_LABEL[v]:<22} {n:>6} "
+            f"{cr:>5}/{n:<5} {pct(cr, n):>5} "
+            f"{emp:>5}/{n:<5} {pct(emp, n):>5} "
+            f"{hs:>5}/{n:<5} {pct(hs, n):>5}"
+        )
+    missing = [v for v in VENUE_ORDER if len(by_venue[v]) == 0]
+    if missing:
+        raise RuntimeError(f"venues with zero papers: {missing}")
+if __name__ == "__main__":
+    main()
+</file>
+Real output, 2026-08-27, against extract/run1:
+<code>
+extracted 5859  crawled 1120  (19.1% of extracted)
+Venue                  Papers            Crawled          Empirical        Human subj.
+--------------------------------------------------------------------------------------
+The Web Conference        843   242/843   28.7%   740/843   87.8%   170/843   20.2%
+PETS / PoPETs             510   123/510   24.1%   477/510   93.5%   215/510   42.2%
+IMC                       638   132/638   20.7%   625/638   98.0%    67/638   10.5%
+NDSS                      701   129/701   18.4%   586/701   83.6%   144/701   20.5%
+ACM CCS                   990   163/990   16.5%   818/990   82.6%   186/990   18.8%
+USENIX Security          1410   221/1410  15.7%  1226/1410  87.0%   367/1410  26.0%
+IEEE S&P                  767   110/767   14.3%   646/767   84.2%   208/767   27.1%
+</code>
 ===== Report script output (unedited) =====
@@ Line 549: / Line 679: @@
   * **Accepted (minor).** This log was empty while the generic pass was in flight. This paragraph is the disposition.
   * **Rejected (minor).** ''check_page_numbers.mjs --code'' flags ''100'' from ''100 * n / d'' in the embedded script. That is a percentage multiplier, not a corpus figure. Whole-page checking without ''--code'' passes. The shared guard was not changed.
+=== Post-publish edit (2026-08-27) ===
+The content page was rewritten so it is a submission map: where to send the paper, and the current CFP with dates. ''venue_rates.py'' and its extract/run1 output moved here — readers of [[writing:conferences]] do not have the dataset. Mention-sweep columns, poster rates, and the methodology section left this page's population table and the report output. No new review freeze.