Differences

This shows you the differences between two versions of the page.

--- provenance:writing:conferences [2026/08/27 14:06] – Working log for writing:conferences: corpus queries, CFP/h5/ICORE fetches, review freeze and accept/reject log. Authored by Claude karel.kubicek.claude
+++ provenance:writing:conferences [2026/09/17 07:25] (current) – Markup rendering sweep: wrap escapes, CLI flag typography, and heading quotes. Authored by Claude. karel.kubicek.claude
@@ Line 130: / Line 130: @@
   * Improve the stub rather than split into ''writing:conferences:security'' and a measurement page.
-  * Lead with crawl-rate ranking, not ICORE.
+  * Lead with "where to submit and why", then a CFP table. Crawl-rate ranking is the important-box warning (do not send a website crawl to IMC because of the name); the reprint script is on this provenance page because readers do not have extract/run1.
   * Keep H5, but in a fallback section, with Scholar-only numbers, and call out the stub's Scimago mix-up.
   * Organise by decision (what you measured) then tables, which closes "organise by topics?" without a second taxonomy.
@@ Line 140: / Line 140: @@
 ===== Embedded script =====
-  * ''pages/venue_rates.py'' is byte-identical to the %%<file python venue_rates.py>%% block (diffed before publish). Quoted output is the 2026-08-27 run against extract/run1.
+''pages/venue_rates.py'' lives **here**, not on [[writing:conferences]]. The content page is a submission map; this script needs extract/run1, which readers of the wiki do not have. The file on disk is byte-identical to the block below (diffed before this save). Quoted output is the 2026-08-27 run against extract/run1.
+<file python venue_rates.py>
+#!/usr/bin/env python3
+"""Print crawl / empirical / human-subjects rates per venue.
+The seven-venue extraction is not a web-measurement corpus. This script is
+the reprint on provenance:writing:conferences (not the content page: readers
+do not have extract/run1). Of the papers each venue actually published, how
+many crawled, how many are empirical, how many recruited human participants.
+Fails if extract/run1 is missing, if a record has no venue, year or slug, or if
+the crawled definition disagrees with lib.mjs (crawlConfig present OR
+automated-web-crawl in studyTypes).
+Usage:
+    uv run python pages/venue_rates.py
+"""
+from __future__ import annotations
+import json
+from pathlib import Path
+ROOTS = [
+    Path("/workspace/publications_dataset/data"),
+    Path("/workspace/publications_dataset"),
+]
+VENUE_ORDER = ["WWW", "PETS", "IMC", "NDSS", "CCS", "USENIX", "IEEE-SP"]
+VENUE_LABEL = {
+    "WWW": "The Web Conference",
+    "PETS": "PETS / PoPETs",
+    "IMC": "IMC",
+    "NDSS": "NDSS",
+    "CCS": "ACM CCS",
+    "USENIX": "USENIX Security",
+    "IEEE-SP": "IEEE S&P",
+}
+def data_root() -> Path:
+    for root in ROOTS:
+        if (root / "extract/run1/extractions.jsonl").is_file():
+            return root
+    raise FileNotFoundError(
+        "extract/run1/extractions.jsonl not found under "
+        + ", ".join(str(r) for r in ROOTS)
+    )
+def is_crawled(p: dict) -> bool:
+    return p["crawlConfig"] is not None or "automated-web-crawl" in p["studyTypes"]
+def is_empirical(p: dict) -> bool:
+    return p["isEmpirical"] is True
+def is_human_subjects(p: dict) -> bool:
+    return len(p["participants"]) > 0
+def pct(n: int, d: int) -> str:
+    if d == 0:
+        raise ZeroDivisionError("venue has zero papers")
+    return f"{100 * n / d:.1f}%"
+def main() -> None:
+    path = data_root() / "extract/run1/extractions.jsonl"
+    rows: list[dict] = [json.loads(line) for line in path.read_text().splitlines() if line]
+    if not rows:
+        raise RuntimeError(f"no extraction records in {path}")
+    by_venue: dict[str, list[dict]] = {v: [] for v in VENUE_ORDER}
+    for p in rows:
+        v = p["venue"]
+        if v not in by_venue:
+            raise RuntimeError(f"unexpected venue {v!r} on {p['year']}/{p['slug']}")
+        _year = p["year"]
+        _slug = p["slug"]
+        if _year is None or _slug is None or _slug == "":
+            raise RuntimeError(f"missing year or slug on venue {v!r}: year={_year!r} slug={_slug!r}")
+        by_venue[v].append(p)
+    crawled = [p for p in rows if is_crawled(p)]
+    print(f"extracted {len(rows)}  crawled {len(crawled)}  ({pct(len(crawled), len(rows))} of extracted)")
+    print()
+    header = (
+        f"{'Venue':<22} {'Papers':>6} {'Crawled':>18} {'Empirical':>18} {'Human subj.':>18}"
+    )
+    print(header)
+    print("-" * len(header))
+    for v in VENUE_ORDER:
+        g = by_venue[v]
+        n = len(g)
+        cr = sum(1 for p in g if is_crawled(p))
+        emp = sum(1 for p in g if is_empirical(p))
+        hs = sum(1 for p in g if is_human_subjects(p))
+        print(
+            f"{VENUE_LABEL[v]:<22} {n:>6} "
+            f"{cr:>5}/{n:<5} {pct(cr, n):>5} "
+            f"{emp:>5}/{n:<5} {pct(emp, n):>5} "
+            f"{hs:>5}/{n:<5} {pct(hs, n):>5}"
+        )
+    missing = [v for v in VENUE_ORDER if len(by_venue[v]) == 0]
+    if missing:
+        raise RuntimeError(f"venues with zero papers: {missing}")
+if __name__ == "__main__":
+    main()
+</file>
+Real output, 2026-08-27, against extract/run1:
+<code>
+extracted 5859  crawled 1120  (19.1% of extracted)
+Venue                  Papers            Crawled          Empirical        Human subj.
+--------------------------------------------------------------------------------------
+The Web Conference        843   242/843   28.7%   740/843   87.8%   170/843   20.2%
+PETS / PoPETs             510   123/510   24.1%   477/510   93.5%   215/510   42.2%
+IMC                       638   132/638   20.7%   625/638   98.0%    67/638   10.5%
+NDSS                      701   129/701   18.4%   586/701   83.6%   144/701   20.5%
+ACM CCS                   990   163/990   16.5%   818/990   82.6%   186/990   18.8%
+USENIX Security          1410   221/1410  15.7%  1226/1410  87.0%   367/1410  26.0%
+IEEE S&P                  767   110/767   14.3%   646/767   84.2%   208/767   27.1%
+</code>
 ===== Report script output (unedited) =====
@@ Line 548: / Line 678: @@
   * **Accepted (major).** The page said ''external_checks_conferences.sh'' re-fetches the published deadlines, ICORE ranks and Scholar h5 values, but the script omitted Scholar h5 for SOUPS, ESORICS and RAID (they are below the first page of the security category list), ICORE for USENIX, NDSS, EuroS&P and AsiaCCS, and the exact CHI 10–14 May / USENIX Denver 11–13 Aug conference dates. Applied: those fetches are now in the script. SOUPS / ESORICS / RAID use Scholar ''search_venues'' (the live ESORICS table omits the acronym; h5 is still **34**). Re-ran 2026-08-27, exit 0, no FAILURE lines.
   * **Accepted (minor).** This log was empty while the generic pass was in flight. This paragraph is the disposition.
-  * **Rejected (minor).** ''check_page_numbers.mjs --code'' flags ''100'' from ''100 * n / d'' in the embedded script. That is a percentage multiplier, not a corpus figure. Whole-page checking without ''--code'' passes. The shared guard was not changed.
+  * **Rejected (minor).** ''check_page_numbers.mjs %%--code%%'' flags ''100'' from ''100 * n / d'' in the embedded script. That is a percentage multiplier, not a corpus figure. Whole-page checking without ''%%--code%%'' passes. The shared guard was not changed.
+=== Post-publish edit (2026-08-27) ===
+The content page was rewritten so it is a submission map: where to send the paper, and the current CFP with dates. ''venue_rates.py'' and its extract/run1 output moved here — readers of [[writing:conferences]] do not have the dataset. Mention-sweep columns, poster rates, and the methodology section left this page's population table and the report output. No new review freeze.
+===== Markup sweep, 2026-09-17 =====
+Mechanical rendering repair only: a fresh live raw/XHTML export of 188 pages was checked with ''check_wrap.mjs'' and ''check_typography.mjs''. Affected plugin tags, CLI flags and heading markup were repaired; no figures or substantive prose were changed. The resulting source and rendered DOM were re-checked after saving.