Differences

This shows you the differences between two versions of the page.

--- provenance:artifacts [2026/09/17 08:13] – Qualify root start link; Authored by Claude karel.kubicek.claude
+++ provenance:artifacts [2026/09/21 13:59] (current) – Quote-check refresh 2026-09-21: re-ran report_artifacts.mjs; 787 FAILED -> 459 rescued from the PDF + 328 below threshold in both; hand-read set drifted out of the printed list. Authored by Claude karel.kubicek.claude
@@ Line 60: / Line 60: @@
 ===== Evidence quotes checked =====
-,539 papers with an artifacts record. ''artifacts.evidence.quote'' against ''paper.cols.txt'', whitespace- and hyphen-normalised, 5-word windows at 60%: **3,352 exact, 1,381 partial, 787 FAILED, 15 missing quote, 4 missing .cols**.
+,539 papers with an artifacts record. ''artifacts.evidence.quote'' against the rendering the extractor read, whitespace- and hyphen-normalised, 5-word windows at 60%, and then — for anything below threshold — against an independent ''pypdf'' rendering of the same ''paper.pdf''. **Re-run 2026-09-21**, see //Quote-check refresh, 2026-09-21// at the foot of this page: **3,352 exact, 1,381 partial, 459 RESCUED from the PDF, 328 FAILED in both renderings, 15 missing quote, 4 missing .cols**.
-The first 15 FAILED rows were read. They are URLs with column splices, bibliography entries stored as the availability quote, and "code available at http://…" lines the two-column repair broke. Same class ''quote_check.mjs'' warns about. None looked fabricated. No FAILED quote is used on the content page. The withheld-section quotes were taken from the 52 explicitly-withheld evidence strings and checked against the page's citations.
+**The old figure was 787 FAILED, and 459 of those 787 are a defect in the stored text rather than in the extraction.** 328 are below threshold in both renderings. The FAILED rate over the 5,539 is **5.9%**, not 14.2%.
+**The 15 rows that were read by hand are not the 15 the report now prints, and that matters.** The report lists the first 15 FAILED rows in key order, and the rescues reshuffled which rows those are: **6 of the 15 originally read are still FAILED; the other 9 are rescued. The 9 new rows now printed have not been read.** Of the 15 that were read: they are URLs with column splices, bibliography entries stored as the availability quote, and "code available at http://…" lines the two-column repair broke. Same class ''quote_check.mjs'' warns about. None looked fabricated. No FAILED quote is used on the content page. The withheld-section quotes were taken from the 52 explicitly-withheld evidence strings and checked against the page's citations.
 ===== External and industry verification =====
@@ Line 477: / Line 479: @@
 Union dropped = 251 (every poster is also ≤4 pages, so the union is the short-record count). The own-link rate moves 56.7% → 58.4%.
-## L. Quote check of artifacts.evidence.quote against paper.cols.txt
+## L. Quote check of artifacts.evidence.quote against paper.cols.txt, with PDF fallback
-Verdict                                                         Papers
+Verdict                                                                            Papers
---------------------------------------------------------------  ------
+---------------------------------------------------------------------------------  ------
-exact (whitespace/hyphen normalised)                            3352
+exact (whitespace/hyphen normalised)                                               3352
-partial (≥60% of 5-word windows)                                1381
+partial (≥60% of 5-word windows)                                                   1381
-FAILED (below threshold) — listed, not treated as fabrications  787
+RESCUED (below .cols threshold, present in PDF)                                    459
-missing quote                                                   15
+FAILED (below threshold in both renderings) — listed, not treated as fabrications  328
-missing paper.cols.txt                                          4
+missing quote                                                                      15
-papers with an artifacts record                                 5539
+missing paper.cols.txt                                                             4
+papers with an artifacts record                                                    5539
 First FAILED quotes (max 15). Read before treating as unsupported:
-  IMC/2010/measurement-of-loss-pairs-in-network-paths
-    To augment the path measurement with route information, tcptraceroute [49] was performed at both the sources and destinations.
   IMC/2010/measurement-and-analysis-of-real-world-802-11-mesh-networks
     The data set analyzed in this thesis includes measurements collected from 110 different production Meraki wireless mesh networks located around the world
   CCS/2010/dismantling-securememory-cryptomemory-and-cryptorf
     Sample code available at http://www.libnfc.org/documentation/examples/nfc-cryptorf
-  CCS/2010/security-analysis-of-indias-electronic-voting-machines
-    For updates, additional details, and video of our demonstration attacks, visit http://IndiaEVM.org.
   CCS/2010/the-security-of-modern-password-expiration-an-algorithmic-framework-and-empirica
     We employed various approaches to crack passwords: dictionary-based password cracking using "John the Ripper" (http://www.openwall.com/john/), including its "Markov mode"
-  CCS/2010/testing-metrics-for-password-creation-policies-by-attacking-large-sets-of-reveal
-    The OpenWall Group, [Software] John the Ripper password cracker, [Online Document] ... Available HTTP http://www.openwall.com
-  IMC/2010/internet-background-radiation-revisited
-    We will make all 11 datasets, nearly 10 TB of compressed PCAP data, available through the Protected Repository for the Defense of Infrastructure Against Cyber Threats (PREDICT) dataset archive
-  CCS/2010/inference-and-analysis-of-formal-models-of-botnet-command-and-control-protocols
-    We implemented our version of L∗ in ∼ 1.7 KLOC of C++ and the bot emulator and experimental infrastructure in ∼ 2.3 KLOC of Python and Bash scripts.
-  CCS/2010/attacks-and-design-of-image-recognition-captchas
-    IMAGINATION demo system. http://goldbach.cse.psu.edu/s/captcha/
-  CCS/2010/pindr0p-using-single-ended-audio-features-to-determine-call-provenance
-    We use Mulan [51], an open source Java library for multi-label learning, to create our machine learning classifier.
-  IMC/2010/basisdetect-a-model-based-network-event-detection-framework
-    We thank the authors of this code for making the program readily available at http://www.eecs.umich.edu/∼cscott/code/mnscann.zip
   IMC/2010/primitives-for-active-internet-topology-mapping-toward-high-frequency-characteri
     Acknowledgments ... CAIDA for measurement infrastructure support
@@ Line 517: / Line 504: @@
   USENIX/2010/making-linux-protection-mechanisms-egalitarian-with-userfs
     DokuWiki. http://www.dokuwiki.org/dokuwiki.
-  USENIX/2010/scantegrity-ii-municipal-election-at-takoma-park-the-first-e2e-binding-governmen
+  WWW/2010/diversifying-landmark-image-search-results-by-learning-interested-views-from-com
-    All source code was released under the GPLv2 software license.
+    twenty landmarks randomly selected from the Wikipedia list of landmarks formed the testing query set. 1 http://en.wikipedia.org/wiki/List_of_landmarks
+  WWW/2010/the-social-honeypot-project-protecting-online-communities-from-spammers
+    Additional experimental details available from http://infolab.tamu.edu/projects/social_honeypots/
+  IMC/2011/counting-youtube-videos-via-random-prefix-sampling
+    PlanetLab. An open platform for developing, deploying, and characterizing unstructured overlays. http://www.planet-lab.org.
+  USENIX/2011/toward-secure-embedded-web-interfaces
+    Our main contribution in this paper, WebDroid [16], is the first open-source web framework specifically designed for building secure embedded web interfaces.
+  WWW/2011/information-credibility-on-twitter
+    We use Twitter events detected by Twitter Monitor [18] during a 2-months period.
+  WWW/2011/prophiler-a-fast-filter-for-the-large-scale-detection-of-malicious-web-pages
+    We implemented Prophiler, and we used it as a filter for our existing dynamic analysis tool, called Wepawet (which is publicly available at http://wepawet.cs.ucsb.edu/).
+  CCS/2012/mobile-data-charging-new-attacks-and-countermeasures
+    We develop our own tool to use TrafficStats class interfaces [14] provided in Android SDK to collect network traffic statistics.
+  CCS/2012/operating-system-framed-in-case-of-mistaken-identity-measuring-the-success-of-we
+    Go to: http://saucers.cups.cs.cmu.edu/yacot/mnt/wtk/survey.php?i=workerID
+  CCS/2012/towards-measuring-warning-readability
+    W. Metze. Forschungsmanual zum Stolperwörter-Lesetest. http://www.agprim.uni-siegen.de/lust/stolpermanual.pdf, 2004.
 ## Y. Arithmetic-derived figures the page may print
@@ Line 621: / Line 624: @@
 The fresh rendered-DOM sweep found this page's bare ''start'' link resolved to ''provenance:start''. It is now root-anchored; the post-save DOM was re-checked for red links. No figures or citations changed.
+===== Quote-check refresh, 2026-09-21 =====
+The 2026-09-04 ''cols''-vs-PDF audit on [[:provenance:literature:corpus]] showed that 73.1% of evidence quotes that cannot be located in ''paper.cols.txt'' **are** present in an independent ''pypdf'' rendering of the same ''paper.pdf''. ''report_artifacts.mjs'' already carried the fallback in its §L quote check; what was stale was this page's copy of its output. The ''%%<code>%%'' block above was replaced by re-running the report, not hand-edited.
+^ Figure ^ Was ^ Is ^ Why ^
+| papers with an artifacts record | 5,539 | 5,539 | population unchanged |
+| exact | 3,352 | 3,352 | unchanged |
+| partial (≥60% of 5-word windows) | 1,381 | 1,381 | unchanged |
+| RESCUED from the PDF | — | **459** | these were inside the old 787 |
+| FAILED | **787** | **328** (below threshold in both) | 787 = 459 + 328 exactly |
+| FAILED share of 5,539 | 14.2% | **5.9%** | 328 of 5,539 |
+| missing quote / missing ''.cols'' | 15 / 4 | 15 / 4 | unchanged |
+**Everything else in the report reproduced byte-for-byte.** A line-by-line diff of the published block against the fresh run differs **only** inside §L — the heading, the verdict table and the printed FAILED list. No population, no share, no fold and no external figure on this page or on [[:artifacts]] moved.
+**The hand-read set drifted out from under the printed list.** The report prints the first 15 FAILED rows in key order. Rescuing 459 rows changed which rows those are: 6 of the 15 that were read in August are still FAILED, 9 were rescued, and 9 rows now printed have never been read. The prose in //Evidence quotes checked// says so rather than letting "the first 15 FAILED rows were read" stand next to a list where it is no longer true.
+^ Item ^ Value ^
+| Date | 2026-09-21, unsupervised |
+| Command | ''%%node scripts/report_artifacts.mjs%%'' |
+| Artifact | ''out/qc0921b/rep_artifacts.txt'' |
+| Script changes | none |
+| Reviewers | one ''sonnet'' figures-vs-script pass |
+| Pages saved | this page only — [[:artifacts]] publishes none of these counts |
+| Open | the 9 newly-printed FAILED rows, and the 328 as a whole, have not been read by hand |