| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| start [2026/08/27 14:18] – Gloss the Design:Automated measurements start link: crawl, scan, or unpack an app. Authored by Claude karel.kubicek.claude | start [2026/09/07 07:03] (current) – Rewrite: index stops being a catalogue — namespace pages own their own page lists. Adds Start here entry points, provenance/corpus reading note, and links the new roadmap. Authored by Claude karel.kubicek.claude |
|---|
| Empirical studies on the web require researchers to navigate a complex landscape of experimental design choices, ranging from selecting a representative sample of websites to choosing the appropriate crawling technology. Similarly, analyzing results involves critical decisions, such as website categorization and statistical methodology. Too often, these decisions are made based on limited guidance, informal advice, or trial and error, despite their profound impact on research outcomes and their applicability. | Empirical studies on the web require researchers to navigate a complex landscape of experimental design choices, ranging from selecting a representative sample of websites to choosing the appropriate crawling technology. Similarly, analyzing results involves critical decisions, such as website categorization and statistical methodology. Too often, these decisions are made based on limited guidance, informal advice, or trial and error, despite their profound impact on research outcomes and their applicability. |
| |
| Measure The Web aims to bridge this gap, by providing evidence-based guidance for empirical web measurement studies. The platform evaluates design choices and their implications by referencing relevant academic publications or conducting original studies where necessary. | Measure The Web aims to bridge this gap, by providing evidence-based guidance for empirical web measurement studies. The platform evaluates design choices and their implications by referencing relevant academic publications or conducting original studies where necessary. It is written for someone starting a PhD in web privacy or security measurement, and for experienced researchers looking for current practice or for a citation that justifies a design they have already chosen. If you are in a mentoring position, consider sharing it with your mentees — and consider [[contributing]]: even a small edit that backs a subjective claim with a citation makes the site stronger. |
| |
| For novice researchers in the web measurement field, Measure The Web should provide a complete knowledge base to conduct their study. But it should be helpful also to experienced researchers, who can find here newest practices or proper arguments (or citations) to justify their design. If you are in a mentoring position, consider sharing the website (or directly [[Research journey]]) to your mentees and also consider reviewing and contributing to the page. Even small edits, such as supporting some subjective claim, can help make this website stronger. | ===== Start here ===== |
| |
| | ^ If you… ^ Start at ^ |
| | | have a study idea and no instrument yet | [[Design:Automated measurements]] — crawl, scan, or unpack an app, and what each can answer | |
| | | have data and need to turn it into labels | [[Design:Website classification]] for the protocol; [[Privacy]] or [[Security]] for the axis | |
| | | have results and need to defend them | [[Statistics:Biases]] — what your frame, vantage and denominator already did to the number | |
| | | want to know what order to do this in | [[Research journey]] — the decisions that are expensive to revisit later | |
| |
| ===== Outline ===== | ===== The seven namespaces ===== |
| |
| The website is organized as follows. Note that it is ordered by website structure, for order by research design, navigate to [[Research journey]]. | Each namespace page carries its own annotated list of the pages inside it, what a student needs each one for, and where the namespace stops. That list lives there rather than here, so it stays current. |
| |
| * [[Design|Research design]] clarifies choices needed to conduct the measurements. Example pages: | ^ Namespace ^ What it is for ^ |
| * [[Design:Automated measurements]] — crawl, scan, or unpack an app; the instrument is the design decision — or [[Design:User studies]] | | [[Design]] | Choices made **before the crawler runs**: which list stands in for the web, how you draw from it, where you appear to be, live or archived, and what you hold fixed between waves. | |
| * [[Design:Website selection]] and [[Design:Sampling|Representative sampling methods]], [[Design:Website classification]], and [[Design:IP classification]] | | [[Programming]] | **The instrument**: which browser, which control protocol, what happens after ''page.goto'', whether the profile survives, and what file you write. | |
| * [[Design:Crawling location]] and [[Design:Archives|Crawling live or using archives]] ([[Programming:Traffic files|using traffic files, e.g., HAR]]) | | [[Privacy]] | **Classifying what a crawl observed** on the privacy axis — requests, cookies, scripts, fingerprints, syncing, consent — and the two ways the instrument lies to you. | |
| * [[Design:Longitudinal|Repeating a measurement over time]] — pinning the list, the browser, the vantage point and the classifier so wave two is comparable to wave one | | [[Security]] | **Web security as it is deployed**, not how to break it: TLS, headers, label sources for "malicious", phishing in the wild. | |
| * [[Design:Mobile and app measurement|Measuring mobile apps]] — store scraping, static and dynamic analysis, and the certificate-pinning problem that decides whether you saw the traffic at all | | [[Statistics]] | **What these methods do to web-measurement data**: the unit-of-analysis error, the hypothesis family a crawl invents, hand-coded ground truth. Not a textbook. | |
| * [[Design:Platforms|Research of specific large platforms]] such as [[Design:Platforms:Facebook]], [[Design:Platforms:Twitter]], [[Design:Platforms:TikTok]], [[Design:Platforms:Amazon]] | | [[Practices]] | **Around the study rather than inside it**: ethics review, notifying operators, regulators, and the sentence that gets quoted once the paper is public. | |
| * [[Programming]] | | [[Writing]] | **Where the paper goes**, and how the related-work section is allowed to count. | |
| * [[Programming:Crawler|Comparison of crawling libraries]] such as [[Programming:Crawler:OpenWPM]], [[Programming:Crawler:Tracker Radar Collector]], [[Programming:Crawler:PageGraph]], etc. | |
| * [[Programming:Multilingual support]] | Outside the seven: [[Artifacts]] — a GitHub URL is not an archival artifact — and [[literature:bibliography|the bibliography]] every citation on this site resolves against. |
| * [[Programming:Interaction|Interaction with websites]] | |
| * [[Programming:Registration|Automating login and registration]] | ===== How to read a page ===== |
| * Multitude of pages linked from elsewhere documenting specific technologies, e.g., [[Programming:Tranco]], [[Programming:Cloudflare Radar]], [[Programming:Docker]], [[Programming:CrUX]], [[Programming:Similarweb]]. | |
| * [[Privacy]] | Pages whose figures come from the publication corpus carry a working log under ''provenance:'', mirroring the page id — [[provenance:privacy:fingerprinting]] for [[Privacy:Fingerprinting]]. Each log holds every query with its denominator, the report script and its unedited output, the name-folds and their unmapped residue, the review passes, and what could not be established. |
| * Classifying [[Privacy:Requests|Web requests]], [[Privacy:Cookies]], [[Privacy:Fingerprinting]], or [[Privacy:JavaScript]] | |
| * [[Privacy:Server side tracking|Measuring server-side tracking]] — when the tracking request never reaches the tracker, and every request-level method quietly stops working | The corpus is seven venues — CCS, IMC, NDSS, PETS, USENIX Security, TheWebConf, IEEE S&P — 2010–2026, **5,859** extracted papers. **Every figure is about the population its query defines**: only **1,120** of those papers ran a crawl, **1,762** ran statistical inference, **402** assess a law. Two pages quoting different populations correctly are not in conflict. 2025 is thin and 2026 is provisional; the corpus-wide caveats are on [[Literature:Corpus]]. |
| * [[Privacy:Cookie syncing|Measuring cookie and ID syncing]] — how third parties learn that their two identifiers are the same person, and the crawl configuration that decides whether you see it at all | |
| * [[Privacy:consent|Granting consent to websites]] | ===== What is coming, and what was ruled out ===== |
| * [[Privacy:TCF consent strings|Decoding TCF consent strings]] — the IAB TC string and Google's Additional Consent string as measurement artefacts, and what a decoded string does and does not prove | |
| * [[Privacy:darkpatterns|Dark patterns]] in the interfaces you measure — and in the way they defeat your crawler | [[Roadmap]] lists the pages that are scoped and queued but not yet written, with the evidence behind each one — and the proposals that were assessed and **deliberately** not written, with their paper counts, so the next gap pass does not re-propose them. |
| * [[Security]] — measuring web security as it is deployed, not how to break it. Children, proposed from the corpus and not yet written: | |
| * [[Security:TLS certificates|TLS and the web PKI]] | |
| * [[Security:Headers|CSP, HSTS and the other security headers]] | |
| * [[Security:VirusTotal|VirusTotal as a label source]] | |
| * [[Security:Phishing]] | |
| * [[Security:Web vulnerabilities|Measuring web vulnerabilities in the wild]] | |
| * Statistics: [[Statistics:Study preregistration]], [[Statistics:Hypothesis testing]] suitable for web measurements, [[Statistics:Regression]], [[Statistics:Pvalue corrections|P-value corrections]], [[Statistics:Biases]], [[Statistics:Interrater agreement|Inter-rater agreement]] for hand-coded data | |
| * [[Writing]] | |
| * [[Writing:Conferences]] — where to send a web-measurement paper, and the current CFP dates. | |
| * [[Writing:Literature review]] — the keyword-derived denominator trap, measured on the SoKs in this corpus. | |
| * [[Artifacts]] — a GitHub URL is not an archival artifact. What to put in a crawl deposit, where 2026 Available badges will accept it, and what to withhold when the data has people in it. | |
| * Pages whose figures come from the publication corpus carry a working log under ''provenance:'' — every query with its denominator, the report script and its unedited output, the folds and their residue, and what could not be established. The corpus-wide caveats are on [[Literature:Corpus]]; the per-page logs mirror the page id, e.g. [[provenance:privacy:fingerprinting]] for [[Privacy:Fingerprinting]]. | |
| * After publication steps to improve the impact of your work: | |
| * [[Practices:Notifying websites]] to remediate the security vulnerabilities and privacy threats. | |
| * [[Practices:Legal enforcement]] with information about local data protection authorities. | |
| * [[Practices:Public relations]] regarding how to present your results to public and journalists. | |
| * [[Practices|Other research practices]] explains various aspects affiliated to the research, but not directly the study design. | |
| * [[Practices:Ethics]] | |
| * [[https://github.com/pliang279/awesome-phd-advice|PhD and research advice]] | |
| * [[https://www.khstats.com/blog|Statistics]] | |
| |
| ===== Contributing ===== | ===== Contributing ===== |
| |
| We welcome contributions! Editing functionality is limited to registered users. You might want to check [[Contributing]] page, which helps with the syntax. | We welcome contributions. Editing is limited to registered users; [[Contributing]] covers the syntax, the namespace structure, and the rule that every published figure names its query. External reading we point at rather than duplicate: [[https://github.com/pliang279/awesome-phd-advice|PhD and research advice]] and [[https://www.khstats.com/blog|khstats]] on statistics. |
| |
| /* | /* |
