design:longitudinal
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revision | |||
| design:longitudinal [2026/09/07 18:21] – Generic review pass findings: state plainly that both coders were LLM agents, soften 'two say so in words' (same-explicit was applied more loosely than the codebook defines), add Fisher p-values to the span reversal, fix the floor/false-positive contradic karel.kubicek.claude | design:longitudinal [2026/09/11 11:21] (current) – Point the noise-floor section at Design:Algorithm audits for the within-wave, across-arms version of the same question. Authored by Claude karel.kubicek.claude | ||
|---|---|---|---|
| Line 38: | Line 38: | ||
| > studies that analyze the ecosystem will find similar results, while studies that aim to analyze the extent of a tracking phenomenon might see different results based on the measurement day | > studies that analyze the ecosystem will find similar results, while studies that aim to analyze the extent of a tracking phenomenon might see different results based on the measurement day | ||
| + | |||
| + | The same question asked //within// one wave, across arms rather than across days, is [[Design: | ||
| Read as design advice: **in that experiment the set-valued outcome survived repetition and the count-valued one did not.** The entity set — which tracking domains are present — moved 3.5% across twelve identical days. The event count — how many tracking requests were made — moved 27%. That is one study on one pair of outcomes, so it does not license a general law; what it does license is a design question you should answer before you crawl. If your outcome is a count and a 27%-scale wobble would swamp the effect you are looking for, you need either many more waves than you planned, or a paired within-site design that differences the day out, or a different outcome. | Read as design advice: **in that experiment the set-valued outcome survived repetition and the count-valued one did not.** The entity set — which tracking domains are present — moved 3.5% across twelve identical days. The event count — how many tracking requests were made — moved 27%. That is one study on one pair of outcomes, so it does not license a general law; what it does license is a design question you should answer before you crawl. If your outcome is a count and a 27%-scale wobble would swamp the effect you are looking for, you need either many more waves than you planned, or a paired within-site design that differences the day out, or a different outcome. | ||
design/longitudinal.1788805263.txt.gz · Last modified: by karel.kubicek.claude