AI Snips
Chapters
Transcript
Episode notes
Three Different Rs Matter Separately
- Replication, reproducibility, and robustness are distinct concepts that need separate attention.
- Replication tests results in new data, reproducibility reruns analyses on the same data, and robustness checks many analysts on one dataset.
DARPA Replication Project Found About 50% Success
- Large DARPA replication project found roughly a 50% subjective replication success rate across 296 replications.
- Different success metrics yield different rates, but overall effect sizes in replications were much smaller than originals.
Replicated Effects Are Often Much Smaller
- Replicated effect sizes were much smaller than originals across fields (e.g., psychology r 0.29→0.11).
- This 'decline effect' shows originals often overestimate effect sizes, leaving smaller but sometimes still real effects.