Open Science, Psychology, and the Art of Not Quite Claiming Causality with Julia Rohrer

18 snips

Jan 30, 2026

Julia Rohrer, a psychologist at Leipzig University focused on open science and causal reasoning. She discusses the state of psychology after the replication crisis. Conversation covers limits of open-science reforms, why causal thinking matters even for association studies, how experiments can mislead via post-treatment bias, and practical steps to state causal questions and assumptions clearly.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Post-Treatment Exclusions Undo Randomization

Excluding participants post-randomization introduces post-treatment bias and breaks the experiment's causal guarantee.
Rohrer warns that routine exclusion (e.g., failed manipulation checks) often invalidates causal claims.

ADVICE

Frame Papers Around Explicit Assumptions

State the causal question you care about, list identification assumptions, then present conclusions conditional on those assumptions.
Put uncertainty into conclusions, not into vague framing of the research question.

INSIGHT

Transparency Feels Risky But Beats Hidden Assumptions

Openness about assumptions can feel self-defeating because reviewers attack them, yet hiding assumptions is worse.
Rohrer notes economics enforces open identification norms and psychology could emulate that.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

In a rare departure from our usual diet of online weirdos, this episode features an academic who is very much not a guru. We’re joined by Julia Rohrer, a psychologist at Leipzig University whose work straddles the disciplinary boundaries of open science, research transparency, and causal inference. Julia is also an editor at Psychological Science and has spent much of the last decade politely pointing out that psychologists often don’t quite know what they’re estimating, why, or under which assumptions.

We talk about the state of psychology after the replication crisis, whether open science reforms have genuinely improved research practice (or just added new boxes to tick), and why causal thinking is unavoidable even when researchers insist they are “only describing associations.” Julia explains why the standard dance of imply causality → deny causality → add boilerplate disclaimer is unhelpful, and argues instead for being explicit about the causal questions researchers actually care about and the assumptions required to answer them.

Along the way we discuss images of scientists in the public and amongst the gurus, how post-treatment bias sneaks into even well-intentioned experimental designs, why specifying the estimand matters more than running ever-fancier models, and how psychology’s current norms can potentially punish honesty about uncertainty. We also touch on her work on birth-order effects and offer some possible reasons for optimism.

With all the guru talk, people sometimes ask us to recommend things that we like, and Julia's work is one such example!

Links

Julia Rohrer’s website
The 100% CI blog
Rohrer, J. M. (2024). Causal inference for psychologists who think that causal inference is not for them. Social and Personality Psychology Compass, 18(3), e12948.
Rohrer, J. M., Tierney, W., Uhlmann, E. L., DeBruine, L. M., Heyman, T., Jones, B., ... & Yarkoni, T. (2021). Putting the self in self-correction: Findings from the loss-of-confidence project. Perspectives on Psychological Science, 16(6), 1255-1269.
Rohrer, J. M., Egloff, B., & Schmukle, S. C. (2015). Examining the effects of birth order on personality. Proceedings of the National Academy of Sciences, 112(46), 14224-14229.
BEMC MAY 2024 - Julia Rohrer - "Causal confusions correlate with casual conclusions"
Dr. Tobias Dienlin - Less casual causal inference for experiments and longitudinal data: Research talk by Julia Rohrer