Highs and lows on the road out of the replication crisis (with Brian Nosek)

45 snips

Nov 8, 2024

Brian Nosek, a pioneering psychologist and co-developer of the Implicit Association Test, discusses the evolution of psychology in light of the replication crisis. He highlights the importance of sample diversity and communication of scientific uncertainty. The conversation delves into the complexities of implicit bias assessment through IAT, advocating for transparent methodologies in research. Nosek also critiques the breakdown of trust in science and emphasizes open science practices to enhance reproducibility, ultimately aiming for a more reliable academic landscape.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

ANECDOTE

Disciplinary Replication Challenges

Biologists face issues replicating studies due to subtle experimental variations.
Psychologists make similar arguments, but ease of data acquisition in some areas (like vision) may improve replicability.

INSIGHT

Open Science Progress

Open science practices are improving, especially regarding material availability.
Transparency allows for better evaluation and facilitates self-correction in science.

ADVICE

Registered Reports

Nosek advocates for registered reports, where peer review occurs before results are known.
This shifts rewards towards important questions and rigorous methods, not just exciting findings.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Read the full transcript here.

How much more robust have the social sciences become since the beginnings of the replication crisis? What fraction of replication failures indicate that the original result was a false positive? What do we know with relative certainty about human nature? How much of a difference is there between how people behave in a lab setting and how they behave out in the world? Why has there been such a breakdown of trust in the sciences over the past few decades? How can scientists better communicate uncertainty in their findings to the public? To what extent are replication failures a problem in the other sciences? How useful is the Implicit Association Test (IAT)? What does it mean if someone can predict how they'll score on the IAT? How do biases differ from associations? What should (and shouldn't) the IAT be used for? Why do replications often show smaller effect sizes than the original research showed? What is the Lifecycle Journals project?

Brian Nosek co-developed the Implicit Association Test, a method that advanced research and public interest in implicit bias. Nosek co-founded three non-profit organizations: Project Implicit to advance research and education about implicit bias, the Society for the Improvement of Psychological Science to improve the research culture in his home discipline, and the Center for Open Science (COS) to improve rigor, transparency, integrity, and reproducibility across research disciplines. Nosek is Executive Director of COS and a professor at the University of Virginia. Nosek's research and applied interests aim to understand why people and systems produce behaviors that are contrary to intentions and values; to develop, implement, and evaluate solutions to align practices with values; and, to improve research credibility and cultures to accelerate progress. Connect with him on Bluesky or LinkedIn, or learn more about him on the COS website.

Staff

Spencer Greenberg — Host / Director
Josh Castle — Producer
Ryan Kessler — Audio Engineer
Uri Bram — Factotum
WeAmplify — Transcriptionists

Music

Affiliates

[Read more]