Survivorship Bias in Prophecy Cases

Ever heard of cognitive dissonance? That thing a psychology lecturer might have explained to you once upon a time, likely using the same UFO cult example everyone else uses. Well, a new paper by Thomas Kelly suggests that the UFO cult example might have been ever so slightly oversold.

Kelly's archival work suggests that the researchers didn't just observe the cult as reported. Instead, they infiltrated it, faked supernatural experiences, assumed quasi-leadership roles, and then wrote up the results as if the group had spontaneously doubled down on their failed prophecy, which they had not. Because the leader recanted, and the group fell apart shortly after the failed prophecy. Minor details.

Matt and Chris discuss this paper, a 2024 multilab replication, and some other papers by Kelly, considering the ever-reliable tendency of researchers to find exactly what they are looking for.

It's cognitive dissonance all the way down, folks.

The full episode is available to Patreon subscribers (1 hour, 10 minutes).

Join us at: https://www.patreon.com/DecodingTheGurus

Decoding Academia 34: When Prophecy Fails Debunked?

00:00 Introduction

02:04 Cognitive Dissonance Theory

06:41 Classic lab evidence: effort justification & the ‘severe initiation’ study

08:33 When Prophecy Fails: The Original Account

10:54 The debunking: archival evidence, misconduct claims, and ethical red flags

20:22 Replication reality check: multi-lab results and ‘strong vs weak’ dissonance

31:40 Beyond one case: survivorship bias, failed prophecies, and early Christianity parallels

35:51 Christianity as Historical Anomaly or Cognitive Dissonance Exemplar?

41:48 Thomas Kelly: Interesting biosafety takes and a possible Christian lens

45:43 The importance of seeking for disconfirming evidence

50:23 Conspiracy-theory dynamics & narrative elaboration

56:30 Classical Psychological Theories and Personal Motivations

01:03:07 Steps that can be taken to reduce biases

01:05:01 Stay tentative, check evidence, and don’t pick sides too fast

01:06:30 A lesson from Scott Alexander!

Sources

Academic Papers and Books

1. Festinger, L. (1957). A theory of cognitive dissonance. Stanford University Press.
2. Festinger, L., Riecken, H. W., & Schachter, S. (1956). When prophecy fails. University of Minnesota Press.
3. Festinger, L., & Carlsmith, J. M. (1959). Cognitive consequences of forced compliance. The Journal of Abnormal and Social Psychology, 58(2), 203–210. https://doi.org/10.1037/h0041593 (The original induced-compliance/$1/$20 study)
4. Kelly, T. (2026). Debunking "When Prophecy Fails." Journal of the History of the Behavioral Sciences, 62(1), e70043. https://doi.org/10.1002/jhbs.70043
5. Kelly, T. (2025). Failed prophecies are fatal. International Journal for the Study of New Religions, 14(1), 48–71. https://doi.org/10.1558/ijsnr.33085
6. Aronson, E., & Mills, J. (1959). The effect of severity of initiation on liking for a group. The Journal of Abnormal and Social Psychology, 59(2), 177–181. https://doi.org/10.1037/h0047195
7. Vaidis, D. C., Sleegers, W. W. A., van Leeuwen, F., DeMarree, K. G., Sætrevik, B., Ross, R. M., ... & Priolo, D. (2024). A multilab replication of the induced-compliance paradigm of cognitive dissonance. Advances in Methods and Practices in Psychological Science, 7(1), 1–26. https://doi.org/10.1177/25152459231213375
8. Croyle, R. T., & Cooper, J. (1983). Dissonance arousal: Physiological evidence. Journal of Personality and Social Psychology, 45(4), 782–791. https://doi.org/10.1037/0022-3514.45.4.782 (The study that the Vaidis et al. 2024 multilab replication was based on)

Podcasts Referenced

1. The Studies Show [formerly Science Fictions] podcast. Episode 90: Cognitive dissonance.
2. QAA Podcast. Episode 350: “When ‘When Prophecy Fails’ Fails.” Interview with Thomas Kelly.
3. Conspirituality podcast. Episode 284: “When Prophecy-Science Fails” (w/ Thomas Kelly), 20 Nov 2025.

Blog Posts & Other Sources

1. Alexander, S. (2023, February 14). Contra Kavanagh on fideism. Astral Codex Ten. (Contains the PMDD / Slate vs. Vox example discussed near the end of the episode)
2. Kavanagh, C. (2023). Am I a fideist? Medium. (Chris’s response to Scott Alexander)
3. Alexander, S. (2023, February 15). Trying again on fideism. Astral Codex Ten.
4. Kelly, T. Open Science Framework repository containing scanned archival materials from the Festinger papers (Bentley Historical Library, University of Michigan).
5. Kelly, T. (2025, March 18). Yet another White House says it won't fund engineered deadly viruses. Tablet Magazine.
6. Kelly, T. Christians for Impact. Politics and policy.