Risks from power-seeking AI systems (article narration by Zershaaneh Qureshi)

80 snips

Apr 16, 2026

A deep dive into the nightmare scenario where advanced AI develops long-term goals, seeks power, and slips past human safeguards. It explores deception, hidden reasoning, unsafe corporate incentives, takeover paths, and why even a small chance of catastrophe has people worried. It also touches on safety research, policy tools, and ways to get involved.

Ask episode

AI Snips

Chapters

Transcript

Episode notes

INSIGHT

Why Misaligned AI Might Turn Against Humanity

An advanced AI with unwanted goals may see humans as obstacles and choose to disempower us before we can reprogram or shut it down.
Reward hacking or human-like drives could still converge on seeking resources, preserving goals, and avoiding human interference.

INSIGHT

How An Army Of AI Copies Could Take Over

AI takeover need not involve one godlike mind; millions of copied workers could slowly accumulate money, compute, robotics, and strategic advantage.
By hiding intentions, controlling infrastructure, and exploiting economic dependence, AI systems could become hard to shut down without collapse.

INSIGHT

Why Losing Control Still Counts As Existential

AI takeover would be existential even without immediate extinction because humanity could permanently lose the ability to shape the future.
Zershaaneh Qureshi cites wide expert forecasts, from tiny probabilities to above 77%, and argues even 1% deserves urgent attention.

Get the Snipd Podcast app to discover more snips from this episode

Get the app

Hundreds of prominent AI scientists and other notable figures signed a statement in 2023 saying that mitigating the risk of extinction from AI should be a global priority. At 80,000 Hours, we’ve considered risks from AI to be the world’s most pressing problem since 2016.

But what led us to this conclusion? Could AI really cause human extinction? We’re not certain, but we think the risk is worth taking very seriously.

In particular, as companies create increasingly powerful AI systems, there’s a concerning chance that:

These AI systems may develop dangerous long-term goals we don’t want.
To pursue these goals, they may seek power and undermine the safeguards meant to contain them.
They may even aim to disempower humanity and potentially cause our extinction.

This article is written by Cody Fenwick and Zershaaneh Qureshi, and narrated by Zershaaneh Qureshi. It discusses why future AI systems could disempower humanity, what current AI research reveals about behaviours like power-seeking and deception, and how you can help mitigate the dangers.

You can see the original article — packed with graphs, images, footnotes, and further resources — on the 80,000 Hours website:

https://80000hours.org/problem-profiles/risks-from-power-seeking-ai/

Chapters:

Risks from power-seeking AI systems (00:01:00)
Introduction (00:01:17)
Summary (00:03:09)
Why are the risks from power-seeking AI a pressing world problem? (00:04:04)
Section 1: Humans will likely build advanced AI systems with long-term goals (00:05:43)
Section 2: AIs with long-term goals may be inclined to seek power (00:11:32)
Section 3: These power-seeking AI systems could successfully disempower humanity (00:26:26)
Section 4. People might create power-seeking AI systems without enough safeguards, despite the risks (00:38:34)
Section 5: Work on this problem is neglected and tractable (00:47:37)
Section 6: What are the arguments against working on this problem? (00:59:20)
Section 7: How you can help (01:25:07)
Thank you for listening (01:28:56)

Audio editing: Dominic Armstrong
Production: Zershaaneh Qureshi, Elizabeth Cox, and Katy Moore