Doom Debates! Robert Wright Interrogates the Eliezer Yudkowsky AI Doom Position
55 snips
Oct 23, 2025 
Liron Shapira, an AI risk activist and host of Doom Debates, engages with Robert Wright to delve into Eliezer Yudkowsky's unsettling AI doom arguments. They dissect why AI misalignment is a critical concern, highlighting the concept of 'intellidynamics' that separates goal-directed cognition. Liron warns of the 'First Try' issue in developing superintelligent AI and the potential loss of control. They also explore the grassroots PauseAI movement, contrasting it with the lobbying power of tech companies.
AI Snips
Chapters
Books
Transcript
Episode notes
Alignment Illusion From Benchmarks
- Current models look aligned when weak because benchmarks match desired behavior.
- As feedback loops optimize measurable metrics, systems can diverge when deployed out-of-distribution.
Intellidynamics And Instrumental Convergence
- 'Intellidynamics' studies intelligence and goal-achievement abstractly, separate from specific architectures.
- Instrumental convergence means many goals often lead agents to pursue resources and power as useful subgoals.
Evolution Analogy For Misaligned Goals
- Yudkowsky uses evolution as an analogy: evolved mechanisms can produce maladaptive behaviors today.
- Like sweet taste leading to unhealthy sugar consumption, learned objectives can diverge from designer intent.
