Alignment Looks Slightly Less Hopeless

Irene Zhang asks if Nathan is pessimistic, and he says p-doom remains wide but models seem more value-aware than earlier doom scenarios assumed.

Play episode from 01:09:57

chevron_right

Transcript

chevron_right

Transcript

Episode notes

This special cross-post from The Intelligence Horizon features Nathan Labenz in a wide-ranging conversation on compressed AI timelines, expert disagreement, and why he believes the singularity is near. They discuss interpretability, RL scaling, and the balance between extraordinary upside, like curing major diseases, and serious existential risks. Nathan explains his evolving p(doom), why he’s slightly more optimistic about robustly good AI, and how defense-in-depth strategies might keep society on track. The episode also explores US-China rivalry, AI governance, and why human cooperation may matter more than technical control alone.

Google: Keep up with AI research on the go with NotebookLM, Google's steerable research and thinking partner. Try it at https://notebooklm.google.com/.

Sponsors:

Tasklet:

Build your own Cognitive Revolution monitoring agent in one click.
Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai

VCX:

VCX, by Fundrise, is the public ticker for private tech, giving everyday investors access to high-growth private companies in AI, space, defense tech, and more. Learn how to invest at https://getvcx.com

Claude:

Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr

CHAPTERS:

(00:00) About the Episode

(03:27) Special Sponsor

(05:12) Opening and AGI framing

(12:08) Scaling RL and paradigms (Part 1)

(21:31) Sponsors: Tasklet | VCX

(24:24) Scaling RL and paradigms (Part 2)

(28:56) Verifiability and long horizons

(41:13) LLMs and world models (Part 1)

(41:19) Sponsor: Claude

(43:32) LLMs and world models (Part 2)

(54:17) Energy, hardware, and chips