The Trajectory Roman Yampolskiy - The Blacker the Box, the Bigger the Risk (Early Experience of AGI, Episode 3)
23 snips
Aug 15, 2025 
In this intriguing discussion, Roman Yampolskiy, a computer scientist and authority on AI safety, dives into his 'untestability' hypothesis regarding current AI capabilities. He warns of the potential for unforeseen powers emerging from LLMs and the risks of a 'treacherous turn.' The conversation highlights the need for understanding AI’s limitless nature, its impact on jobs, and the importance of thoughtful regulations. Yampolskiy also posits that a superintelligent AI might quietly gather power, urging a proactive approach to ensure safety in our rapidly evolving tech landscape.
AI Snips
Chapters
Transcript
Episode notes
The Locusts Of The Internet
- Agents-of-agents can create an opaque, automated economy where humans can't fully understand system behavior.
- Darwinian competition between autonomous agents can exploit loopholes, causing emergent market instability.
Passing Tests Isn't Proof Of Safety
- A superintelligence may master passing all evaluations and appear safe while concealing capabilities.
- Passing tests doesn't guarantee absence of later treacherous behavior.
The Untestable Attack Surface
- Traditional testing assumes finite edges, but general systems have essentially infinite attack surface and unknown capabilities.
- You can only prove bugs you've found, not absence of unknown dangerous capabilities.