AXRP - the AI X-risk Research Podcast 49 - Caspar Oesterheld on Program Equilibrium
12 snips
Feb 18, 2026 Caspar Oesterheld, a PhD student and assistant director at CMU's Foundations of Cooperative AI Lab, studies multi-agent AI safety and program equilibria. He explains program games where programs read each other's code. They cover simulation-based versus proof-based cooperation, epsilon-grounded simulation tricks, shared randomness fixes, and why correlated versus private randomness changes what cooperation is possible.
AI Snips
Chapters
Transcript
Episode notes
Proof Search And Löb's Theorem Enable Robust Cooperation
- Proof-based bots search for proofs about opponent behavior (e.g., "if I cooperate, you cooperate"), enabling robust cooperation.
- Löb's theorem explains how such mutual proofs can be consistent and yield cooperation.
Break Simulation Loops With Tiny Randomized Moves
- Use randomized "epsilon-grounded" simulation: with tiny probability act unconditionally and otherwise simulate the opponent.
- This avoids infinite recursion while approximating mutual simulation-based cooperation.
Optimize Simulations Instead Of Running Deep Recursion
- Compiler-style optimizations can collapse long recursive simulations into sampling a depth and querying base actions, reducing runtime cost.
- That turns many recursive simulations into efficient sampled computations.