The Data Exchange with Ben Lorica World Models Are Here—But It’s Still the GPT-2 Phase
Mar 19, 2026
Jeff Hawke, CTO of Odyssey, builds general-purpose world models that generate interactive visual simulations from images or text. He explains how continuous video-like models are trained, early use cases like games and robotics, compute and latency challenges, stability limits on long runs, and the path toward scalable, real-time and on-device deployments.
AI Snips
Chapters
Transcript
Episode notes
World Models Are In The GPT-2 Phase
- World models are early and exploratory—Jeff Hawke compares the current period to the GPT-2 era of LLMs.
- He expects many experimental use cases (games, retail, live events, robotics) before mass commercialization.
Use Transformers But Train For Dynamics
- Use transformer architectures as the backbone but design training regimes for dynamics rather than static generation.
- Jeff Hawke stresses transformers underpin frontier models, but framing and training determine whether you learn world dynamics.
Current World Models Struggle With Long Rollouts
- Major limitations today are short-range prediction and instability over long rollouts; state-of-the-art contiguous prediction moved from 15–30 seconds (mid‑2025) to roughly 1–2 minutes.
- Instability appears as visual degradation or 'psychedelic' noise when models go out of distribution.