Chain of Thought | AI Agents, Infrastructure & Engineering Why LLMs Are Plausibility Engines, Not Truth Engines | Dan Klein
Apr 8, 2026
Dan Klein, co-founder and CTO of Scaled Cognition and UC Berkeley CS professor known for NLP and conversational AI, explains why large language models are plausibility engines, not truth machines. He covers why prototypes fail to ship, the limits of prompting, APT1’s action-first architecture, metacognition to curb hallucinations, and why stacking models and benchmarks can give a false sense of reliability.
AI Snips
Chapters
Transcript
Episode notes
Modularity Enables Trustworthy Systems
- Traditional software relies on modular decomposition with defined contracts; end-to-end ML often discards that, reducing ability to guarantee behavior.
- Klein argues you need architectural alignment between model structure and desired guarantees for shipping high-stakes systems.
Build Models Around Actions Not Tokens
- Design models with actions and information as first-class objects instead of only tokens to get semantic control and enforceable policies.
- Klein says actions have semantics (allowed/forbidden, data flow) enabling stronger guarantees than token outputs.
Agentic Data Requires Goals Actions And Metacognition
- Agentic data must include goals, actions (APIs/tools), and human intent; such data rarely exists and must be created for training trustworthy agents.
- Klein emphasizes metacognition: models should track what they know, what they can do, and information provenance to avoid hallucinations.