Astral Codex Ten Podcast Next-Token Predictor Is An AI's Job, Not Its Species
6 snips
Apr 2, 2026 A deep take on whether advanced AI is merely a next-token predictor or something with layered cognition. A layered analogy compares evolution, learning, and chips to argue levels of explanation matter. Weird representations like helical manifolds and mechanistic interpretability show internal complexity. The discussion rebuts the ‘stochastic parrot’ view and teases practical implications for AI design.
AI Snips
Chapters
Transcript
Episode notes
Prediction Algorithms Create World Models
- The brain and LLMs are both shaped by outer optimization processes yet operate with internal world models rather than explicit outer goals.
- Scott Alexander compares evolution→predictive coding→world model to company incentives→next-token training→LLM world model to show parallel levels.
Next Token Objective Is Not The Model's Inner Thought
- Next-token prediction is the training objective, but the model's internal computations don't literally think in token-probability steps when solving tasks.
- Scott explains math or world reasoning arises from abstracted representations, not token-by-token arithmetic reasoning.
Outer Optimization Doesn't Appear In Everyday Thought
- Being optimized by a process (evolution or companies) doesn't mean each thought references that process; proximate cognition can be far removed from distal objectives.
- Scott uses the example of solving a math problem without thinking about reproduction to illustrate levels.