LessWrong (Curated & Popular) "You can’t imitation-learn how to continual-learn" by Steven Byrnes
Mar 23, 2026
Steven Byrnes, author and essayist on ML theory, argues for a sharp difference between imitation learning and true continual learning. He sketches model-based reinforcement learning and lifelong weight updates. He contrasts in-context tricks with decades-long within-lifetime learning, explores thought experiments like a sealed genius country, and explains why a frozen transformer cannot reproduce ongoing learning dynamics.
AI Snips
Chapters
Transcript
Episode notes
Continual Learning Builds New Conceptual Abilities
- Continual learning means acquiring new knowledge and conceptual frameworks, not just tracking facts over time.
- Steven Byrnes contrasts RL agents and human development as examples that build new conceptual abilities from scratch through weight updates.
Context Windows Aren't A Substitute For Weight Updates
- Extending context windows or better scratchpads addresses forgetting but doesn't substitute for within-lifetime learning that changes model weights.
- Byrnes argues no context window can turn GPT-2 into GPT-5 or replace decades of human learning.
Sealed Country Thought Experiment Shows Open-Ended Growth
- Byrnes gives a thought experiment sealing a country of geniuses in VR for 100 years to show emergent new sciences and philosophies.
- He asks whether LLMs with only context changes could reproduce that open-ended cultural and conceptual growth.