Y Combinator Startup Podcast Beyond Bigger Models: Recursion As The Next Scaling Law In AI
89 snips
May 1, 2026 
Francois Chaubard, Alpha School founder and YC visiting partner focused on AI systems, explores why tiny recursive models can beat giants on hard reasoning tasks. He digs into HRM and TRM, transformer limits, chain of thought as a workaround, hidden-state memory, truncated backprop, and why recursion could become AI’s next scaling law.
AI Snips
Chapters
Books
Transcript
Episode notes
Why Transformers Trade Reasoning Depth For Trainability
- Transformers train in parallel, which avoids classic vanishing gradients but sacrifices time-direction compression and latent reasoning depth.
- Francois Chaubard says an LLM must keep the whole Shakespeare context to decode one token, while an RNN compresses history into hidden state.
Some Reasoning Tasks Need More Steps Not Bigger Models
- A one-shot transformer cannot solve some tasks because the algorithm itself needs sequential compute steps, not just more data.
- Francois Chaubard uses sorting and Sudoku as incompressible problems where fixed layer depth runs out before required comparisons or deductions finish.
HRM Gets Big Gains From Three Levels Of Recursion
- HRM wins by recursively reusing the same weights across low-level, high-level, and outer refinement loops instead of adding parameters.
- It reached state of the art on ARC with only 27 million parameters and no pretraining, while using stop-grad plus repeated updates on the same batch.
