Learning Bayesian Statistics #139 Efficient Bayesian Optimization in PyTorch, with Max Balandat
5 snips
Aug 20, 2025 Max Balandat, who leads the modeling and optimization team at Meta, discusses the fascinating world of Bayesian optimization and the BoTorch library. He shares insights on the seamless integration of BoTorch with PyTorch, enhancing flexibility for researchers. The conversation delves into the significance of adaptive experimentation and the impact of LLMs on optimization. Max emphasizes the importance of effectively communicating uncertainty to stakeholders and reflects on the transition from academia to industry, highlighting collaboration in research.
AI Snips
Chapters
Transcript
Episode notes
BoTorch Began As A Side Project
- Max started BoTorch as a side project in early 2017 to overcome iteration friction in internal tooling.
- Prototypes quickly showed modular PyTorch+GPyTorch enabled much faster research cycles.
Open Source Accelerated Development
- BoTorch originated to speed internal R&D where previous stacks made prototyping slow and brittle.
- Open-sourcing leveraged Meta's culture and increased external feedback, hires, and bug fixes.
Scale With Engineering And Model Choice
- Scale BoTorch usage via software engineering: rigorous tests, benchmarks, code review, and on-call rotations.
- For huge data use SVGPs or trust-region methods instead of naive GP fitting.