Eye On A.I. #261 Jonathan Frankle: How Databricks is Disrupting AI Model Training
76 snips
Jun 12, 2025 In this engaging discussion, Jonathan Frankle, Chief Scientist at Databricks and co-founder of MosaicML, shares insights into innovative AI training techniques. He introduces TAO (Test-time Adaptive Optimization), a method enabling model tuning without expensive labeled data. Jonathan discusses the advantages of synthetic data and reinforcement learning, and how Databricks' reward model enhances performance while minimizing costs. The conversation highlights the potential for transforming AI deployment in enterprises, making it faster and more efficient.
AI Snips
Chapters
Transcript
Episode notes
Tau Trades Training for Inference Efficiency
- Tau uses synthetic data generation at training time to lower inference compute costs.
- Training uses extra compute up front, allowing normal inference speed without extra runtime cost.
Test Models Before Deployment
- Deploy models conservatively and collect interaction data before retraining with Tau.
- Test thoroughly and consider A-B testing to avoid deploying models that worsen production metrics.
Tau and Forgetting Challenges
- Tau assists in reducing the need for labeled data, but does not solve catastrophic forgetting.
- Parameter-efficient fine-tuning methods like LoRa also act as regularizers reducing forgetting and overfitting.