689: Observing LLMs in Production to Automatically Catch Issues
Super Data Science: ML & AI Podcast with Jon Krohn
00:00
Ensuring Model Safety and Performance in Production with ML Observability
The chapter focuses on the role of ML observability in detecting biases, ensuring fairness, and monitoring model performance in production environments. It discusses the importance of measuring fairness metrics such as recall parity and false positive rate parity to evaluate model decisions across different demographic groups. The conversation highlights the evolution of ML observability from a luxury to a crucial element in ensuring model safety and performance.
Play episode from 47:10
Transcript
