689: Observing LLMs in Production to Automatically Catch Issues
Super Data Science: ML & AI Podcast with Jon Krohn
00:00
Monitoring Drift in Machine Learning Models
This chapter explores detecting embedding drift for unstructured cases in machine learning models and setting up monitors for structured data to detect deviations from baseline distributions. It covers various types of drift like covariant, feature, data, and metadata drift, emphasizing the importance of feature drift measurement using metrics like PSI and introduces the platform Arise for drift monitoring with automatic schema detection and customizable options.
Play episode from 11:43
Transcript
