The Data Exchange with Ben Lorica Why Traditional Observability Falls Short for AI Agents
13 snips
Jan 22, 2026 Lior Gavish, CTO and co-founder of Monte Carlo Data, dives into the shift from data observability to agent observability. They explore how AI is transforming data teams into data-and-AI teams and discuss the broad adoption of agents across industries. Lior emphasizes the importance of capturing granular telemetry to understand complex agent decisions and the challenges of measuring output quality with traditional methods. He introduces automated troubleshooting agents and highlights the critical role of observability for optimizing AI performance.
AI Snips
Chapters
Transcript
Episode notes
Customers Across Industries Run Thousands Of Agents
- Monte Carlo's customers span tech, manufacturing, media, and education and some run thousands of agents.
- Lior reports that 5-10% of organizations have scaled agents to production but many more are about to.
Unstructured Outputs Increase Observability Complexity
- Agent interactions often produce unstructured text and touch many data sources, increasing variability.
- Interpreting and evaluating this telemetry requires new techniques, including LLM-based judging and deterministic checks.
Protect Telemetry And Measure Output Quality
- Treat agent telemetry as highly sensitive and keep it under customer control to meet security and compliance needs.
- Analyze agent outputs with specialized methods to measure quality beyond classic observability metrics.