The Data Exchange with Ben Lorica The Truth About Agents in Production
40 snips
Dec 31, 2025 
Join Samuel Colvin, founder of Pydantic, Aparna Dhinakaran from Arize AI, Adam Jones at Anthropic, and Jerry Liu of LlamaIndex in a fascinating conversation about Agentic AI. They explore impressive agent architectures, the advantages and challenges of multi-agent systems, and innovative memory and state management strategies. Aparna emphasizes the importance of observability with evals, while the group shares thoughts on bridging technical and non-technical users through no-code solutions. They also discuss future capabilities and realistic expectations for agent technology.
AI Snips
Chapters
Transcript
Episode notes
Live Tracing Outweighs Offline Evals
- Offline evals have a role, but tracing and live observability often give faster, actionable feedback.
- Samuel Colvin notes coding agents get immediate feedback via tests, reducing reliance on offline evals.
Pair Analytics With User Research
- Watch users interact with your agent and pair analytics with user research to find real issues.
- Adam Jones emphasizes product analytics and user sessions over abstract metrics alone.
Single-Agent Universality Is Limited
- The one-agent-for-all-tools fantasy works for coding but fails broadly; many domains need specialized components.
- Samuel Colvin cautions against giving one agent every tool and expecting robust results everywhere.