Deployed: The AI Product Podcast The Secrets To Being A Great AI PM: Evals, Quality & More | Apollo.io AI Product Lead, Tyler Phillips
Mar 27, 2025
Tyler Phillips, AI Product Lead at Apollo.io, dives into the art of crafting successful AI products for sales. He shares valuable insights on the often-overlooked tasks of AI product management, like evaluating AI outputs and creating quality frameworks. Highlighting the significance of prompt writing from domain experts, Tyler discusses how personalized outreach enhances customer experiences. He also reveals strategies for ensuring data quality and constructing effective evaluation criteria, making for a treasure trove of tips for aspiring AI PMs.
AI Snips
Chapters
Transcript
Episode notes
Agents vs Workflows Clarified
- Agents and deterministic workflows are distinct; not all automation needs an agentic AI.
- Deterministic workflows are better solved step-by-step, reserving agents for less structured tasks.
Augmentation Beats Automation
- AI in sales should augment humans, not fully automate the process.
- AI's value lies in creating relevant, personalized outreach based on better data synthesis.
AI PMs Must Lead Quality Control
- AI PMs should deeply engage in AI evals, creating golden datasets and scoring outputs on criteria like accuracy and relevance.
- Without PM involvement in quality, building highly retentive AI products is very hard.