Behind the Craft Is This the First AI Analyst That Actually Works? | James Evans (Amplitude)
45 snips
Jul 6, 2025 James Evans, Head of AI for Amplitude, explores groundbreaking AI analytics that could revolutionize product monitoring. He discusses the current shortcomings of AI data analysts and shares insights on developing 'goal-specific agents' for optimized decision-making. Evans emphasizes the significance of user feedback in crafting effective AI tools and addresses the evolving nature of AI in analytics. He also tackles concerns about AI potentially displacing jobs in data science while advocating for human creativity in decision-making.
AI Snips
Chapters
Transcript
Episode notes
Keep Human Approval in AI Loop
- Let AI agents run continuously but keep humans in the loop for approval.
- Human oversight helps avoid friction and reinforces trust in AI-generated experiments.
Live Discovery of Real UI Issue
- A real user interface issue with dead clicks on the pricing page was discovered by the AI agent.
- This problem was not artificially created, showcasing agent practical impact during a live demo.
Session Replay Insights Aid Trust
- AI can analyze session replays more efficiently than humans, highlighting key moments linked to insights.
- This enhances trust by allowing users to quickly verify agent conclusions with visual evidence.