The Analytics Power Hour #294: Adapting an Analytics Team to an AI World
15 snips
Mar 31, 2026 John Lovett, VP of Analytics at SEER Interactive and author focused on KPIs and operationalizing AI, joins to talk practical AI adoption. He covers companywide mandates and training, building guardrails and trust, personal experiments that scale to production, conversational analytics with GA4 and BigQuery, and using agents and GEO tactics to drive quick client wins.
AI Snips
Chapters
Books
Transcript
Episode notes
Bot Traffic Discovery Prompted Guardrails
- Early wins came from an analyst discovering a viral traffic spike via conversational analytics, which turned out to be bot traffic.
- That incident prompted John to add anomaly checks (Twyman's Law) to agents to prevent false-positive insights.
Force Deterministic Proofs To Counter LLM Overconfidence
- LLM outputs often overconfidently present plausible but incorrect conclusions, so add verification steps.
- John applied Twyman's Law and required deterministic checks: show SQL/math behind any number and flag anomalous spikes for verification.
Create A Release Path For Prototype Agents
- Provide a production pathway so prototypes become company tools with engineering hardening and governance.
- Seer runs release days where vibe-coded agents are stress-tested and productionalized by engineering into the SEER MCP for company use.
