The Official SaaStr Podcast: SaaS | Founders | Investors SaaStr 850: The Agents, Episode 1: Who Maintains All This?
80 snips
Apr 15, 2026 They dig into who actually maintains agentic and vibe‑coded apps in production. They recount a preview environment outage that confused both agents and humans. They describe micro hallucinations and a mysterious model regression that broke a pitch deck analyzer. They cover an agent that recommended wildly expensive credits and an AI CS agent catching placeholder sponsor graphics.
AI Snips
Chapters
Transcript
Episode notes
AI VP Marketing Got Dates Wrong And Drifted
- The AI VP of Marketing reported conflicting year-over-year facts, claiming 44% ahead one day and 11% the next due to wrong-year comparisons.
- Jason now spends ~15 minutes daily correcting these micro-hallucinations to prevent data drift.
Model Regression Broke Pitch Deck Analyzer
- The pitch deck analyzer began hallucinating startup revenue (e.g., $100k ARR growing 500%) after a model update, despite no code changes.
- Jason tracked it to subtle model/regression changes in Claude causing 5% of decks to produce anomalous results.
Continuously Monitor And Retrain Customer Facing Agents
- Read agent conversations and retrain agents continuously to prevent costly mistakes and bad recommendations.
- Amelia's Clay example shows agents can recommend expensive models or plan upgrades if not updated on pricing.