
Experiment Nation: The Podcast S6E6 - The Uncomfortable Truth About AI, ROI, and CRO Careers
Mar 27, 2026
Conversations about where AI helps experimentation and where it quietly breaks workflows. Practical debates on measuring MDE for continuous revenue metrics like ARPU. Tips for carving out time for customer research when schedules are overloaded. Warnings about unpaid interview audits and how to handle volume hiring for CRO roles. Real talk on escaping the endless ROI loop and why some agencies lack designers.
AI Snips
Chapters
Transcript
Episode notes
AI Speeds Work But Breaks On Specificity
- AI can accelerate routine CRO tasks but fails when outputs must meet tight, specific standards.
- Ronald Santiago found AI builds quickly but produces brittle, spaghetti results that require constant spot-checking and regression tests.
Prompt AI To Be Brutally Honest
- Do prompt AI to be blunt and rigorous when you need candid critique rather than reassuring fluff.
- Ronald uses a ChatGPT prompt like "don't worry about my feelings give it to me straight" to avoid safe, repetitive suggestions.
Prefer Binary Metrics Over Revenue For MDE
- Continuous revenue metrics like ARPU are noisy and hard to model for MDE without deep data understanding.
- Ramil and Ronald recommend preferring binary outcomes (e.g., add-to-cart) because they reduce variance and sample size needs.
