Lenny's Reads Listen: Building AI product sense, part 2
16 snips
Feb 10, 2026 A practical ritual for surfacing AI failure modes before users do. Techniques include forcing models to be wrong, probing ambiguity, and stress-testing to find first breakpoints. Discussion of defining minimum viable quality with three thresholds and five context factors that shift quality expectations. Advice on estimating per-call cost early and building simple guardrails to protect user trust.
AI Snips
Chapters
Transcript
Episode notes
AI Product Sense Is Core PM Skill
- AI product sense is the ability to map model capabilities and failures into trustworthy products.
- It shifts PM work from idea validation to predicting real-world model behavior and user trust.
Do Weekly 15-Minute Failure Tests
- Run three short weekly rituals to surface failure modes before users do.
- Compare bad and good model outputs to find what constraints or context fix hallucinations.
Ask The Model To Produce Obviously Wrong Outputs
- Intentionally ask models to do something obviously wrong to reveal hallucination tendencies.
- Then rerun with a short constraint that says 'only include items explicitly mentioned' to see needed signals.