The LLM treadmill and build vs buy

Fathom was built on the assumption that transcription would become commoditized and generative models would steadily improve. Rather than training proprietary models, Richard focused on building the infrastructure around them and waiting for model capabilities to reach the right threshold.

In this conversation, he explains why AI has made effort and impact harder to predict, and why that shifts product development from roadmap execution toward experimentation. He describes separating an exploratory AI team from core engineering, structuring that team to prototype and write specs, and expecting a meaningful portion of experiments not to work.
Richard introduces his Jenga model for AI development, testing different models and use cases to find where resistance is lowest. He also discusses the operational realities of rapid model updates, hallucination rates, and what he calls the LLM treadmill.

The discussion explores qualitative QA, organizational design, buy versus build decisions, and why leadership taste plays an increasingly important role as AI lowers the barrier to generating outputs.

Key takeaways: 

• Estimating effort and impact is becoming harder
  As model capabilities improve quickly, features that require months today may take far less time in the near future. This makes traditional planning assumptions less stable.
• Product development increasingly resembles R&D
  With shifting capabilities and uncertain outcomes, teams must experiment, prototype, and iterate rather than rely solely on long term roadmaps.
• Organizational structure must reflect experimentation
  Separating exploratory AI work from core engineering can allow faster iteration while maintaining stability elsewhere.
• Rapid model updates create operational pressure
  Frequent improvements and changing performance levels can require teams to revisit and adjust features more often than in traditional software cycles.
• Qualitative judgment plays a larger role
  As AI lowers the cost of generating outputs, evaluating quality and deciding what to ship becomes increasingly important.

Fathom: fathom.ai
Fathom LinkedIn: linkedin/company/fathom-video/
Richard's LinkedIn: linkedin/in/rrwhite/

00:00 Intro: Why AI Breaks Roadmaps
00:19 Meet Richard White (Fathom AI)
02:16 From Roadmaps to R&D
04:49 Designing AI Teams for Speed
07:11 The Jenga Model
09:56 Failing 50% & AI Team Psychology
13:40 LLMs as Interns & Anti-Planning
21:01 QA, Data Pain & Developing Taste
24:59 Executive Taste & Culture Rules
27:20 Reacting to AI Waves
28:50 Fathom’s 4-Step Product Plan
30:47 What New Models Unlock
32:13 From Scribe to Second Brain
40:32 Build vs Buy in AI
45:32 The Debrief

📜 Read the transcript for this episode: from-roadmaps-to-rd-how-ai-is-changing-product-development-with-richard-white-founder-of-fathom-ai/transcript

For more prompts, tips, and AI tools. Check out our website: https://www.beyondtheprompt.ai/ or follow Jeremy or Henrik on Linkedin:

Henrik: https://www.linkedin.com/in/werdelin
Jeremy: https://www.linkedin.com/in/jeremyutley

Show edited by Emma Cecilie Jensen.