Pragmatism in Practice Measuring what matters in the age of AI
Sep 16, 2025
Chris Westerhold, Global Practice Director at ThoughtWorks, and Abhi Noda, CEO and co-founder of DX, dive into the complexities of measuring AI's impact on developer productivity. They discuss the challenges of evaluating knowledge-worker outputs and the importance of balancing leading and lagging metrics. The duo debates the quality trade-offs of AI-generated code and emphasizes prioritizing people and processes over tools in engineering. Finally, they outline practical steps for effective metric adoption and the anticipated insights from the upcoming DORA report.
AI Snips
Chapters
Transcript
Episode notes
Pair Leading And Lagging Metrics
- Use both leading indicators (waste, friction) and lagging indicators (scorecards) to diagnose and improve engineering performance.
- Balance them so scorecards tell you where you are and leading metrics tell you what to fix next.
Fix People And Process First
- Break problems into people, process, and technology before choosing interventions.
- Prioritize process and people improvements over adding more technology or tools.
AI Changes Some Metrics, Not All
- Core engineering metrics (speed to customer, friction) remain stable pre- and post-AI to allow comparisons.
- New working patterns from AI require additional measures like amount of work offloaded to AI and AI-specific quality.