High Signal: Data Science | Career | AI Episode 15: Why Good Metrics Still Lead to Bad Decisions — and How to Fix It
51 snips
Apr 24, 2025 Eoin Mahony, data science partner at Lightspeed and former Uber science lead, shares his insights on effective metrics. He argues that metrics can mislead if their underlying mechanisms aren’t understood, a lesson he learned while optimizing NYC's Citi Bikes. Eoin discusses the pitfalls of relying too heavily on simplistic data and emphasizes the need for rigorous analysis in data-driven decision-making. He also dives into how generative AI can improve workflows while navigating the hype surrounding tech adoption, blending practical advice with real-world examples.
AI Snips
Chapters
Transcript
Episode notes
Nuance of Measurement in Networks
- Measurement can flip the sign of effects in networked experiments if not designed carefully.
- Long-term impacts differ from short-term behaviors, requiring thoughtful evaluation.
Adapting Uber Marketplace During COVID
- During COVID, Uber’s roadmap was scrapped as market conditions shifted dramatically.
- The team made large system adjustments to operate in a sparse, uncertain environment.
Sell the Problem, Not Solution
- Always sell the problem clearly before proposing a solution.
- Align everyone with why solving the problem matters and what success looks like.