DataFramed #289 How I Nearly Got Fired For Running An A/B Test with Vanessa Larco, Former Partner at New Enterprise Associates
19 snips
Mar 6, 2025 
In this engaging discussion, Vanessa Larco, a former partner at NEA with an impressive background in product leadership from giants like Microsoft and Disney, shares her insights on A/B testing. She explores the challenges of balancing short-term gains with long-term user experience, particularly in gaming. Vanessa also highlights the pitfalls of misaligned metrics and the need for clear communication and cross-team collaboration. Her anecdotes underline the importance of understanding user behavior and leveraging data to drive meaningful changes in product development.
AI Snips
Chapters
Transcript
Episode notes
Unpredictable User Preferences
- A/B testing revealed user preferences that the team wouldn't have predicted.
- The Gnome Town theme unexpectedly won, highlighting the data's power.
A/B Testing Limitations
- A/B testing has limitations, especially regarding long-term strategy.
- It excels in optimizing existing elements, but not for developing entirely new products.
Deepening User Understanding
- Use A/B testing to understand user behavior deeply.
- Supplement with focus groups for richer insights.