Ecommerce Playbook: Numbers, Struggles & Growth The Measurement Roadmap: How it Works
Sep 25, 2025
Dive into the intricacies of the Measurement Roadmap, a framework that guides what to test in e-commerce. Discover the significance of incrementality testing and how it can illuminate vast discrepancies in platform reporting. Hear about the surprising results of Meta’s ad effectiveness and its influence on Amazon sales strategies. Unravel the importance of narrowing error margins for confident decision-making, and learn how to anchor choices in financial realities while interpreting unexpected test outcomes with caution.
AI Snips
Chapters
Transcript
Episode notes
Meta's 20% Underreporting Benchmark
- For Meta, CTC uses a 1.2 factor on seven-day click as a starting benchmark.
- That 20% upward adjustment matches House's published results and raises confidence.
Prioritize Tests By Potential Delta
- Prioritize tests where spend and uncertainty create the largest business delta.
- Use worst/best case bounds to quantify the potential upside and downside before testing.
Shrink Error Bars With Repeated Tests
- Repeated tests shrink error bars and produce a brand-specific distribution of incrementality.
- Taylor Holiday frames this as a progressive truth-seeking process, not a one-time fix.