The Analytics Power Hour #293: Tool Selection and the Unhelpfulness of Feature Comparisons
20 snips
Mar 17, 2026 Jason Packer, founder of Quantible Analytics and author of Google Analytics Alternatives, brings concise analytics tool evaluation expertise. He describes testing tools with real data and avoiding vendor demos. Conversation covers proof of concept strategies, hidden costs of DIY and open source, the influence of product philosophy and vendor roots, and how price and total cost of ownership should guide choices.
AI Snips
Chapters
Books
Transcript
Episode notes
Validate Tools With Real Data
- Use a tool with real data to learn its behavior instead of relying on vendor demos or videos.
- Jason installed each product on real websites (including one bought site full of bot traffic) to reveal real-world quirks and limits.
Bought Test Site Exposed Bot Problems
- Jason used a bought website that was 'littered with bots' to test tools' handling of messy traffic.
- That site’s 404s and bot hits exposed how different platforms handle filtering, sessionization and data hygiene.
Do Multiple POCs Or Toy Tests
- Run multiple POCs or at least lightweight toy tests to stress features you care about before committing.
- Moe pushed for multiple POCs (wanted four, settled for two) and Jason suggests toy tests like using a personal site when full POCs aren't feasible.
