Software Engineering Daily Turbopuffer with Simon Hørup Eskildsen
103 snips
Sep 30, 2025 
Simon Hørup Eskildsen, co-founder of TurboPuffer and former Shopify infrastructure engineer, joins security expert Gregor Vand to discuss the fascinating world of vector databases. They dive into how TurboPuffer was shaped by early AI experiments and the challenge of storage costs. Simon explains its design for lightning-fast queries and the importance of unique indexing strategies. He shares success stories from companies like Cursor and Notion, emphasizes a focus on commercial clients, and hints at exciting features in the pipeline, all while sporting a fun pixel-art aesthetic.
AI Snips
Chapters
Transcript
Episode notes
Choose Index Type By Storage Latency
- Graph-based ANN works great in memory but fails on high-latency storage due to many random round trips.
- Cluster/centroid approaches minimize round trips and suit disk and S3 access patterns for sub-second cold queries.
Treat Object Storage As Canonical And Prewarm
- Design for a cache hierarchy: memory, disk, then object storage as the canonical source of truth.
- Use pre-warming heuristics to reduce cold-query latency for user-facing queries.
Continuously Sample Production Queries
- Measure real-world recall by sampling production queries rather than relying solely on academic benchmarks.
- Alert and act if a customer's recall drops below thresholds like 90% to maintain quality.
