Data Science at Home cover image

There Is No AI. There's a Stateless Function on 10,000 GPUs Pretending to Know You (Ep. 299)

Data Science at Home

00:00

KV cache and batching basics

Francesco describes attention keys/values, KV cache growth, and why caching matters for generation efficiency.

Play episode from 08:03
Transcript

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app