Breaking the Memory Wall in the Age of Inference
The Data Exchange with Ben Lorica
00:00
External Memory for Growing Model Size
Sid discusses why external DRAM (LPDDR) is needed as models and KV caches grow beyond on-chip SRAM capacity.
Play episode from 09:02
Transcript
