Super Data Science: ML & AI Podcast with Jon Krohn 921: NPUs vs GPUs vs CPUs for Local AI Workloads, with Dell’s Ish Shah and Shirish Gupta
82 snips
Sep 9, 2025 
Ish Shah and Shirish Gupta from Dell Technologies share their expertise in AI hardware innovation. They explore the competitive landscape of NPUs versus GPUs and the advantages of using Windows for AI development. Listeners learn about Dell's cutting-edge products, including the new Pro Max mobile workstation with a discrete NPU. The conversation delves into optimizing local versus cloud AI workloads, decision-making in hardware investments, and the importance of future-proofing technology for evolving AI applications.
AI Snips
Chapters
Books
Transcript
Episode notes
Colonoscopy Inference Demo On A Laptop
- Dell demoed a discrete NPU in the Pro Max mobile workstation to run medical imaging inference locally on colonoscopy video.
- The demo processed live video on-device with no internet uplink or cloud queueing required.
Models Shrink While Hardware Grows
- Modern client hardware is converging: models are shrinking while hardware accelerators grow more capable.
- This convergence lets cloud-scale models approach practical on-device usage within months.
GPU Scale Versus NPU Efficiency
- GPUs provide broad parallelism and far greater scale today, making them the go-to for large model training and high-performance workloads.
- NPUs currently lead on power efficiency, but GPUs retain higher absolute performance and scalability.
