On-Prem Tokens vs Cloud Spend

Austin and Vik break down a packed week in semiconductors, covering GTC, OFC, and Micron earnings. The conversation kicks off with Jensen Huang's bold claim that engineers should spend $250K/year on AI tokens, and whether companies will buy tokens or token generators (i.e., on-prem hardware like the Dell Pro Max with GB300). They dig into the CapEx vs OpEx tradeoffs, data security concerns, and how sharing GPU resources might end up looking a lot like the old EDA license model.

Next up: Micron crushed earnings and appears to be designed into Vera Rubin for HBM4 — despite months of rumors saying otherwise. Austin and Vik unpack the nuance around HBM pin speeds, memory node base dies, and what Micron's massive new fab investments in Taiwan, Singapore, Idaho, and New York mean for the memory cycle.

The back half of the episode dives into optical interconnects for AI scale-up. A new industry consortium (OCI-MSA) has formed with Meta, Broadcom, NVIDIA, and OpenAI to standardize optical components. Vik explains why traditional indium phosphide lasers might be overkill for short-reach scale-up, and makes the case for micro LEDs — a "slow but wide" approach that could fill the gap between copper and conventional optics. They also touch on Credo's expanding product portfolio (and the infamous purple-to-orange cable saga), plus Lumentum's new VCSEL work for scale-up.

Vik - https://www.viksnewsletter.com/
Austin - https://www.chipstrat.com/

CHAPTERS
0:00 Intro & GTC/OFC Conference Overload
2:09 Jensen's $250K Token Budget Per Engineer
5:08 On-Prem Inference vs. Cloud Token Spending (Dell Pro Max, CapEx vs OpEx)
6:44 Sharing GPU Resources Like EDA Licenses
8:16 Data Security & On-Prem Privacy Concerns
9:53 Matthew Berman's Fine-Tuned Open Claw Agent
10:35 Vik Sets Up Open Claw on a Home Server
11:53 Always Be Clauden (ABC) – Managing Agents from Your Phone
13:34 Micron Earnings & HBM4 in Vera Rubin
16:39 HBM Pin Speeds & the Micron Design-In Debate
20:17 Micron's New Fab Investments & Memory Cycle Fears
23:49 Why AI Drives a Step Change in Memory Demand
26:30 Optical Compute Interconnect MSA (OCI-MSA)
29:48 Scale-Up Optics: Do We Need New Technology?
30:58 Micro LEDs – The "Slow but Wide" Approach
35:45 Micro LEDs vs. Copper vs. Traditional Optics
36:55 Credo's Product Spectrum & the Purple Cable Story
39:31 VCSELs & Lumentum's 1060nm Scale-Up Play