Austin and Vik react to OpenAI's Jalapeño announcement at Hot Chips. Plus extra spicy questions like should OpenAI sell it, how much of this was AI-written RTL, and where is Anthropic's chip?
Key Takeaways:
- The chip's core design philosophy is "dark silicon is cheaper than idle accelerators" — using one balanced chip and power-gating unused blocks is more efficient than a two-chip (e.g. GPU + LPU) solution.
- The unprecedented nine-month RTL-to-tapeout cycle was enabled by AI for EDA tools, serving as a wake-up call that small, expert teams can now develop Rubin-class chips in under a year.
- Jalapeño's key innovation is a NUMA-style architecture that gives each accelerator a local HBM slice, solving the memory contention that throttles performance in unified memory systems.
- OpenAI chose Broadcom's ESUN for its scale-up network to connect 128 chips in the rack at 600 GB/s and up to 2,048 chips across 16 racks at 200G --- all scale up!
- The design's "regret factor" principle justifies generality — the opportunity cost of being unable to support a future model is far higher than the marginal cost of adding hardware flexibility upfront.
Chapters: 0:00 Hot Chips Reaction 2:32 Designing for User Experience 11:16 A Generalized Inference Chip 14:18 The Foundry-IDM Analogy 18:42 The 'Regret Factor' 21:02 The 9-Month Design Cycle 23:45 Challenging the Two-Chip Solution 35:08 Solving HBM Underutilization 36:46 The NUMA Architecture Solution 39:28 System-Level ESUN Networking 42:06 Dark Silicon vs. Idle Accelerators 49:08 A Wake-Up Call for the Industry 52:59 Where's Anthropic's Chip?
Podden och tillhörande omslagsbild på den här sidan tillhör
Vikram Sekar and Austin Lyons. Innehållet i podden är skapat av Vikram Sekar and Austin Lyons och inte av,
eller tillsammans med, Poddtoppen.
