Skip to content

Frontier Chips for Frontier AI Labs, with Walter Goodwin, Founder/CEO of Fractile

No Priors

35m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

This episode features a conversation between the host and Walter Goodwin, founder and CEO of Fractal, about the evolving landscape of AI inference chips, workload-driven architectures, and the structural dynamics of the semiconductor market. Goodwin explains how his company transitioned from SRAM-based designs to working closely with DRAM memory vendors to achieve high bandwidth and high capacity for running multi-trillion parameter models. They discuss the potential for AI-accelerated chip design to compress timelines while navigating physical and financial bottlenecks like foundry cycle times and three-year amortization windows. The discussion also covers how custom accelerators fit alongside Nvidia, AMD, and internal hyperscaler efforts at the AI frontier.

  • Optimize chip designs for inference speed by prioritizing memory bandwidth to handle massive multi-trillion parameter models effectively. 10:58
  • Expect chip design cycles to compress through AI-driven tooling, though physical fab cycle times and amortization windows will remain foundational constraints. 18:53
  • Carve out a competitive advantage in the AI chip market by structurally securing a three to six month lead over rivals. 0:34

Chapters

  1. 0:00
    Podcast Intro and Host Greeting

    Hosts introduce the episode and welcome listeners.

  2. 0:46
    Walter Goodwin Introduces Fractal

    Walter explains Fractal’s mission and focus on fast inference chips.

  3. 2:35
    Chip Landscape Overview

    Discussion of the current AI chip ecosystem and the prevalence of similar designs.

  4. 7:32
    Full-Stack Design Approach

    Fractal’s end-to-end design strategy and team structure.

  5. 9:49
    Key Technical Bets and Memory Bandwidth

    Fractal’s focus on high‑bandwidth memory and scaling beyond SRAM.

  6. 11:25
    Speed vs. Capacity Trade‑off in LLMs

    Exploring the mismatch between high bandwidth and low capacity in current chips.

  7. 15:44
    Accelerating Architecture Iteration

    How Fractal shortens design cycles to stay ahead of evolving workloads.

  8. 31:24
    Market Structure and Frontier Labs

    Discussion of chip supply diversity and the role of third‑party vendors for frontier AI labs.

Highlights

  • 0:01

    Challenging Nvidia System Complexitydata

    The guest points out that an Nvidia system contains between six and nine custom chips, highlighting the ambition for smaller companies to build potent integrated responses.

  • 1:24

    Building Very Fast Inference Chipsinsight

    The guest explains that Fractal builds very fast inference chips for the world's largest models, focusing on pouring more compute into models at test time to achieve superhuman capabilities.

  • 2:41

    The AI ASIC Industry Zooinsight

    The guest notes that the current landscape of AI ASICs features many identikit chips across hyperscalers like Google, Meta, Microsoft, and OpenAI, often delivered via ASIC houses like Broadcom.

  • 10:58

    Moving From SRAM to DRAMstory

    The guest describes moving away from SRAM-based designs due to context length scaling concerns, partnering with memory vendors to achieve high bandwidth to higher capacity DRAM memories.

  • 14:09

    Running a Lean Chip Startupdata

    With a team of about 150 people, Fractal handles front-end design, physical design, backend implementation, and advanced packaging internally to maintain an agile closed loop.

  • 31:48

    Purpose of First-Party Effort Chipsinsight

    The guest mentions a joke in the industry that the primary purpose of first-party silicon efforts by hyperscalers is to reduce the prices they pay to Nvidia.

  • 34:20

    Risks of Proprietary Hardware Betsdebate

    The guest argues that frontier labs going all-in on proprietary silicon face catastrophic risk if a competitor discovers a breakthrough that only works on a different platform.

Quotes

  • “We build very, very fast inference chips for the world's largest models.”
    Unnamed speaker · 1:26
  • “Memory bandwidth has gone up about 40x in the same timeframe.”
    Unnamed speaker · 30:58
  • “The snappier chatbot is kind of the faster horses of kind of fast inference.”
    Unnamed speaker · 13:21

About this episode

Founder and CEO of full-stack AI chip company Fractile, Walter Goodwin, joins Sarah Guo to discuss the bets he’s made on the future of the chip market as other major players like Broadcom, NVIDIA, and AMD try to accelerate their workloads. They discuss the difference in Fractile’s newer approach on model architecture with a full-stack team in the current chip landscape and the technical bets they’re making in that direction. Walter also talks about compressing the gap between the chip design cycle and its payoff period, and making a generational leap in AI inference to realize the bet in volume against the value to be captured. Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @goodwin_ml | @fractile_ai Chapters: 00:46 – Walter Goodwin and Fractile Introduction 02:29 – The Chip Landscape Now 04:56 – Common Handoffs From Architecture-Focused Players 07:18 – Full Stack Approach and Team Setup 09:49 – Fractile’s Most Important Technical Bets 15:32 – Workload Predictions and Compressing the Chip Design Cycle 23:03 – Architect Intent to Output Bottlenecks and Accelerating Trials 28:16 – Workload Bets on Model Architectural Shifts 31:20 – The Future of AI Chip Players and Market Structure 35:14 – Conclusion

Transcript

0:00
35:38

6,853 words · 2 speakers

Topics

  • Fractal
  • Nvidia
  • Broadcom
  • TSMC
  • TPU
  • HBM
  • DRAM
  • AMD
  • Google
  • SRAM

Page updated · transcript and summary are machine-generated