Skip to content

Why Diffusion Will Win AI Inference with Inception Co-Founder and CEO Stefano Ermon

No Priors

Hosted by Sarah Guo

38m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

In this episode, the guest Stefano Ermon, co-founder and CEO of Inception and longtime Stanford professor, discusses why diffusion models will challenge autoregressive transformers in AI inference. The guest explains how his lab pioneered score-based generative models in 2019 and achieved a 2024 research breakthrough matching GPT-2 scale performance with a diffusion-based language model while generating text ten times faster. The discussion covers why diffusion workloads map better to GPU parallelization, how inference-time scaling and RL post-training bottlenecks favor parallel architectures, and how Inception is currently serving production workloads with its Mercury models. The conversation also explores broader industry dynamics, including the economics of compute efficiency, data efficiency trade-offs, and the vital role of academic research in contrarian bets like FlashAttention and DPO.

  • Diffusion-based language models can generate text significantly faster by outputting multiple tokens simultaneously in parallel. 5:36
  • Autoregressive models suffer from being memory-bound during inference because generation remains strictly sequential. 8:58
  • Diffusion models allow easier steering and external control because generation happens through coarse-to-fine iterative denoising. 26:04
  • Academic research remains fertile ground for contrarian bets that eventually reshape the broader artificial intelligence industry. 37:12

Chapters

  1. 0:05
    Podcast Intro and Host Greeting

    The host welcomes listeners and introduces Stefano Ermon as the guest.

  2. 0:49
    Birth of Diffusion Models

    Stefano details how score-based denoising led to diffusion models and their impact on image generation.

  3. 5:36
    Inference Efficiency: Diffusion vs. Autoregressive

    The conversation explains why diffusion models are inherently faster at inference and better suited for GPU workloads.

  4. 16:00
    Future Outlook and Talent at Inception

    The host and Stefano discuss the company’s growth plans, research focus, and why it attracts ambitious engineers.

  5. 18:04
    Speed as a Competitive Edge in AI Services

    They talk about how latency matters for customers, especially in voice and real‑time applications.

  6. 19:14
    Hardware and Software Synergy for Diffusion Models

    Stefano explains how software optimizations complement hardware advances to deliver high‑performance inference.

  7. 31:14
    Challenges and Ecosystem Gaps for Diffusion Deployment

    He highlights the lack of mature tooling and the need to build custom serving stacks for diffusion models.

Highlights

  • 0:42

    Origins of Generative Modelingstory

    The guest describes starting research on generative models at Stanford in 2014 when the field was unpopular and researchers had to justify training them merely to learn unsupervised features for supervised learning.

  • 3:29

    The 2019 Denoising Breakthroughstory

    The guest explains how his lab pioneered score-based generative models in 2019 by training neural networks to denoise images, which laid the foundation for modern diffusion models.

  • 5:17

    Ten Times Faster Generationdata

    The guest details a 2024 research breakthrough where a diffusion transformer matched GPT-2 perplexity while generating text ten times faster than an autoregressive model.

  • 7:20

    Inference Time Scaling Limitsinsight

    The guest argues that autoregressive inference workloads are memory-bound and fail to map efficiently to the parallel processing strengths of GPUs.

  • 9:34

    Parallel Workloads on GPUsstrategy

    The guest explains that diffusion models process many tokens in parallel during inference, matching the training workload and utilizing hardware much better than sequential models.

  • 12:33

    Production Deployment of Mercurydata

    The guest notes that Inception has successfully deployed its Mercury diffusion models in production, matching performance tiers of flash and mini models while running custom serving infrastructure.

Quotes

  • “We published a paper basically showing that for the first time, it was possible to match the quality of an autoregressive model at the GPT-2 scale.”
    Unnamed speaker · 5:07
  • “Because it's diffusion, because you're outputting many tokens at the same time, we were able to generate text like 10x faster compared to the autoregressive model.”
    Unnamed speaker · 5:36
  • “The more parallel solution is the one that is eventually going to win.”
    Unnamed speaker · 11:07
  • “We think it works really well. So we've been able to train diffusion-based LLMs that are comparable in quality with the speed-optimized models for Frontier Labs.”
    Unnamed speaker · 12:23
  • “Diffusion models are typically easier to control compared to autoregressive models. And the reason is that if you think about an autoregressive model, you kind of have to wait until you've generated the full object to know whether or not it's satisfied.”
    Unnamed speaker · 26:04

About this episode

As generative AI hits hardware and latency bottlenecks, Stanford professor, diffusion pioneer, and Inception co-founder and CEO Stefano Ermon is betting on a radical new architecture. Stefano joins Sarah Guo to talk about Inception, and how his team is applying diffusion architecture beyond images and video into discrete text and code generation. Stefano explains the limitations of autoregressive LLMs, as well as why parallel token generation in diffusion models offers superior inference scaling and hardware utilization on standard GPUs. He also shares details about Inception’s Mercury models, real-world voice agent applications, the software stack required to serve diffusion-based models at scale, academia’s role at the frontier of AI innovations, and why the next era of AI competition will be defined by efficiency. Sign up for new podcasts every week. Email feedback to show@no-priors.com Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @StefanoErmon | @_inception_ai Chapters: 00:00 – Stefano Ermon Introduction 00:35 – Research Background 02:54 – Starting Inception 05:59 – Why Diffusion Beats Autoregressive 11:10 – Discrete vs. Continuous Modalities 13:19 – Inception Today 16:45 – Where Speed Wins 17:31 – Inception Customer Base 18:49 – Interaction with Hardware Landscape 19:34 – Inception and the Broader Industry 21:41 – Data Compression and Structure 24:45 – Controllability of Diffusion Modeles 27:25 – Emergent Capabilities at Scale 29:02 – Future Workload Split Between Diffusion vs. Traditional 30:03 – Adoption Challenges 31:44 – Hiring and Team Organization 32:50 – Recursive Self Improvement 34:02 – Resource Allocation 35:10 – Impact of Academia 38:13 – Conclusion

Transcript

0:00
38:13

6,875 words · 2 speakers

Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.

Topics

  • diffusion models
  • autoregressive models
  • Inception
  • Frontier Labs
  • Stanford
  • GANs
  • OpenAI
  • Flash
  • Mercury
  • Mini Nano

Page updated · transcript and summary are machine-generated