Skip to content

The Race to Recursive Self-Improving AI

a16z Podcast

Hosted by Erik Torenberg

33m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

MakerMaker.ai co-founders Daval Ajuda and Owen Heh join a16z general partner Martín Casado to discuss recursive self-improvement, or RSI, in artificial intelligence. The guests distinguish true RSI from traditional automation or autocatalytic development, defining it specifically as a closed loop where the model serving as the optimizer is also the object being optimized. They examine how optimizing both the numerator of capability and the denominator of cost can yield compounding gains across components like models, harnesses, and infrastructure. Along the way, the discussion covers the empirical challenges of measuring these gains, dealing with reward hacking and variance, and why fields with fast, digital evaluations will accelerate faster than physical domains.

  • Differentiate true recursive self-improvement from autocatalytic tool use by asking whether the AI model itself acts as the optimizer. 3:37
  • Track the cost of experimental runs alongside capability gains using a bleeder board to avoid throwing unmeasured capital at problems. 11:53
  • Recognize that intelligence is multi-dimensional and that overall system speedups are fundamentally constrained by the slowest component according to Amdahl's law. 28:18
  • Avoid relying on theoretical takeoff predictions and instead test systems empirically to account for noise, variance, and reward hacking. 8:20

Chapters

  1. 0:00
    Defining Recursive Self-Improvement

    Hosts introduce the concept of AI systems that can improve themselves across multiple stack levels.

  2. 4:29
    Distinguishing RSI from Autocatalysis

    They clarify the difference between true self-improvement and human-driven AI development.

  3. 7:31
    Closed-Loop vs Open-Loop Improvement

    Discussion of closed-loop systems, mathematical modeling, and empirical challenges in RSI.

  4. 18:14
    Practical RSI in Inference Optimization

    The team focuses on speeding up inference as a concrete domain for RSI gains.

  5. 22:15
    Human-AI Co-Optimization

    They discuss the role of human researchers as agents within a multi-agent RSI framework.

  6. 23:51
    Limits and Asymptotes of RSI

    Speculation on physical and informational limits that may cap RSI growth.

  7. 26:15
    Measuring Numerator and Denominator

    Exploration of how to quantify performance gains versus resource costs in RSI experiments.

  8. 31:43
    Closing with Takeoff Outlook

    Final recap, future expectations, and a humorous anecdote about RSI breakthroughs.

Highlights

  • 3:37

    Closed Loops Versus Autocatalysisdebate

    The host and guest discuss how historical tools like steam engines or compilers featured autocatalytic effects but lacked the closed loop where the improver is also being improved.

  • 4:46

    Defining Recursive Self-Improvementinsight

    The guest explains that recursive self-improvement occurs when the language model itself becomes the optimizer, allowing cognitive improvement to be reinvested directly into the model.

  • 5:58

    Differential Equation of Capabilitystrategy

    The guest outlines a mathematical framework where capability growth is driven by performance in the numerator and cost or difficulty in the denominator.

  • 12:13

    NanoGPT Benchmark Improvementsdata

    The guest notes that retraining the GPT-2 model on a single GPU has dropped significantly in cost over time due to algorithmic and agent-driven techniques.

  • 16:21

    Stamp Collections Versus Physicsinsight

    The guest invokes Richard Feynman to argue that AI research without a unified mathematical framework resembles stamp collecting rather than true physics.

  • 19:05

    Compounding Inference Optimizationinsight

    The participants discuss the compounding impact of using self-improving systems to make inference faster, which in turn allows the system to generate more improvements more rapidly.

  • 26:15

    The Viceroy of Variancestory

    The guest describes an internal company role dedicated to rigorously destroying experimental results and rooting out hidden reward hacking and variance.

Quotes

  • “The fundamental idea here is to build AI that improves itself.”
    Unnamed speaker · 0:00
  • “The simpler thing that closes the loop is the thing that scales.”
    Unnamed speaker · 14:37
  • “We have a culture of tracking where things go, cost, time, all that kind of stuff.”
    Unnamed speaker · 11:45
  • “The real way to know what's happening is to test things.”
    Unnamed speaker · 8:24
  • A16Z 32:21–33:10

About this episode

a16z General Partner Martin Casado sits down with MakerMaker. AI co-founders Dhaval Adjodah and Owen He to unpack recursive self-improvement, or RSI, and separate the increasingly popular term from what it would actually mean for an AI system to improve itself. Dhaval and Owen distinguish true RSI from simply using AI to automate pieces of AI development. Their goal is a closed loop where the system being improved is also doing the improving, across components including models, harnesses, and eventually more of the AI stack. They explain why closing that loop still doesn’t guarantee runaway “takeoff,” and why measuring real progress requires accounting for both capability gains and the resources required to achieve them. They also share what they’re seeing in their own research, including early signs that self-improving systems can increase performance while improving efficiency. And they dig into the practical challenges of proving those gains are real, from variance and reward hacking to the enormous difficulty of distinguishing genuine improvement from noise.

Transcript

0:00
33:27

5,992 words · 4 speakers

Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.

Topics

  • GPU
  • Kimi K three
  • multi-agent system
  • recursive self-improvement
  • MakerMaker.ai
  • DeepMind
  • MIT Media Lab
  • NanoGPT
  • reinforcement learning
  • scaling laws

Page updated · transcript and summary are machine-generated