
Summary
Written by a model from the transcript; not yet reviewed.
This episode explores why current AI scaling methods, particularly reinforcement learning atop large language models, may fall short of achieving true artificial general intelligence without major breakthroughs in continual learning. The host argues that labs are currently forced to pre-bake specific skills and workflows into models because the underlying systems lack the human-like ability to learn on the job from unstructured experience. While models continue to impress on benchmarks, their current inability to seamlessly acquire company-specific knowledge explains why they are not yet capturing tens of trillions of dollars in global labor value. The discussion also examines why RL scaling lacks the predictable power law of pre-training and how fierce competition among labs will likely prevent any single company from establishing a permanent runaway advantage once continual learning breakthroughs begin.
- Current AI models require pre-baked skills and customized training pipelines because they lack the human ability to learn on the job. 2:40
- True economic diffusion of AI labor is limited because models currently lack the capabilities required to replace broad human knowledge work. 5:15
- Scaling reinforcement learning from verifiable reward does not share the same clean, predictable power law trend as pre-training loss. 8:23
- The future of AI capability will likely rely on continual learning agents that share their specialized domain experiences back to a central model. 9:44
Chapters
- 0:00Short Timelines vs RL Scaling
The host questions why people expect rapid RL scaling while remaining skeptical of human-like learning timelines.
- 0:45Baron Millich’s Blog Insight
Reference to Millich’s blog highlighting the cost of expert‑generated training data.
- 2:36Skill Baking vs On‑Job Learning
Comparison of building in skills versus enabling generalization and on‑job learning.
- 3:12Biology Dinner Example
Illustrates the difficulty of creating custom training loops for niche tasks.
- 5:06AGI’s Economic Diffusion
Speculation on how brain‑like AGI would diffuse faster than current models and change hiring dynamics.
- 7:01Goalpost Shifting in AI Progress
Acknowledges shifting benchmarks and the need to broaden AGI definitions.
- 8:25Scaling Laws vs Continual Learning
Contrast between pre‑training scaling laws and the lack of a clear trend for RL from verifiable reward.
- 9:44Future of Continual Learning
Prediction that continual learning will be the main driver of AGI progress and its gradual rollout.
- 11:56Competition Among Model Companies
Discussion of fierce competition, talent poaching, and the lack of runaway advantage.
Highlights
- 0:00
Short Timelines vs RL Scalinginsight
The host questions why proponents of extremely short timelines are bullish on reinforcement learning atop language models, arguing that if models were truly human-like learners, training on verifiable outcomes would be pointless.
- 0:45
Baron Millich's Blog Insightinsight
Quoting Baron Millich, the host notes that frontier model improvements are heavily driven by billions of dollars paid to human experts to write targeted questions and example answers rather than pure algorithmic leaps.
- 5:15
AGI's Economic Diffusiondebate
The host argues that if models actually possessed human-level capabilities, they would integrate into companies instantly and command trillions of dollars in token purchases, unlike the current slower market adoption.
- 6:49
Goalpost Shifting in AI Progressinsight
While AI bulls rightly criticize bears for moving goalposts, the host explains that this shifting is justified because solving previous bottlenecks like reasoning has simply revealed deeper requirements for intelligence.
- 8:25
Scaling Laws vs Continual Learningdata
Pre-training followed a clean power law in loss reduction across compute scales, but researchers lack a similar publicly known trend for reinforcement learning from verifiable reward.
- 9:08
Toby Board's RL Benchmark Datadata
Toby Board's analysis of O-series benchmarks suggests that achieving a boost equivalent to a single GPT level requires roughly a millionfold scale-up in total reinforcement learning compute.
- 9:50
Future of Continual Learningstrategy
Baron Millich suggests the future involves specialized continual learning agents that perform different jobs, extract value, and bring their learnings back to a core hive mind model.
- 11:33
Competition Among Model Companiesinsight
Fierce industry competition, driven by talent poaching, the San Francisco rumor mill, and reverse engineering, continuously neutralizes any temporary runaway advantage a single AI lab might achieve.
Quotes
“When we see frontier models improving at various benchmarks, we should think not just about the increased scale and the clever ML research ideas, but the billions of dollars that are paid to PhDs, MDs, and other experts.”
Dwarkesh Patel · 0:48 “Humans don't have to go through the special training phase or they need to rehearse every single piece of software that they might ever need to use on the job.”
Dwarkesh Patel · 0:39 “If we actually had a human-like learner, robotics would be, in large part, a solved problem.”
Dwarkesh Patel · 1:20 “People are really underrating how much company and context-specific skills are required to do most jobs.”
Dwarkesh Patel · 2:59 “I expect that by 2030, the labs will have made significant progress on my hobby horse of continual learning.”
Dwarkesh Patel · 7:52
About this episode
Read the essay here https://www.dwarkesh.com/p/thoughts-on-ai-progress-dec-2025-video. Timestamps 00:00:00 What are we scaling? 00:03:11 The value of human labor 00:05:04 Economic diffusion lag is cope00:06:34 Goal-post shifting is justified 00:08:23 RL scaling 00:09:18 Broadly deployed intelligence explosion This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit www.dwarkesh.com https://www.dwarkesh.com?utm_medium=podcast&utm_campaign=CTA_1
Transcript
2,381 words · 1 speaker
Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.
Topics
- continual learning
- in-context learning
- GPT-3
- reinforcement learning
- Gemini 3
- Language Models Are Few-Shot Learners
- O-series benchmarks
- LLMs
- verifiable reward
- power law
Page updated · transcript and summary are machine-generated