
Summary
Written by a model from the transcript; not yet reviewed.
This episode features a discussion on physical AI, dexterity, and robotics foundations with the co-founder and CEO of Generalist. The guest explains how the company's latest foundation model, Gen 1.5, demonstrates emergent one-shot and few-shot physical learning and zero-shot tool improvisation. The conversation explores why dexterity remains the most challenging hurdle in robotics, drawing parallels to evolutionary biology, human tool-making, and language models. The guest also shares insights on data scaling laws, custom hardware development, and the long-term vision of empowering robots to accelerate scientific research and physical labor.
- Gen 1.5 demonstrates one-shot and few-shot physical learning without explicit training for those tasks. 3:04
- Generalist uses simple training recipes and hardware setups to maximize the speed of model intelligence research. 29:54
- Data in robotics is not fungible; collecting diverse data targeted at specific physical tasks drives model performance. 38:41
- Dexterity is considered the physical equivalent of coding, acting as the fundamental driver of robotic capability. 45:15
Chapters
- 0:00Intro and Context of Robotics Challenges
Mario introduces the episode and sets the stage by highlighting the Fukushima disaster and the lack of robots capable of physical tasks.
- 0:46Dexterity as the Key to General Intelligence
The discussion turns to why dexterity is the hardest yet most crucial skill for robots to master, drawing parallels to coding.
- 11:34Evolutionary Roots of Dexterity
Pete explores how human evolutionary pressures on tool use and hand development inspire robotic dexterity research.
- 38:09Data Collection and Scaling Laws
The conversation covers how Generalist gathers massive robotics data, the challenges of valuing data, and the role of scaling laws.
- 41:24Building the Physical AI Stack
Pete outlines Generalist’s hardware strategy, emphasizing simplicity, modularity, and the importance of custom hands for rapid iteration.
- 47:40Robotics Application Layers and Physical Durability
Explores how foundational models affect physical-world robotics applications and their lasting value.
- 48:20Hardware, Workflows, and Durable Value in Robotics
Discusses the importance of better hardware and industry knowledge for making robots useful.
- 49:10Open Source Models and Data Flywheels in Robotics
Examines the potential of open-source robotics models and the challenges of data sharing and value capture.
- 50:53Geopolitical and Policy Implications for Robotics AI
Considers how national interests and policy might shape the future of robotics AI and open-source ecosystems.
- 52:02Safety, Alignment, and Early Investment in Robotics AI
Highlights the need for early research into safety and alignment as robotics AI matures.
- 52:42Teaching, Coaching, and Parallels with Robot Training
Draws parallels between human teaching methods and robot training, emphasizing learning from failures.
- 58:15Novel Tool Use Improvisation as a Milestone
Illustrates how robots can adapt to new tools, demonstrating a key step toward general intelligence.
Highlights
- 3:04
Releasing Gen 1.5 Foundation Modelinsight
The guest introduces Gen 1.5, noting its new capabilities in one-shot and few-shot physical learning and zero-shot generalization. While success rates are modest, the fact that a robot can learn a physical task from a single demonstration is a major milestone.
- 4:21
Parallels with GPT-3 In-Context Learninginsight
The guest explains that just as GPT-3 introduced in-context learning through text data distribution and burstiness, Gen 1.5 exhibits physical burstiness where tasks like taking bills from a wallet follow predictable sequences.
- 6:57
Emergence of One-Shot Capabilitiesdata
The guest reveals that the model was not explicitly trained to perform one-shot or few-shot examples. Instead, physical world burstiness led the model to deduce what actions to take next based on its context.
- 18:10
Evolutionary Roots of Dexterityinsight
The guest discusses how evolutionary biology suggests tool-making and dexterity created the pressure for brain growth and cognition. This biological inspiration drives the focus on dexterous manipulation in robotics.
- 37:01
Carrying Hard Drives in Backpacksstory
To overcome poor internet bandwidth in remote locations, the team resorted to physically transporting hard drives in backpacks and building custom server machines to process data uploads.
- 45:15
Dexterity as the Coding Analogyinsight
The guest argues that dexterity is the physical equivalent of coding because touching and moving the world is the foundation of creating physical value. Cracking dexterity allows robots to build subsequent generations of robots.
- 57:01
Novel Tool Use and Improvisationinsight
The guest highlights Gen 1.5's zero-shot physical generalization through tool improvisation, such as replacing a brush with a dustpan to sweep an object into a bowl, demonstrating true physical reasoning.
Quotes
“If you're not touching and moving the world and you're a robot, what are you doing?”
Unnamed speaker · 0:16 “The success rates of all these are modest today, but the fact that these success rates are anything above zero is the incredible thing.”
Unnamed speaker · 0:33 “If we can really crack dexterity, then the robots can build the robots and the robots can start building the next generation of robots.”
Unnamed speaker · 1:04 “The best analogy would be dexterity.”
Unnamed speaker · 1:03 “If you don't touch the world at all, what can you do?”
Unnamed speaker · 46:21 “There will be these application layers almost in the physical world where you have construction or data center cabling or whatever it might be.”
Mario Gabriele · 47:40 “The zero‑shot physical generalization that we're seeing, that's more of the equivalent of like asking a language model to just write a poem that has never existed before.”
Unnamed speaker · 57:40 “I think that there will definitely be, you know, better and better open source models. And there will be better and better models that are not open.”
Unnamed speaker · 50:39 “I do think, you know, even as amazing as a model of gen 1.5, there's still things that like a very young toddler is just much better than gen 1.5 out today.”
Unnamed speaker · 55:53 “I love these two books that basically like put together in a much better way than I probably could myself.”
Unnamed speaker · 1:03:47
About this episode
Pete Florence https://www.peteflorence.com/ and I began by agreeing that his company has an excellent name. Generalist https://generalistai.com/ builds AI for robots, including models that can begin learning a new task from a single demonstration. We talked about dexterity, the strange business of gathering physical experience for a machine, and what we might build if robots could do much more of the work. — Every timestamp 00:00 Intro 02:24 GEN-1.5 and one-shot learning 08:55 What makes a task difficult 10:53 Dexterity in robotics 14:53 Matt Mason’s definition of a hand 17:34 Giving robots a sense of touch 20:30 Growing up in a family of technologists and scientists 22:46 What drew Pete into robotics 27:38 Generalist’s founding thesis and what sets it apart 32:17 Protecting simplicity 35:01 How Generalist’s data collection has evolved 35:30 Generalist’s scaling philosophy 39:53 Generalist’s build-versus-buy philosophy 42:15 Balancing general intelligence with commercial deployment 44:24 Why Pete is betting on dexterity 47:33 The durability of the robotics application layer 49:09 Open-weight robotics models and data flywheels 50:52 Geopolitics, safety, and alignment 52:42 The parallels between teaching humans and robots 1:00:05 The timeline for robotics progress and adoption 1:03:37 Book recommendations — Pete’s complete reading list The Score Takes Care of Itself: My Philosophy of Leadership: https://www.amazon.com/Score-Takes-Care-Itself-Philosophy/dp/1591843472Wooden on Leadership: How to Create a Winning Organization: https://www.amazon.com/Wooden-Leadership-Create-Winning-Organization/dp/0071453393— Other resources Generalist: https://generalistai.com https://generalistai.com/GEN-1.5: https://generalistai.com/blog/gen-1.5Proprioception: https://en.wikipedia.org/wiki/ProprioceptionCutaneous receptor: https://en.wikipedia.org/wiki/Cutaneous_receptorFukushima nuclear accident: https://en.wikipedia.org/wiki/Fukushima_nuclear_accidentThe bitter lesson: https://www.cs.utexas.edu/~eunsol/courses/data/bitter_lesson.pdfGEN-1: https://generalistai.com/blog/gen-1— Where to find Pete Florence LinkedIn: https://www.linkedin.com/in/peteflorence X: https://x.com/peteflorence Website: https://www.peteflorence.com https://www.peteflorence.com/ — Production and marketing by penname.co https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.
Transcript
12,446 words · 4 speakers
Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.
Topics
- Gen 1.5
- gen 1
- GPT-3
- Generalist
- Fukushima
- foundation model
- OpenAI
- Code
- Anthropic
- American companies
Page updated · transcript and summary are machine-generated