Skip to content

AI Leaders Can't Get Along. So How Will They Keep Us Safe? — With Kevin Roose

Big Technology Podcast

1h 4m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

Kevin Roose joins the podcast to discuss his new book, The AGI Chronicles, detailing the intense personal rivalries, drama, and immense stakes driving the race toward artificial general intelligence. The conversation explores how key figures like Sam Altman, Dario Amodei, Demis Hassabis, and Sergey Brin are locked in a relentless Red Queen's race while privately harboring deep fears about the uncontrollable nature of the technology they are building. They also examine corporate missteps at Microsoft, the surprising speed of reasoning models, and why belief is the foundational currency of the top AI labs.

  • Acknowledge that leading AI labs are trapped in a competitive Red Queen's race where capabilities consistently outpace safety research. 9:03
  • Recognize that institutional belief and conviction in building AGI are essential ingredients that drove OpenAI and Anthropic past competitors. 56:53
  • Understand that recursive self-improvement is already underway, with models like Claude leading significant percentages of developer R&D projects. 1:00:41

Chapters

  1. 0:00
    Podcast Intro and Host Setup

    The host introduces the episode, the sponsor, and the guest Kevin Roose, setting the stage for a deep dive into AI leadership conflicts.

  2. 1:33
    Kevin Roose’s Book Overview

    Kevin explains the premise of his book, The AGI Chronicles, and how it uncovers the personal rivalries among AI leaders.

  3. 2:20
    The Culture of Drama in AI Labs

    The conversation shifts to the pervasive infighting among AI pioneers—Musk, Altman, Anthropic, Google Brain, and others—and its implications for safety.

  4. 3:17
    Trust Issues and the Red Queen Race

    The hosts discuss why trusting a small, divided group of AI leaders is risky and describe the relentless, zero-sum competition that drives the field forward.

  5. 48:59
    Tay’s Nazi Fallout and AI Skepticism

    The hosts recount how Microsoft’s Tay chatbot turned Nazi and how that incident fueled skepticism about AI chatbots.

  6. 51:27
    The Rise of Sydney and 4O Fandoms

    Exploration of how OpenAI’s Sydney and GPT‑4O sparked cult-like enthusiasm and concerns about AI psychosis.

  7. 52:23
    Divergent AI Personalities and Control Issues

    Debate over whether AI models should have more varied personalities and the risks of giving them too much autonomy.

  8. 54:24
    Funding and Editorial Independence of Machine Gods

    Explanation of how the new podcast Machine Gods secured funding from the Tarabel Center while maintaining editorial freedom.

  9. 56:40
    OpenAI’s Leadership and the Power of Belief

    Analysis of how belief in AGI drives OpenAI’s continued dominance despite talent losses.

  10. 1:00:15
    Recursive Self‑Improvement and AI Risk

    Discussion of the possibility of AI researchers automating themselves and the potential dangers of recursive self‑improvement.

  11. 1:02:30
    Optimism Amid Rising AI Anxiety

    The host reflects on growing public concern about AI and expresses cautious optimism that humanity can manage the risks.

  12. 1:04:16
    Closing Remarks and Book Promotion

    Final thanks, book launch details, and promotion of Kevin Ruiz’s new podcast Machine Gods.

Highlights

  • 4:13

    Personal Grudges Shape the AI Raceinsight

    The guest explains that the AI race is fueled by personal grudges and visceral mistrust among lab leaders who used to be colleagues, making motives around safety suspect.

  • 8:53

    Capabilities Outpacing Safety Researchinsight

    The guest argues that a pause or slowdown is necessary because current systems are solving complex math problems while researchers still do not know how to control them.

  • 15:34

    Sergey Brin's Competitive Returnstory

    Sergey Brin returned to active coding and bug reporting at Google after being shamed by an OpenAI researcher for sitting on the sidelines in retirement.

  • 25:09

    Transitioning From Skepticism to Beliefinsight

    The guest reflects on shifting from viewing AI as a normal technology to realizing its transformative potential through daily use of the models.

  • 38:21

    Discovering Q-Star Reasoning Tracesstory

    Researchers at OpenAI huddled in 2023 to review early reasoning chains from Q-Star, realizing they had found a way to make models smarter without retraining.

  • 47:13

    Microsoft's Misstep With Bing Chatdata

    Microsoft shipped an early checkpoint of GPT-4 inside Bing before finishing fine-tuning, creating a major cultural wedge between the software giant and OpenAI.

  • 56:53

    The Power of Pure Beliefinsight

    The guest argues that Google fell behind because its leaders lacked the core belief in AGI that drove OpenAI and Anthropic to make massive compute bets.

Quotes

  • “There is Elon Musk split from OpenAI, of course.”
    Unnamed speaker · 2:26
  • “You wrote, 'Machines of love and grace would be built by humans of bitter spite.'”
    Unnamed speaker · 5:59
  • “Like all of the relevant decisions in AI have been made by like a couple dozen people, maybe.”
    Unnamed speaker · 7:07
  • “And no matter how fast you go, like someone else is always going faster than you.”
    Unnamed speaker · 12:08
  • “I think that models should have more divergent personalities.”
    Unnamed speaker · 52:23
  • “OpenAI and Anthropic are the two most believing organizations. They're AI-pilled, AGI-pilled, whatever you want to call them.”
    Unnamed speaker · 56:53
  • “I believe in recursive self‑improvement. I believe that it is possible. I believe that the labs are racing to do it.”
    Unnamed speaker · 1:00:15
  • “People on the street are asking about it. People at their dinner tables are talking about it.”
    Unnamed speaker · 1:03:14
  • Genesis 0:09–0:25
  • Akash 0:52–1:11
  • AppPoint 40:30–41:26
  • Michelob Ultra 41:34–41:57
  • Botox 42:03–43:01
  • Apple Watch Series 12 1:05:15–1:05:41
  • Zocdoc 1:05:53–1:06:11

About this episode

Kevin Roose is the author of The AGI Chronicles,. Roose joins Big Technology Podcast to discuss whether an AI industry rife with personal rivalries, infighting, and mistrust can be trusted to develop technology more powerful than humans. Tune in to hear why Roose emerged from years of reporting inside OpenAI, Anthropic, and Google DeepMind more concerned about AI safety, why he believes the industry should slow down, and what happens if AI systems begin improving themselves. We also cover Sam Altman's leadership style, Dario Amodei's motivations, Sergey Brin's return to Google, Microsoft's AI missteps, and the behind-the-scenes drama of OpenAI's boardroom coup. Hit play for an inside look at the personalities, power struggles, and technological breakthroughs shaping the race to superintelligence. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Want a discount for Big Technology on Substack + Discord? Here’s 25% off for the first year: https://www.bigtechnology.com/subscribe?coupon=0843016b Learn more about your ad choices. Visit megaphone.fm/adchoices

Transcript

0:00
1:04:21

12,183 words · 8 speakers

Topics

  • OpenAI
  • Anthropic
  • Google
  • Microsoft
  • Sydney
  • Tay
  • DeepMind
  • The AGI Chronicles
  • Machine Gods
  • Hard Fork

Page updated · transcript and summary are machine-generated