
Summary
Written by a model from the transcript; not yet reviewed.
This episode explores the rapidly expanding and often misunderstood market of AI inference with Tuan Srivastava, co-founder and CEO of Base10. The conversation details how Base10 evolved from early machine learning tooling for data scientists into a specialized infrastructure provider operating across twenty different clouds and handling massive production workloads. The discussion examines why inference represents the definitive future market for AI, the complexities of managing heterogeneous compute during severe supply shortages, and how enterprises are moving toward a portfolio approach utilizing both closed models and customized open-weight models.
Listeners get a behind-the-scenes look at the operational realities of scaling AI infrastructure, including the financial engineering, five-year terms, and thirty percent prepayments required to secure billions of dollars in compute. Srivastava explains the concept of owned intelligence and continuous learning loops where companies capture data from user interactions to retrain proprietary models. The episode stands out for its clear, highly specific breakdown of the multi-step journey organizations take as they transition from basic API routing to dedicated deployments and custom fine-tuning.
- Anticipate long-term forecasting gaps by planning for rapid growth that can outpace initial projections by severalfold in emerging markets. 7:17
- Recognize that serving models at scale requires a dedicated, heterogeneous stack rather than traditional cloud or training infrastructure. 17:46
- Build compounding organizational advantages by establishing continuous learning loops that integrate proprietary data, user feedback, and model retraining. 25:15
- Navigate severe hardware shortages through architectural flexibility and multi-cloud strategies that stitch together regional compute resources. 29:44
Chapters
- 0:00Podcast Introduction and Inference Overview
Hosts introduce the episode, framing inference as the next critical AI challenge.
- 0:35The Complexity of Inference at Scale
Discussion of why inference is hard to scale, the need for capacity, and the software challenges involved.
- 2:39Early Years of Base10 and Product Pivot
The founders recount starting in 2019, the initial focus on data science tools, and the shift to serving inference as the core product.
- 7:38Market Dynamics and the Supply Crunch
Exploration of the rapidly growing inference market, the limited compute supply, and the impact on pricing and growth.
- 20:55Customer Journey: From Closed to Open Models
Illustrates how customers move from using closed models to incorporating open‑weight models, routing, and post‑training loops.
- 29:44Heterogeneous Compute Strategy
Base10’s approach to using multiple clouds and regions to achieve reliability, low latency, and geographic flexibility.
- 31:33Financial and Operational Challenges
Details the complexities of financing large compute spend, project‑finance style deals, and the need for long‑term commitments.
- 36:54Hiring Philosophy and Customer Success
Tuan shares what he looks for in talent and how a hospitality mindset drives customer relationships and product excellence.
- 42:34Future Outlook and Experimentation
Speculation on future model portfolios, the role of post‑training, and the potential for unlimited compute resources.
- 43:42Closing Remarks and Call to Action
Final thanks, podcast subscription reminders, and a brief sign‑off from the host.
Highlights
- 2:44
Pivoting From Early ML Toolsstory
The guest explains how the company originally built scaffolding for data scientists using small models and internal tools, eventually discarding three years of work in late 2022 to focus entirely on high-scale inference serving.
- 7:17
Revenue Forecasts Consistently Offdata
The guest reveals that the company's revenue forecasts for consecutive years have been completely off, including being off by 4x against initial January projections for the year 2025 due to exponential market demand.
- 17:46
Inference As The Final Marketinsight
The discussion frames inference as the final market of the AI economy because it represents the actual running of intelligence and compute cost of goods, projected to reach trillions of dollars as AI scales.
- 21:26
The Evolution Of Customer Workflowsstrategy
The guest outlines the typical customer journey from using closed APIs to adopting open-weight models, implementing dedicated deployments, and finally building custom post-trained models with specialized routing layers.
- 26:53
Enterprise Shift To Open Modelsinsight
The host and guest discuss how companies ranging from AI natives like Sierra to large enterprises like AT&T are increasingly relying on a portfolio of models to optimize costs and protect proprietary data.
- 34:49
The Financial Reality Of Computedata
The guest breaks down the capital intensity of modern inference infrastructure, noting that securing billions in compute typically requires five-year terms and thirty percent prepayments.
- 37:47
Unreasonable Hospitality In Softwareinsight
The conversation connects the company's client relationship philosophy to Danny Meyer's principles of hospitality, highlighting high-touch service and deep integration with customer engineering teams.
Quotes
“Inference in a lot of ways, I think, is maybe like one of the most misunderstood markets in the world.”
Unnamed speaker · 0:08 “The only thing left to do will be inference.”
Unnamed speaker · 0:00 “We are critical path. Like if Base10 is down, a lot of our businesses aren't working.”
Unnamed speaker · 0:25 “Inference is the cause of AI.”
Unnamed speaker · 18:36
About this episode
Tuhin Srivastava https://www.linkedin.com/in/tuhin-srivastava is the co-founder and CEO of Baseten https://www.baseten.co/, a company building infrastructure for training and inference. When an AI model runs in production (responding to a user, powering an app, making a decision), Baseten is one of the companies that makes that happen reliably, quickly, and at scale. The company counts Abridge, Cursor, and OpenEvidence among its customers, spans 20 clouds and 90 regions, and was most recently valued at $13 billion following a $1.5 billion funding round. Tuhin sees inference as AI’s “final market”: as AI spreads through the economy, the cost of running intelligence grows with it. Baseten’s breakout began when it discarded three years of other product work in roughly six weeks to focus on an inference engine it had been giving away. In this conversation, we explore why inference has become one of the most consequential and least understood markets in AI, what it actually takes to run a model in production at scale, and why the compute squeeze may be the single largest constraint on AI growth right now. We also examine the customer journey from closed models to custom fine-tuned deployments, the concept of owned intelligence, and why Tuhin believes luck accounts for roughly 90% of startup success, while insisting that the 10% skill is still indispensable. — Tuhin’s bookshelf When I asked Tuhin what book he’d assign to everyone, he named Breakneck https://www.amazon.com/Breakneck-Chinas-Quest-Engineer-Future-ebook/dp/B0DXQGL4X9. He also pointed to Danny Meyer’s book about Union Square Cafe, Setting the Table https://www.amazon.com/dp/0060742763. — Episode transcript: https://www.generalist.com/i/223344398/transcript — Every timestamp (00:00) Intro (02:06) What Baseten is building and how it began (04:47) Finding balance in a fast-growing startup (06:45) Why inference is the “final market” (08:34) Tuhin’s path to founding Baseten (12:06) What Australia gets right (13:43) The opportunity Baseten found in inference (20:34) How customers use Baseten (29:00) Managing compute across clouds (30:40) The compute crunch (32:11) Financing inference at scale (35:40) Why inference is hard to build in-house (36:41) Who Baseten wants to hire (39:46) Hard work and skill vs. luck (42:20) Final meditations — Resources Baseten: https://www.baseten.co https://www.baseten.co/Sierra: https://sierra.ai https://sierra.ai/Coinbase: https://www.coinbase.com https://www.coinbase.com/No Priors Ep 56 | With Baseten CEO and Co-Founder Tuhin Srivastava: https://www.youtube.com/watch?v=oe3OSlsf4YAEtched: https://www.etched.com https://www.etched.com/Cerebras: https://www.cerebras.ai https://www.cerebras.ai/NVIDIA Groq 3 LPX: https://www.nvidia.com/en-us/data-center/lpxUnreasonable Hospitality: The Remarkable Power of Giving People More Than They Expect: https://www.amazon.com/Unreasonable-Hospitality-Remarkable-Giving-People/dp/0593418573Blackbird VC: https://www.blackbird.vc https://www.blackbird.vc/Greylock: https://greylock.com https://greylock.com/Conviction: https://www.conviction.com https://www.conviction.com/— Where to find Tuhin X: https://x.com/tuhinone LinkedIn: https://www.linkedin.com/in/tuhin-srivastava — Production and marketing by penname.co https://penname.co/. For inquiries about sponsoring the podcast, email jordan@penname.co.
Transcript
8,839 words · 3 speakers
Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.
Topics
- Base10
- Nvidia
- Greylock
- Conviction
- Refusion
- Sierra
- Abridge
- Cursor
- OpenEvidence
- Amazon Web Services
Page updated · transcript and summary are machine-generated