Skip to content

Rebooting Enterprise AI with MCP and Kubernetes

Practical AI

48m
Transcript

Original podcast

Where to listen

Summary

Written by a model from the transcript; not yet reviewed.

Craig McLuckie, CEO of Stacklock and co-creator of Kubernetes, explains why MCP is becoming the enterprise infrastructure layer for AI agents. He draws a parallel to Docker, which both solved a packaging problem and revealed the need for orchestration. Similarly, MCP both describes how LLMs invoke tools and reveals the need for enterprise controls: secure runtimes, registries, gateways, and control planes. He walks through ToolHive, an Apache 2 open-source project that packages MCP servers in Linux containers, provides a vetted registry, a virtual MCP gateway for user-specific tool views, and a Kubernetes control plane for scaling.

The episode also covers identity challenges, including token exchange patterns and the three-legged stool of agent identity (service account, role claims, on-behalf-of claims). Craig reports that tool pollution from 150 MCP servers can burn 20,000–30,000 tokens per interaction, reducible by 80–90% via a proxy layer. He shares that Stacklock sees 50% month-over-month growth in Kubernetes deployment of MCP servers, and that his own engineering team saw a 60% throughput increase in one week by running 5–15 agents concurrently.

  • MCP is analogous to Docker in that it both solves an immediate problem (tool invocation for LLMs) and reveals a larger infrastructure need for enterprise orchestration and control. 3:00
  • Tool pollution from 150 MCP tools can burn 20,000–30,000 tokens per interaction; a proxy layer with find and work tools reduces input token consumption by 80–90%. 29:35
  • Agent identity requires a three-legged stool: a service account identity for the endpoint, role-based claims from the agent owner, and on-behalf-of claims inherited from the user. 23:12
  • Stacklock is seeing 50% month-over-month growth in Kubernetes deployment of MCP servers, with millions of tool invocations from the Kubernetes side. 38:09
  • Craig's engineering team saw a 60% throughput increase in a single week by running 5–15 agents concurrently with controlled tool access. 46:01

Highlights

  • 3:00

    MCP as the Docker Momentinsight

    Craig draws a parallel between Docker (which solved packaging and revealed the need for Kubernetes) and MCP (which solves tool invocation and reveals the need for enterprise orchestration). He calls it rare 'double duty' that a technology occupies two spaces simultaneously.

  • 15:28

    Four Pieces of an MCP Platformstrategy

    Craig identifies four components of an MCP platform: a secure runtime for hosting, a registry for vetted services, a gateway as a single endpoint for clients, and a control plane for scaling from one to thousands of servers with user-group mapping.

  • 23:12

    Three-Legged Stool of Agent Identityinsight

    Craig explains that agent identity requires three components: a service account identity identifying the agentic endpoint, role-based claims provided by the agent's owner, and on-behalf-of claims inherited from the user accessing the agent, which can be chained through multiple systems.

  • 29:35

    Tool Pollution Burns 30K Tokensdata

    With 150 tools from multiple MCP servers, organizations can burn 20,000–30,000 tokens per interaction just describing available tools. A proxy layer with find and work endpoints reduces input token consumption by 80 to 90 percent, which is critical for smaller models that are bad at tool invocation.

  • 33:02

    ToolHive: Building the Yellow Brick Roadstory

    Craig describes ToolHive as Apache 2 licensed open source, with the philosophy that while Anthropic, OpenAI, and Google describe the Emerald City, someone needs to build the yellow brick road. It packages MCP servers in Linux containers, runs them through full SDLC pipelines, and provides registry, gateway, and Kubernetes control plane.

  • 38:09

    50% Monthly Growth on Kubernetesdata

    Craig reports 50% month-over-month growth in Kubernetes deployment of MCP servers, with millions of tool invocations coming from the Kubernetes side, signaling rapid enterprise adoption of running MCP servers in containerized orchestration environments.

  • 40:58

    Stochastic Reconciliation for Self-Healinginsight

    Craig envisions a future where when a Kubernetes reconciliation loop fails (like a crash loop backoff), a stochastic AI system is invoked to reason about why the system is out of conformance and drive it back, enabling self-healing and self-optimizing infrastructure.

  • 46:01

    60% Throughput Gain from Concurrencydata

    Craig's engineering team saw a 60% throughput increase in a single week by running 5–15 agents concurrently with controlled tool access. He calls developers 'performance athletes' and describes the pattern as a path toward a dark factory with a human operator spinning plates.

About this episode

What happens when AI agents start acting less like chatbots and more like coworkers? In this episode, Dan and Chris sit down with Craig McLuckie, CEO of Stacklok to explore MCP, Kubernetes, ToolHive, enterprise AI, and the emerging infrastructure powering AI-native applications. From identity management to agent orchestration and system architecture, this conversation dives into how organizations may soon manage entire fleets of AI agents working behind the scenes. Featuring: Craig McLuckie – LinkedIn https://www.linkedin.com/in/craigmcluckie/Chris Benson – Website https://chrisbenson.com/, LinkedIn https://www.linkedin.com/in/chrisbenson, Bluesky https://bsky.app/profile/chrisbenson.bsky.social, GitHub https://github.com/chrisbenson, X https://x.com/chrisbensonDaniel Whitenack – Website https://www.datadan.io/, GitHub https://github.com/dwhitena, X https://x.com/dwhitenaLinks: Stacklok https://stacklok.com/?utm_source=linkedin&utm_medium=social&utm_campaign=profileToolhive https://stacklok.com/download/Sponsors: Prediction Guard: A self-hosted AI control plane for running agents in high impact environments. predictionguard.com/practicalai https://predictionguard.com/practicalaiUpcoming Events: Register for upcoming webinars here https://practicalai.fm/webinars! Midwest AI Summit 2026 https://midwestaisummit.com/

Transcript

0:00
48:08

9,944 words · 4 speakers

Topics

  • MCP
  • Kubernetes
  • AI Infrastructure
  • Agent Identity
  • ToolHive
  • Enterprise AI
  • Agentic Concurrency
  • Token Optimization

Page updated · transcript and summary are machine-generated