
AI Can Write the Code. Can It Run the System? | Grepr CEO Jad Naous
Hosted by Erik Torenberg
- 1h 0m
- Transcript
Summary
Written by a model from the transcript; not yet reviewed.
In this episode, A16Z general partner Martín Casado talks with Grepper CEO Jad Naus about how AI is moving beyond code generation into the much harder domain of operations and infrastructure. The guest explains how his 10-person team uses coding agents and new workflows to build distributed systems in days rather than months, multiplying the output of top engineers. However, the host and guest debate whether AI can ever truly reason about production systems, manage distributed clusters, and handle the heavy-tailed, exception-based nature of the real world where state spaces grow exponentially. While code generation benefits from a training set that matches the solution space, operations require understanding complex real-world variables, leading to a discussion on whether AI should rely on predictive anomaly detection or deterministic workflows and experimentation.
Ultimately, the conversation highlights the gap between the digital world where AI excels and the messy physical and operational realities of running enterprise systems. They examine why buyers still prioritize immediate cost-reduction use cases like telemetry savings over nascent AI-driven SRE tools, and how economic equilibrium ultimately dictates what succeeds. This makes for a compelling listen for engineers and founders trying to understand the actual limits and leverage points of AI agents in production software development and infrastructure management.
- Top software engineers can multiply their output by integrating AI agents into workflows, completing projects in days that once took months. 24:35
- Instead of relying on lagging indicators to debug incidents, effective AI operations tools should focus on capturing changes and early warning messages. 30:57
- Rather than trying to predict complex operational failures, AI can use deterministic change detection to run structured debugging workflows and hypotheses. 42:19
- Enterprise software buyers prioritize immediate, measurable cost reduction, such as slashing telemetry bills, over unproven AI reliability features. 57:51
Chapters
- 0:00Podcast Intro and AI Coding Origins
The episode opens with a brief introduction to AI’s evolution from simple autocomplete to complex code generation, setting the stage for a deeper dive into operations.
- 0:32From Code to Operations: The AI Gap
The conversation turns to the challenges of applying AI to operational systems, where state space and rare failure modes make modeling difficult.
- 10:08Grepr’s Mission: Telemetry Reduction and AI Layer
Jad explains Grepr’s two‑layer approach: first cutting telemetry costs by 90%, then feeding distilled signals into an AI for troubleshooting and optimization.
- 18:41AI in Code Writing: Gains and Workflow
Jad shares concrete examples of how AI has accelerated development, turning months of work into days, and details his team’s AI‑augmented coding workflow.
- 28:42Query Optimization and AI Reasoning
Jad recounts how AI helped optimize complex query pipelines, illustrating the potential for AI to replace human reasoning in performance tuning.
- 32:32AI‑Driven Observability and Anomaly Detection
They discuss building predictive models that flag anomalies, then using AI to hypothesize causes and run experiments to resolve issues.
- 37:01Limits of AI in Distributed Cluster Management
The hosts debate whether AI can fully manage distributed clusters, concluding that while AI can assist, human oversight remains essential.
- 46:27Future Outlook: Code vs Operations
They wrap up by reflecting on the differing complexities of coding and operations, pondering how AI might evolve to tackle the hardest problems.
- 47:44VC Perspective on AI’s Role
The host discusses how AI can assist in VC analysis but emphasizes that human relationships remain essential.
- 50:15AI vs. Human VC Decision-Making
Debate over whether AI can replace the relational aspects of venture capital.
- 52:41AI’s Path Beyond Code
Discussion of future AI applications after code generation, including self‑driving and other domains.
- 55:11Economic Equilibrium and AI Adoption
Analysis of how economic forces will determine which AI solutions succeed in real‑world problems.
Highlights
- 18:41
Building Products 10x Fasterstory
The guest shares that a complex data lake auto-promotion feature which required distributed metadata coordination took just 10 days to build alone, compared to six months previously.
- 19:44
Good Engineers Gain More Leverageinsight
The guest explains that as AI tools evolve, the productivity gap between average engineers and truly great engineers expands by a factor of ten because complex distributed systems require human mental bandwidth.
- 30:57
Flaws in Lagging Indicator SREinsight
The guest argues that traditional AI SRE tools fail because they try to backtrack into root causes from lagging metrics like traffic spikes, whereas effective systems start by detecting specific environment changes.
- 33:24
The Hidden Costs of Software Stateinsight
The host points out that every time a bit is added to a program, its operational state space increases by a factor of two, creating more possible states than atoms in the universe and making chaos engineering difficult.
- 35:34
Running Parallel Claude Code Sessionsstrategy
The guest details his workflow of running three to four command-line session windows simultaneously with persistent subagents, having a tech lead reviewer agent check PR documents and algorithms before human review.
- 54:33
The Cost of Autonomous Drivingdata
The host notes that after Sebastian Thrun won the DARPA Grand Challenge, self-driving cars took 20 years and $170 billion in investment to achieve worse unit economics than Uber.
- 57:49
Buyers Want Cost Cutting Over AIstory
The guest shares an anecdote where prospective customers expressed excitement about cutting their DataDog telemetry bills by 90% rather than buying the proposed AI reliability and troubleshooting layer.
Quotes
“When we started doing AI for coding, it was autocomplete.”
Unnamed speaker · 0:05 “But writing software and running it are two very different problems.”
Erik Torenberg · 1:01 “First, people were spending somewhere between 5% and 20% of what they spend on infrastructure on observability.”
Unnamed speaker · 7:55 “And so you put us in the data stream, plug and play, pass the data through, reduce your cost by 90%.”
Unnamed speaker · 10:12 “It's like every time you add a bit to a program, the state space of like, what it can do operationally increases by a factor of two.”
Martin Casado · 33:08 “I think over time it'll all, it's so funny that I'm actually telling you this”
Unnamed speaker · 47:44 “the distribution of the solution space is the same as the distribution of the training set”
Martin Casado · 53:57 “It took 20 years and $170 billion of investment to get to worse unit economics than Uber”
Martin Casado · 54:38
About this episode
a16z General Partner Martin Casado sits down with Grepr CEO Jad Naous to ask what comes after AI learns to write code: can it also debug and operate the systems that code creates? Jad shares how AI has transformed engineering inside his 10-person team, cutting some projects from six months to just 10 days. But operating production systems is a harder challenge, with unpredictable workloads, failures, and countless possible states. They discuss whether AI can learn to diagnose problems, test hypotheses, and debug systems like a human engineer. They also explore AI SREs, security, query optimization, and why the best engineers may be gaining the most leverage from coding agents.
Transcript
11,416 words · 3 speakers
Speaker names marked are inferred from speaking time and the show’s known hosts — they are not voice-verified.
Topics
- AI
- code
- A16Z
- Stanford
- DARPA Grand Challenge
- SRE
- universal function approximator
Page updated · transcript and summary are machine-generated