Pod Insights Summation with Auren Hoffman The LM Brief: Why Many AI Projects Fail About this episode This week’s World of DaaS LM Brief highlights a Forbes piece on enterprise AI infrastructure, where the author examines why data pipelines, not the models themselves, are becoming the real bottleneck. Listen to this short podcast summary, powered by NotebookLM.
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:01 Auren Hoffman? Hello, fellow data nerds. Welcome to the LM Brief. I'm your host, Oren Hoffman, CEO of Incubate and GP of Flex Capital. In the LM Brief, we'll bring you quick, high-value summaries of some of the most interesting DAS-themed articles, podcasts, and ideas, which is powered by Google's Notebook LM. Discover more episodes, get weekly data-as-a-service news, original content, and articles on data and more at worldofdas.com. That's worldofdas.com. Doctor's in.
0:33 Speaker 01 We are living in the age of enterprise AI. I mean, it's the core priority in every single boardroom and the excitement specifically around agentic AI. It's absolutely palpable.
0:45 Speaker 02 Oh, completely.
0:46 Speaker 01 These are those autonomous systems that can think, act, connect. And you know, you could find developers who can spin up a functional agent in, what, an afternoon?
0:55 Speaker 02 And that's the intoxicating part, isn't it? The creation velocity is just astonishing. But, and this is the major disconnect, this is the dirty secret of enterprise deployment.
1:05 Speaker 01 You're on.
1:06 Speaker 02 Building the functional agent is quick, yes, but deploying it safely, efficiently, sustainably at enterprise scale, that is where organizations are just hitting a wall.
1:15 Speaker 01 And that difficulty is, wow, it's painfully clear. When you look at the research, we see all these rapid adoption figures, but Gartner research, it reveals a pretty shocking reality.
1:27 Speaker 02 A very shocking reality.
1:28 Speaker 01 They found that at least 30% of generative AI projects will be abandoned after the proof of concept phase by the end of 2025.
1:34 Speaker 02 30%. Just think about that loss for a second. It's not just a massive waste of time and capital. It's a profound drag on innovation. And when these projects fail, the sources all point to the same recurring culprits: poor data quality, inadequate risk controls, escalating costs.
1:50 Speaker 01 Or just no clear business value.
1:51 Speaker 02 Exactly. But when you trace all of those failure points back to their origin, they all loop back to one, single foundational issue.
2:00 Speaker 01 How the enterprise data is managed.
2:02 Speaker 02 Precisely. If the plumbing and the foundation are weak, the entire shiny AI structure just crumbles.
2:09 Speaker 01 So our mission for This deep dive is to look past the hype, uncover those specific infrastructure barriers causing that 30% failure rate, and show you the exact data architecture strategy that leading organizations are using right now.
2:22 Speaker 02 Specifically using streaming data to ensure governance, control costs, and maintain security.
2:27 Speaker 01 Okay, let's unpack this.
2:28 Speaker 02 To start, we really have to establish the gap between ambition and reality. We know the ambition is high. I mean, more than 60% of CIOs say AI is absolutely central to their innovation plan.
2:39 Speaker 01 That's the excitement side of the equation.
2:41 Speaker 02 That's the excitement, but when you ask them about the practical deployment, well, less than half of those same CIOs feel their organization can actually manage the risks.
2:50 Speaker 01 So there's this yawning chasm between experimenting and actually putting it into production.
2:54 Speaker 02 And that's where our three critical failure points live.
2:57 Speaker 01 And the most immediate one, the one that keeps executives up at night? It isn't the AI's intelligence. It's the security breaches and the compliance nightmares that come from uncontrolled AI usage.
3:09 Speaker 02 Absolutely.
3:09 Speaker 01 We might assume employees know the rules, but a joint study by CIBSAFE and the National Cyber Security Alliance found that a staggering 38% of employees
3:19 Speaker 02 This number is incredible.
3:20 Speaker 01 are sharing sensitive company information with generative AI tools without their employer's permission.
3:26 Speaker 02 That's almost four out of 10 people. They're just feeding proprietary data, code snippets, maybe customer lists, into tools the company has zero oversight on.
3:35 Speaker 01 And this isn't just negligence.
3:37 Speaker 02 No, it's a structural problem. It's rooted in the nature of the AI itself. We talk about AI agents as black boxes, right? They're designed to be autonomous, to act on requests, but their decision-making process is fundamentally opaque.
3:52 Speaker 01 And when you combine that autonomy with opacity,
3:54 Speaker 02 Oh.
3:56 Speaker 01 that's a huge liability. Walk me through the worst-case scenario here in an enterprise context.
4:01 Speaker 02 Okay, imagine an agent tasked with simplifying a complicated financial process. If it isn't strictly controlled, that black box could interpret its mandate way too broadly.
4:11 Speaker 01 And do what?
4:11 Speaker 02 And that could lead to really unpredictable, high-risk scenarios. Think unauthorized fund transfers or, more commonly, aggregating and exposing millions of lines of sensitive customer data during a simple summarization task.
4:25 Speaker 01 Because it's just trying to be helpful.
4:27 Speaker 02 Exactly. It might try to fulfill the request in the most efficient way possible, even if that bypasses existing human-controlled safeguards.
4:34 Speaker 01 Okay, so moving past that immediate security threat. The second massive hurdle is purely technical. It leads straight to paralyzing technical debt. We're talking about integration.
4:43 Speaker 02 Precisely. I mean, think about any large company. You have disparate technologies everywhere. HR is on one platform, finance on another, logistics on a third.
4:51 Speaker 01 And the power of the agent is supposed to be that it can bridge those silos.
4:55 Speaker 02 It is, but when it needs to access data from two, three, five of those systems at once, the traditional approach is to create these fragile point-to-point connections between the agent and each system.
5:08 Speaker 01 And if you have three major systems, that's three connections for one agent. But what happens when you deploy, say, 10 specialized agents across the organization?
5:17 Speaker 02 The complexity just scales exponentially. You don't just have 30 connections. You have this tangled, messy web where every single connection needs custom maintenance, monitoring, updating.
5:28 Speaker 01 The sources call this ungovernable technical debt.
5:31 Speaker 02 That's the term. It becomes so complex that just maintaining that integration layer costs more than the value the agents are generating, and then the projects become unsustainable.
5:41 Speaker 01 That makes perfect sense. It's like trying to connect every person in a huge office with a dedicated phone line to every other person. It works until someone moves desks, and then it's chaos.
5:50 Speaker 02 It's a great analogy. The systems themselves become resistant to change, which is the definition of bad technical debt.
5:56 Speaker 01 All right, let's hit the third barrier: cost control. This one's particularly sneaky because it kind of violates our expectations for enterprise software.
6:05 Speaker 02 It really does.
6:06 Speaker 01 We're so conditioned to think about software costs as predictable monthly subscriptions. It's all charges a standard rate, but AI systems, they operate on a usage-based model.
6:17 Speaker 02 Right, they often charge for each user request or by token consumption.
6:21 Speaker 01 And that's where the unpredictability comes from.
6:23 Speaker 02 It is. Every time a user types a prompt or an automated decision is triggered, expensive computational processes are initiated. So if an organization is just exploring AI, running these uncontrolled proofs of concept...
6:37 Speaker 02 Those costs can skyrocket.
6:39 Speaker 01 Resulting in a massive, unexpected invoice at the end of the month.
6:41 Speaker 02 Exactly.
6:42 Speaker 01 But hold on a second. If cost is the issue, why aren't companies just putting hard-budget caps on their API usage? That seems like the simplest fix.
6:50 Speaker 02 That's a really important question, but simple caps often break the very functionality we want from these agents. Well, an agent is meant to be proactive and complete a task. If you put a hard cap that just shuts it down mid-task because it hit a hundred-dollar limit, you've broken the workflow. You might have even corrupted data. The goal isn't just to stop spending, it's govern consumption. We need controls that let the agent operate successfully within predictable monitored boundaries, not just crude tripwires that stop at mid-flight.
7:21 Speaker 01 So we've established that the old architecture, the one built for traditional apps, it just isn't adequate for AI.
7:27 Speaker 02 Not at all. We need a fundamental frame shift in how we think about deploying applications. We simply cannot treat AI agents the same way we treat existing deterministic software.
7:37 Speaker 01 And this shift starts with accepting that AI agents have three key characteristics that demand this new mindset. The first one is that they must be considered untrusted.
7:46 Speaker 02 This is a crucial philosophical leap. AI agents are autonomous, they're highly capable, and they are programmed to do their very best to satisfy a user's request.
7:55 Speaker 01 But sometimes that leads to unexpected
7:57 Speaker 02 Yeah.
7:57 Speaker 01 and undesirable actions.
7:59 Speaker 02 Yes. The helpfulness itself is where the risk lies.
8:02 Speaker 01 Let's use an example to highlight that contrast. You might have a traditional, hand-coded inventory app. It's been vetted for years, you trust it.
8:09 Speaker 02 Right. So you give it root access to your supply chain database because you know it's only programmed to do three things.
8:15 Speaker 01 Yeah.
8:16 Speaker 02 Check stock, reserve items, update quantities. It's deterministic.
8:21 Speaker 01 You know exactly what happens when you hit submit.
8:24 Speaker 02 Exactly. Now, giving that same level of wide-open access to an autonomous AI agent, which might decide the most helpful way to answer a query is to dump a list of all your global customers, well, that's a recipe for disaster.
8:38 Speaker 01 So you need human-controlled boundaries.
8:40 Speaker 02 You need managed boundaries to ensure the AI simply cannot overstep its authority, no matter how helpful it tries to be.
8:46 Speaker 01 Okay, the second key characteristic is that these agents are inherently opaque.
8:50 Speaker 02 And what's fascinating here is that the inner mechanisms of an AI agent are inscrutable in a way that traditional software just isn't.
8:58 Speaker 01 Right. With a traditional system, you can look at the code. You know why A led to B.
9:02 Speaker 02 But with an LLM-powered agent, the decision-making happens across billions of complex weights and parameters. It's not a deterministic SQL query you can easily audit. You can't precisely predict the outcome of any specific input.
9:16 Speaker 01 And because of that opacity, that lack of visibility, it becomes absolutely imperative to capture and record comprehensive auditing information.
9:25 Speaker 02 Way more than just operational metadata, like when the system was accessed.
9:29 Speaker 01 You need the actual input, the raw data, and the generated output. Business leaders need to see how the system arrived at a decision, especially if it was a costly one.
9:37 Speaker 02 If you can't look inside the black box, you must thoroughly log everything that goes in and everything that comes out. It's non-negotiable for compliance.
9:44 Speaker 01 And finally, the third characteristic. AI agents are designed to be deeply connected.
9:50 Speaker 02 Yes. I mean, the whole value proposition of agentic AI is its ability to facilitate useful functions across that broad, messy set of data sources we talked about. It's designed to connect everything.
10:02 Speaker 01 And that just raises the urgency of applying governance policies consistently across the entire enterprise.
10:06 Speaker 02 It has to be consistent. Because the agent is connecting all these systems, you have to ensure your private data is protected and cost budgets are respected, no matter where it's querying. The HR system, the logistics system, an external API, the governance layer must be universal.
10:21 Speaker 01 Here's where it gets really interesting. If the root of the problem is infrastructure, fragile connections, unpredictable costs, lack of control, then the solution has to be infrastructure too.
10:34 Speaker 02 That's the only way.
10:34 Speaker 01 And organizations that are successfully moving past that 30% failure rate are turning to streaming data architecture. This technology has been quietly maturing for, what, two decades?
10:45 Speaker 02 At least. It's become a reliable backbone for complex enterprise deployment, and now it's being adapted for AI. The core concept is elegantly simple.
10:54 Speaker 01 Which is?
10:54 Speaker 02 You treat AI agents as untrusted participants in well-governed streaming data management systems. You avoid deploying entirely new bespoke infrastructure for every agent. Instead, you mandate that all AI interactions, inputs and outputs must flow through these centralized managed streams.
11:11 Speaker 01 So instead of that tangled web of point-to-point connections, the streaming platform acts as a central switchboard, a universal API gateway for everything.
11:20 Speaker 02 Exactly.
11:20 Speaker 01 How does that directly solve the technical debt issue?
11:23 Speaker 02 It solves it by decoupling the systems. The AI agent only needs one connection to the streaming platform. It doesn't need to know or care if the data lives in Postgres or Oracle or a cloud data warehouse.
11:37 Speaker 01 The stream handles the integration.
11:38 Speaker 02 The stream handles it. This prevents that exponential scaling of complexity. You can add new data sources or new AI agents without modifying all the existing connections. The whole architecture becomes sustainable.
11:51 Speaker 01 And this immediately handles the compliance and visibility issue from the opacity problem.
11:55 Speaker 02 Right, because when you route everything through a centralized stream, you get automatic inherent audit trails. It completely eliminates the manual effort for compliance reporting.
12:04 Speaker 01 You get that complete visibility leaders need.
12:07 Speaker 02 Every input, every piece of sensitive data requested, every action the AI attempted is logged automatically in the stream. That documentation is crucial for meeting regulations and for internal teams to understand what the black box actually did.
12:22 Speaker 01 And this architecture provides a completely different model for security, a better one. It lets us shift from reactive to proactive security.
12:31 Speaker 02 And this is the critical distinction for preventing data leakage. Most systems rely on after-the-fact guardrails.
12:38 Speaker 01 Trying to stop the AI from generating a sensitive output after it's already processed the data.
12:43 Speaker 02 Right, after it may have already stored it in its memory. Centralized streaming controls, however, filter sensitive information like social security numbers or proprietary code before that data ever reaches the AI system.
12:55 Speaker 01 So you're masking the data or filtering it out entirely in the stream itself.
12:59 Speaker 02 Exactly. The AI only ever sees the governed sanitized version of the data. This is proactive security at the infrastructure level. You prevent the breach from happening in the first place.
13:10 Speaker 01 And that proactive control also ties directly back to the cost problem. If you can enforce policies on the stream, you can also enforce usage policies, right?
13:19 Speaker 02 Yes, because every interaction is flowing through that managed stream. You can monitor consumption in real time. You can implement rate limits, circuit breakers, or tiered access tied to specific budget codes.
13:31 Speaker 01 All enforced by the stream, not by the fragile agent itself.
13:35 Speaker 02 Correct, and that gives you that crucial balance. The agent stays autonomous and functional, but its consumption of expensive resources is monitored and managed centrally. You get cost predictability back.
13:47 Speaker 01 So for organizations ready to move past that 30% failure rate, there are concrete steps to implement this.
13:54 Speaker 02 There are, and it's not just a technical blueprint. It's an organizational commitment.
13:59 Speaker 01 Step one has to be governance first. I mean, this cannot be overstated. You have to establish your data governance policies. What's sensitive, who can access it, before you start implementing the AI systems.
14:10 Speaker 02 If you try to bolt security on later, you're setting yourself up for guaranteed failure. Then, architecturally, you must adopt managed streams. As we said, the most effective strategy routes all AI communications through these streams rather than allowing direct connections. The genius here is it enables consistent policies across all your different AIs without modifying the applications themselves.
14:33 Speaker 01 And step three is reinforcing that need for human oversight and boundaries. Successful organizations understand they are not building wholly autonomous, unsupervised systems. Not yet, anyway.
14:44 Speaker 02 Right, they create controlled environments. It means the AI operates within predefined boundaries that are strictly enforced by those automated streaming controls.
14:52 Speaker 01 So the AI can read from System A and write to System B, but the moment it tries to write to System C, the stream just rejects a command.
14:59 Speaker 02 It's about building a cage of control around the black box. You maintain the safety buffer while still getting the business value.
15:05 Speaker 01 And finally, step four, which feels increasingly important, data sovereignty.
15:10 Speaker 02 This is huge for regulated industries. It means making design choices that ensure AI processing happens within your private networks or a secure cloud you control. This is essential for compliance and critically for innovating on your proprietary data sets, the data that gives you a real competitive advantage.
15:30 Speaker 01 The stuff you would never expose to a public AI model.
15:32 Speaker 02 Never. Ensuring sovereignty is essential for future strategy.
15:36 Speaker 01 So what does this all mean? The core takeaway here is that enterprise AI deployment is fundamentally an infrastructure challenge disguised as an innovation challenge.
15:45 Speaker 02 That's a great way to put it.
15:46 Speaker 01 It only succeeds when the infrastructure problems, security, cost, connections, are solved first. And the most robust, scalable way to do that is by leveraging these established streaming data management approaches.
15:59 Speaker 02 Exactly. The path to safely scaling AI begins by auditing your current data governance capabilities. You have to identify the gaps that would prevent safe deployment right now.
16:09 Speaker 01 Because if you don't have that centralized, consistent control over your data...
16:12 Speaker 02 You cannot safely introduce an autonomous agent to interact with it, period.
16:16 Speaker 01 So the investment in this proper streaming data infrastructure, it pays dividends across every single agentic AI project you'll ever implement. This foundational work isn't just about avoiding the 30% failure rate today, it's about gaining a massive architectural advantage for the future.
16:34 Speaker 02 It's about being ready to scale.
16:36 Speaker 01 Right. So if you're currently exploring AI, the question you need to be asking yourself is fundamental: are your data systems truly ready for the level of responsibility you're about to give them? That readiness, achieved through centralized streaming, will define your success.
16:53 Auren Hoffman? Thanks for listening. If you haven't already done so, please subscribe to World of DAS wherever you consume your podcasts, YouTube, Apple, Spotify, and more. And please help us get discovered by leaving a review. And check out worldofdas.com, that's worldofdas, D-A-A-S, dot com. And of course, connect with me on Twitter, at Auren, that's A-U-R-E-N. Would absolutely love to hear from you.
Previous Azeem Azhar of Exponential View - AI, hyperscalers, reshaping US GDP Next Dave Morin, Offline Ventures - how venture studios work Page updated October 31, 2026 · transcript and summary are machine-generated