Pod Insights Summation with Auren Hoffman The LM Brief: Integration gaps and budget missteps stall enterprise AI About this episode This week’s World of DaaS LM Brief looks at new research from MIT, which finds that 95% of generative AI pilot programs inside companies are failing to deliver meaningful financial impact. The issue is not model quality but enterprise integration and a steep organizational learning gap. The report shows ROI is strongest in back-office automation even as most budgets flow to sales and marketing, and that success comes from buying specialized solutions and forming strategic partnerships rather ...
Transcript Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.
0:01 Auren Hoffman? Hello, fellow data nerds. Welcome to the LM Brief, a new series from World of DAS. I'm your host, Oren Hoffman, CEO of Incubate and GP of Flex Capital. In the LM Brief, we'll bring you quick, high-value summaries of some of the most interesting DAS-themed articles, podcasts, and ideas, which is powered by Google's Notebook LM. Discover more episodes, get weekly data-as-a-service news, original content, and articles on data and more at worldofdas.com. That's worldofdas.com. Talk to you soon.
0:35 Speaker 01 Generative AI, it's absolutely everywhere, right? Dominating the news, boardroom discussions, there's just this massive buzz. Companies are pouring money in, making huge promises about transformation. But are those promises actually delivering real results? Okay, let's unpack this because the research we're diving into today really helps cut through that noise. We're looking at brand-new work from MIT's NANDA Initiative. That's their network for AI development and adoption. The report's called "The Gen AI Divide: State of AI in Business 2025." And our mission today, really, is to figure out why, despite all the hype, all the investment, why are most company Gen AI projects kind of, well, stuck at the starting line? What's that mean for business, for work? This deep dive should give you a really clear shortcut to understanding what's actually happening.
1:20 Speaker 02 It really is a surprising headline finding, isn't it? I mean, the promise of Gen AI for businesses is huge, undeniable. But yeah, this report shows that most projects aiming for that quick revenue boost, they're falling flat. And we're going to dig into some insights here that go way beyond the usual suspects, the common assumption.
1:38 Speaker 01 Okay, let's jump right into that headline finding, then, because it is, frankly, a bit of a bombshell. The MIT report says about 95% 95% of generative AI pilot programs in companies are basically failing, failing to speed up revenue, failing to show a real measurable impact on the P and L, the bottom line. That's a huge Gen AI divide, as they call it. It paints a picture of a near-universal struggle.
2:03 Speaker 02 It does. And it's important to know, this isn't just like a few stories they heard. The research is pretty rigorous. They did 150 interviews with leaders, surveyed 350 employees. Oh, and they analyzed 300 public AI deployments, too. So that gives a really comprehensive look at what's working versus what's just stuck.
2:20 Speaker 01 Wow, okay. So if 95% are struggling, what are companies saying is the problem? Is it the usual things, like... Regulations being too tough, or maybe the AI models themselves just aren't quite there yet.
2:33 Speaker 02 Yeah, those are definitely the things you hear a lot, but Aditya Talapalli, he's the lead author on this MIT report. He pushes back on that directly. He says the core issue is not the quality of the AI models. They're actually quite powerful. Instead, it's what he calls a learning gap, both for the tools and for the organizations using them. He points right at flawed enterprise integration.
2:54 Speaker 01 Flawed integration, so how does that play out?
2:56 Speaker 02 Well, think about it. Generic tools, maybe like ChatGPT, they're great for individuals because they're so flexible. But they often stall out in the company setting because they don't learn from or adapt to the specific, often complex workflows already in place.
3:10 Speaker 01 Ah.
3:11 Speaker 02 So it's less about the AI's raw intelligence and more about its, let's call it, social skills within company system. Can it actually fit in and learn the ropes?
3:20 Speaker 01 That's a great distinction. Individual flexibility versus enterprise integration and that learning gap. Okay. Which actually leads nicely to another maybe surprising finding you mentioned. It's about where companies are putting their AI money. The data shows a real mismatch, doesn't it? More than half the gen AI budget is going towards sales and marketing tools.
3:40 Speaker 02 Right. And this is where the report really kind of flips the script.
3:43 Speaker 01 Yeah.
3:44 Speaker 02 The biggest return on investment, the highest ROI. It's actually found in, let's be honest, the less flashy area. Back office automation.
3:53 Speaker 01 Back office, so what does that mean exactly?
3:55 Speaker 02 It means things like eliminating the need for business process outsourcing, cutting costs from external agencies, just generally streamlining internal operations. It's almost like, you know, everyone's rushing to deck out the front of the shop, make it look good. But the real money is being saved or made by fixing the plumbing and the wiring in the back.
4:14 Speaker 01 That's fascinating.
4:16 Speaker 02 So it really makes you wonder why the focus on sales and marketing tools, if the big, quick wins seem to be elsewhere. What does that imply for strategy going forward?
4:26 Speaker 01 Yeah, why do you think that is? Is it just because sales and marketing seem more directly linked to bringing money in? Easier to sell internally, perhaps?
4:34 Speaker 02 It's probably a mix of things. Sales and marketing applications that are often more visible, maybe easier to grasp conceptually. And yeah, maybe they seem like the most direct path to revenue growth, at least on the surface.
4:45 Speaker 01 Right.
4:46 Speaker 02 But this report clearly shows the deeper, often more immediate benefits, cost savings, efficiency are hiding in the operational backend.
4:54 Speaker 01 Okay, so the picture so far is, well, a bit grim. Lots of struggle, maybe misplaced focus. But what about the exceptions? That small 5% who are succeeding. This is where it gets really interesting, right? What makes that 5% different? What are they getting right?
5:08 Speaker 02 Yeah, definitely. And Chalapalli, the lead author, shares some clear insights on that. They saw, for instance, some really young startups, leaders, maybe only 19 or 20 years old, achieving incredible growth. Like, $0 to $20 million in revenue in just a year.
5:23 Speaker 01 Wow.
5:23 Speaker 02 Their formula seems pretty simple, deceptively so. They pick one specific pain point, just one. Then they execute really well on solving it. And crucially, they partner smartly with the companies that end up using their tools. Focus, execution, partnership.
5:40 Speaker 01 That's a powerful lesson in focus, for sure. But, okay, for our listeners, maybe in bigger, more established companies, how does that startup approach apply? Can large organizations adopt those same principles?
5:51 Speaker 02 Absolutely. And the data backs this up with some crucial insights on how companies adopt AI, regardless of size. The report found that buying AI tools from specialized vendors or building strong partnerships, that approach succeeds about 2/3 of the time, like 67%.
6:08 Speaker 01 Okay.
6:08 Speaker 02 Whereas trying to build everything internally from scratch, that only worked about 1/3 of the time.
6:13 Speaker 01 Only a third? That's a big difference.
6:14 Speaker 02 It really is. And it's super relevant for sectors like financial services or anywhere highly regulated. A lot of those firms right now, in 2025, are trying to build their own proprietary gin AI systems.
6:25 Speaker 01 Yeah, you hear about that a lot, control, security.
6:28 Speaker 02 Exactly. But this MOT research suggests that very approach, the internal build, leads to more failures. Interestingly, the report noted companies were often reluctant to share their failure rates on internal projects.
6:43 Speaker 01 Hmm, not surprising, I guess.
6:44 Speaker 02 Right. But the data leaned towards purchased or partnered solutions being more reliable. It suggests it's often smarter to buy or partner for the specialized stuff rather than trying to reinvent that wheel internally.
6:57 Speaker 01 So buy or partner overbuild generally. What else did this successful 5% do?
7:02 Speaker 02 A couple of the key things stood out. Empowering line manager, the people actually running the teams and dealing with the day to day to drive adoption, not just leaving it to some central AI ladder innovation hub.
7:13 Speaker 01 Get it closer to the ground.
7:14 Speaker 02 Exactly. And also selecting tools that can integrate deeply and actually adapt over time. Which brings us back to that learning gap idea we talked about earlier. The tool needs to evolve with the business.
7:24 Speaker 01 Right, it can't just be static. Okay, so that covers the pilots and what leads to success there. What about the bigger picture, like the impact on the workforce? What does the report suggest about jobs?
7:33 Speaker 02 Well, workforce disruption is definitely happening. No question. Particularly in areas like customer support, administrative roles.
7:40 Speaker 01 So layoffs.
7:41 Speaker 02 Not necessarily mass layoffs, interestingly. What they're seeing more is companies choosing not to backfill positions when someone leaves.
7:49 Speaker 01 Uh, attrition.
7:51 Speaker 02 Exactly. It's a quieter shift. And it seems concentrated in those jobs that were often outsourced before. Maybe seen as lower value add internally.
7:58 Speaker 01 Okay. And what about other challenges? You hear a lot about employees using tools like ChatGPT on the sly. Did the report touch on that, shadow AI?
8:07 Speaker 02 It did, yeah. That's highlighted as a real challenge. The widespread use of unsanctioned tools. Employees using things like ChatGPT without official approval or oversight.
8:16 Speaker 01 Which creates risks, obviously.
8:18 Speaker 02 Huge risks. Data security, compliance, you name it. And tied to that is the ongoing difficulty companies have in accurately measuring AI's true impact. It's hard to pin down the exact effect on productivity, on profit, especially with unsanctioned use clouding the picture.
8:34 Speaker 01 Makes sense. Anything about the future, what's next?
8:36 Speaker 02 Well, the report mentions that the really advanced organizations, the leading edge, are already experimenting with something called agentic AI systems.
8:44 Speaker 01 Agentic AI, what's that?
8:46 Speaker 02 Think of AI that doesn't just respond to prompts, but can actually learn, remember information over time, and then act independently based on goals, with inset boundaries, of course.
8:56 Speaker 01 Wow, like a digital team member, almost?
8:59 Speaker 02 Sort of, yeah. It gives us a glimpse into how the next phase of AI in the enterprise might look much more autonomous, potentially much more powerful.
9:07 Speaker 01 Okay, so pulling this all together, if you're listening and trying to make sense of the gen AI hype, the big takeaway from this deep dive seems to be it's not really about how powerful the AI model is in isolation. Success hinges on how companies integrate it, how they actually use it day to day. And the path seems to involve really focused problem solving, smart partnerships, and maybe looking beyond the obvious places like sales and marketing, looking towards that back office automation for some potentially bigger, quicker wins. Precisely, and I think understanding this, digging into the details like we have, helps you see past the breathless headlines. You start to grasp the real world complexities, but also the genuine opportunities of adopting AI in a business context.
9:51 Speaker 01 It reinforces that idea, you know, that knowledge isn't just about knowing something, it's about understanding it well enough to apply it.
9:57 Speaker 02 Yeah, absolutely. So as we wrap up, maybe here's a final thought to leave you with. Given this finding that a staggering 95% of current gen AI pilots are falling short of that rapid revenue acceleration goal, what does that suggest about how companies, and maybe even us as individuals interacting with these systems, might need to redefine what success even looks like for AI in business? Especially as these more advanced agetic systems start to emerge. Something to think about, keep exploring, and thanks for diving deep with us today.
10:29 Auren Hoffman? Thanks for listening. If you haven't already done so, please subscribe to World of DAS wherever you consume your podcasts, YouTube, Apple, Spotify, and more. And please help us get discovered by leaving a review. And check out worldofdas.com. That's worldofdas, D-A-A-S, dot com. And of course, connect with me on Twitter, at Auren, that's A-U-R-E-N. Would absolutely love to hear from you.
Previous Zip CEO Rujul Zaparde - procurement, AI agents and YC insights Next CMO Confidential: Auren Hoffman l Why Vendor Management Is A Skill You Need to Master Now Page updated November 1, 2026 · transcript and summary are machine-generated