Pod Insights Voices of Search // A Search Engine Optimization (SEO) & Content Marketing Podcast How AI Search Is Reshaping Retail Media and Discovery About this episode Enterprise retailers index only 40-50% of their product pages. Joe Doran, Chief Product Officer at Botify, analyzes AI search readiness across retail sites where 9 of 10 products contain three to seven feed errors or missing fields. The conversation covers crawl efficiency and rendering gaps that leave LLM bots seeing just 30-40% of JavaScript-dependent content, structured feed optimization for the Agentic Commerce Protocol and Universal Commerce Protocol, and a shift toward holistic, purchase-anchored attribution over session-based models.
See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.
Transcript 0:00 Speaker 00 The Voices of Search Podcast is a proud member of the I Hear Everything Podcast Network. Looking to launch or scale your podcast? I Hear Everything delivers podcast production, growth, and monetization solutions that transform your words into profit. Ready to give your brand a voice? Then visit ihereeverything.com. Welcome to the Voices of Search Podcast and I Hear Everything production. In this podcast, we'll share the news, knowledge, and strategies you need to navigate the ever-changing world of SEO. Ready to expedite your company's organic growth efforts? Sit back, relax, and get ready for your daily dose of search engine optimization wisdom. Here's today's host
0:42 Speaker 00 of the Voices of Search Podcast, Tyson Stockton.
0:46 Speaker 01 My name's Tyson, and today I'm joined with Joe Doran, Chief Product Officer at Botify. Welcome to the podcast, Joe.
0:54 Speaker 02 It's great to be here. Super excited to talk about the future of agentic search and what's happening in the world and how retailers can get ready for it and actually help them sell more product.
1:08 Speaker 01 Yeah, and I like that we're diving into like a specific kind of industry and vertical within it. Because a lot of times on the podcast, we're talking about the industry as a whole. We all know the industry is changing, all these differences going on. But today, we want to kind of like dive more into like commerce and retail, which is an area we've both spent quite a bit of our career in. So to start things off, kind of like, give us the lay of the land. Like, how is commerce and e-commerce in particular, changing?
1:42 Speaker 02 Yeah, it is changing dramatically, and I think it has a lot of leaders in the retail, e-commerce, DTC sections really scared about what will be the future of how consumers shop, right? And how much control a retailer can actually have over that shopping experience when it's driven by an agent, right? Or a bot that's actually there, whether that is how. Happening within the chat GPT and Gemini or whether it's happening outside of that and in and around their site. So thi- this has a lot of retailers, a lot of leadership at retailers and e-commerce providers, thinking about their overall strategy
2:26 Speaker 02 and their overall investment so that they can control what's most precious to them is their brand and the shopping experience of their brand between them and their consumers.
2:35 Speaker 01 And I mean, we we we've talked about it before on the show a little bit as far as like how consumers are both going through like the buyer's journey as well as just kind of like how people are actually like interacting with it like from what you're seeing today like is are are conversions happening yet within kind of like these environments or are you seeing it more as like the discovery like research, early stages of the cycle?
3:06 Speaker 02 Yeah, I think it's mostly the early stage of the cycle or what we commonly refer to as the upper funnel or the product discovery component and and the thing that's changing about that that journey is really these chatbots, whether it's chat GPT Gemini or perplex or any of them. They are surrounding the consumer with additional context that search engines could never provide. Right? It's allowing them to go and look through their specific use case, their specific shopping use case. I need a a certain outfit for a certain event. I need to have it within a certain budget. Those multiple constraints are really hard in a traditional search
3:50 Speaker 02 solution to go and solve for. The chatbots do a great job of that product discovery, narrowing it down and allowing you to go and find what the full portfolio of products are out there. Right? And and what scares retailers is, you know, are they showing up when their competitors' brands are showing up? How are they showing that their products actually outperform or are clearly differentiated and have a unique selling proposition vis a vis their competitors? And this is what Really drives that fear among the retailer, they need to lean in or they're going to miss out on that. But one of the things that we are seeing is that the transactions, as you said,
4:29 Speaker 02 are not happening within the L L M's. It's not happening within the chat experience, even though they're trying to get there. But we believe that that that that awareness, that consideration, that interest and intent is being formed. And what happens when a referral comes to a retailer's site? It's going right to the product description page and it's leading to a higher conversion. So this channel looks like it can change the nature of the conversion rates, right? For the retailer, but they but they're they're not in control of it. So that that's what like, oh, my goodness, so much promise, so much risk.
5:08 Speaker 01 Yeah, well, it's like there's like a black box piece because it's like, if you're looking just at like your internal metrics, it's almost like the you have this impression that the buyer's journey is getting compressed down. Because from when someone enters the site to then maybe that transaction or that conversion happening, it's a shorter time period. But it's like, I don't think that necessarily like our behavior as humans are changing as much as sure where we're getting the information is. But it's like, I'm still going to want to research and learn about different variances of products, make my own opinions and do it. But when you actually like land on in the site,
5:50 Speaker 01 there's a very short time period. And so I think it is something that's like both creating a lot of fear and uncertainty. But at the same time, it's like, hey, e commerce isn't dead, like people are still more than ever buying online. It's just like how we're competing and how we're being discovered is changing.
6:10 Speaker 02 Exactly. And and one of the things that's really interesting, Tyson, is really when we talk to retailers, like about half of them will say what you said, which is what the industry talks about is that the. The buyer's cycle has been very compressed, right? You have a very compressed buyer's cycle. The other half of the retailers will say, well, the old search experience was full of a lot of false positives. I would make a query, I would go to a product page and it wasn't what I wanted. I needed to go back, do another query. So it's actually more efficient and you got rid of the bad clicks. The clicks that were never leading to a conversion. So it's actually, traditional search is wasteful.
6:55 Speaker 02 And, and, uh, you know, the, the AI search is actually more efficient. Not really sure if there's data to prove which one is actually truly correct, but we do know that, that this zero click environment is happening. And it's, and it's like, what does it mean to me? And what does it mean to me as a retailer? Am I shopping experience? And how do I make sure that my products are getting the best chance to actually show up? And, and be
7:25 Speaker 01 Well, like to that, to that last point, like, let's talk a little bit on like indexation. Cause I feel like this is an area, I mean, one, if you're, if you're talking enterprise, these are substantial, really large websites. And I mean, and if you just think about the amount of like filters and facets, it's like, you have this potential of just ballooning websites. And then that becomes a challenge as far as like, what are actual search engines, now LLMs seen and discovering? So like, from your experience and your perspective, like,
8:03 Speaker 01 are retailers aware and like, I guess, how familiar do you think like the industry is with like, the actual indexation and kind of crawl efficiency of their website?
8:18 Speaker 02 Yeah, I think there's a lot of retailers that believe that their website is fully crawlable, that that they are indexed at a high rate, which, you know, because they've been doing this for a long time, fifteen, twenty years against this. But the challenge is, a lot of these merchants and a lot of these retailers, just like what you said, are exploding their product portfolios, right? It is very rare that you see a tier one or tier two or even a tier three retailer with less than a million products. That means that their their their website is going to be well over a million pages that's there. When we look at retailers across like size and scale,
9:03 Speaker 02 we're only seeing them, and this is reported through Google Search Console, their indexation sits between forty to fifty percent, which means only forty to fifty percent of their web pages are actually being indexed, let alone crawled, right? So they're and their crawlability is never at a hundred percent. The crawlability is always really, really low, probably like maybe ten to twenty points above their indexation rate, right? And that is really a a clear indicator that you're not, you know, your your products aren't even showing up, let alone and and and Google's got the most sophisticated bot in the world,
9:43 Speaker 02 and they're not showing up with the most sophisticated bots and and the most extensive crawl budget, right? All the chat bots and crawlers that are coming from ChatGPT and the rest of the LLMs are twenty years behind in uh uh being able to crawl with a significant amount of efficiency and scale that Google has. So they're just not as sophisticated. They have a hard time spending that budget efficiently and getting to what would be the most profitable or the most productive pages that a retailer has. So they're missing out, and this can be a major, if if they fix that problem,
10:26 Speaker 02 that's a first step that they need to do in fixing the problem to drive visibility, because they're they're not giving their products a chance, right?
10:35 Speaker 01 Well, I mean, I want to double down on that point because it's like forty or fifty percent like that is crazy. As far as like, it doesn't matter how good your content is, it doesn't matter how fast your website loads. If they're only seen like that, then that means almost half of your products don't even have a chance. And let alone, and I mean, to me, it's not necessarily like, oh, this is like an AI search problem. It's like, no, this is a fundamental then of just online discovery. And one of the things that like I've seen too is like, you also have like varying, like working with international e-commerce sites.
11:19 Speaker 01 Like we did a study with one of our clients who operates in a bunch of different countries. Each country had a significantly different percentage of indexation. And some were a little stronger, some were really bad. But it's like, to me, that's like the lowest hanging fruit from an e-commerce side. To actually have a significant gain, if like, you don't change your performance at all, but you just improve that indexation. Like that has a potential to just exponentially increase like your bottom line.
11:55 Speaker 02 Yep. No, it it does. And we see that significantly. But you know, a lot of our retail customers or a lot of the retail industry, they've spent decades investing in their e-commerce site. Really to support human traffic, right? And the non-human traffic was was always seen as waste with the exception of Google bot. Now, there's fifteen to twenty other bots that matter or crawlers that matter. That you need to go and support and drive against, and they're not serving the Google crawler efficiently today by itself. So this this complexity is significantly increasing. So how do you go and and make the investments in the infrastructure to make sure that you're
12:39 Speaker 02 you're all of your content and all your product information is fully ready and readable by the? Bots, right? And that that is not only just like you said, performance time. So it has to be very, very fast in loading and and delivering it. But it has to be able to fully read and fully render that page, right? And a lot of those pages that our retailers use are very heavy JavaScript or API laden that that really cause problems for a bot or a crawler to truly fully read and render. So they give up because they're not going to, if I can't resolve this very, very fast or efficiently, I'm going to reduce the number of pages in your budget, right, that I'm going to crawl.
13:23 Speaker 02 And that that just, it just becomes a compounding effort, right? And you'll flow out of the index and you will flow out of impressions, you'll flow out of clicks and you'll lose organic revenue, which is really, really a shame, right? Because they, the retailers really need to get, they really need that to power the rest of the e-commerce engine.
13:45 Speaker 01 Well, let's, let's dive in a little bit more because you made in the comment a little earlier on how like the, the bots and the crawlers of some of these emerging platforms, companies are not, they don't have the same sophistication or, I mean, essentially experience that Google has in this space. So like, tell us a little bit more on like, what are some of those differences? How can we better understand these kind of, I mean, really like these bots that are growing in importance and significant to a business's health?
14:20 Speaker 02 Yeah, I think there, there's a couple of them. First off, just in full
14:26 Speaker 02 rendering, if you want to be able to render it, the, the biggest challenge is JavaScript on a page. Retailers use JavaScript very materially to have a performance side
14:38 Speaker 02 lift for consumers using a browser, right? So it, it is a very great tool for human traffic to increase performance and, and have the the right responsiveness on the page for what a consumer is actually looking for. Our friends at the crawlers and bots cannot render them. So what ends up happening is they see thirty to forty percent of the content that's on the page. They miss a tremendous amount of the content that's not really fully rendered. So what do they miss? They miss reviews. Reviews is almost always in JavaScript because it's really populated by a third-party solution.
15:15 Speaker 02 A lot of times their specs and descriptions tend to get pulled through JavaScript and not kind of like directly pushed into the page like on HTML. Any offers or promotions tend to be rendered through JavaScript. So you get a like thirty to forty percent of what's on the page. And and when we show this to a retailer and say, this is what a crawler sees and this is what a consumer sees, it's it's they just get, you know, very irate. Because it's like, oh, I need to fix this because they they're missing like half the information that's that's there. So how do I go and and and do that?
15:53 Speaker 02 And show that. So that that's what is is one of the biggest challenges that that really is is part of the problem. The other big challenge is the LLMs or the agentic search players like ChatGPT and those, they operate differently. When you do a prompt into ChatGPT, it will pull from its sources of index and what it actually has stored. But it's going to do a confirmation crawl against that page to go read it. If they can't read it, it's not going to be able to verify that data. That and it and it probably will lead to, oh, it it doesn't match what's in my index. So I'm not going to. It's not verified. I can't trust it. I'm not going to show it.
16:39 Speaker 02 No mention, no citation.
16:42 Speaker 02 And it it. So this goes differently. Google does that, but very, very specifically, Google does that really around Google Shopping, right? And when they're presenting a Google carousel, because they're trying to confirm price and availability from what you've given to them in structured data. So common challenge, but because they operate a little bit differently, it's it's an additional complexity, right? That you, as a retailer, have to ensure that that you're serving these these this non-human traffic in the same way with the same care that you do for your human traffic. Because it it it is part of the shopping experience.
17:19 Speaker 01 That's it. That's super interesting and it's in some ways, it's like it has like some deja vu elements of it where it's like. I mean, yeah, JavaScript up until now, I mean. Early days SEO, huge concern, like that was a huge topic. We talked about it a ton. Kind of started to fade away as Google became more and more sophisticated and how they were able to render pages. And like now it feels like we're kind of coming back. So it's like, even though we're in this new space in this new world. It's still like some familiarity to it. But I think your comment of like, hey.
18:05 Speaker 01 Sometimes these systems or these bots are only seeing maybe thirty percent of the page. I would assume in addition of like, hey, maybe there's important information that they're not seeing and it's not going to be included. There probably has also been like a perceived lower quality of content. Which then could influence the amount of bandwidth that these crawlers are going to spend on those sites. So it's like. It it it feels like that's like a compounding issue, not just like, oh, hey, like they didn't see that our product was on sale. It's like, okay, maybe we would have had sixty percent indexation instead of forty.
18:45 Speaker 02 Yes, that is exactly the the challenge, right? So i- if I as a crawler can efficiently crawl the breadth of your site, I'm going to continually pull back on it. Because there's other parts that I could probably spend on. The the the big challenge with anybody that has a crawler like and they're trying to index the interweb. The the internet is massive, right? E- even Google will have a very difficult time indexing the full internet. So they have to make choices, right? And they're making and they set a budget limit for what they can actually do. And if you were uh have content that's important and you make it easy for that bot to go consume it with a high degree of efficiency.
19:30 Speaker 02 Right? They're they're going to spend more money on you. than on the things where it's harder to go do. So you have to play this, this, this game because you have to work within the constraints that, that these big, you know, marketplaces, search engines, AI search engines are, are playing for us, right? So, and even though Google Bot can, can do a very good job, they're the best at rendering, you know, JavaScript, they would prefer not to because rendering JavaScript is super expensive. Right? So it, and, and it will eat in to part of the budget. So if you're relying on, oh, well, Google can crawl me because they're, they're the most sophisticated.
20:12 Speaker 02 That, that is a false promise because they only have so many dollars to spend and a JavaScript page is going to cost three to five X to, to render than, than a fully rendered, simple page. Uh, for, for the client with all the information that they, they need. So.
20:28 Speaker 01 Yeah. And it's like that, that point is also like, that, like, the amount of cost and energy that's going into that area is so significant. And it's like, naturally there's gonna be trade offs to it.
20:45 Speaker 02 Yep.
20:45 Speaker 01 And I would say especially with like, as the barriers to building have been decreasing, you'd almost expect like the volume and the size and breadth of like, I mean, pages on the internet. So it's like, that's going to become more and more of a challenge, I would assume for all these types of systems. Now I wanna, I wanna double back a little bit on kind of like some of your previous experience. So part of your career, you spent with Microsoft, you worked with them from like the paid search side.
21:23 Speaker 01 I think that's probably pretty interesting with also in this new era because paid search, you have a lot of like structured data elements with product feeds and like, to me, like, that is becoming more and more the competition ground, not just in paid search, but also in this new form of discovery.
21:45 Speaker 02 Absolutely, absolutely. So within a paid search ecosystem, I think there is a lot of things that we thought about very, very clearly for our consumers, for the retailers, trying to get them the best associated like sponsored ad to go associate with their query and how do we go get it right around the ability to go bid efficiently to boost against like individuals that were most important, but also kind of like what was a structured element and the structured data that I would bring in that would be matching criteria to those queries as well as what we get displayed on the page and
22:28 Speaker 02 then what was the right balance between paid and unpaid. But I think that the thing that becomes and the other thing that we had is we would also for every one of our advertisers, we had our own crawler because we would crawl all the advertiser sites to make sure that there is a great connection between the landing page and all that fun stuff. So I think there is a big connection there. The other thing that we spent a lot of time is having the right balance between organic results and paid results and what was the right balance between that, which I think is going to be something that we need to think about more in the world of AI and what you are thinking about and thinking
23:09 Speaker 02 about the contributions of organic traffic and search to the overall. But the product information is, I think, is critical, especially as a structured data associated with the product for making recommendations and making ranking that would be good predictors of a, in the paid search world, a click, right? And a lot of times, we were just optimizing against those clicks based on what observed behaviors we had seen and we weren't taking into account the quality of the product, the fitment of the product to the prompt, other additional data that would be there that would be a better way to recommend a product that may lead to the ultimate goal instead of, instead of the ultimate goal
23:54 Speaker 02 being a click and ultimate goal being the consumer buying the product. Right? Because it was always the that paid mechanism. It's a shortcoming is because it's a step removed. Because Google doesn't care if you convert or not. They they kind of do because they want the return on ad spend. But they get paid on a click, right? So that's what they care about, right?
24:20 Speaker 01 Yeah, which is and it's like an interesting little nuance because it's like, yeah, if then that ultimately like the attribution slightly off. It's like you're going to have a different perspective on like your decision making will be different. And I think you see it sometimes within like organizations where it's like, maybe an engineering team is focused more on kind of like
24:46 Speaker 01 user behavior on it, which is going to be very different. So it's like they're looking more of like what happens when the person is on the site. When if you're, I don't know, SEO marketing, you might be thinking a little bit more of like, well, how do we get more people into the site? And it's like both of them are going to play into the ultimate business goal. But I've seen that like conflict between perspectives that ultimately traces back to that attribution. Like what what are you optimizing for essentially?
25:21 Speaker 03 Time for a one minute break to hear from our sponsor, Previsible. So you're looking for SEO help? And you got a couple of options. You could start replying to spam from agencies that claim they can get you to rank number one on Google. You can pay an hourly rate for a consultant who will inevitably nickel and dime you with hourly charges. Or you can work with a cookie cutter agency to quickly launch a strategy less project with low success rate. None of those sound very good, now do they? Well, that's where Previsible's integrated consulting model comes in. Previsible draws from a collective forty years of SEO and digital marketing experience to unlock your organic growth opportunities. They build custom solutions that combine strategy, technical expertise, content and reporting to effectively operationalize SEO for your business.
26:07 Speaker 03 Previsible's four stage approach ensures that Your SEO programs thrive by starting off with a strategy-first approach. Then they support you in your efforts to create quality content, help you identify technical issues, and most importantly, they'll work with your cross-functional teams to integrate your SEO strategies to make sure that your SEO budget actually drives results, not just your agency's bottom line. So join brands like Yelp, eBay, Canva, Atlassian, Square, all who rely on the SEO consultants at PreVisible. For more information, go to previsible.io. That's previsible, P-R-E-V-I-S-I-B-L-E dot I O.
26:49 Speaker 02 Exactly, and I think that's what led us to, like, we really, we as an industry, the practitioners, as well as kind of like the industry itself, have kind of like laid in, when you think about paid search, really on a session-based conversion, right? So that is a simple conversion. There is no, um, no real look at how much does that organic traffic contribute to a paid search? What other factors happened to a paid search transfer that came over to, that led to a conversion? How do you think about that from a multi-channel or kind of like universal measurement, uh, kind of like standpoint? And I think the, with this transition with AI discoverability
27:32 Speaker 02 and the genetic commerce and everything that's happening now, there's an opportunity to redefine what metrics matter, right? That drive the outcome, which is a, uh, especially in retail or commerce activities at all, is just getting the product sold, right? Having that purchase done? And, and what are the touchpoints that kind of like really, really matter? And how, how do they contribute or, or attribute to kind of like that transaction? Right? So they have a more holistic view because our world, because of everything, looks at, you know, session-based. You know, the paid search people don't really talk to the organic people, maybe a little bit. They don't talk to the, uh, the open web paid media. They don't talk to the paid social, right?
28:16 Speaker 02 The e-mail is set up separate. So the, the, I think the, the opportunity of a lot of touchpoints being redefined by having, um, AI search happening, I think is just very good and very healthy for our industry.
28:32 Speaker 01 Hundred percent. Now, let's, let's go a little bit more into these kind of like agentic feeds and like, as far as like AI product catalog discoverability, like how, how should we be looking at that? How should we be thinking about that? Whether it's from like, preparation or even like emphasis and prioritization on kind of some of our roadmaps?
29:00 Speaker 02 Yeah, absolutely. I, I think that anyone that's a retailer should be leaning in very heavily to go deliver full, accurate, complete product feeds uh to the L L M's, right? Any of them that'll actually take it. And I think they should really double down and relook at what they're feeding into Google Merchant Center, right? Because Google Merchant Center will support Google Shopping, but it's also going to be the place where that feed is going to support A A overviews, right? Um, A mode and Gemini, right? From those components. The thing that's changed in the world of AI search is that they want
29:42 Speaker 02 structured product data to come in that's full and complete, not just to inform a product carousel that shows up there or a product card. They're using it to inform the context in and around that response, right? Very, very important. And, and they're using it to validate and trust based on their crawl data that they're looking at crawling the data in association with what they have in the product feed, right? Now, the, the product feed, like when you look at the agentic commerce protocol from OpenAI, when you look at uh universal commerce protocol from Google, their required fields as well as um um optional fields are a superset.
30:30 Speaker 02 Of what had been historically pushed for Google Shopping or for other paid ads. Because the, because of what I said, they need all that additional information to provide context for a proper answer. The, the product feeds before were really about filling up an ad.
30:50 Speaker 02 Product title, product description, price, reviews, image. They only cared about like five fields within that feed. Now they need the rest of the information. Because they're, these answer engines are actually looking for comparison. Is it the right fitment? Is it, is it the right size? Is it going to fit my use case? Does it have other additional reviews there that I can see? Not just the star rating, but actually the context of the review that fits the question that's actually there. So this, this means it means a retailer, all that rich information that is on my product description page
31:31 Speaker 02 needs to be exposed, needs to be read, needs to be pushed, right? Um, you know, needs to be crawlable and it needs to be pushed to these search engines. Or they're not going to have a good shopping experience. They're not going to show up, right? As, as they should. So this is really important. And the other thing I think, uh, Tyson, that it makes this really hard is it for the last, you know, 20 or 30 years, we've been adding a number of, uh, data elements to product description pages that don't sit in the retailer's product information system, right? Or their information management system around the product. Cause they've augmented with third party data. They have additional first party data that sits in and around it.
32:16 Speaker 02 That's not in the core product information system. So it, it ends up being a problem where they've, they've built this, this castle of a beautiful product description page. There's not one source of it.
32:31 Speaker 02 So it ends up being a huge problem for how do I go and so when the retailers get asked for additional attributes and fields in these new product feeds, they don't know how to source it. And that and they can't source it at scale. That ends up being a challenge for a lot of these retailers. And that's why I think this is a very interesting like time in and around it. And I think what those AI search engines want and what they're asking for today is going to grow exponentially. Because they're going to want more and more information. The when ACP launched from OpenAI, they had 75 fields.
33:12 Speaker 02 They're down to 42 because they knew that the the additional 30 fields or 33 fields were too hard for the retailer to fill out, but they still want them.
33:23 Speaker 01 And what like, can you give us a few examples of like, what are those attributes? Or what are those fields that, you know, almost like we should be expecting and getting ahead of?
33:34 Speaker 02 Yeah, so first, the common one is review content. So they want the review content, not just a star rating. So that is one thing that's actually needs to get pulled in at depth. And they don't just want like, when they go and crawl a site, they usually see the two, three, maybe five that are displayed. They want 10 to 15, right? And it's like, oh, how do I do that? I have to go like that. That ends up being a complicated problem. Q and A is one of the things that comes up time and time again. That retailers just don't have on each product page, right? And and I think a lot of commerce brands are missing
34:17 Speaker 02 questions and answers around their specific products. And this then causes a problem like, shit, I don't have them. Pardon my French. We are a French company. But the how do I go produce them for my 2 million, 5 million, you know, products that I have? I don't have staff or people to go produce that data or that Q and A that's there. The other thing that's really, really interesting, since we've gotten into this, this.
34:49 Speaker 02 Business with our product called Agenna Catalog is it we're seeing a tremendous amount of missing fields, right? So we'll first start with a standard product feed could be coming directly from their system. It could be what they're shipping off to Google Merchant Center today, but we'll see dimensions are missing. We'll see that the category is actually wrong. We'll see that that that additional fields are just not populated because they don't exist in their product information system. It could be the pattern that's associated with it. It could be the material source that's associated with it. So we end up seeing, you know, we'll
35:29 Speaker 02 go and do an analysis for a feed and we'll see nine out of 10 records or products are have at least three to seven errors in them, right? Missing data, incorrect data, data that's inconsistent with what's on the product description page, you know, or fields that, you know, just were never populated.
35:49 Speaker 01 Yeah. And a lot of this, as you're describing it, to me, it's like you almost want to have the same attention or like thought that as SEOs, we used to have more of just like on the page where it's like, as you're describing that I'm thinking like, yeah, I've been in so many conversations where you're talking with retailers about just like product descriptions, the reviews on the page, how you're displaying all this. And it's like you're trying to build a more complete picture. And with how these new systems work, the fees are then basically that same on-page experience. So it's like, yeah, if you are not populating all those
36:34 Speaker 01 things or you don't have the infrastructure in place to be populating them, then it's almost like it comes back to earlier in the conversation where you're only showing maybe 30% of the page or the attributes of the product, which is then going to have all those other snowballing.
36:53 Speaker 01 Impacts that we were previously kind of discussing. So it's like, conceptually as an SEO, it's almost like you have to treat the feeds as your on-page SEO efforts, the same as we've been doing on the actual page.
37:07 Speaker 02 Yeah. And, and, and one of the questions that, that is absolutely right. And one of the questions that, that the, our, our clients ask us is kind of like, well, why is it that you're able to identify all these challenges and, and solve these problems? And it's like, because we crawl 100% of your site at depth and we store that data. So we, as part of our crawling, because we're doing this all the time, we have to have a baseline of what is 100% of your site across your full site maps. We compare and contrast to what Google says in the Google Search Console. We can compare and contrast what we see as being
37:49 Speaker 02 crawled by looking at the logs. We have the full, full view of it. But when we look at their, their product feeds, I can compare it directly to the structured product data that I have from my crawl from your site. And I can tell you what's, what's there and not there. Right? So I can show with you directly. Right? And, and, and that way, I think it's very, very telling to not only the, you know, the, the retailer itself, not only from the, the practitioner, but also from the technologist. I can, I can show you, here are the fields that you don't have, that maybe you need to get your merchant team, your sourcing team to go and ask from this, from whoever's the OEM is, or if it's your product, what needs to be there that would be critical. But I can also tell you
38:32 Speaker 02 just the gaps that your product information system is missing, or whatever sourcing your product feeds is missing today. Cause I see it all on, on your PDP. So I'm, and, and the retailers have spent, right? Decades making those product description pages very robust, right? But we just have to extract that data.
38:56 Speaker 01 Interesting. And I mean, that approach totally makes sense because again, earlier in the conversation, it's almost like the low hanging fruit is like a lot of this information or data is existing. We're just looking then of how are we getting it and actually making sure it's being packaged up and made accessible essentially. So it's like having that comparison of like, we are going to ensure the crawl and then highlight the differences like in as, I don't know, as someone that would be on like the in-house or the e-commerce side. To me, it's like that has to be a little bit of a glass half full because
39:41 Speaker 01 that is a scalable opportunity to have a significant gains rather than it's like, oh, we need to rethink like, and I don't know, creating like product videos. It's like that is a very slow like investment that is going to take a lot longer over time.
40:05 Speaker 01 Now, I could, I could keep going down this rabbit hole with you, but I do want to be mindful of your time here. And I want to hit on a few kind of like lightning round questions for the audience. So I'm going to throw five questions at you, a little more shorter form. And the first question I'd like to ask is, what's the most overrated e-commerce SEO metric today?
40:28 Speaker 02 Yeah, I, I think the most overrated SEO metric for e-commerce today is just tracking organic traffic as a standalone KPI and not comparing it with everything else that is actually happening in other channels. So if I'm an e-commerce writer, if I'm not thinking about my organic traffic from, from search and comparing contrasting what my referral rates and my organic traffic is from the AI sources that are there, I'm missing out. If I'm not looking at my investment pressure that's coming from paid.
41:05 Speaker 02 Or from other channels, I'm missing out. If I'm not thinking about what's happening to my paid and organic Google Shopping compared to my organic traffic, I am, I'm not getting the full picture, right? And I think it's related to where we started. The, the zero click searches, uh, am I getting cited and showing up? Kind of like, what is the intent gap that's kind of like happening?
41:29 Speaker 01 Yeah, and, and, and a lot of ways too, it's like, both in how you described that, but also like in, in the industry as a whole, I feel like we, as S E O's, we do need to have like a wider lens of how we are looking at this because there's also similarities of like, some of our activities are almost like swinging towards like digital P R type activities. So it's like, I do agree that you have to have that broad perspective to then be effective in kind of like this new era. Next question I'd like to ask is what's one A I search trend retailers are overreacting to?
42:09 Speaker 02 Yep, uh, I, I think this really, um,
42:14 Speaker 02 when you look at, at A I search, one of the biggest things people will look at, the first thing they look at is they'll look at their share of voice, which is kind of like how often I'm, I'm mentioned, but the other thing that they overreact to is prompt volume, uh, and how many prompts are actually being executed. Both those metrics are probabilistic metrics. Both of them are synthetic. No one is publishing accurate, transparent data around what is actually being answered by the search engines and, and what is the volume on there? There's panel providers that populate that data on behalf of a lot of the native A I, kind of like visibility tools that are actually out there.
42:58 Speaker 02 But all these panels are small. They may not be directly representative of, of the retailer's consumers because it's all internet like data, you know, picked up from Chrome plugins or extensions or i- i- and I.
43:15 Speaker 02 Really question like, are you really having an appropriate sample? And then they extrapolate these small panels to a large population. And, you know, when we've looked at it, it's like that we have a really struggle. We really struggle to go and represent to the marketer that this data is meaningful and is representative of their shoppers. Right? And, and it's like, and then on top of that, there's no way to connect the share of voice or your prompt volume to business outcomes. Where we go and show them directly, it's like now your citation rate, I can look at your citation rate and I can look at your crawl volume.
43:56 Speaker 02 I can do a correlation. If you increase citation rate, I see a correlated increase in crawl volume, which means that they're doing a retrieval crawl to go and, and get it. I'd say I, I can see that, but your crawl volume is more representative of, of your mentions and your, your impressions than anything else. Right? So that is one of the metrics that we, and, and that our industry is so hot and moving right now that, that people are willing to go and they, and, and accept it without thinking about it.
44:28 Speaker 01 I, I love that you brought up that point. I've been struggling with that too. And it's like,
44:35 Speaker 01 I feel like so much time is probably wasted in that area where it's like, unless you're trying to gain some like competitive understanding, or maybe like managing some stakeholders in your organization. It just tends to be that time vacuum. And there's going to be so much more personalization and variance that it's like the integrity of that too is like, I'm just not fully there on as much as I probably was. Yeah. Five, 10 years ago and probably more like 10, 15 years ago.
45:11 Speaker 01 Next question.
45:14 Speaker 01 Product contents or product feeds, which will matter more in the next two years?
45:20 Speaker 02 Product feeds, no doubt, no doubt. I believe, like we discussed earlier, a lot of retailers have very rich and robust product content today that sits on their PDPs. It's just fragmented across your ecosystem. So consolidating it, making sure it's full, complete and accurate within a product feed is incredibly important right now. One, for the primary use case of like getting and pushing that structured data into the LLM so that the OpenAI and Google across Gemini AI can better represent your products and services. But I think there's additional uses that you could do this. It's like, once you get that data like up and set and ready, maybe that data has to get fed
46:05 Speaker 02 back into your product information systems where the distribution actually serves to get there. It's like, how do you go and get that? I think everyone is trying to figure out how they bring a chat experience or an AI search experience to their site search, right? One of the first things that they ask for is structured product data that is bigger and better than what they have. That is a clear application. And saying, I can go and help you do that and redeploy this is where you you build once, make the investment once and deploy it in many locations to go and add to your thing. But product feed, I think, is what matters next over the next couple of years.
46:44 Speaker 01 Yeah, and it's the key to scalability in the same lens. And I think, like we mentioned earlier, it's almost like, you know, I'd encourage SEOs to think of the feeds as like the new on-page SEO and take the same critical eye to those feeds that we've been applying to when we're actually looking at the visible page.
47:08 Speaker 01 What's one infrastructure investment every retailer should make today?
47:13 Speaker 02 Yeah, I think every retailer should make a investment in their infrastructure about how they're specifically serving non-human traffic. How do I serve the bots? Right now, they they they have a very robust CDN or Content Delivery Network play. They usually kind of like have multiple parties that are actually doing that to make sure that they deliver the best human experience possible. They're not making that investment to do it in the bot traffic. Right? And those bots and non-humans have very different needs. Right? And I and the shift has to be from thinking of non-human traffic is something that I kind of have to do.
47:57 Speaker 02 But it should be something where I have to invest here to have the best shopping experience. I have to service my non-human traffic, those crawlers, very effectively, very efficiently with a high quality. Right? If they think that way, that would be the one thing that they they would need to do. And, you know, it can it can help them. It can help them drive revenue and bottom line and not just be about cost containment.
48:25 Speaker 01 Yeah, and it's the it's the fundamental starting point that we hit on earlier of like, that is the initial like table stakes of getting the seat at the table to then compete in all the other areas. Yet we fixate on those more like visible competitive areas rather than like, well, are we even in the game?
48:50 Speaker 01 Last question I have for you. Five years from now, what will e-commerce teams wish they had started doing earlier?
48:58 Speaker 02 Yep, I think the number one thing that an e-commerce team should be thinking about and measuring today is attribution, right? How do I think about the outcome I'm trying to drive and the contribution of each of my channels to that outcome? So I'm trying to get somebody to buy a product and I should not leverage the past 10 to 20 years of thinking and think everything is based on a isolated session, right? That starts with the referral traffic to that. I think humans are complex and messy people.
49:40 Speaker 02 I do not believe they just go to a single chatbot, make a request and then kind of like follow through and purchase. I think the shopping experience is messy. I think they do some things on their phone. I think they do some things on their desktop. I think they do some things in traditional search, some things in agentic search. And I think they do some things, they're impacted by paid media, whether it's a TV ad, CTV ad, a video ad they saw on their phone or a retargeting ad, right? Or something I see on site. So I think they need to lift their head and think broader, stay anchored on the purchase
50:20 Speaker 02 and look at the contribution or the attribution of all these efforts to that purchase.
50:28 Speaker 01 Yeah, and for me too, like, the way that I keep thinking about this is like, in some ways, I find myself keep coming back to like more traditional like consumer behavior elements. And it's like, that area within our practice to me is like, always going to be the case because it's like, you are still dealing with humans and yes, we're competing in these other arenas and we have to think of bot traffic and all these other areas. But then also it's like, yeah, humans are fickle, like, we're not going to make decisions immediately. Some people are going to stress over them, some people are going to impulsively make them.
51:08 Speaker 01 But there's that human element at the end of the day, that it's like, yeah, we're operating in a new arena, but we are still dealing with humans at the same time. So it's like, those two pieces come together. And I think your point on attribution is... In some ways, it's almost like the product being where it's like you're creating those scalable kind of infrastructure that then you can build and make decisions off from that.
51:38 Speaker 02 Yep, absolutely. Hundred percent.
51:42 Speaker 01 Well, greatly appreciate the conversation. That's going to wrap up this episode of the Voice of Search Podcast. Thanks again to Joe Doran, Chief Product Officer at Botify for joining us. If you'd like to contact Joe, you can find a link to his LinkedIn profile in the show notes, or you can go on over and check out the company's website, botify.com. And if you haven't subscribed yet and you want a daily stream of SEO and content marketing knowledge in your podcast feed, hit that subscribe button in your podcast app or on YouTube and we'll be back in your feed in the following day. That's all for today. Thanks for stopping by and we'll see you in the next episode.
Previous Is the quality of traffic different today for local businesses? Next The most overrated ecommerce SEO metric today Page updated November 14, 2026 · transcript and summary are machine-generated