THE GHOSTS IN THE MACHINE: HOW HIDDEN CONTENT BLOAT DEGRADES ECOMMERCE SEARCH PERFORMANCE
- Dan White
- 7 days ago
- 6 min read
Updated: 6 days ago
A lot of my SEO work involves expansion - adding new content, building on what's there - fundamentally growing websites so that they are more visible, more often, so your store can make more money.
In the past 2 years though, time focussed on subtraction has been steadily increasing. Why? The leftovers of all your marketing activities that are still buried in a store's website. The ghosts of your past marketing. But those ghosts can come back to haunt you; affecting how your appearing in both AI search and regular search. Here's how to prune your old content and put those ghosts to bed.
The Ghosts in the Machine
Generally, it's true that bigger websites mean more ranking opportunities. But what's often forgotten is that SEO is a game of quality as well as quantity. Build a bigger website for sure, but each and every URL you add in will help or hinder your SEO performance and the bigger the website, the bigger the potential bloat.
We're talking old seasonal campaigns, discontinued products, job ads, PDF's (there's always PDF's) and orphaned products that all float around your website like ghosts. (When I write ghosts, I think of the ghosts from the BBC's Ghosts - harmless, occasionally helpful, occasionally annoying but a kind of stuck where they are. Not the creepy kind.)
This kind of content bloat isn't anything new. What is new though is that both Google and your LLM of choice seem to be increasingly good at finding this old content and surfacing it. Here's the example that sparked this post...
I was searching for 'exeter airport flight destinations'. The second result 'Where we fly to' felt like the best result as the first result looked like the airport's home page. I click, the PDF loads and oh wait, the PDF is a ghost from 2022. Entirely out of date but happily ranking for a major search term. The result is useless and I go look elsewhere (in this instance, I jumped to a different airport's website).
The Business & Revenue Impact of Ghosts
Let's be clear. When this happens, this has a business impact -
Expired products or out of date discount codes are a direct barrier to purchase. Customers will bounce and find what they want on a competitor's website.
It feeds information hungry LLM's with inaccurate or mixed messaging, resulting in inaccurate or hallucinated answers - reducing trust and increasing confusion from the humans reading
Ghost pages at scale, can collectively drag down the overall quality of the site which can leading to diminished rankings. Poorer rankings mean less traffic and fewer sales. (Source)
How Google & AI Crawlers unearth these ghosts
It's not a surprise that bots will find these ghosts. It's always been their job. What can increase the chances of finding or holding onto ghost content are old backlinks, broken XML sitemaps, internal search parameter leaks, poor page management or a combination of all of these.
What is different though is that LLM's like ChatGPT seem to be more reliant on historic data sets. So an old promo code it found 6 months ago will be presented as fact simply because it hasn't yet found any newer information. So, clearing away old content is increasingly important.
Then there's also an issue of logic. Right now, even with all things AI, no systems seem to be particularly good at catching ghosts which have the right technical signals but are by no means the right result. The top organic result for 'ecommerce conferences in the south west' is Ecommerce south west which will be back, bigger than ever...in 2019.
So, while AI Search continues to evolve at a scary pace, it's still very much our responsibility to keep our websites as ghost free as possible. Here's how I've approached this.
Step by Step Strategic Cleanup
Step 1: Data Collection
What you'll need to do is cross reference your Google Search Console data with CMS exports, backlink data and as much as possible from your GA4 / analytics profile. This will give you as comprehensive record as you can get for every URLs that's ever existed.
Step 2: Pattern recognition
The next step is to spot patterns or look for common areas where ghost URLs can exist. Starting points for this include:
Identifying orphan URLs
Manually trawling the CMS 'page' and 'post' sections
Reviewing the least well performing indexed URLs in Google Search Console
Working through the Page Indexing error reports in Google Search Console
Using Screaming Frog's visualisations to spot pockets of content you don't recognise
You might want to look for URLs which have:
Random status numbers
Random URL constructions
Random canonical or redirect URL destinations
The fundamental issue with this kind of challenge is that the data will only tell you so much. Like with the ecommerce south west conference, technically, that website looks fine. But from a user's perspective it's in a desperate need of an overhaul, so you will need to manually review a lot of pages. As a result this isn't quite the same as a content audit which traditionally cross-references performance data with a specific website section (e.g. a blog) but a much wider project.
Doing this can sometimes turn up single URLs which are unique on the site. Covid 19 pandemic response pages are a good example of this. Other's will form a cluster of out of date material that lives in the same directory. A great example of this was when I discovered 25 PDF catalogues for an ecommerce client dating back over 15 years all stored in a random ophaned part of the site. Or you can make much larger discoveries.
Is doing this boring? Hell yes. Is it also valuable? Hell yes once again. Looking at these patterns gives you a fantastic insight into how your store functions and more often than not will help flag other issues which will affect SEO. So, congratulations, you're about to make your to-do list even longer.
Step 3: Actioning affected URLs
We have 3 options for what to do with pages which no longer serve us:
Option 1: Update / refresh the content. Could a time sensitive piece be made evergreen? Could a archived product be brought back into circulation?
Option 2: Consolidation. If there are duplicates across the site, or if the URL isn't worth keeping then we can look to 301 / 307 redirect the URL to something more valuable. That way we can funnel any authority these pages carry to things that matter.
Option 3: Remove them. Depending on the page in question we can set and leave them as a 404. Or list them as a 410 so Google knows they're officially gone.
Step 4: Preventative Measures
There's always going to be processes to put into place which mitigate content bloat in the future but what these are and how often they're applied depends so much on the size and scale of your store. If you've never looked at what's sitting on your website then an SEO Audit could be a good place to start. Broadly speaking when good SEO is being implemented and regularly maintained, you'd expect to see fewer of these kinds of issues.
Saying Goodbye to your Ghosts
Cleaning up content like this is a technical chore, but but it's also a high ROI revenue activity. Removing the ghosts of pages past clears the way for Google's crawl budget to go further on the rest of your site. It aides overall site quality. It limits cannibalisation. It also means fewer factual inaccuracies with AI Engines. All in all, it presents your store in the best possible light.
If you want someone to lead on SEO projects such as this for your store, then contact me for an honest chat about how I can best help.