In this post, I’m going to take you through my own process of identifying and fixing redirect chains and chains and chains and chains.
Redirect chains are a well-documented area of SEO that in most cases fall into a nice category of being a relatively low-hanging fruit opportunity that doesn’t require huge swathes of technical SEO nous to identify and audit.
That being said, when looking at what is now quite a mature (albeit ever-evolving) industry in terms of knowledge and tools, there is the inevitable by-product of information overload and varying tool accuracy that can muddy the process of this well-traversed SEO exercise.
To hopefully cut through some of this slack, I’d like to take you through my step by step process of redirect chain auditing and the tools and resources I use. This may be nothing new for most seasoned SEOs, though I hope there are some learnings for most of you; you lovely grateful lot.
For the purposes of this blog post, we’re going to assume the context of redirect chain auditing being part of a technical SEO audit on a new client, so things are nice and fresh.
For simplicity, (and potentially with the thrifty in mind) I’m also just going to use as few SEO tools as possible for this exercise (see previous point about tool overload). I know the likes of Sitebulb and Ahrefs have fantastically competent redirect chain auditing features within their products, though I just want to keep this as streamlined as possible. Hopefully you’ll see why.
Now of course, redirect chain auditing is something you should do regularly as part of your technical SEO ventures particularly if you’re working on large ecommerce sites that undergo migrations or CMS changes.
At any rate, I love getting stuck down the rabbit hole with these nice and early on in a project especially given the potential rewards in site speed and crawl budget optimisation if fixed sufficiently. In addition, it can be a super quick exercise to do.
Step 1: Gathering Data Part 1
Crawl your website with Screaming Frog (duh). The default setup of Screaming Frog’s spider configuration is fine if we want to jump in and get a good overarching view of the situation across the site.
However, if we’re working on a large site that we know has different product teams working on different product areas, we may want to compartmentalise our redirect chain auditing by subfolders.
If we’re crawling a subfolder on our site, say mysite.com/shoes/, the default setup of the tool will only crawl his subfolder. However redirect chains to any given URL can get messy and come from all areas of your site, so you may want to consider the “Crawl outside of start folder” configuration. This will begin your crawl from your selected subfolder and will pull in URLs across the site that link to URLs in this subfolder that may eventually show up in our redirect chain report which I’ll get to shortly.
For this, within the tool we need to go to Configuration>Spider>Crawl>Crawl Outside of Start Folder and check the box.
Redirect chain auditing of complex sites may also require us to pull in data from related subdomains to complete our report. For that, we’ll need to head to Configuration>Spider>Crawl>Crawl All Subdomains and check the box.
One final thing I’d recommend is also telling Screaming Frog to crawl links with nofollow attributes in them. Old legacy URLs like often rear their heads and contribute to redirect chain issues, so we want to make sure we’re picking up on these as well. Head to Configuration>Spider>Crawl and check the Follow Internal “nofollow” and Follow External “nofollow” boxes for this.
There’s probably a bunch of other Screaming Frog configurations you should be doing prior to crawls that identify redirect chains, though I won’t list them here. Go here instead. For more details on auditing redirects in general using the Screaming Frog tool, check this piece that the guys themselves have written.
Once your crawl has completed, it’s a simple case of then heading to Reports>Redirects>Redirect Chains and save the file (Screaming Frog also compiles canonical chain reports which are handy in their own right though I won’t address that here).
Step 2: Gathering Data Part 2
Once you’re done with Screaming Frog, it’s next time to head to a companion tool for this. As I’ve referred to above, there are many out there that have their own ways of flagging redirect chains. For this one, I’m going to use SEMRush’s Site Audit feature, which is found in the SEO Toolkit section of their dashboard.
Assuming you have a domain set up in the “Projects” section and have already run crawls (if you haven’t and don’t know how, read here), then head on over to the “Site Audit” section which can be found under the “On Page & Tech SEO” submenu.
Once you’re here, you’ll be presented with an all-singing, all-dancing technical SEO site health dashboard. To get what we want, click on the “Issues” tab and, depending on the level of severity of it on your site according to SEMRush, you should be able to locate “redirect chains and loops” somewhere in his list.
Click through to this and then export this list.
Note: depending on your level of subscription with SEMRush, you may not be able to crawl the same number of URLs as you could with Screaming Frog. Consider tweaking crawling limits with the latter so you end up with roughly the same URLs for both. This won’t of course mean that the two respective tools crawl the same areas of the site however, though it at least allows for that similar amount of secondary layers of data that we’re after.
Step 3: Combining Your Data
Now comes the fun stuff – Excel consolidation.
Screaming Frog
We’ll start off with the Screaming Frog export first. The raw export of SF’s redirect chain report, while thorough, contains a few superfluous data columns that we can happily chop out to get the export into an easily-digestible deliverable. We want to end with an Excel file that has as few columns as possible and simply shows the source URL at the beginning of the chain, the final destination URL, and the URLs in between that form the chain that we need to remove.
Firstly, and of course there are other ways around this, but we want to ensure that we’re using separate tabs for the different “Number of Redirects” we may have in any given audit. Create copies of your full export across different tabs, and then apply some filtering on the “Number of Redirects” column that one tab addresses issues where you have 2 hops in a chain, one accounts for 3 hops, one for 4, and so on.
Returning back to our first tab, firstly remove (or hide) the columns “Chain Type” and “Loop”.
Then, filter the column “Temp Redirect in Chain” so it displays only “false” values. Temporary redirects should be segmented out from this cleaning up exercise given they’re a different animal that are complicated by context and should be treated on a more granular, case by case basis.
Following this, filter the “Final Indexability” column so the values only read “Indexable”.
Then, filter out the “Final Content” column so the values display “text/html”. Ensure the “Final Status Code” column is showing only “200”.
Now, to clean up the appearance of the worksheet further, go ahead and hide the columns you’ve just applied filters to.
We are now in a safe enough place to return to my favourite exercise of brutally removing Excel columns without the end result being a complete catastrophe.
We can remove the column entitled “Final Address” as this is repeated in a column further down the line. “Final Status Code”? yeah, get rid of that too. And all the individual “Content”, “Status Code”, “Status” and “Redirect Type” columns. Our previous filtering should have refined our current list to URLs that are indexable and fall under the category of being a 301, so these should be good to go.
What we should have now is a nice refined worksheet which features just the columns “Source” (aka where the redirect chain link begins), the link in this source where the chain starts (“Address”) the redirects in between and the final destination, which will be sequentially named “Redirect URL 1”, “Redirect URL 2” and so on.
Repeat this entire process across your individually-arranged “Number of Redirects” tabs and you should be getting to a pretty good place.
SEMRush
Moving onto our SEMRush export, begin with the same process of creating tab copies for different numbers of redirects (named “Length” in their report).
Remove (or hide) the columns entitled “Redirect Type” and the final “Discovered” column, despite how interesting timestamps can be. The SEMRush redirect chain report template also seems to assume you’ve got chains of up to six in each report, which in my case has left me with a bunch of empty columns, so remove these as well.
Now, filter out all the “Status Code” columns so they’re only displaying 301 URLs, with the final “Status Code” column in the report showing only 200 in their values. Hide these columns. You should then have something that looks like this:
Consolidation
The next step shouldn’t be too tricky.
Starting with our Screaming Frog report, copy the cells in the “Source” column into the “Page URL with Redirect Link” column in the SEMRush doc, working across each of the respective redirect chain count tabs in both documents.
Frog’s “Address” values go into SEMRush’s “Initial Redirect URL” tab, while “Redirect URL 1” and “Redirect URL 2” etc go into “URL 1” and “URL 2” etc in the SEMRush doc. Finally, ensure that the last “Redirect URL” numbered column in the Screaming Frog doc go into SEMRush’s “Final Destination URL” where we are consolidating all of our URLs at the end of the chain. Rename these columns prior to doing this if it’s easier.
Now it’s just a just of filing down the duplicate source links we may have inherited from either of these tools’ reports.
In Excel, go to Data>Remove Duplicates and then ensure you’re only addressing duplicate source URL values in the “Page URL with Redirect Link” column.
Here’s another bland Excel screenshot to break up the blog post:
Step 4: Sense Checking
While we’re keeping third party SEO tools at a minimum here, I like to bring in a couple of old favourites for the sense check phase. These include httpstatus.io and the Redirect Path browser plugin by Ayima.
These tools don’t have the capability to audit thousands of URLs or entire domains like Screaming Frog or SEMRush do, but I like to use them as a QA tool while I audit redirect chains to flag any potential anomalies. This is probably recommended if we’re bringing in data from different third party tools.
httpstatus.io will allow you to check up to 100 URLs at a time, which is a decent enough sample if we’re grabbing a batch from our finalised list in Excel. Line up the export from your chosen sample inputted into httpstatus.io and see how the chain compares with your consolidated list.
If you’re not satisfied with this, then use the Redirect Path browser plugin to manually check the URL status live on any given page.
Ultimately this all about data validation at this stage, as ideally want to avoid mistakenly removing URLs from a chain and not break our site entirely in the process.
One final step we should incorporate here is using Screaming Frog to crawl our final list one more time validate. You may have your preferred SEO tools for the sense checking phase (or auditing phase for that matter) though I always return to the old favourite.
You’ll want to crawl the URLs in the “Initial Redirect URL” column here using the tool’s List Mode. Check the Response Code tab here for details on your redirect chains, all of which are readily exportable.
Step 5: Off Into the Ether
Once you’re ready with your deliverable, get ready to send it off to your relevant development or product teams. I’ll leave the level of required explanation and justification to your own discretion based on your ability to read the room as a fully grown adult. Well done.
Nota bene: As you’ll probably discover by now, a lot of redirect chains can be fixed by simply updating the source links to the final destination shown in the chain and then removing the hops in between from the server if this is safe and feasible. Still, I retain the opinion that is a quick and easy technical SEO exercise that can return notable gains.