Broken Links

How to analyse and manage corrupt links to improve UX and not waste the Crawling Budget.

Broken Links Analysis

One of the analyses that never fails in my Seo Audits is regarding broken links that generate 404 errors and inexorably waste crawling budget. Finding this criticality with Screaming Frog is very simple and requires only a few steps:

  1. Start scanning your website.
  2. Select the “Response Codes” tab.
  3. Set the filter “Client Errors (4XX)” to display broken links.

Alternatively, you can select the “Clients Errors” item from the sidebar.

In the example, 9 errors with status code 404 were found, corresponding to 0.18% of the links discovered in the crawl.

4. View the source of the broken links (Broken Links) by clicking on the ‘Inlinks tab; by selecting a URL in the upper window you can populate the “Inlinks” tab.

As you can see in the next example, there is a broken link to the BrightonSEO site (https://www.brightonseo.com/people/oliver-brett/), which has as its source: https://www.screamingfrog.co.uk/2018-a-year-in-review/.

Here is a close-up view of the bottom pane of the window detailing the ‘inlinks’ data

The “From” column identifies the source where the 404-generating link is, while the “To” column displays the broken link (destination). In the following columns you can also see the anchor text, the alt text (if it is a hyperlinked image), and whether the link has a “follow” (true) or “nofollow” (false) attribute.

  • 5. With the data collected, you can export the results.

Bulk Export > Response Code > Client Errors (4XX)

Export Client Errors

With this process you are able to cover most cases where there are broken links but it may happen that the “source” URL that is the last page to link to the 404 error page is a redirect ( or a node in a redirect chain) and should be handled as such. To find out if the source is a redirect you can look at the ‘Type-Type’ column and check if identified as ‘HTTP Redirect’.

To quickly find the source page of these errors, I recommend using the “All Redirects” report.

Reports > Redirects > All Redirects

This way you can isolate the source page, redirects, and the last source generating 4xx errors for proper link optimization.

Finally, if you have a list of URLs on which you want to check for broken links instead of scanning a website, you can simply upload them in list mode.

Related Tab: Response Code | Sidebar | Report

Broken Links Video Analysis

Seo Spider Tab