How to Verify Your Backlinks with Screaming Frog SEO Spider

Your backlink data from Ahrefs, Semrush, or Majestic lags behind reality. A site removes a link, deletes a page, or goes dark entirely, and it can take those indexes weeks or months to notice. If you need to know which of your backlinks are actually live right now, the fastest way is to crawl them yourself.

Screaming Frog's custom extraction feature does exactly this: point it at your list of backlink URLs, and it tells you whether each page still links to you, what the anchor text says, and whether the link carries a nofollow attribute. Here's the full setup, what the results actually mean, and what to do with them.

What this checks, and what it doesn't

This confirms whether a backlink still exists on the page it's supposed to be on, right now, and whether it's set to nofollow via the link attribute. It does not tell you the link's authority, the referring domain's traffic, or anything about link value, that's what your backlink index (Ahrefs, Semrush, Majestic) is for. Screaming Frog is a crawler, not a backlink database. Use the index to find your links, and use this to confirm they're actually still there.

Setup, step by step

1. Export your backlink list. Pull the referring page URLs from Ahrefs, Semrush, or Google Search Console into a plain list, one URL per line.
2. Check your license. Custom extraction is a paid-license feature. The free version caps crawls at 500 URLs and disables custom extraction entirely, so this only works with a licensed copy (£199/year as of 2026).
3. Switch to List mode. Open Screaming Frog and select List mode instead of the default Spider mode. List mode crawls only the URLs you give it, not everything they link to.
4. Set up custom extraction. Go to Configuration > Custom > Extraction and add three XPath extractors. Replace example.com with your own domain in all three, this is the step people skip and then wonder why every result comes back empty:

//a[contains(@href, 'yourdomain.com')]/@href
//a[contains(@href, 'yourdomain.com')]
//a[contains(@href, 'yourdomain.com')]/@rel

• //a[contains(@href, 'yourdomain.com')]/@href → name it Link
• //a[contains(@href, 'yourdomain.com')] → name it Anchor
• //a[contains(@href, 'yourdomain.com')]/@rel → name it Link rel attribute

1. Paste your URL list and crawl. Upload the list from step 1 and start the crawl. You'll see pages processing in real time in the Internal tab.
2. Read the results. Switch to the Custom tab and the Extraction filter. Each row is a page from your list, with the extracted link, anchor text, and rel attribute if the link was found.

What an empty result actually means

A blank row in the Extraction tab isn't automatically “link removed.” Check the status code in the Internal tab first, since a few different things produce the same blank result:

• 404 or 5xx: the page itself is gone or broken. The link died with the page.
• Blocked by robots.txt: Screaming Frog couldn't crawl the page at all, so it can't confirm either way. This shows up as blocked, not missing.
• 200 but genuinely blank: the page loaded fine and the link to you is actually gone. This is the case worth acting on.

One gap in this method worth knowing: the rel-attribute extractor catches nofollow set directly on the link tag, but it won't catch nofollow applied sitewide through a meta robots tag or an X-Robots-Tag header. Check those separately in the Internal tab if a link's value matters enough to verify properly, I cover both in X-Robots-Tag.

A real number, and what to do with it

Running this on my own backlink list, 270 out of 280 came back active. The 10 that didn't split into two buckets: pages that 404'd entirely, and pages that loaded fine but no longer contained the link. That second bucket is the one worth spending time on. Not every dead link is worth chasing, if the referring page had little authority or traffic to begin with, let it go. But for links from sites that were actually driving value, a short follow-up email asking whether the link can be restored costs a few minutes and occasionally works.

Frequently asked questions

Do I need a paid Screaming Frog license for this?

Yes. Custom extraction is disabled in the free version, along with JavaScript rendering and crawls over 500 URLs. The paid license is £199/year as of 2026.

Does this replace Ahrefs or Semrush for backlink tracking?

No. Screaming Frog doesn't discover backlinks or score their value, it only verifies links you already know about. Use a backlink index to find and evaluate links, then use this to confirm they're still live.

Why did every result come back empty?

The most common cause is forgetting to swap the placeholder domain in the XPath expressions for your actual domain. Double check all three extractors reference your real domain before running the crawl.

Can this detect a nofollow link?

It detects nofollow, sponsored, or ugc set directly on the link's rel attribute. It does not detect a sitewide nofollow applied via a meta robots tag or an X-Robots-Tag header, check the Internal tab separately for those.

Where to go next

Backlink reports from any index are only as good as their last refresh. Running your own list through custom extraction takes a few minutes and gives you a live answer instead of a cached one. Do it quarterly on your important links, check the status code before assuming a blank result means the link is gone, and spend your outreach time on the links that were actually worth having in the first place.