We’re delighted to announce Screaming Frog SEO Spider version 24.0, codenamed internally as ‘bolus’.
In this release we have introduced a number of much demanded new features, as well as smaller quality of life improvements and updates.
Let’s take a look at what’s new.
1) Screaming Frog SEO Spider MCP
You can now run crawls, analyse, export and manipulate data using the SEO Spider and node.js within Claude, LM Studio and other AI chat assistants.
The MCP allows you to connect to the SEO Spider and automate countless crawl related activities using natural language. From the basic, such as running and summarising a crawl.
To more fun, and advanced tasks.
Such as manipulating or visualising the data the way you want to view it.
As well as performing the leg work for exporting, combining data, running the analysis pipeline and installing relevant scripts for interactive visuals for greater insight.
Like this pretty link equity analysis we made earlier using the Ahrefs integration in a crawl.
Integrating with Claude Cowork, also opens up lots of possibilities.
There are plenty of use cases, ultimately the idea of the integration is to make daily tasks and workflows that much more efficient. This is not a replacement for an experienced SEO professional, obviously.
To set this up, please refer to our documentation on the MCP Server.
Please note – Not every feature and function is currently supported in the SEO Spider MCP. However, we plan to extend functionality based upon user feedback.
So let us know what else you’d like to see!
2) Auto Compare Crawls
You’re now able to select ‘Auto Compare Crawls’ for scheduled (and CLI) crawls.
This can be set up in the scheduled crawl task, which will then auto compare the last two crawls performed in a project.
This allows you to quickly view a crawl comparison of the last two crawls, including all changes that have occurred since the last crawl.
Read more in our crawl comparison tutorial.
3) View Crawl Changes in Email Notifications
If you have enabled crawl completion notifications, you will already be familiar with the email summary upon crawl completion.
This has been improved to include all issues found within the crawl in a table below the summary.
If you have selected to ‘Auto Compare Crawls’, you will see a crawl comparison of issues in the email – highlighting any changes in issues found between the last two crawls.
This is a great way to monitor and be alerted to website changes and SEO issues – such as pages being added or removed, a sudden spike in non-indexable pages, or page titles disappearing.
All without having to open up the crawl.
4) Send Crawl Export Attachments by Email
You’re now also able to send emails with export attachments automatically upon crawl completion.
This can be set up in the scheduled crawl task under ‘Notifications‘.
For example, if you need to send a weekly broken links export to a developer, or a crawl issues summary to a stakeholder, you can input their email and the report of choice and it will appear in their inbox.
Attachments are zipped and the email will warn the receiver if there is an error, such as attachments being too large in size for the email provider.
5) Find Uncrawlable Links
You can now find common uncrawlable link types in the SEO Spider.
These can be viewed in the Links tab and the new ‘Pages With Uncrawlable Internal Outlinks‘ filter.
There is a new ‘Link Crawlability’ column in the lower Outlinks tab, which helps to identify any of these link types as ‘Uncrawlable’. They can be exported in bulk via the ‘Bulk Export > Links > Pages With Uncrawlable Internal Outlinks’ export.
Uncrawlable links are links found in the HTML, but do not conform to link best practices outlined by Google. For example:
<span href="https://example.com">
<div href="https://example.com">
<a onclick="goto('https://example.com')">
<a href="javascript:goTo('products')">
To find uncrawlable link types, please ensure ‘Store’ is enabled in the ‘Config > Spider > Crawl’ menu.
There isn’t an option to crawl these currently, but this might be available in the future as we refine how we parse some of these link types.
Please note – Our research indicates Google will crawl anything in the HTML that looks like a link. They will attempt to parse it and use for discovery. So strictly speaking, many of these link types will likely be ‘crawlable’.
However they are not recommended, and should not be relied upon. They might not be treated in the same way as a recommended hyperlink for passing link signals.
6) Usage Stats
If you’ve ever wondered how much time you spend crawling, then you can now see your own usage data via ‘Help > Usage Stats’.
This has limited practical use (!), but is fun and might give you a better idea of how much value you’re getting from the tool each year. This data is stored locally on your machine.
Other Updates
Version 24.0 also includes a number of smaller updates and bug fixes.
That’s everything for version 24.0!
Thanks to everyone for their continued support, feature requests and feedback. Please let us know if you experience any issues with this latest update via our support.
Small Update – Version 24.1 Released 8th June 2026
We have just released a small update to version 24.1 of the SEO Spider. This release is mainly bug fixes and small improvements –
