Doing a search engine optimization SEO audit is no joke. It used to take time, the patience of a saint and too many spreadsheets. You may grow white hair or two in the process. But, thanks to technical SEO audit tools, we are no longer doing those insane manual checks that we did in the past. Most SEO experts arm themselves with these tools so they’re no longer rummaging through raw data but making strategic decisions instead.
In this article, I’ll share four of my go-to tools for performing a technical SEO audit: Netpeak Spider, DeepCrawl (a cloud-based tool), Screaming Frog (a desktop-based tool) and Search Console (the free web-based tool from Google themselves). They all have their different strengths and use cases. Depending on your requirements, you may need to choose one — or you may find all three useful in conjunction.
1. Netpeak Spider
Netpeak Spider is a desktop tool for day-to-day SEO audit, a quick search for issues, systematic analysis, and website scraping.
It finds broken links and redirects, text images of your website or broken images, escapes duplicate content: Titles, Meta Descriptions, H1 Headers, and presents all the data in graphics and diagrams.
- Checks 50+ key on-page SEO parameters of crawled URLs;
- Consider indexation instructions (Robots.txt, Meta Robots, X-Robots-Tag, Canonical);
- Calculate internal PageRank to improve website linking structure;
- Set custom rules to crawl either the entire website or its certain part;
When analyzing a medium website (10K pages) Netpeak Spider is leading in RAM consumption (the closest rival consumes 2 times more), and it has the same crawling speed as Screaming Frog SEO Spider (in Memory mode).
And Netpeak Spider is completely leading in analyzing large websites (100К pages) with the fastest crawling that consumes the smallest RAM amount. It can also show the data of various SEO factors that might be hurting your rankings.
I really like DeepCrawl, because of its flexibility and the depth of the reports it provides. When starting a crawl, you can choose from numerous crawl types, right up to a full gap analysis of my entire site. You can also auto-schedule crawls, which is really useful. Crawls are highly customizable, so you can set the criteria for maximum / minimum links per page, content, title / URL length, load time, etc.
Here are the three things I like the most about DeepCrawl:
- It can easily handle crawling millions of pages, so if you need to crawl a huge site, look no further.
- It provides competitor reports — not just the basic headlines, but the real nitty-gritty details on content, pricing and promotions, site architecture and even the brand’s key influencers.
- It allows you to project manage your SEO team, creating tickets for issues and assigning them to people. It also alerts you if any issues pop up consistently in crawls. Plus, it maintains a history of all changes, so you can look back and monitor performance and progress over time.
If I had to improve one thing, I’d ask for more mobile-focused reports. (Their support guys told me to expect more of these in their next product update, so looks like that’ll be solved soon anyway).
3. Screaming Frog
When it comes to desktop crawlers, Screaming Frog is an undisputed leader. The tool has been around for quite some time now, and webmasters managing sites of all sizes swear by it. If you are looking at crawling less than 500 URLs, you can even use it for free.
However, if you have a large website with over 10,000 pages, be wary, as desktop crawlers can cause server-response problems. Besides, it doesn’t come with collaboration features, which makes it far less attractive for SEO teams these days.
That said, in this price range, it’s one of the most useful crawlers, and here are the reasons it’s in my top three:
- It doesn’t die off when your memory is running low and alerts you beforehand. I especially like this feature, because this has happened to me a few times. All you need to do is save the project, increase random access memory (RAM) and restart it to continue.
- Their bulk export option really makes life easier, as you can export all data, including internal links, outbound links, anchor text, image alt text, etc.
- There is an option to accept cookies, so you can also crawl websites that make it compulsory to accept cookies.
And, though I like the tool overall, if I had to change one thing about it, I’d want them to improve the user experience to make it easier to customize crawls.
4. Google Search Console
While SEO veterans might find it funny to see this tool in the list, many SEOs are relying on it more than ever. The tool has come a long way since its earlier days and can offer a fair amount of insights. These are the three things I do love about it:
- It gives you estimates on your position for a keyword, plus the number of impressions and clicks for your site on that keyword in Google search results. That may be basic, but it’s important and useful.
- It gives a good summary of things that matter — things like broken links, number of pages indexed, correctness of HTML markup, page loading speed, etc.
- It’s free — and it comes from the horse’s mouth! (OK, that’s two things, but they’re both major plus points.)
The only thing I don’t like about Search Console is that it doesn’t always give a complete picture.
Remember, these tools may not be the best fit for your specific needs. All three have their particular unique selling points and solve specific pain points well. You should review them all and choose the one that’s right for you. The main things to consider are the size of your website, the volume of new pages you generate and the kind of insights you are looking for. Source.