68% of online experiences begin with a search engine, while the first three listings to appear in a Google search receive roughly 50-60% of clicks. For this reason, brands need to get search engine optimisation (SEO) right to stand out online. Tools like Screaming Frog have become vital to achieving that.
Centred around key SEO considerations like page load speeds and website security, Screaming Frog SEO Spider is a comprehensive website crawling tool. It can identify over 300 different SEO issues, leading to actionable improvements that can transform the search standing of your entire site.
In this article, we’ll consider top tips for seamlessly finding your most pressing technical issues with the help of Screaming Frog.
What is Screaming Frog?
Screaming Frog is an SEO-based website crawling tool that views your website in much the same way as Google does. Screaming Frog will crawl things like a page’s indexability, size, meta descriptions, and even text ratios. In doing this, it can identify over 300 different performance issues. Regardless of website size, Screaming Frog can crawl your main pages and subcategories with real-time results and analysis.
Screaming Frog is available in both a free downloadable version that can crawl 500 URLs and a paid licence. This paid version unlocks unlimited crawling, as well as benefits such as crawl comparisons, scheduled crawls, and Google Analytics integration.
Issues You Can Identify With Screaming Frog
Some of the most damaging SEO problems identified by Screaming Frog include –
- Broken links
- Audit redirects
- Duplicate content
- Missing or ineffective page titles and meta descriptions
- And more
Screaming Frog is a tool that can feel overwhelming at some times; that’s why it is important to dedicate some time to learning how to use it properly and find the most relevant issues of your website. But time is something that we sometimes lack, that’s why you can always rely on an SEO marketing agency to help you with your SEO technical audits and your strategy.
How to Crawl Your Website Using Screaming Frog
Screaming Frog is available in an easily downloadable format for Windows, Mac OS, Ubuntu, and Fedora. Once you have Screaming Frog on your desktop, you should crawl your main page domain, and website subcategories to identify your most pressing technical issues. You can do that in the following ways –
- Crawling a Subdomain: Begin crawling your subdomain by typing it into the ‘enter URL’ box. When you do this, Screaming Frog will treat all other pages as external links by default, unless you select the ‘all subdomains’ filter.
- Crawling a subfolder: You can crawl a subfolder by entering it and then selecting ‘subfolder’ from the top switcher.
- Crawling a list of URLs: Crawl a list of URLs for purposes such as site migrations and URL auditing by selecting ‘List’ from the mode menu, and either pasting or uploading your URL list.
How to Find Your Most Pressing Technical Issues With Screaming Frog
Whether you have a large website or a relatively small page, there’s a lot of information to crawl through. Make sure Screaming Frog can identify the most pressing technical issues using the following tips –
# 1 – Limit Your Crawls
Large-scale URL crawls make it far harder to identify fixable individual issues. As such, aim to limit your crawls to smaller sections of web content that you can analyse more thoroughly. There are a few different ways to do this, including eliminating certain types of URLs in the configuration menu, running separate, segmented crawls using keywords with the ‘include’ feature, or even segmenting your crawl using the ‘segments’ column.
# 2 – Extract Content Snippets
Sometimes, certain snippets of content, such as meta descriptions or weak product descriptions, will let your SEO down. Screaming Frog includes a ‘custom extraction’ tool that makes it possible to identify problematic snippets on even high-performing pages. Rather than requiring complete content rewrites, this tool makes it easier to see precisely where optimisations need to happen, and what impact even small changes have overall.
# 3 – Audit Inactive Pages
There’s an obvious draw to focus on your active pages when using Screaming Frog. Yet, inactive pages like broken links of 4xx pages can significantly harm SEO scores. By auditing your inactive pages using Screaming Frog’s extraction and custom search tools, you can reduce the prevalence of 4xx redirects, remove broken links, and more effectively identify issues like out-of-stock products, lacking product reviews, and so on.
