Version 20.0.1 Windows • macOS • Linux 500+ features

Screaming Frog SEO Spider

The gold standard for technical SEO auditing. Full control over your site structure, link profile, and search visibility.

Download for free

Free version: up to 500 URLs • Premium: unlimited

Informational Resource

This is an independent informational website about Screaming Frog SEO Spider. We are not affiliated with, endorsed by, or officially connected to Screaming Frog Ltd. All product names, logos, and brands are property of their respective owners. All download links lead to the official website.

Technical Site Audit

Error Detection

  • 404, 500 and all HTTP status codes
  • Broken links (internal/external)
  • SSL/TLS certificate errors
  • Robots.txt & .htaccess issues

Meta Data

  • Duplicate titles & meta descriptions
  • Meta tag length (SEO optimum)
  • Missing H1/H2 headers
  • Image alt texts

Performance

  • Page load time analysis
  • HTML and resource size
  • Unoptimized images
  • PageSpeed recommendations

JavaScript SEO

JS Rendering

  • Rendered HTML preview
  • JS-loaded content detection
  • Googlebot visibility check

Bot Emulation

  • Googlebot Desktop
  • Googlebot Smartphone
  • Bingbot, Yandex and more

Structured Data

  • Schema.org validation
  • JSON-LD, Microdata, RDFa
  • Structured data export

Log File Analyser

Analyse server logs to understand how search bots interact with your site. Optimize your crawl budget.

Bot Analysis

  • Googlebot (Desktop/Mobile)
  • Bingbot, Yandex, Baidu
  • Social media bots (Facebook, Twitter)
  • Custom user agents

Crawl Budget

  • Crawl frequency analysis
  • Daily/weekly trends
  • Time spent per page
  • Blocked resources

Technical Analysis

  • HTTP status codes in logs
  • Redirects in logs
  • Most popular URLs
  • 5xx server errors
1M+
log lines/minute
50+
bot types
CLF
W3C • NCSA • IIS

📊 Analysis Example:

Googlebot
47% of traffic
/blog/*
most popular
3.2 sec
average time

Sitemap & robots.txt

Sitemap Generation

  • XML, HTML, TXT sitemaps
  • Content type segmentation
  • Existing sitemap validation
  • Priorities and frequencies

robots.txt Analysis

  • Directive validation
  • Blocked important pages
  • Sitemap directives
  • Syntax error detection

Rendering

Visualization

  • Force-directed graph
  • Link heat maps
  • Tree structure visualization
  • 3D visualization

Page Preview

  • Page source code
  • Visual page rendering
  • Rendered vs raw comparison

Export Options

  • Excel, CSV, SQL
  • Google Sheets integration
  • API access

Advanced Features

Integrations

  • Google Analytics 4
  • Google Search Console
  • PageSpeed Insights API
  • Ahrefs, Majestic, Moz

Custom Scripts

  • JavaScript extraction
  • XPath & CSS selectors
  • Regular expressions
  • Custom filters

Bulk Operations

  • Search & replace in URLs
  • Crawl comparison
  • Task automation
  • Scheduled crawling

Ready for professional audit?

Download Screaming Frog

Free version for sites up to 500 pages