Free SEO crawler, website audit and AI crawler checker. Self-hosted and open source: crawls a site for technical SEO issues and checks that Googlebot, Bingbot and AI crawlers (GPTBot, PerplexityBot, OAI-SearchBot) get the same readable page. CMS-aware, XLSX export, free Screaming Frog alternative.
Audit any website for LLM discoverability and agent readiness. Checks AI crawler access, llms.txt, JSON-LD, MCP/A2A discovery, scrapability, and the SEO fundamentals AI retrieval still depends on. Local, free, no API keys.
robots.txt for AI crawlers: copy-paste templates that allow AI search bots (OAI-SearchBot, Claude-SearchBot, PerplexityBot) and let you block AI training (GPTBot, ClaudeBot, Google-Extended), a verified AI crawler list, an llms.txt template and access tests. For generative engine optimization (GEO) and AI SEO.
Free, open-source AI visibility and SEO audit. Crawl your whole site with 643 checks for technical SEO, AEO/GEO, 40 AI crawlers and agentic-web readiness (llms.txt, MCP, A2A), then track whether ChatGPT, Claude, Gemini and Perplexity name you. Fixes for your stack. CLI, dashboard and MCP server; runs locally.