henu-wang/ai-crawlers-reference
ai-crawlers-reference
0 catalogued skills
Complete database of AI search engine crawler user-agents (GPTBot, PerplexityBot, ClaudeBot) with robots.txt configuration examples
1HTMLLicense unavailable
GITHUB PROJECTS / RECORDED SOURCE METADATA
Discover projects by topic and publisher. Inspect the source, license and activity before using their code or skills.
henu-wang/ai-crawlers-reference
0 catalogued skills
Complete database of AI search engine crawler user-agents (GPTBot, PerplexityBot, ClaudeBot) with robots.txt configuration examples
intikhab49/whofetched
0 catalogued skills
Prove which AI fetched your page: secret canary links, a Cloudflare Worker log and published-IP checks. Tested on ChatGPT, Claude, Gemini, Grok, Perplexity and Copilot (Oct 2026).