GITHUB PROJECTS / RECORDED SOURCE METADATA

Projects worth exploring

Discover projects by topic and publisher. Inspect the source, license and activity before using their code or skills.

Clear

henu-wang/ai-crawlers-reference

ai-crawlers-reference

0 catalogued skills

Complete database of AI search engine crawler user-agents (GPTBot, PerplexityBot, ClaudeBot) with robots.txt configuration examples

1HTMLLicense unavailable