File tree Expand file tree Collapse file tree 2 files changed +38
-5
lines changed Expand file tree Collapse file tree 2 files changed +38
-5
lines changed Original file line number Diff line number Diff line change 1- User-agent: *
1+ User-agent: GPTBot
2+ User-agent: ChatGPT-User
3+ User-agent: ChatGPT-User /2.0
4+ User-agent: anthropic-ai
5+ User-agent: ClaudeBot
6+ User-agent: claude-web
7+ User-agent: Google-Extended
8+ User-agent: Amazonbot
9+ User-agent: CCBot
10+ User-agent: cohere-ai
11+ User-agent: cohere-training-data-crawler
12+ User-agent: PerplexityBot
13+ User-agent: Perplexity-User
14+ User-agent: MistralAI-User
15+ User-agent: AI 2 Bot
16+ User-agent: Diffbot
17+ Disallow: /
18+
19+ User-agent: OAI-SearchBot
20+ Disallow: /
21+
22+ User-agent: Applebot-Extended
23+ Disallow: /
24+
25+ User-agent: Bytespider
226Disallow: /
327
4- User-agent: Googlebot
5- Allow : /
28+ User-agent: DuckAssistBot
29+ Disallow : /
630
7- User-agent: Bingbot
8- Allow: /
31+ User-agent: YouBot
32+ Disallow: /
33+
34+ User-agent: TimpiBot
35+ Disallow: /
36+
37+ User-agent: DataForSeoBot
38+ Disallow: /
39+
40+ User-agent: *
41+ Allow: /
You can’t perform that action at this time.
0 commit comments