Skip to content

Commit 2f3ae43

Browse files
author
rhiannon
committed
revamp
1 parent f03fbdf commit 2f3ae43

File tree

2 files changed

+38
-5
lines changed

2 files changed

+38
-5
lines changed

images/emergency.png

48.1 KB
Loading

robots.txt

Lines changed: 38 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,41 @@
1-
User-agent: *
1+
User-agent: GPTBot
2+
User-agent: ChatGPT-User
3+
User-agent: ChatGPT-User/2.0
4+
User-agent: anthropic-ai
5+
User-agent: ClaudeBot
6+
User-agent: claude-web
7+
User-agent: Google-Extended
8+
User-agent: Amazonbot
9+
User-agent: CCBot
10+
User-agent: cohere-ai
11+
User-agent: cohere-training-data-crawler
12+
User-agent: PerplexityBot
13+
User-agent: Perplexity-User
14+
User-agent: MistralAI-User
15+
User-agent: AI2Bot
16+
User-agent: Diffbot
17+
Disallow: /
18+
19+
User-agent: OAI-SearchBot
20+
Disallow: /
21+
22+
User-agent: Applebot-Extended
23+
Disallow: /
24+
25+
User-agent: Bytespider
226
Disallow: /
327

4-
User-agent: Googlebot
5-
Allow: /
28+
User-agent: DuckAssistBot
29+
Disallow: /
630

7-
User-agent: Bingbot
8-
Allow: /
31+
User-agent: YouBot
32+
Disallow: /
33+
34+
User-agent: TimpiBot
35+
Disallow: /
36+
37+
User-agent: DataForSeoBot
38+
Disallow: /
39+
40+
User-agent: *
41+
Allow: /

0 commit comments

Comments
 (0)