阻止所有 AI 机器人抓取 WordPress

作者：本站编辑发布时间：2025-09-23 来源:本站原创点击数：

随着科技的进步，未来会出现各种像 ChatGPT 一样的文字类人工智能 AI 会抓取网站并抓取你的内容，收集并使用你的数据来训练 ChatGPT、OpenAI、DeepSeek 等软件以及数千种其他 AI 创作。

如果你想阻止人工智能机器人抓取你的网页，我这里给大家一个比较全面的人工智能阻止列表，防止它们窃取你的内容。

为什么要拦截 AI 爬虫？

很多 AI 机器人会爬取网站的内容，用于训练人工智能模型，如果不加以限制，你的网站内容可能会被未经许可地用于 AI 训练，而你无法控制或获利。AI 机器人爬取网站的方式可能非常频繁且占用大量服务器资源，影响网站的访问速度，甚至导致服务器负载过高，影响正常用户的体验。

它们还可能会抓取你的网站内容，并将其用于其他平台。这可能导致内容被盗用，甚至被搜索引擎误认为是抄袭，影响你的 SEO 排名。如果你的站点上有用户评论、论坛、私人信息或其他敏感数据，可能会泄露用户隐私。

使用 robots.txt 屏蔽人工智能爬虫

想要屏蔽 AI 机器人（人工智能爬虫）最简单的方法就是通过 robots.txt 文件。它是一个包含规则的文件，规定了搜索引擎爬虫和其他类似机器人应该遵守的爬取规则。我们可以在其中添加规则，限制机器人访问特定页面或整个网站。

Robots.txt 的规则仅仅是建议，机器人并不一定会百分百遵守。正规的搜索引擎爬虫会遵守规则，而恶意爬虫可能会直接忽略并继续爬取网站内容。

以下是用于 robots.txt 文件的规则，可用于屏蔽 AI 机器人：

1.
#Block AI bots
2.
User-agent: Agent GPT
3.
User-agent: AgentGPT
4.
User-agent: AIBot
5.
User-agent: AI2Bot
6.
User-agent: AISearchBot
7.
User-agent: AlexaTM
8.
User-agent: Alpha AI
9.
User-agent: AlphaAI
10.
User-agent: Amazon Bedrock
11.
User-agent: Amazon Lex
12.
User-agent: Amazonbot
13.
User-agent: Amelia
14.
User-agent: anthropic-ai
15.
User-agent: AnyPicker
16.
User-agent: Applebot
17.
User-agent: AutoGPT
18.
User-agent: AwarioRssBot
19.
User-agent: AwarioSmartBot
20.
User-agent: Brave Leo AI
21.
User-agent: Bytespider
22.
User-agent: CatBoost
23.
User-agent: CC-Crawler
24.
User-agent: CCBot
25.
User-agent: ChatGPT
26.
User-agent: Chinchilla
27.
User-agent: Claude-Web
28.
User-agent: ClaudeBot
29.
User-agent: cohere-ai
30.
User-agent: cohere-training-data-crawler
31.
User-agent: Common Crawl
32.
User-agent: commoncrawl
33.
User-agent: Crawlspace
34.
User-agent: crew AI
35.
User-agent: crewAI
36.
User-agent: DALL-E
37.
User-agent: DataForSeoBot
38.
User-agent: DeepMind
39.
User-agent: DeepSeek
40.
User-agent: DepolarizingGPT
41.
User-agent: DialoGPT
42.
User-agent: Diffbot
43.
User-agent: DuckAssistBot
44.
User-agent: FacebookBot
45.
User-agent: Firecrawl
46.
User-agent: Flyriver
47.
User-agent: FriendlyCrawler
48.
User-agent: Gemini
49.
User-agent: Gemma
50.
User-agent: GenAI
51.
User-agent: Google Bard AI
52.
User-agent: Google-CloudVertexBot
53.
User-agent: Google-Extended
54.
User-agent: GoogleOther
55.
User-agent: GPT-2
56.
User-agent: GPT-3
57.
User-agent: GPT-4
58.
User-agent: GPTBot
59.
User-agent: GPTZero
60.
User-agent: Grok
61.
User-agent: Hugging Face
62.
User-agent: iaskspider
63.
User-agent: ICC-Crawler
64.
User-agent: ImagesiftBot
65.
User-agent: img2dataset
66.
User-agent: IntelliSeek.ai
67.
User-agent: ISSCyberRiskCrawler
68.
User-agent: Kangaroo
69.
User-agent: LeftWingGPT
70.
User-agent: LLaMA
71.
User-agent: magpie-crawler
72.
User-agent: Meltwater
73.
User-agent: Meta AI
74.
User-agent: Meta Llama
75.
User-agent: Meta.AI
76.
User-agent: Meta-AI
77.
User-agent: Meta-ExternalAgent
78.
User-agent: Meta-ExternalFetcher
79.
User-agent: MetaAI
80.
User-agent: Mistral
81.
User-agent: OAI-SearchBot
82.
User-agent: OAI SearchBot
83.
User-agent: omgili
84.
User-agent: Open AI
85.
User-agent: OpenAI
86.
User-agent: PanguBot
87.
User-agent: peer39_crawler
88.
User-agent: PerplexityBot
89.
User-agent: PetalBot
90.
User-agent: RightWingGPT
91.
User-agent: Scrapy
92.
User-agent: SearchGPT
93.
User-agent: SemrushBot
94.
User-agent: Sidetrade
95.
User-agent: Stability
96.
User-agent: The Knowledge AI
97.
User-agent: Timpibot
98.
User-agent: VelenPublicWebCrawler
99.
User-agent: WebChatGPT
100.
User-agent: Webzio
101.
User-agent: Whisper
102.
User-agent: x.AI
103.
User-agent: xAI
104.
User-agent: YouBot
105.
User-agent: Zero GTP
106.
Disallow: /

强制阻止 AI 机器人对网站的访问

要完全屏蔽 AI 机器人，可以将以下规则添加到 Apache 配置文件或 .htaccess 主文件（网站根目录）。这种方法比 robots.txt 更权威，能够有效阻止 AI 机器人访问你的站点。

1.
#Block AI bots
2.

3.
RewriteEngine On
4.
RewriteCond %{HTTP_USER_AGENT} (Agent\ GPT|AgentGPT|AIBot|AI2Bot|AISearchBot|AlexaTM|Alpha\ AI|AlphaAI|Amazon\ Bedrock|Amazon\ Lex|Amazonbot) [NC,OR]
5.
RewriteCond %{HTTP_USER_AGENT} (Amelia|anthropic-ai|AnyPicker|Applebot|AutoGPT|AwarioRssBot|AwarioSmartBot|Brave\ Leo\ AI|Bytespider|CatBoost) [NC,OR]
6.
RewriteCond %{HTTP_USER_AGENT} (CC-Crawler|CCBot|ChatGPT|Chinchilla|Claude-Web|ClaudeBot|cohere-ai|cohere-training-data-crawler|Common\ Crawl) [NC,OR]
7.
RewriteCond %{HTTP_USER_AGENT} (commoncrawl|Crawlspace|crew\ AI|crewAI|DALL-E|DataForSeoBot|DeepMind|DeepSeek|DepolarizingGPT|DialoGPT|Diffbot) [NC,OR]
8.
RewriteCond %{HTTP_USER_AGENT} (DuckAssistBot|FacebookBot|Firecrawl|Flyriver|FriendlyCrawler|Gemini|Gemma|GenAI|Google\ Bard\ AI|Google-CloudVertexBot) [NC,OR]
9.
RewriteCond %{HTTP_USER_AGENT} (Google-Extended|GoogleOther|GPT-2|GPT-3|GPT-4|GPTBot|GPTZero|Grok|Hugging\ Face|iaskspider|ICC-Crawler|ImagesiftBot) [NC,OR]
10.
RewriteCond %{HTTP_USER_AGENT} (img2dataset|IntelliSeek\.ai|ISSCyberRiskCrawler|Kangaroo|LeftWingGPT|LLaMA|magpie-crawler|Meltwater|Meta\ AI|Meta\ Llama) [NC,OR]
11.
RewriteCond %{HTTP_USER_AGENT} (Meta\.AI|Meta-AI|Meta-ExternalAgent|Meta-ExternalFetcher|MetaAI|Mistral|OAI-SearchBot|OAI\ SearchBot|omgili|Open\ AI) [NC,OR]
12.
RewriteCond %{HTTP_USER_AGENT} (OpenAI|PanguBot|peer39_crawler|PerplexityBot|PetalBot|RightWingGPT|Scrapy|SearchGPT|SemrushBot|Sidetrade|Stability) [NC,OR]
13.
RewriteCond %{HTTP_USER_AGENT} (The\ Knowledge\ AI|Timpibot|VelenPublicWebCrawler|WebChatGPT|Webzio|Whisper|x\.AI|xAI|YouBot|Zero\ GTP) [NC]
14.
RewriteRule (.*) - [F,L]
15.

16.
#END Block AI bots