What is ClaudeBot?

What does ClaudeBot do with your content?

Crawls pages for Claude's models and search index. Like most AI crawlers, ClaudeBot reads your pages as plain HTML — it does not run JavaScript — so content that only appears after scripts execute is invisible to it.

Should you block ClaudeBot?

This is a genuine trade-off. Allowing ClaudeBot lets future Anthropic models learn your business exists — useful when customers ask those models for recommendations. Blocking it keeps your content out of training data at the cost of that future visibility. Neither answer is wrong; it depends on what you value more.

How do you allow or block ClaudeBot in robots.txt?

Add one of these snippets to the robots.txt file at the root of your site. The first explicitly allows ClaudeBot everywhere; the second blocks it completely.

# Allow ClaudeBot
User-agent: ClaudeBot
Allow: /

# Block ClaudeBot
User-agent: ClaudeBot
Disallow: /

Is robots.txt enough?

Not always. CDNs and firewalls (Cloudflare in particular) can block crawlers at the network level regardless of what robots.txt says — Cloudflare blocks AI crawlers by default for many accounts. If you want ClaudeBot to reach your site, check your CDN's bot settings too. SeeGeo's free audit checks both.

What is ClaudeBot?

ClaudeBot is Anthropic's AI training crawler. Crawls pages for Claude's models and search index.

Should I block ClaudeBot?

This is a genuine trade-off. Allowing ClaudeBot lets future Anthropic models learn your business exists — useful when customers ask those models for recommendations.

Does ClaudeBot run JavaScript?

No. ClaudeBot, like most AI crawlers, reads raw HTML without executing JavaScript. If your content only renders client-side, it is effectively invisible to it.