What is Claude-User?
Claude-User is a user fetcher run by Anthropic. Here is what Anthropic says it is for, and how to allow or block it.
What it is for
What Anthropic says: User fetches. Visits a page when a Claude user asks a question that needs it.
User fetchers visit a page because a person asked an assistant something that needs it. Because a person asked, some companies say these fetchers may not apply robots.txt rules the way a crawler does; check the operator's documentation.
Is Claude-User blocked on your site?
Free. Checks your robots.txt for Claude-User and 13 other AI crawlers.
How to block Claude-User
Add this group to the robots.txt file at the root of your site:
User-agent: Claude-User
Disallow: /How to allow Claude-User
If nothing in your robots.txt names Claude-User and your User-agent: * group does not say Disallow: /, it is already allowed. If an older rule blocks it, remove that group, or say so explicitly:
User-agent: Claude-User
Allow: /robots.txt is a request, not a lock, and a "block AI bots" setting at your host or CDN can turn Claude-User away whatever robots.txt says. More on writing the rules
Other Anthropic agents
- ClaudeBot: a training crawler
- Claude-SearchBot: a search crawler
Operators publish these names and change them. This page reflects what Anthropic has said publicly; check Anthropic's own documentation before you rely on it. See the full AI crawler list.