What is OAI-SearchBot?
OAI-SearchBot is a search crawler run by OpenAI. Here is what OpenAI says it is for, and how to allow or block it.
What it is for
What OpenAI says: Search. Finds pages to show and link to in ChatGPT's search features.
Search crawlers build an index that an AI assistant or search product uses to find and link to pages when it answers questions. Blocking one can mean that product cannot show or link to your pages.
Is OAI-SearchBot blocked on your site?
Free. Checks your robots.txt for OAI-SearchBot and 13 other AI crawlers.
How to block OAI-SearchBot
Add this group to the robots.txt file at the root of your site:
User-agent: OAI-SearchBot
Disallow: /How to allow OAI-SearchBot
If nothing in your robots.txt names OAI-SearchBot and your User-agent: * group does not say Disallow: /, it is already allowed. If an older rule blocks it, remove that group, or say so explicitly:
User-agent: OAI-SearchBot
Allow: /robots.txt is a request, not a lock, and a "block AI bots" setting at your host or CDN can turn OAI-SearchBot away whatever robots.txt says. More on writing the rules
Other OpenAI agents
- GPTBot: a training crawler
- ChatGPT-User: a user fetcher
Operators publish these names and change them. This page reflects what OpenAI has said publicly; check OpenAI's own documentation before you rely on it. See the full AI crawler list.