Should you block OAI-SearchBot? OpenAI's search crawler
OAI-SearchBot is operated by OpenAI. It crawls websites so OpenAI can surface them in ChatGPT search results and citations. It is separate from GPTBot training controls.
Who should block OAI-SearchBot, and who should not?
Leave it open if you want ChatGPT search to cite and link to you; it is the OpenAI crawler tied to those answers. Blocking it removes you from ChatGPT search answers, apart from navigational links, and does nothing about training, which GPTBot governs. One mistake to check for: a CDN or firewall bot rule that blocks OpenAI's addresses wholesale, so robots.txt says yes while the network says no. Compare your bot-management settings with the published IP list before assuming you are open.
The verified facts
| User-agent token | OAI-SearchBot/1.4 |
|---|---|
| Full user agent in logs | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot |
| Operator | OpenAI |
| Purpose | Search and AI-search indexers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Published IP ranges | openai.com/searchbot.json |
What does OpenAI’s documentation add?
- OpenAI says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers, but can still appear as navigational links.
- A robots.txt change can take about 24 hours to reach ChatGPT search, according to OpenAI.
- OpenAI recommends allowing both the token in robots.txt and requests from its published IP ranges.
How do you block OAI-SearchBot?
Add this to your robots.txt:
User-agent: OAI-SearchBot
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
How do you tell a real OAI-SearchBot request from a fake one?
A user-agent match proves nothing, since any client can send OAI-SearchBot/1.4. Check the IP against the published range file above, re-fetched on a schedule. A request that fails is not OpenAI. The log verification guide has a script for the IP check and a worked reverse-DNS example, or paste the IP into the AI crawler IP verifier, which runs these checks against OpenAI’s current data.
What does blocking OAI-SearchBot cost you?
Your pages will not be shown in ChatGPT search answers, though OpenAI says they can still appear as navigational links.
What else does OpenAI document?
OpenAI also documents GPTBot, ChatGPT-User and OAI-AdsBot, each with its own robots.txt token. How OpenAI’s crawlers fit together.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for OAI-SearchBot does nothing to crawlers with other tokens. In the same group, the directory also covers Claude-SearchBot (Anthropic), Googlebot (Google), Bingbot (Microsoft), Applebot (Apple), PerplexityBot (Perplexity) and DuckAssistBot (DuckDuckGo), each with its own token and its own documented cost of blocking.