Should you block PerplexityBot? Perplexity's search crawler
PerplexityBot is operated by Perplexity. It gathers and indexes public web content so Perplexity can surface and link websites in search results.
Who should block PerplexityBot, and who should not?
Leave it open if you want Perplexity to cite and link you: it is the agent behind Perplexity's search results, and Perplexity says it is not used to crawl content for AI foundation models. Blocking it removes you from those results but does not stop Perplexity-User fetching a page a user asks about. If you use a WAF, follow Perplexity's pattern: an allow rule that matches both the user agent and the published IP list. It keeps impostors out only if you also block other requests that use the same user agent.
The verified facts
| User-agent token | PerplexityBot/1.0 |
|---|---|
| Full user agent in logs | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot) |
| Operator | Perplexity |
| Purpose | Search and AI-search indexers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Published IP ranges | www.perplexity.com/perplexitybot.json |
What does Perplexity’s documentation add?
- Perplexity says PerplexityBot surfaces and links websites in its search results and is not used to crawl content for AI foundation models.
- Perplexity recommends allowing it in robots.txt and permitting requests from its published IP ranges; for a web application firewall it suggests a rule that matches both the user agent and the IP list.
- A robots.txt change may take up to 24 hours to apply.
How do you block PerplexityBot?
Add this to your robots.txt:
User-agent: PerplexityBot
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
How do you tell a real PerplexityBot request from a fake one?
A user-agent match proves nothing, since any client can send PerplexityBot/1.0. Check the IP against the published range file above, re-fetched on a schedule. A request that fails is not Perplexity. The log verification guide has a script for the IP check and a worked reverse-DNS example, or paste the IP into the AI crawler IP verifier, which runs these checks against Perplexity’s current data.
What does blocking PerplexityBot cost you?
Perplexity will not surface or link your pages in its search results. Perplexity-User can still visit a page when a user asks about it.
What else does Perplexity document?
Perplexity also documents Perplexity-User, which has its own robots.txt token. How Perplexity’s crawlers fit together.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for PerplexityBot does nothing to crawlers with other tokens. In the same group, the directory also covers DuckAssistBot (DuckDuckGo), DuckDuckBot (DuckDuckGo), Meta-WebIndexer (Meta), Amzn-SearchBot (Amazon), YouBot (You.com) and Diffbot (Diffbot), each with its own token and its own documented cost of blocking.