Should you block Applebot? Apple's search crawler
Applebot is operated by Apple. It crawls pages for search in Spotlight, Siri, and Safari. Apple says the data may also help train its foundation models and add context to AI-generated answers in its products.
Who should block Applebot, and who should not?
Leave it open on pages you want found in Siri, Spotlight, and Safari; there is rarely a reason to block the crawler itself. Apple separates the concerns publishers usually have: disallow Applebot-Extended to keep content out of foundation-model training, and use nosnippet, or isAccessibleForFree set to false on paywalled pages, to keep it out of AI-generated answers. The cost of nosnippet: Apple says suggestions for that page then show only its title. One trap: a site with Googlebot rules and no Applebot section gets the Googlebot rules applied to Apple, so check that those rules suit Apple too.
The verified facts
| User-agent token | Applebot |
|---|---|
| Full user agent in logs | Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot) |
| Operator | Apple |
| Purpose | Search and AI-search indexers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Published IP ranges | search.developer.apple.com/applebot.json |
| Reverse DNS ends in | applebot.apple.com |
What does Apple’s documentation add?
- If robots.txt does not mention Applebot but does mention Googlebot, Apple says Applebot follows the Googlebot rules.
- Applebot does not follow Crawl-delay. Apple says it slows down on its own when a site slows or returns errors.
- The nosnippet tag, in a meta tag or an X-Robots-Tag: applebot header, stops Apple generating a description or web answer for a page and stops it using that content as context for AI-generated output.
- Pages marked isAccessibleForFree: false in structured data stay eligible for search, but Apple will not use them as context for AI-generated output.
How do you block Applebot?
Add this to your robots.txt:
User-agent: Applebot
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
How do you tell a real Applebot request from a fake one?
A user-agent match proves nothing, since any client can send Applebot. Check the IP against the published range file above, re-fetched on a schedule. Or reverse-resolve the IP, check the name ends in applebot.apple.com, and forward-resolve that name back to the same IP. A request that fails is not Apple. The log verification guide has a script for the IP check and a worked reverse-DNS example, or paste the IP into the AI crawler IP verifier, which runs these checks against Apple’s current data.
What does blocking Applebot cost you?
Your pages drop out of Apple's search results in Spotlight, Siri, and Safari. Training and AI-answer use have their own controls, so a block is not needed for those.
What else does Apple document?
Apple also documents Applebot-Extended, which has its own robots.txt token. How Apple’s crawlers fit together.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for Applebot does nothing to crawlers with other tokens. In the same group, the directory also covers PerplexityBot (Perplexity), DuckAssistBot (DuckDuckGo), DuckDuckBot (DuckDuckGo), Meta-WebIndexer (Meta), Amzn-SearchBot (Amazon) and YouBot (You.com), each with its own token and its own documented cost of blocking.