Should you block Applebot-Extended? Apple's training control
Applebot-Extended is operated by Apple. It is a robots.txt control token, not a crawler. Apple uses it to decide whether content Applebot crawls may train Apple's foundation models for Apple Intelligence, Services, and Developer Tools.
Who should block Applebot-Extended, and who should not?
Block it if you do not want Apple training its foundation models on your content; Apple says the block costs nothing in search ranking or appearance. It is the cheapest training opt-out in this directory, next to Google-Extended. It does not stop Apple using your pages as context for AI answers in Siri; for that, add nosnippet to the content you want withheld. Because it never visits, your logs will not show it: the robots.txt rule is the only record.
The verified facts
| User-agent token | Applebot-Extended |
|---|---|
| Operator | Apple |
| Purpose | Model training crawlers |
| Honors robots.txt | Yes, per the operator's documentation. |
| Verification | Applebot-Extended is a robots.txt control token, not an HTTP user-agent string |
What does Apple’s documentation add?
- Apple says Applebot-Extended does not crawl webpages and is used only to decide how data crawled by Applebot may be used.
- Pages that disallow Applebot-Extended can still appear in search results, and Apple says its rules are not considered in search ranking.
- It does not cover AI-generated answers in Siri and Search. Apple's opt-out for those is the nosnippet tag on specific content.
How do you block Applebot-Extended?
Add this to your robots.txt:
User-agent: Applebot-Extended
Disallow: /Once you give a crawler its own group, it stops reading your User-agent: * rules, so run the whole file through the AI crawler robots.txt tester before you deploy it.
What does blocking Applebot-Extended cost you?
Apple will not use your content to train its general-purpose foundation models. Search appearance in Spotlight, Siri, and Safari is unaffected.
What else does Apple document?
Apple also documents Applebot, which has its own robots.txt token. How Apple’s crawlers fit together.
Which crawlers in the same group should you decide on at the same time?
A robots.txt group for Applebot-Extended does nothing to crawlers with other tokens. In the same group, the directory also covers AI2Bot (Allen Institute for AI), MistralAI-Training (Mistral), GPTBot (OpenAI) and ClaudeBot (Anthropic), each with its own token and its own documented cost of blocking.