Apple · Training
Applebot-Extended
There is nothing here to catch in a log file, and that fact is worth more than a verification badge would be: Apple states plainly that "Applebot-Extended does not crawl webpages" and that it "is only used to determine how to use the data crawled by the Applebot user agent." Because no request is ever issued under this name, it sits at no rung of the verification ladder at all (not IP range, not reverse DNS, not even the user-agent-only floor), so any inbound hit carrying this string is spoofed by definition, with no benign explanation available. What the token actually governs is downstream usage: whether the material Applebot already collected may train Apple's general-purpose foundation models behind Apple Intelligence, Services and Developer Tools. The page is still fetched; Apple simply declines to feed what it holds into training.
- Operated by
- Apple
- Purpose
- Training
- robots.txt token
- Applebot-Extended
- Verification
- Not a crawler
Collects pages into a corpus used to train models.
Can Applebot-Extended be verified?
Applebot-Extended is a robots.txt permission token, not a crawler. No request ever arrives carrying it as a user agent, so there is nothing to verify and nothing to see in traffic logs. It exists so you can state a preference that Apple reads separately, using the crawlers it does operate.
This token issues no requests, so there is nothing to confirm. This entry was last reviewed against Apple's own documentation on .
User agent
None. Applebot-Extended is only ever written into a robots.txt file. It never appears in a request header, so searching your access logs for it will always come back empty.
robots.txt for Applebot-Extended
The relationship runs the other way: this is itself a robots.txt directive, written in the form Apple documents: ``` User-agent: Applebot-Extended Disallow: /private/ ``` Apple calls it "a secondary user agent ... that gives web publishers additional controls over how their website content can be used by Apple," layered on top of Apple following all robots.txt rules and directives with its actual crawler.
Block
User-agent: Applebot-Extended
Disallow: /Allow
User-agent: Applebot-Extended
Allow: /robots.txt is a request, not an enforcement mechanism. It is honoured by convention, and a crawler that ignores it is stopped at your edge, not in a text file.
What blocking Applebot-Extended costs you
Close to nothing on the discovery side, and Apple commits to that in writing: "Webpages that disallow Applebot-Extended can still be included in search results." Even combined with `nosnippet` tags, your content stays discoverable through Spotlight, Siri and Safari as well as other system-wide features. The only stated cost lands on Apple: allowing the token "will help improve the capabilities and quality of Apple's generative AI models over time." The thing to get right is that the three levers are distinct and do not imply one another: `Disallow: Applebot` stops the fetch, `Disallow: Applebot-Extended` stops training use, and `nosnippet` or `isAccessibleForFree: false` stops use as grounding context in generated answers.