Apple · Training
Applebot
One crawler, three downstream uses, each with its own separate off switch, which is why a blanket disallow here is almost always the wrong instrument. The crawl powers Apple's search layer across Spotlight, Siri and Safari; the same data "may also be used to help train Apple foundation models powering generative AI features," which is governed by Applebot-Extended rather than by any rule on this token; and it may be used as grounding context when models generate output, including Siri and Search answers that link back to their sources, which `nosnippet` controls. This is also the best-verified token in this group: Apple documents full forward-confirmed reverse DNS under `applebot.apple.com` with worked `host` examples, plus a CIDR JSON range file (note Apple's own markup links that file over `http://`, so use HTTPS).
- Operated by
- Apple
- Purpose
- Training
- robots.txt token
- Applebot
- Verification
- IP verified
Collects pages into a corpus used to train models.
How to verify Applebot
Apple publishes the IP ranges Applebot crawls from, and we fetch that list on a schedule. A request claiming to be Applebot can therefore be checked against 24 published ranges: one that does not match is not this crawler.
Published ranges last confirmed by us on .
Apple documents reverse-DNS verification: a genuine request resolves to a hostname under .applebot.apple.com that forward-resolves back to the same address.
User agent
Apple publishes this user agent for Applebot. Match on the Applebot product token rather than the whole string: vendors revise the surrounding version and URL fragments without notice.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Safari/605.1.15 (Applebot/0.1; +http://www.apple.com/go/applebot)robots.txt for Applebot
Yes. Apple respects standard robots.txt directives in general search crawls targeted at Applebot. Two behaviours to know: "If robots instructions don't mention Applebot but mention Googlebot, the Apple robot will follow Googlebot instructions," so silence about Apple is not a block; and Applebot does not follow `crawl-delay`. Beyond robots.txt, Apple supports the `noindex`, `nosnippet`, `nofollow`, `none` and `all` meta tags, the `X-Robots-Tag` header in the form `X-Robots-Tag: applebot: nosnippet`, and the schema.org `isAccessibleForFree` JSON-LD property for paywalled pages. Apple also states it does not crawl pages behind a login or a paywall.
Block
User-agent: Applebot
Disallow: /Allow
User-agent: Applebot
Allow: /robots.txt is a request, not an enforcement mechanism. It is honoured by convention, and a crawler that ignores it is stopped at your edge, not in a text file.
What blocking Applebot costs you
The heaviest block available here, because it is not one surface but the whole Apple discovery layer at once: Apple states that enabling Applebot in robots.txt "allows website content to appear in search results for Apple users around the world in these products," meaning Spotlight, Siri and Safari together. You also forfeit the source links Apple attaches to world-knowledge answers in Siri and Search, which is a live citation surface that can return a visit. The reason to hesitate is that the granular controls already exist: `nosnippet` removes you from AI-generated answers and an Applebot-Extended disallow removes you from training, both while leaving indexing intact. Blocking the crawler to solve a training concern throws away the indexing too.