Perplexity · Search index
PerplexityBot
Two things separate this token from its sibling: it chooses when to visit, and it honours robots.txt when it does. Perplexity describes it as "designed to surface and link websites in search results on Perplexity" and walls it off from model training ("It is not used to crawl content for AI foundation models"), backed by a structural claim rather than a promise, that "Perplexity does not build foundation models, so your content will not be used for AI model pre-training." On verification it reaches an IP match and stops there: a published range file, no Web Bot Auth, no documented reverse DNS. Perplexity's own WAF guidance is to "combine both User-Agent string matching and IP address verification," which is the right instinct, though the file carries a creation timestamp of February 2025 and roughly a dozen single-host `/32` addresses, so any infrastructure added since is indistinguishable from a spoof.
- Operated by
- Perplexity
- Purpose
- Search index
- robots.txt token
- PerplexityBot
- Verification
- IP verified
Crawls continuously to build and refresh an AI search index.
How to verify PerplexityBot
Perplexity publishes the IP ranges PerplexityBot crawls from, and we fetch that list on a schedule. A request claiming to be PerplexityBot can therefore be checked against 8 published ranges: one that does not match is not this crawler.
Published ranges last confirmed by us on .
User agent
Perplexity publishes this user agent for PerplexityBot. Match on the PerplexityBot product token rather than the whole string: vendors revise the surrounding version and URL fragments without notice.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)robots.txt for PerplexityBot
Yes, but the disallow is partial rather than absolute, and Perplexity spells out the remainder: the crawler "will not index the full or partial text content of any site that disallows it via robots.txt," however "if a page is blocked, we may still index the domain, headline, and a brief factual summary." So a blocked page is de-texted, not deleted. Perplexity also warns that "it may take up to 24 hours for our systems to reflect changes."
Block
User-agent: PerplexityBot
Disallow: /Allow
User-agent: PerplexityBot
Allow: /robots.txt is a request, not an enforcement mechanism. It is honoured by convention, and a crawler that ignores it is stopped at your edge, not in a text file.
What blocking PerplexityBot costs you
You lose the page text that Perplexity's citations are drawn from, which is the surface that sends clicks back to you, and the residue is the awkward part. Because a blocked page can still be indexed by domain, headline and short factual summary, the achievable end state is being described in an answer without being the linked source: your material used, your link absent. Training exclusion is not on offer here either, since Perplexity says it builds no foundation models and does not pre-train on this crawl, so anyone reaching for this lever for that reason has the wrong lever. And the block does not extend to `Perplexity-User`, which can still fetch the page live when someone asks about it.