OpenAI · Training
GPTBot
Corpus collection, and nothing else: this is the token that decides whether your pages can end up inside a future GPT model's weights. It sweeps continuously on OpenAI's own schedule, unconnected to any individual conversation, which is what separates it from ChatGPT-User's single on-demand fetch. Its reach now extends past its own crawl: OpenAI honours a GPTBot disallow for page content that Atlas picked up while a user was browsing, so the token works as a standing training-consent flag rather than merely a crawler identity.
- Operated by
- OpenAI
- Purpose
- Training
- robots.txt token
- GPTBot
- Verification
- IP verified
Collects pages into a corpus used to train models.
How to verify GPTBot
OpenAI publishes the IP ranges GPTBot crawls from, and we fetch that list on a schedule. A request claiming to be GPTBot can therefore be checked against 21 published ranges: one that does not match is not this crawler.
Published ranges last confirmed by us on .
User agent
OpenAI publishes this user agent for GPTBot. Match on the GPTBot product token rather than the whole string: vendors revise the surrounding version and URL fragments without notice.
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbotrobots.txt for GPTBot
Yes, and the disallow carries a declared meaning rather than only an access effect: OpenAI states that disallowing GPTBot indicates a site's content should not be used to train generative AI foundation models.
Block
User-agent: GPTBot
Disallow: /Allow
User-agent: GPTBot
Allow: /robots.txt is a request, not an enforcement mechanism. It is honoured by convention, and a crawler that ignores it is stopped at your edge, not in a text file.
What blocking GPTBot costs you
Close to zero in traffic terms, and this is the most useful thing to tell a site operator, because the intuition that blocking GPTBot hides you from ChatGPT is wrong. Search citations, live user fetches and ad review all run on separate tokens with independently evaluated robots.txt rules, so a disallow here touches none of them. What you actually give up is presence in the training corpus: no attribution, no referral and no revenue attached either way. One caveat cuts the other way: OpenAI says that where both bots are allowed it may reuse a single crawl for both purposes, so allowing both can mean fewer total requests than allowing one.