xAI · Answers
xAI-SearchBot
Grok can ground an answer in a live lookup rather than in model memory: xAI's consumer FAQ confirms real-time web search as a product feature, and its developer docs confirm that fetched sources come back to the user as inline citations with full traceability. A token by this name would, by naming convention, be the fetcher behind that step. That mapping is inference, not documentation: xAI attaches no agent string to the behaviour on any page it publishes, and the name is a near-exact structural clone of a documented OpenAI token, which is one plausible route by which it was coined. The retrieval surface is real and vendor-confirmed; the claim that this particular string performs it is neither.
- Operated by
- xAI
- Purpose
- Answers
- robots.txt token
- xAI-SearchBot
- Verification
- User agent only
Fetches a page in real time to answer a question someone just asked.
How to verify xAI-SearchBot
xAI publishes no list of IP addresses for xAI-SearchBot, so nobody (us included) can prove that a request carrying this user agent really came from xAI. The name is trivial to copy. Treat it as a claim the visitor is making about itself, not as an identity anyone has checked.
There is no range list to confirm. This entry was last reviewed against xAI's own documentation on .
User agent
xAI has not published a full user-agent string for xAI-SearchBot. Requests are identified by the xAI-SearchBot product token appearing in the User-Agent header; we match that token rather than a whole string, because the rest of the header varies and matching it would miss real traffic.
robots.txt for xAI-SearchBot
Block
User-agent: xAI-SearchBot
Disallow: /Allow
User-agent: xAI-SearchBot
Allow: /robots.txt is a request, not an enforcement mechanism. It is honoured by convention, and a crawler that ignores it is stopped at your edge, not in a text file.
What blocking xAI-SearchBot costs you
There are two outcomes and no way to tell which one you are in. If the token is genuine and reads robots.txt, a disallow costs you citation eligibility: your URL stops appearing in the source list Grok attaches to an answer, which is the only place Grok sends a click. If it is not genuine, or is genuine but ignores robots.txt, the identical line costs you that slot and blocks nothing, which is the worse half of both cases at once. Nothing in your logs will settle it: xAI publishes no IP range for any agent it operates, and its Web Bot Auth key directory path returns 404, so a request claiming this name cannot be confirmed or refuted as xAI.
Vendor documentation
xAI does not publish documentation for xAI-SearchBot that we could find. Everything on this page comes from what they do publish elsewhere and from observed behaviour, so treat it accordingly.