Google · Training
Google-CloudVertexBot
The only token here that a site owner asks for. Google states that preferences addressed to it affect crawls requested by site owners for building Vertex AI agents, with no effect on Search or other products. Someone with Google Cloud credentials (usually you, sometimes a vendor building on your behalf) points Vertex AI at a domain to construct a grounded agent or search app over it. The crawl is therefore consensual and bounded by a specific project, an entirely different posture from GoogleOther's unattributed sweeps or Google-Extended's corpus-wide permission.
- Operated by
- Purpose
- Training
- robots.txt token
- Google-CloudVertexBot
- Verification
- IP verified
Collects pages into a corpus used to train models.
How to verify Google-CloudVertexBot
Google publishes the IP ranges Google-CloudVertexBot crawls from, and we fetch that list on a schedule. A request claiming to be Google-CloudVertexBot can therefore be checked against 2,163 published ranges: one that does not match is not this crawler.
Published ranges last confirmed by us on .
Google documents reverse-DNS verification: a genuine request resolves to a hostname under .googlebot.com or .geo.googlebot.com that forward-resolves back to the same address.
User agent
Google has not published a full user-agent string for Google-CloudVertexBot. Requests are identified by the Google-CloudVertexBot product token appearing in the User-Agent header; we match that token rather than a whole string, because the rest of the header varies and matching it would miss real traffic.
robots.txt for Google-CloudVertexBot
Yes, a common crawler, addressable as `Google-CloudVertexBot`. It also falls back to the `Googlebot` group, so a broad Googlebot disallow catches it too. That fallback is easy to miss, and means some sites already block it without intending to.
Block
User-agent: Google-CloudVertexBot
Disallow: /Allow
User-agent: Google-CloudVertexBot
Allow: /robots.txt is a request, not an enforcement mechanism. It is honoured by convention, and a crawler that ignores it is stopped at your edge, not in a text file.
What blocking Google-CloudVertexBot costs you
What breaks is a project you or a partner deliberately started. Disallow it and the Vertex AI data store never populates, so the support assistant, internal search or grounded chatbot being built over your content returns nothing or quietly goes stale: a build failure with a confusing cause, since whoever is debugging the agent in Google Cloud is often not whoever edited robots.txt. Search, Discover, Images and AI Overviews are all explicitly untouched. If nobody at your organisation uses Vertex AI Search or Agent Builder, blocking it costs precisely nothing.