Microsoft · Search index
Bingbot
“Our standard crawler,” in Microsoft's words, handling “most of our crawling needs each day”: the general-purpose discovery and refresh crawl behind the Bing index, in desktop and mobile variants. Its reach runs well past the blue links: Microsoft Learn describes Copilot Studio's public-website grounding as query-time retrieval through Bing Custom Search over that same index, and states that “Crawling is how Bingbot (Bing crawler) discovers new and updated pages and content to add to search index,” so the fetch that sets your Bing ranking also decides whether Copilot has anything of yours to ground on. Two checkable caveats attach to the verification material Microsoft publishes: `bing.com/toolbox/bingbot.json` carries 28 IPv4 prefixes and no IPv6, and its own `creationTime` reads 2024-01-03 despite Microsoft telling operators to refresh the list daily because it can change at any time. The crawler table has a second defect worth knowing: its BingPreview row lists user-agent strings that say `bingbot/2.0`, so by Microsoft's own published strings the two are indistinguishable at the UA level.
- Operated by
- Microsoft
- Purpose
- Search index
- robots.txt token
- Bingbot
- Verification
- IP verified
Crawls continuously to build and refresh an AI search index.
How to verify Bingbot
Microsoft publishes the IP ranges Bingbot crawls from, and we fetch that list on a schedule. A request claiming to be Bingbot can therefore be checked against 28 published ranges: one that does not match is not this crawler.
Published ranges last confirmed by us on .
Microsoft documents reverse-DNS verification: a genuine request resolves to a hostname under search.msn.com that forward-resolves back to the same address.
User agent
Microsoft publishes this user agent for Bingbot. Match on the Bingbot product token rather than the whole string: vendors revise the surrounding version and URL fragments without notice.
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)robots.txt for Bingbot
Yes. Microsoft names robots.txt as one of two supported controls, the other being hourly Crawl Control in Bing Webmaster Tools. It additionally honours directives written for the older `msnbot` token, with `bingbot` rules taking precedence where both are present.
Block
User-agent: Bingbot
Disallow: /Allow
User-agent: Bingbot
Allow: /robots.txt is a request, not an enforcement mechanism. It is honoured by convention, and a crawler that ignores it is stopped at your edge, not in a text file.
What blocking Bingbot costs you
Two surfaces fall at once, which makes this the most expensive block Microsoft offers. You leave the Bing web index, and with it Yahoo, DuckDuckGo and the other engines that syndicate Bing results; and because Microsoft documents Copilot's public-web grounding as reading that index rather than crawling independently, you also remove yourself from Copilot's citable pool without ever having written a Copilot rule. That second effect surprises people, because there is no separate AI opt-out held in reserve: one decision covers both.