Meta · Answers
meta-externalfetcher
Ask Meta AI a question and this is the request that lands on the one page needed to answer it. Meta describes it as fetching “individual links at a user's request” and supporting “agentic AI capabilities — including helping AI navigate websites to complete tasks for users”; the 2025 wording covered only user-initiated fetches, so task-completion navigation is new, and this is now the token behind software operating your site on somebody's behalf. Traffic is bursty and demand-shaped, one URL at a time, against meta-externalagent's continuous sweep. It is also the Meta token with the strongest incentive to spoof: no IP list, no reverse-DNS convention, and a documented reason to appear on paths robots.txt forbids.
- Operated by
- Meta
- Purpose
- Answers
- robots.txt token
- meta-externalfetcher
- Verification
- Network origin only
Fetches a page in real time to answer a question someone just asked.
How to verify meta-externalfetcher
We hold no list of IP addresses that Meta publishes itself for meta-externalfetcher. What we hold instead is 574 prefixes that a third party (a public routing record, not Meta) reports Meta's network announcing. A request from inside one of those tells you where it came from, not who sent it: anything else on that network can send the same packet, and nothing we hold ties meta-externalfetcher to those addresses beyond the network they sit on. A request from OUTSIDE them is not thereby a fake either: this describes one network's announcements, not every address Meta can crawl from. Corroboration, not proof.
We hold no range list Meta publishes itself for meta-externalfetcher, so there is no such fetch to confirm. This entry was last reviewed against Meta's own documentation on .
User agent
Meta publishes this user agent for meta-externalfetcher. Match on the meta-externalfetcher product token rather than the whole string: vendors revise the surrounding version and URL fragments without notice.
meta-externalfetcher/1.1 (+/documentation/sharing/webmasters/web-crawlers)robots.txt for meta-externalfetcher
No. Meta states it may ignore your rules. Verbatim: “Accordingly, this crawler may bypass robots.txt rules,” repeated in the robots.txt section as “The Meta-ExternalFetcher crawler may bypass robots.txt because it performs fetches that were requested by the user.” A `Disallow` here is a preference the token is documented as free to override.
Block
User-agent: meta-externalfetcher
Disallow: /Allow
User-agent: meta-externalfetcher
Allow: /robots.txt is a request, not an enforcement mechanism. It is honoured by convention, and a crawler that ignores it is stopped at your edge, not in a text file.
What blocking meta-externalfetcher costs you
The bypass clause makes the cost partly hypothetical, because adding a rule may simply not stop the requests. Where the rule is honoured, what you lose is live retrieval: a person who pastes your URL into Meta AI, or points an agent at your site to finish a task, gets an answer assembled without your current page. There is no cached surface to degrade gracefully into and no ranking consequence anywhere else, so the whole loss lands inside that single interaction.