Skip to main content

What is Meta-ExternalAgent?

Meta · AI training · class ai

Crawls for Meta AI model training and product improvement.

Verdict: Charge it or block it. It does not pay yet.
On the mugshot wall: No published way to verify a request is really this crawler

Profile

Compiled by Crawler Atlas from the operator's published documentation. Not yet declared by the operator. JSON

OperatorMeta · developers.facebook.com
PurposeAI training crawlers. Collect pages to train models. Bulk, periodic, and the ones most owners want to charge or block.
User-agent containsmeta-externalagent · meta-externalfetcher · facebookbot
robots.txtUser-agent: Meta-ExternalAgent
Disallow: /
How to verifyNo published verification method. Treat the name as a claim, not proof.
Should you block it?Charge it or block it. It does not pay yet.
Does it pay?No. It does not carry a wallet and cannot answer a 402. A site that charges simply serves it nothing.
Docshttps://developers.facebook.com/docs/sharing/webmasters/web-crawlers

Observed on the network

Anonymous daily counts from sites running Pay-per-Crawl that opted in, plus the ForgeMesh fleet. Last 7 days. Small sample: 1 sites reporting.

No sightings reported yet. Meta-ExternalAgent appears here the day an opted-in site sees it.

Do something about it

Also in ai training crawlers

ForgeMesh Crawler Atlas is maintained by ForgeMesh. Names and marks belong to their operators and are used to identify their crawlers.