What is Meta-ExternalAgent?
Meta-ExternalAgent is a training crawler run by Meta. Here is what Meta says it is for, and how to allow or block it.
What it is for
What Meta says: Training and related uses. Meta says it collects content for uses such as training AI models.
Training crawlers collect pages that may be used to train AI models. Blocking one does not stop the same company's search or user fetchers.
Is Meta-ExternalAgent blocked on your site?
Free. Checks your robots.txt for Meta-ExternalAgent and 13 other AI crawlers.
How to block Meta-ExternalAgent
Add this group to the robots.txt file at the root of your site:
User-agent: Meta-ExternalAgent
Disallow: /How to allow Meta-ExternalAgent
If nothing in your robots.txt names Meta-ExternalAgent and your User-agent: * group does not say Disallow: /, it is already allowed. If an older rule blocks it, remove that group, or say so explicitly:
User-agent: Meta-ExternalAgent
Allow: /robots.txt is a request, not a lock, and a "block AI bots" setting at your host or CDN can turn Meta-ExternalAgent away whatever robots.txt says. More on writing the rules
Operators publish these names and change them. This page reflects what Meta has said publicly; check Meta's own documentation before you rely on it. See the full AI crawler list.