AI crawler study · UK trade suppliers
Does roofingmegastore.co.uk block AI crawlers?
roofingmegastore.co.uk refused our request for its robots.txt. Checked 22 October 2026.
The site turned our checker away (bot protection), so we cannot see the rules it gives real AI crawlers. Check it live.
From robots.txt rules for the homepage (RFC 9309). robots.txt is a request; this does not show what any AI company does. Part of Kavel's AI crawler study, built with AI.
Show this on a page or README
Is this your site? SiteCheck shows what else AI agents need: readable text, structured data and contact details.