Blocks Facebook crawling.
https://www.facebook.com/
Robots.txt · AI Bot
Get access to data on 4,020,709 websites that are Facebook Bot Disallow Customers. We know of 3,340,030 live websites using Facebook Bot Disallow and an additional 680,679 sites that used Facebook Bot Disallow historically and 2,335,827 websites in the United States.
Get a list of 4,020,709 websites using Facebook Bot Disallow which includes location information, hosting data, contact details, 3,340,030 currently live websites and an additional 1,490,436 domains that redirect to sites in this list. 680,679 sites that used this technology previouslyand 2,335,827 websites in the United States currently using Facebook Bot Disallow.
Countries
Financial
Group
Region