Blocks Facebook crawling.
https://www.facebook.com/
Robots.txt · AI Bot
Get access to data on 4,187,777 websites that are Facebook Bot Disallow Customers. We know of 3,519,291 live websites using Facebook Bot Disallow and an additional 668,486 sites that used Facebook Bot Disallow historically and 2,480,908 websites in the United States.
Get a list of 4,187,777 websites using Facebook Bot Disallow which includes location information, hosting data, contact details, 3,519,291 currently live websites and an additional 1,550,783 domains that redirect to sites in this list. 668,486 sites that used this technology previouslyand 2,480,908 websites in the United States currently using Facebook Bot Disallow.
Countries
Financial
Group
Region