Blocks Facebook crawling.
https://www.facebook.com/
Robots.txt · AI Bot
Get access to data on 4,107,950 websites that are Facebook Bot Disallow Customers. We know of 3,384,465 live websites using Facebook Bot Disallow and an additional 723,485 sites that used Facebook Bot Disallow historically and 2,383,928 websites in the United States.
Get a list of 4,107,950 websites using Facebook Bot Disallow which includes location information, hosting data, contact details, 3,384,465 currently live websites and an additional 1,490,226 domains that redirect to sites in this list. 723,485 sites that used this technology previouslyand 2,383,928 websites in the United States currently using Facebook Bot Disallow.
Countries
Financial
Group
Region