Robots.txt Usage Distribution in Peru

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Google AdsBot Disallow	4,388	12.94
Googlebot Disallow	4,186	12.34
Robots Disallow	2,347 ^*	6.92
Ahrefs Bot Disallow	2,293	6.76
Google AdsBot Mobile Disallow	2,167	6.39
Microsoft BingBot Disallow	1,536	4.53
Google Media Partners Disallow	1,523 ^*	4.49
Baidu Bot Disallow	1,499	4.42
Sogou Bot Disallow	1,330	3.92
Yandex Bot Disallow	1,326	3.91
Yahoo Slurp Disallow	1,187	3.5
Alexa Bot Disallow	946	2.79
Some Robots Disallow	899	2.65
Soso Bot Disallow	817	2.41
AppleBot Disallow	687	2.03
GPTBot Disallow	675	1.99
Common Crawl Bot Disallow	646	1.9
Anthropic Claude Bot Disallow	609	1.8
Google Bard Disallow	584	1.72
Facebook Bot Disallow	571	1.68
Apple AI Bot Disallow	560	1.65
Perplexity	552	1.63
Cohere AI Disallow	527	1.55
Semrush Bot Disallow	418	1.23
Google AdsBot Mobile Apps Disallow	380	1.12
Majestic Bot Disallow	358	1.06
Google Image Bot Disallow	162	0.48
Microsoft MSNBot Disallow	117	0.34
ByteDance Bytespider Disallow	91	0.27
Exalead Bot Disallow	91	0.27
Google Video Bot Disallow	91	0.27
Webz Omgili Disallow	78	0.23
YOU Disallow	51	0.15
Diffbot Disallow	50	0.15
360 Bot Disallow	41	0.12
BuiltWithBot Disallow	28	0.08
Youdao Bot Disallow	27	0.08
Google News Bot Disallow	22	0.06
Pipl Disallow	22	0.06
DuckDuckGo Bot Disallow	14	0.04
Google Favicon Disallow	6	0.02
Google API Bot Disallow	4	0.01
Google DuplexWeb Disallow	4	0.01
LLMS Text	2	0.01
Google Read Aloud Disallow	1	0