Robots.txt Usage Distribution in Peru

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Google AdsBot Disallow	5,630	17.42
Googlebot Disallow	3,955	12.23
Ahrefs Bot Disallow	3,801	11.76
Robots Disallow	2,611	8.08
Google AdsBot Mobile Disallow	2,250	6.96
Google Media Partners Disallow	1,693 ^*	5.24
Baidu Bot Disallow	1,339	4.14
Yandex Bot Disallow	927	2.87
Microsoft BingBot Disallow	830	2.57
Yahoo Slurp Disallow	821	2.54
Some Robots Disallow	778	2.41
Alexa Bot Disallow	757	2.34
Sogou Bot Disallow	720	2.23
GPTBot Disallow	691	2.14
Soso Bot Disallow	678	2.1
Google AdsBot Mobile Apps Disallow	561	1.74
Common Crawl Bot Disallow	560	1.73
Semrush Bot Disallow	438	1.35
Majestic Bot Disallow	380	1.18
Google Bard Disallow	360	1.11
Anthropic Claude Bot Disallow	346	1.07
Facebook Bot Disallow	329	1.02
Cohere AI Disallow	306	0.95
AppleBot Disallow	277	0.86
Apple AI Bot Disallow	253	0.78
Perplexity	240	0.74
Microsoft MSNBot Disallow	186	0.58
Google Image Bot Disallow	155	0.48
Exalead Bot Disallow	94	0.29
Google News Bot Disallow	51	0.16
Google Video Bot Disallow	51	0.16
360 Bot Disallow	49	0.15
ByteDance Bytespider Disallow	38	0.12
Youdao Bot Disallow	37	0.11
Webz Omgili Disallow	29	0.09
BuiltWithBot Disallow	28	0.09
DuckDuckGo Bot Disallow	17	0.05
Google Favicon Disallow	16	0.05
Google API Bot Disallow	12	0.04
Google DuplexWeb Disallow	12	0.04
Diffbot Disallow	6	0.02
YOU Disallow	6	0.02
Pipl Disallow	5	0.02
Google FeedFetcher Disallow	3	0.01