Robot.txt Distribuzione dell'utilizzo a Mauritius

Distribuzione per siti web che utilizzano Robot.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	534	16,6
Googlebot Disallow	431	13,4
Google AdsBot Mobile Disallow	394	12,25
Robots Disallow	346	10,76
Ahrefs Bot Disallow	194	6,03
Common Crawl Bot Disallow	89	11,65
AppleBot Disallow	88	2,74
Anthropic Claude Bot Disallow	85	11,13
GPTBot Disallow	84	10,99
Google Bard Disallow	84	10,99
Facebook Bot Disallow	82	10,73
Apple AI Bot Disallow	82	10,73
Perplexity	75	9,82
Cohere AI Disallow	73	9,55
Alexa Bot Disallow	64	8,38
Google AdsBot Mobile Apps Disallow	61	1,9
Yahoo Slurp Disallow	59	1,83
Soso Bot Disallow	58	1,8
Yandex Bot Disallow	54	1,68
Microsoft BingBot Disallow	48	1,49
Google Media Partners Disallow	48 ^*	1,49
Some Robots Disallow	47	1,46
Semrush Bot Disallow	43	1,34
Majestic Bot Disallow	23	0,71
Webz Omgili Disallow	13	1,7
ByteDance Bytespider Disallow	9	1,18
Exalead Bot Disallow	8	0,25
Microsoft MSNBot Disallow	7	0,22
Google Image Bot Disallow	6	0,19
360 Bot Disallow	3	0,09
Diffbot Disallow	3	0,39
BuiltWithBot Disallow	3	0,09
DuckDuckGo Bot Disallow	3	0,09
YOU Disallow	3	0,39
Google News Bot Disallow	3	0,09
Google Video Bot Disallow	3	0,09
Google Favicon Disallow	3	0,09
Google API Bot Disallow	2	0,06
Google DuplexWeb Disallow	2	0,06