Robots.txt Répartition des utilisations en Martinique

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Googlebot Disallow	515	31,6
Google AdsBot Disallow	470	28,83
Ahrefs Bot Disallow	100	6,13
Robots Disallow	97	5,95
Some Robots Disallow	36	2,21
GPTBot Disallow	33	2,02
Yahoo Slurp Disallow	29	1,78
Majestic Bot Disallow	29	1,78
AppleBot Disallow	28	1,72
Common Crawl Bot Disallow	27	1,66
Anthropic Claude Bot Disallow	27	1,66
Google Bard Disallow	26	1,6
Perplexity	26	1,6
Facebook Bot Disallow	25	1,53
Cohere AI Disallow	25	1,53
Apple AI Bot Disallow	25	1,53
Google AdsBot Mobile Apps Disallow	22	1,35
Semrush Bot Disallow	19	1,17
Alexa Bot Disallow	14	0,86
Soso Bot Disallow	14	0,86
Microsoft MSNBot Disallow	5	0,31
DuckDuckGo Bot Disallow	5	0,31
Webz Omgili Disallow	4	0,25
Google Image Bot Disallow	4	0,25
ByteDance Bytespider Disallow	3	0,18
Google News Bot Disallow	3	0,18
Google Video Bot Disallow	3	0,18
Google Favicon Disallow	3	0,18
Google API Bot Disallow	3	0,18
Google DuplexWeb Disallow	3	0,18
Exalead Bot Disallow	2	0,12
YOU Disallow	2	0,12
Diffbot Disallow	1	0,06
Youdao Bot Disallow	1	0,06
Google FeedFetcher Disallow	1	0,06