Robots.txt Répartition des utilisations aux Samoa

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Googlebot Disallow	7 327	11,96
Baidu Bot Disallow	7 252	11,84
Sogou Bot Disallow	7 191	11,74
Yandex Bot Disallow	6 899	11,26
Microsoft BingBot Disallow	6 885	11,24
Yahoo Slurp Disallow	6 870	11,21
Some Robots Disallow	5 752	9,39
Soso Bot Disallow	4 978	8,12
Alexa Bot Disallow	4 964	8,1
Google Media Partners Disallow	941 ^*	1,54
Robots Disallow	611 ^*	1
Google AdsBot Disallow	126	0,21
Ahrefs Bot Disallow	99	0,16
AppleBot Disallow	93	0,15
GPTBot Disallow	91	0,15
Common Crawl Bot Disallow	89	0,15
Majestic Bot Disallow	87	0,14
Semrush Bot Disallow	74	0,12
Google Bard Disallow	72	0,12
Anthropic Claude Bot Disallow	69	0,11
Microsoft MSNBot Disallow	69	0,11
Exalead Bot Disallow	61	0,1
Apple AI Bot Disallow	59	0,1
Google Image Bot Disallow	54	0,09
Facebook Bot Disallow	53	0,09
Perplexity	40	0,07
Webz Omgili Disallow	37	0,06
DuckDuckGo Bot Disallow	36	0,06
ByteDance Bytespider Disallow	35	0,06
Google Video Bot Disallow	35	0,06
Google News Bot Disallow	35	0,06
Google Favicon Disallow	35	0,06
360 Bot Disallow	32	0,05
Cohere AI Disallow	30	0,05
Google API Bot Disallow	29	0,05
Google DuplexWeb Disallow	29	0,05
Diffbot Disallow	21	0,03
Google AdsBot Mobile Apps Disallow	20	0,03
YOU Disallow	18	0,03
Google Read Aloud Disallow	18	0,03
Google FeedFetcher Disallow	18	0,03
Google Web Light Disallow	18	0,03
BuiltWithBot Disallow	12	0,02
Pipl Disallow	9	0,01
LLMS Text	1	0
Youdao Bot Disallow	1	0