Robots.txt Répartition des utilisations à Monaco

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Google AdsBot Disallow	394	13,36
Google AdsBot Mobile Disallow	327	11,09
Googlebot Disallow	306	10,38
Robots Disallow	207	7,02
Ahrefs Bot Disallow	120	4,07
GPTBot Disallow	110	12,85
Common Crawl Bot Disallow	107	12,5
Google AdsBot Mobile Apps Disallow	103	3,49
AppleBot Disallow	98	3,32
Anthropic Claude Bot Disallow	97	11,33
Facebook Bot Disallow	97	11,33
Perplexity	97	11,33
Google Bard Disallow	96	11,21
Cohere AI Disallow	96	11,21
Apple AI Bot Disallow	95	11,1
Google Media Partners Disallow	64 ^*	2,17
Baidu Bot Disallow	63	2,14
Yahoo Slurp Disallow	47	1,59
Yandex Bot Disallow	46	1,56
Microsoft BingBot Disallow	39	1,32
Alexa Bot Disallow	37	4,32
Some Robots Disallow	34	1,15
Soso Bot Disallow	30	1,02
Majestic Bot Disallow	24	0,81
Google Image Bot Disallow	24	0,81
Microsoft MSNBot Disallow	21	0,71
Semrush Bot Disallow	20	0,68
Google News Bot Disallow	19	0,64
Google Video Bot Disallow	19	0,64
Google Favicon Disallow	19	0,64
DuckDuckGo Bot Disallow	18	0,61
Google API Bot Disallow	18	0,61
Google DuplexWeb Disallow	18	0,61
Exalead Bot Disallow	14	0,47
ByteDance Bytespider Disallow	5	0,58
360 Bot Disallow	4	0,14
YOU Disallow	4	0,47
Webz Omgili Disallow	3	0,35
Diffbot Disallow	3	0,35
BuiltWithBot Disallow	3	0,1
Youdao Bot Disallow	3	0,1