Robots.txt Répartition des utilisations au Groenland

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Robots Disallow	142	12,13
Google AdsBot Disallow	100	8,54
Googlebot Disallow	82	7
Ahrefs Bot Disallow	71	6,06
Baidu Bot Disallow	55	4,7
Google AdsBot Mobile Disallow	48	4,1
Some Robots Disallow	45	3,84
GPTBot Disallow	42	12,1
Common Crawl Bot Disallow	38	10,95
Anthropic Claude Bot Disallow	38	10,95
AppleBot Disallow	37	3,16
Google Bard Disallow	37	10,66
Facebook Bot Disallow	36	10,37
Yahoo Slurp Disallow	34	2,9
Yandex Bot Disallow	33	2,82
Apple AI Bot Disallow	33	9,51
Perplexity	32	9,22
Google Media Partners Disallow	32 ^*	2,73
Microsoft BingBot Disallow	31	2,65
Cohere AI Disallow	31	8,93
Google AdsBot Mobile Apps Disallow	29	2,48
Alexa Bot Disallow	28	8,07
Soso Bot Disallow	25	2,13
Majestic Bot Disallow	18	1,54
Webz Omgili Disallow	10	2,88
Semrush Bot Disallow	9	0,77
ByteDance Bytespider Disallow	8	2,31
Microsoft MSNBot Disallow	6	0,51
Google Image Bot Disallow	5	0,43
Diffbot Disallow	4	1,15
DuckDuckGo Bot Disallow	4	0,34
Google News Bot Disallow	4	0,34
Google Video Bot Disallow	4	0,34
Google Favicon Disallow	4	0,34
Google API Bot Disallow	4	0,34
Google DuplexWeb Disallow	4	0,34
YOU Disallow	3	0,86
Exalead Bot Disallow	2	0,17
360 Bot Disallow	1	0,09
BuiltWithBot Disallow	1	0,09
Pipl Disallow	1	0,09