Robots.txt Répartition des utilisations en Russie

Distribution pour les sites Web utilisant Robots.txt technologies

Technologie	Sites Web	%
Robots Disallow	880 463	25,59
Ahrefs Bot Disallow	403 488	11,73
Majestic Bot Disallow	396 790	11,53
Semrush Bot Disallow	317 767	9,24
Microsoft BingBot Disallow	232 549	6,76
Baidu Bot Disallow	231 757	6,74
Yandex Bot Disallow	231 236	6,72
AppleBot Disallow	215 980	6,28
Googlebot Disallow	123 725	3,6
Alexa Bot Disallow	81 614	2,37
Some Robots Disallow	77 405	2,25
Microsoft MSNBot Disallow	29 217	0,85
Sogou Bot Disallow	24 288	0,71
Yahoo Slurp Disallow	23 889	0,69
Common Crawl Bot Disallow	23 727	0,69
Soso Bot Disallow	21 349	0,62
GPTBot Disallow	18 025	0,52
Exalead Bot Disallow	15 763	0,46
Google Media Partners Disallow	10 819 ^*	0,31
Google AdsBot Disallow	10 172	0,3
Google Bard Disallow	9 664	0,28
Google AdsBot Mobile Disallow	9 011	0,26
Facebook Bot Disallow	8 156	0,24
Anthropic Claude Bot Disallow	7 798	0,23
Perplexity	6 376	0,19
Webz Omgili Disallow	4 962	0,14
Google Image Bot Disallow	4 776	0,14
360 Bot Disallow	4 277	0,12
ByteDance Bytespider Disallow	3 617	0,11
BuiltWithBot Disallow	3 491	0,1
Cohere AI Disallow	2 752	0,08
Apple AI Bot Disallow	1 111	0,03
Diffbot Disallow	624	0,02
Google AdsBot Mobile Apps Disallow	602	0,02
Google News Bot Disallow	505	0,01
YOU Disallow	499	0,01
Google Video Bot Disallow	418	0,01
DuckDuckGo Bot Disallow	410	0,01
Google API Bot Disallow	254	0,01
Youdao Bot Disallow	250	0,01
Google Favicon Disallow	250	0,01
Google DuplexWeb Disallow	227	0,01
Pipl Disallow	225	0,01
Google FeedFetcher Disallow	79	0
LLMS Text	62	0