Robots.txt Usage Distribution in Uzbekistan

Distribution for websites using Robots.txt technologies

Technology	Websites	%
Robots Disallow	1,879 ^*	31.02
Googlebot Disallow	793	13.09
Yandex Bot Disallow	536	8.85
Ahrefs Bot Disallow	456	7.53
Semrush Bot Disallow	431	7.11
Majestic Bot Disallow	406	6.7
Microsoft MSNBot Disallow	345	5.69
Google AdsBot Disallow	175	2.89
Some Robots Disallow	148	2.44
Google AdsBot Mobile Disallow	124	2.05
Baidu Bot Disallow	81	1.34
Alexa Bot Disallow	81	1.34
Yahoo Slurp Disallow	70	1.16
Microsoft BingBot Disallow	64	1.06
Sogou Bot Disallow	61	1.01
Soso Bot Disallow	51	0.84
GPTBot Disallow	44	0.73
Google Image Bot Disallow	36	0.59
Common Crawl Bot Disallow	29	0.48
Anthropic Claude Bot Disallow	27	0.45
Google Bard Disallow	25	0.41
ByteDance Bytespider Disallow	21	0.35
Exalead Bot Disallow	20	0.33
Google Media Partners Disallow	18 ^*	0.3
AppleBot Disallow	15	0.25
Facebook Bot Disallow	15	0.25
Perplexity	14	0.23
Apple AI Bot Disallow	13	0.21
360 Bot Disallow	13	0.21
Cohere AI Disallow	12	0.2
Webz Omgili Disallow	11	0.18
Google AdsBot Mobile Apps Disallow	9	0.15
Diffbot Disallow	4	0.07
DuckDuckGo Bot Disallow	4	0.07
BuiltWithBot Disallow	4	0.07
Google Video Bot Disallow	4	0.07
Google News Bot Disallow	4	0.07
Google Favicon Disallow	4	0.07
Google API Bot Disallow	4	0.07
Google DuplexWeb Disallow	4	0.07
YOU Disallow	3	0.05
Pipl Disallow	1	0.02