Robots.txt Nutzungsverteilung in Bulgarien

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Ahrefs Bot Disallow	3.669	10,93
Robots Disallow	3.366	10,03
Googlebot Disallow	3.018	8,99
Google AdsBot Disallow	2.535	7,55
Majestic Bot Disallow	1.970	5,87
Yandex Bot Disallow	1.674	4,99
Baidu Bot Disallow	1.582	4,71
Google AdsBot Mobile Disallow	1.445	4,31
Microsoft BingBot Disallow	1.426	4,25
Sogou Bot Disallow	1.419	4,23
Alexa Bot Disallow	1.359	25,73
Yahoo Slurp Disallow	1.252	3,73
Semrush Bot Disallow	1.222	3,64
Soso Bot Disallow	1.160	3,46
Common Crawl Bot Disallow	608	11,51
Some Robots Disallow	578	1,72
GPTBot Disallow	542	10,26
Google Media Partners Disallow	524 ^*	1,56
Anthropic Claude Bot Disallow	460	8,71
Google Bard Disallow	366	6,93
AppleBot Disallow	351	1,05
Facebook Bot Disallow	348	6,59
Perplexity	313	5,93
Apple AI Bot Disallow	304	5,76
Cohere AI Disallow	301	5,7
Google AdsBot Mobile Apps Disallow	271	0,81
Exalead Bot Disallow	268	0,8
Microsoft MSNBot Disallow	179	0,53
ByteDance Bytespider Disallow	167	3,16
Google Image Bot Disallow	154	0,46
360 Bot Disallow	109	0,32
Webz Omgili Disallow	84	1,59
YOU Disallow	71	1,34
DuckDuckGo Bot Disallow	62	0,18
Google News Bot Disallow	60	0,18
Google Video Bot Disallow	59	0,18
Diffbot Disallow	54	1,02
Google Favicon Disallow	53	0,16
Google API Bot Disallow	53	0,16
BuiltWithBot Disallow	52	0,15
Google DuplexWeb Disallow	52	0,15
Pipl Disallow	14	0,04
Youdao Bot Disallow	4	0,01
LLMS Text	3	0,01
Google FeedFetcher Disallow	1	0