Robots.txt Nutzungsverteilung in Kuba

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Googlebot Disallow	47	8,29
Robots Disallow	42	7,41
Google AdsBot Disallow	36	6,35
Google AdsBot Mobile Disallow	35	6,17
Google Media Partners Disallow	32 ^*	5,64
Baidu Bot Disallow	27	4,76
Yahoo Slurp Disallow	24	4,23
Sogou Bot Disallow	21	3,7
Yandex Bot Disallow	20	3,53
Common Crawl Bot Disallow	19	3,35
Microsoft BingBot Disallow	18	3,17
Alexa Bot Disallow	18	3,17
GPTBot Disallow	17	3
Soso Bot Disallow	16	2,82
AppleBot Disallow	16	2,82
Anthropic Claude Bot Disallow	15	2,65
Google Bard Disallow	15	2,65
Facebook Bot Disallow	15	2,65
Cohere AI Disallow	15	2,65
Ahrefs Bot Disallow	14	2,47
Apple AI Bot Disallow	14	2,47
Perplexity	14	2,47
Majestic Bot Disallow	14	2,47
Google AdsBot Mobile Apps Disallow	13	2,29
Semrush Bot Disallow	9	1,59
Microsoft MSNBot Disallow	9	1,59
Google Image Bot Disallow	4	0,71
Exalead Bot Disallow	3	0,53
DuckDuckGo Bot Disallow	3	0,53
Google News Bot Disallow	3	0,53
Google Video Bot Disallow	3	0,53
Google Favicon Disallow	3	0,53
Google API Bot Disallow	3	0,53
Google DuplexWeb Disallow	3	0,53
360 Bot Disallow	2	0,35
BuiltWithBot Disallow	2	0,35
ByteDance Bytespider Disallow	1	0,18
Webz Omgili Disallow	1	0,18
Youdao Bot Disallow	1	0,18