Robots.txt Nutzungsverteilung in Paraguay

Verteilung für Websites mit Robots.txt Technologien

Technologie	Websites	%
Google AdsBot Disallow	569	16,07
Googlebot Disallow	489	13,81
Robots Disallow	468	13,22
Google AdsBot Mobile Disallow	424	11,98
Ahrefs Bot Disallow	215	6,07
GPTBot Disallow	85	2,4
Google Media Partners Disallow	79 ^*	2,23
Semrush Bot Disallow	76	2,15
Anthropic Claude Bot Disallow	71	2,01
Google Bard Disallow	68	1,92
Baidu Bot Disallow	67	1,89
Common Crawl Bot Disallow	66	1,86
AppleBot Disallow	63	1,78
Majestic Bot Disallow	61	1,72
Facebook Bot Disallow	60	1,69
Apple AI Bot Disallow	58	1,64
Yandex Bot Disallow	57	1,61
Cohere AI Disallow	57	1,61
Perplexity	56	1,58
Some Robots Disallow	54	1,53
Yahoo Slurp Disallow	53	1,5
Google AdsBot Mobile Apps Disallow	49	1,38
Sogou Bot Disallow	48	1,36
Microsoft BingBot Disallow	48	1,36
Alexa Bot Disallow	41	1,16
Soso Bot Disallow	37	1,05
Microsoft MSNBot Disallow	21	0,59
Google Image Bot Disallow	16	0,45
Exalead Bot Disallow	15	0,42
ByteDance Bytespider Disallow	14	0,4
Webz Omgili Disallow	9	0,25
YOU Disallow	6	0,17
360 Bot Disallow	5	0,14
Diffbot Disallow	5	0,14
BuiltWithBot Disallow	4	0,11
DuckDuckGo Bot Disallow	4	0,11
Google News Bot Disallow	4	0,11
Google Video Bot Disallow	4	0,11
Google Favicon Disallow	4	0,11
Youdao Bot Disallow	3	0,08
Google API Bot Disallow	3	0,08
Google DuplexWeb Disallow	3	0,08
Pipl Disallow	1	0,03