Robot.txt Distribuzione dell'utilizzo in Nuova Zelanda

Distribuzione per siti web che utilizzano Robot.txt tecnologie

Tecnologia	Siti web	%
Google AdsBot Disallow	56.525	12,36
Google AdsBot Mobile Disallow	43.917	9,61
GPTBot Disallow	27.872	12,11
AppleBot Disallow	26.799	5,86
Common Crawl Bot Disallow	26.589	11,55
Anthropic Claude Bot Disallow	26.495	11,51
Google Bard Disallow	26.355	11,45
Facebook Bot Disallow	26.106	11,34
Apple AI Bot Disallow	26.079	11,33
Perplexity	25.988	11,29
Cohere AI Disallow	25.861	11,23
Googlebot Disallow	25.544	5,59
Google AdsBot Mobile Apps Disallow	17.292	3,78
Ahrefs Bot Disallow	15.163	3,32
Baidu Bot Disallow	10.063	2,2
Microsoft BingBot Disallow	9.270	2,03
Robots Disallow	8.502 ^*	1,86
Yandex Bot Disallow	5.682	1,24
Yahoo Slurp Disallow	4.568	1
Sogou Bot Disallow	3.296	0,72
Alexa Bot Disallow	2.921	1,27
Soso Bot Disallow	2.687	0,59
Majestic Bot Disallow	2.522	0,55
Some Robots Disallow	2.205	0,48
Semrush Bot Disallow	2.175	0,48
ByteDance Bytespider Disallow	901	0,39
Google Media Partners Disallow	888 ^*	0,19
LLMS Text	732	0,16
Microsoft MSNBot Disallow	680	0,15
Google Image Bot Disallow	665	0,15
Exalead Bot Disallow	618	0,14
Webz Omgili Disallow	499	0,22
Diffbot Disallow	283	0,12
YOU Disallow	198	0,09
Google News Bot Disallow	190	0,04
Google Video Bot Disallow	181	0,04
360 Bot Disallow	168	0,04
DuckDuckGo Bot Disallow	134	0,03
BuiltWithBot Disallow	97	0,02
Google Favicon Disallow	93	0,02
Pipl Disallow	84	0,02
Anthropic Claude User Bot Disallow	84	0,04
Anthropic Claude Search Bot Disallow	80	0,02
Google API Bot Disallow	41	0,01
Google DuplexWeb Disallow	40	0,01
Youdao Bot Disallow	37	0,01
Google FeedFetcher Disallow	2	0
Google Read Aloud Disallow	1	0