Robot.txt Distribuzione dell'utilizzo in Giappone

Distribuzione per siti web che utilizzano Robot.txt tecnologie

Tecnologia	Siti web	%
Googlebot Disallow	159.206	18,32
Google AdsBot Disallow	150.571	17,33
Google AdsBot Mobile Disallow	134.040	15,42
Ahrefs Bot Disallow	42.358	4,87
Robots Disallow	38.740 ^*	4,46
Microsoft BingBot Disallow	34.526	3,97
Semrush Bot Disallow	27.492	3,16
Baidu Bot Disallow	27.126	3,12
Majestic Bot Disallow	26.469	3,05
Yandex Bot Disallow	24.450	2,81
Sogou Bot Disallow	23.081	2,66
Yahoo Slurp Disallow	22.003	2,53
Alexa Bot Disallow	18.650	2,15
Soso Bot Disallow	17.148	1,97
Some Robots Disallow	14.719	1,69
GPTBot Disallow	14.440	1,66
AppleBot Disallow	11.303	1,3
Anthropic Claude Bot Disallow	10.484	1,21
Common Crawl Bot Disallow	10.360	1,19
Google Bard Disallow	9.492	1,09
Apple AI Bot Disallow	9.067	1,04
Perplexity	8.210	0,94
Cohere AI Disallow	7.894	0,91
Google Media Partners Disallow	6.515 ^*	0,75
Facebook Bot Disallow	4.888	0,56
Microsoft MSNBot Disallow	2.786	0,32
Google AdsBot Mobile Apps Disallow	2.281	0,26
ByteDance Bytespider Disallow	2.266	0,26
Exalead Bot Disallow	1.682	0,19
Google Image Bot Disallow	1.592	0,18
Webz Omgili Disallow	1.373	0,16
360 Bot Disallow	986	0,11
LLMS Text	800	0,09
Youdao Bot Disallow	390	0,04
Diffbot Disallow	305	0,04
YOU Disallow	207	0,02
DuckDuckGo Bot Disallow	182	0,02
Pipl Disallow	168	0,02
Google News Bot Disallow	139	0,02
Google Video Bot Disallow	126	0,01
Anthropic Claude User Bot Disallow	112	0,01
Anthropic Claude Search Bot Disallow	107	0,01
Google API Bot Disallow	91	0,01
Google Favicon Disallow	88	0,01
BuiltWithBot Disallow	85	0,01
Google DuplexWeb Disallow	83	0,01
Google Read Aloud Disallow	4	0
Google FeedFetcher Disallow	1	0