AI Bot Usage Distribution in the Top 10k Sites

Top In AI Bot Usage Distribution in the Top 10k Sites

Technology	Websites	%
GPTBot Disallow	1,526	15.26
Common Crawl Bot Disallow	1,178	11.78
Anthropic Claude Bot Disallow	1,083	10.83
ByteDance Bytespider Disallow	873	8.73
Google Bard Disallow	785	7.85
Facebook Bot Disallow	780	7.8
Perplexity	780	7.8
Webz Omgili Disallow	729	7.29
Cohere AI Disallow	710	7.1
Apple AI Bot Disallow	695	6.95
AmazonBot Disallow	580	5.8
Diffbot Disallow	534	5.34
Alexa Bot Disallow	440	4.4
YOU Disallow	268	2.68

10,961 Detections

of AI Bot in the Top 10k Sites. Last updated 07th Jun 2025.

GPTBot Disallow is currently the most popular technology in this category.