Log In · Signup for Free
EN  DE  FR  ES  IT  Pусский  한국어  日本語  简体中文  عربي

Semrush Bot Disallow Usage Statistics

S Semrush Bot Disallow

The website has disallow rules for Semrush Bot.

https://www.semrush.com/bot/

Robots.txt

Semrush Bot Disallow · Customers

Get access to data on 4,865,076 websites that are Semrush Bot Disallow Customers. We know of 1,518,445 live websites using Semrush Bot Disallow and an additional 3,346,631 sites that used Semrush Bot Disallow historically and 741,978 websites in the United States

Download Lead List
Technology Awards
Most popular technology 🇙🇙🇘🇕 under Robots.txt.
3rd Most popular technology in China 🇨🇳 under Robots.txt.
3rd Most popular technology in Kazakhstan 🇰🇿 under Robots.txt.
4th Most popular technology in San Marino 🇸🇲 under Robots.txt.
4th Most popular technology in Iran 🇮🇷 under Robots.txt.
4th Most popular technology in Russia 🇷🇺 under Robots.txt.
Investor Center
Competitors and Similar Technologies
Majestic Bot Disallow favicon
Majestic Bot Disallow

1,545,212 live sites

Ahrefs Bot Disallow favicon
Ahrefs Bot Disallow

1,856,245 live sites

ByteDance Bytespider Disallow favicon
ByteDance Bytespider Disallow

1,008,012 live sites

Perplexity favicon
Perplexity

3,351,874 live sites

Cohere AI Disallow favicon
Cohere AI Disallow

3,366,401 live sites

LLMS Text favicon
LLMS Text

677,562 live sites

Facebook Bot Disallow favicon
Facebook Bot Disallow

3,417,528 live sites

Apple AI Bot Disallow favicon
Apple AI Bot Disallow

4,019,690 live sites

Google Bard Disallow favicon
Google Bard Disallow

4,127,089 live sites

Anthropic Claude Bot Disallow favicon
Anthropic Claude Bot Disallow

4,238,680 live sites

Google AdsBot Mobile Apps Disallow favicon
Google AdsBot Mobile Apps Disallow

539,515 live sites

Soso Bot Disallow favicon
Soso Bot Disallow

4,299,023 live sites

Common Crawl Bot Disallow favicon
Common Crawl Bot Disallow

4,324,319 live sites

GPTBot Disallow favicon
GPTBot Disallow

4,430,048 live sites

AppleBot Disallow favicon
AppleBot Disallow

4,439,451 live sites

Exalead Bot Disallow favicon
Exalead Bot Disallow

335,791 live sites

Microsoft MSNBot Disallow favicon
Microsoft MSNBot Disallow

333,596 live sites

Google Image Bot Disallow favicon
Google Image Bot Disallow

290,384 live sites

Webz Omgili Disallow favicon
Webz Omgili Disallow

282,491 live sites

DuckDuckGo Bot Disallow favicon
DuckDuckGo Bot Disallow

278,430 live sites

Google AdsBot Mobile Disallow favicon
Google AdsBot Mobile Disallow

8,297,265 live sites

Google AdsBot Disallow favicon
Google AdsBot Disallow

8,906,110 live sites

Diffbot Disallow favicon
Diffbot Disallow

239,422 live sites

Google Media Partners Disallow favicon
Google Media Partners Disallow

11,796,610 live sites

Sogou Bot Disallow favicon
Sogou Bot Disallow

12,003,031 live sites

YOU Disallow favicon
YOU Disallow

155,170 live sites

Youdao Bot Disallow favicon
Youdao Bot Disallow

141,795 live sites

360 Bot Disallow favicon
360 Bot Disallow

121,377 live sites

BuiltWithBot Disallow favicon
BuiltWithBot Disallow

44,424 live sites

Google Video Bot Disallow favicon
Google Video Bot Disallow

67,367 live sites

Download Lead List

Get a list of 1,577,957 websites using Semrush Bot Disallow which includes location information, hosting data and contact details. The list includes 1,518,445 live websites and 59,512 websites redirecting to those sites. 741,978 of these sites are in the United States.

We also know of 3,346,631 sites that have used Semrush Bot Disallow previously.

Site Totals
Total Live
1,518,445
59,512 additional website redirects?.
Expired
3,346,631
🇺🇸 American Live Sites
741,978
Live and Expired
4,865,076
Expired
3,346,631
Traffic
Top 1m
20,720
Top 100k
3,381
Top 10k
382
Top 1000
31
Outside Top 1m
1,490,618
Technology Spend
$100+
324,583
$500+
122,637
$1000+
70,212
$5k+
10,326
$10k+
4,321
Social Followers
10+
74,483
100+
67,402
500+
50,308
1000+
38,504
5000+
14,267
10k+
9,059
100k+
1,475
1m+
121
Sales Revenue
$100
93,907
$500
89,659
$1000
86,863
$5000
72,444
$10k
47,291
$100k
7,500
$1m
1,353
$10m
421
$100m
140
SKU Products
10+
17,214
50+
11,352
500+
6,252
100+
4,138
5000+
1,282
10k+
517
100k+
20
Live Country Totals
🇺🇸 United States
741,978
🇷🇺 Russia
163,303
🇨🇳 China
116,919
🇩🇪 Germany
74,262
🇬🇧 United Kingdom
33,639
🇮🇹 Italy
26,517
🇫🇷 France
23,507
🇯🇵 Japan
19,913
🇳🇱 Netherlands
19,241
🇨🇦 Canada
14,502
🇪🇸 Spain
11,602
🇧🇷 Brazil
10,378
🇵🇱 Poland
9,685
🇰🇷 South Korea
8,704
🇦🇺 Australia
8,250
🇨🇭 Switzerland
7,716
🇸🇪 Sweden
7,132
🇨🇿 Czech Republic
6,220
🇮🇳 India
5,410
🇧🇪 Belgium
4,893
🇭🇺 Hungary
4,695
🇩🇰 Denmark
4,399
🇨🇨 Cocos Islands
4,264
🇪🇺 European Union
3,877
🇨🇴 .co
3,733
🇹🇼 Taiwan
3,614
🇹🇭 Thailand
3,530
🇦🇹 Austria
3,144
🇮🇩 Indonesia
2,823
🇲🇪 .me
2,795
🇪🇨 Ecuador
2,735
🇹🇷 Turkey
2,683
🇲🇽 Mexico
2,627
🇳🇴 Norway
2,351
🇬🇷 Greece
2,210
🇮🇷 Iran
2,184
🇸🇰 Slovakia
2,009
🇵🇹 Portugal
1,942
🇦🇷 Argentina
1,800
🇫🇮 Finland
1,776
🇻🇳 Vietnam
1,686
🇳🇿 New Zealand
1,655
🇷🇴 Romania
1,629
🇿🇦 South Africa
1,485
🇭🇰 Hong Kong
1,410
🇨🇱 Chile
1,294
🇺🇦 Ukraine
1,263
🇮🇱 Israel
1,224
🇮🇴 .io
1,147
🇧🇬 Bulgaria
1,092
🇲🇾 Malaysia
1,061
🇮🇲 Isle of Man
957
🇸🇬 Singapore
839
🇮🇪 Ireland
791
🇱🇺 Luxembourg
699
🇵🇰 Pakistan
681
🇦🇪 UAE
619
🇹🇻 .tv
616
🇱🇻 Latvia
583
🇸🇮 Slovenia
533
🇳🇺 Niue
520
🇱🇹 Lithuania
508
🇪🇪 Estonia
473
🇦🇮 Anguilla
468
🇰🇿 Kazakhstan
441
🇵🇪 Peru
404
🇳🇬 Nigeria
341
🇷🇸 Serbia
319
🇵🇭 Philippines
301
🇲🇹 Malta
293
🇭🇷 Croatia
291
🇮🇸 Iceland
261
🇧🇾 Belarus
246
🇧🇩 Bangladesh
226
🇨🇾 Cyprus
225
🇱🇦 Lao People's Democratic Republic
211
🇱🇮 Liechtenstein
202
🇷🇪 Reunion
200
🇸🇦 Saudi Arabia
195
🇲🇦 Morocco
187
🇺🇿 Uzbekistan
171
🇺🇾 Uruguay
166
🇩🇴 Dominican Republic
150
🇰🇪 Kenya
149
🇦🇱 Albania
142
🇳🇵 Nepal
138
🇪🇬 Egypt
120
🇱🇰 Sri Lanka
115
🇨🇷 Costa Rica
100
🇵🇼 Palau
98
🇬🇪 Georgia
92
🇬🇹 Guatemala
89
🇬🇬 Guernsey
86
🇧🇦 Bosnia and Herzegovina
82
🇸🇭 Saint Helena
82
🇵🇾 Paraguay
80
🇲🇩 Moldova
77
🇲🇰 Macedonia
75
🇫🇲 Micronesia
74
🇱🇧 Lebanon
74
🇺🇬 Uganda
73
🇹🇿 Tanzania
71
🇬🇭 Ghana
71
🇦🇲 Armenia
70
🇻🇪 Venezuela
70
🇵🇦 Panama
68
🇼🇸 Samoa
65
🇹🇴 Tonga
64
🇶🇦 Qatar
61
🇹🇳 Tunisia
58
🇵🇷 Puerto Rico
56
🇷🇼 Rwanda
54
🇸🇴 Somalia
52
🇯🇴 Jordan
48
🇦🇫 Afghanistan
47
🇸🇨 Seychelles
47
🇨🇮 Cote d'Ivoire
46
🇲🇺 Mauritius
44
🇧🇸 Bahamas
43
🇿🇼 Zimbabwe
43
🇧🇿 Belize
43
🇧🇴 Bolivia
42
🇻🇨 Saint Vincent and the Grenadines
38
🇬🇸 South Georgia and the South Sandwich Islands
35
🇨🇽 Christmas Island
35
🇪🇹 Ethiopia
34
🇸🇳 Senegal
34
🇰🇼 Kuwait
33
🇩🇿 Algeria
33
🇰🇭 Cambodia
31
🇳🇨 New Caledonia
31
🇦🇿 Azerbaijan
30
🇬🇵 Guadeloupe
30
🇳🇦 Namibia
29
🇮🇶 Iraq
29
🇦🇬 Antigua and Barbuda
28
🇴🇲 Oman
26
🇧🇭 Bahrain
26
🇲🇬 Madagascar
24
🇱🇾 Libya
24
🇸🇻 El Salvador
23
🇨🇩 Congo
23
🇨🇲 Cameroon
22
🇧🇯 Benin
22
🇭🇳 Honduras
22
🇲🇿 Mozambique
22
🇲🇶 Martinique
22
🇲🇲 Myanmar
21
🇦🇩 Andorra
20
🇦🇴 Angola
20
🇰🇬 Kyrgyzstan
19
🇹🇬 Togo
19
🇸🇹 Sao Tome and Principe
18
🇧🇲 Bermuda
17
🇧🇫 Burkina Faso
17
🇲🇨 Monaco
17
🇵🇲 Saint Pierre and Miquelon
16
🇲🇳 Mongolia
16
🇹🇫 French Southern Territories
16
🇸🇲 San Marino
15
🇸🇷 Suriname
15
🇵🇬 Papua New Guinea
14
🇯🇲 Jamaica
14
🇿🇲 Zambia
14
🇹🇹 Trinidad and Tobago
14
🇵🇫 French Polynesia
14
🇻🇬 British Virgin Islands
13
🇭🇹 Haiti
13
🇧🇧 Barbados
13
🇲🇷 Mauritania
13
🇦🇸 American Samoa
12
🇲🇸 Montserrat
12
🇸🇩 Sudan
12
🇵🇸 Palestine
11
🇻🇮 United States Virgin Islands
11
🇧🇼 Botswana
11
🇦🇼 Aruba
11
🇲🇻 Maldives
11
🇰🇾 Cayman Islands
11
🇫🇴 Faroe Islands
11
🇲🇼 Malawi
10
🇹🇯 Tajikistan
10
🇨🇻 Cabo Verde
10
🇸🇱 Sierra Leone
10
🇳🇮 Nicaragua
9
🇾🇹 Mayotte
9
🇬🇫 French Guiana
9
🇬🇾 Guyana
9
🇲🇴 Macao
9
🇧🇮 Burundi
8
🇧🇹 Bhutan
8
🇫🇯 Fiji
8
🇲🇱 Mali
8
🇦🇽 Åland Islands
8
🇬🇩 Grenada
7
🇸🇽 Sint Maarten
7
🇨🇺 Cuba
7
🇻🇺 Vanuatu
7
🇹🇨 Turks and Caicos Islands
7
🇬🇮 Gibraltar
6
🇩🇯 Djibouti
6
🇼🇫 Wallis and Futuna
6
🇸🇿 Swaziland
6
🇬🇲 Gambia
5
🇬🇱 Greenland
5
🇹🇩 Chad
5
🇯🇪 Jersey
5
🇾🇪 Yemen
5
🇬🇺 Guam
5
🇨🇬 Congo
4
🇱🇷 Liberia
4
🇱🇨 Saint Lucia
4
🇨🇼 Curacao
3
🇹🇲 Turkmenistan
3
🇬🇳 Guinea
3
🇧🇳 Brunei
3
🇸🇧 Solomon Islands
3
🇹🇱 East Timor
3
🇲🇵 Northern Mariana Islands
3
🇳🇪 Niger
3
🇱🇸 Lesotho
2
🇸🇸 South Sudan
2
🇩🇲 Dominica
2
🇬🇼 Guinea-Bissau
1
🇭🇲 Heard Island and McDonald Islands
1
🇺🇲 United States Minor Outlying Islands
1
🇬🇦 Gabon
1
🇻🇦 Vatican City
1
🇰🇳 Saint Kitts and Nevis
1
🇸🇾 Syria
1
Live Region Totals
North America
756,480
Europe
426,191
Western Europe
168,131
East Asia
150,585
Central Europe
108,466
Southern Europe
46,842
CHAPS
25,753
B Benelux Union
24,833
South America
20,808
Northern Europe
18,274
N Nordics
15,943
Scandinavia
13,882
Balkan States
11,851
Western Asia
7,756
Middle East
7,564
South Asia
6,636
Levant Narrow
4,266
Africa
3,627
Southern Cone
3,340
Sub-Saharan Africa
3,139
Baltic States
1,564
Southern Africa
1,533
East Africa
665
Central Asia
644
Western Africa
607
Northern Africa
447
Central America
354
Central Africa
155
Live Financial Totals
AUKUS
783,867
AMER
777,574
EMEA
434,354
B BRICS
297,495
BRIC
296,010
APAC
184,622
MART
174,374
ASEAN+6
171,357
ASEAN+3
156,042
BASIC
134,192
ASIAN TIGERS
127,872
FRITES
61,626
CANZUK
58,046
ASEAN+5
54,438
FRIT
50,024
ANZUK
43,544
PIGS
42,271
M MIKTA
25,087
LATCAR
24,670
LATAM
23,927
IMEA
16,481
CIVETS
12,530
MEA
11,071
SEA
10,509
VISTA
10,477
ANZ
9,905
Tiger Cub Economies
9,401
MINT
8,474
MEP
8,245
MENA
7,891
CLMV
1,949
Live Group Totals
G G20 Nations
1,258,895
Club de Paris
1,189,222
Asia-Pacific Economic Cooperation
1,094,830
O Organisation for Economic Co-operation and Development
1,057,451
D Development Assistance Committee
1,044,226
N North Atlantic Treaty Organization
995,416
G7 Nations
934,318
I International Solar Alliance
915,568
G2 Nations
858,897
O Organization of American States
780,514
Sovereign states and dependent territories of North America
760,477
C Council of Europe
425,503
A Assembly of European Regions
326,690
S Shanghai Cooperation Organisation
286,954
E European Union, European Economic Area and Switzerland
220,881
E European Economic Area
213,165
E Eastern Partnership
212,129
E Euronest Parliamentary Assembly
211,853
E European Union
210,351
I Interparliamentary Assembly on Orthodoxy
195,158
B Organization of the Black Sea Economic Cooperation
172,591
C Commonwealth of Independent States Members
164,367
E Eurasian Economic Union
164,079
G Group of 77
162,404
EC Inner Six
149,119
Majority German-speaking states of Central Europe
85,122
C Commonwealth of Nations
70,369
EC Outer Seven
60,323
O Organization of Ibero-American States
37,417
V Visegrad Group
22,609
3 Global Governance Group
22,327
I IBSA Dialogue Forum
17,273
O Organisation of Islamic Cooperation
12,823
M Southern Common Market
12,424
C Community of Portuguese Language Countries
12,394
F Federation of Euro-Asian Stock Exchanges
12,035
E European Free Trade Association
10,530
A Association of Southeast Asian Nations
10,506
B Bay of Bengal Initiative for Multi-Sectoral Technical and Economic Cooperation
9,448
P Pacific Alliance
8,058
A Association of Caribbean State Members
7,142
A African Union
7,026
S South Asian Association for Regional Cooperation
6,636
E Economic Cooperation Organization
6,269
O Organization of the Petroleum Exporting Countries
3,553
C Central European Free Trade Agreement Members
3,490
T Organization of Turkic States
3,344
S Southern African Development Community
1,851
A Arab League
1,632
G GUAM Organization for Democracy and Economic Development
1,462
G Gulf Cooperation Council
960
O Organisation of Eastern Caribbean States
625
C Central American Integration System
504
E East African Community
380
C Caribbean Community Members
256
Caribbean Free Trade Association
185
Union of South American Nations
136