Some Robots Disallow Usage Statistics

Some Robots Disallow

Blocks some but not all robots from indexing the website.

https://robotstxt.org

Robots.txt

Some Robots Disallow Customers

Get access to data on 60,830,604 websites that are Some Robots Disallow Customers. We know of 39,138,226 live websites using Some Robots Disallow and an additional 21,692,378 sites that used Some Robots Disallow historically and 596,988 websites in the United States.

Get a list of 60,830,604 websites using Some Robots Disallow which includes location information, hosting data, contact details, 39,138,226 currently live websites and an additional 1,058,844 domains that redirect to sites in this list. 21,692,378 sites that used this technology previouslyand 596,988 websites in the United States currently using Some Robots Disallow.

Site Totals

Total Live
39,138,226
1,058,844 additional website redirects?.
USA Live Sites
596,988
Live and Historical
60,830,604
Top 1m
4.13%
41,316
Top 100k
9.47%
9,473
Top 10k
15.05%
1,505

Countries

United States
596,988
Germany
323,468
United Kingdom
180,511
Russia
103,776
Philippines
88,594
Australia
60,799
.co
60,609
Netherlands
46,433
Spain
43,580
India
41,971
Canada
41,211
France
36,585
China
36,054
.io
33,886
Cocos Islands
30,053
Brazil
27,584
Italy
24,955
.me
24,334
Anguilla
21,988
South Korea
19,191
Switzerland
16,000
Indonesia
15,924
Japan
15,673
Austria
14,761
PL Poland
14,643
ZA South Africa
11,139
BE Belgium
10,777
VG British Virgin Islands
9,028
TV .tv
8,043
SE Sweden
7,241
MX Mexico
6,388
WS Samoa
5,899
TH Thailand
5,473
IR Iran
5,162
DK Denmark
4,581
PT Portugal
4,226
RO Romania
4,137
FI Finland
4,094
CZ Czech Republic
4,073
TR Turkey
3,475
SK Slovakia
3,363
VN Vietnam
3,172
AR Argentina
3,072
GR Greece
2,707
NO Norway
2,678
MY Malaysia
2,622
TW Taiwan
2,532
NZ New Zealand
2,289
CL Chile
2,201
PW Palau
2,166
SH Saint Helena
2,113
HU Hungary
2,073
TO Tonga
2,036
GG Guernsey
2,016
EE Estonia
1,950
UA Ukraine
1,930
VC Saint Vincent
1,766
IM Isle of Man
1,710
HK Hong Kong
1,691
PK Pakistan
1,682
SI Slovenia
1,629
SG Singapore
1,618
AE UAE
1,531
RE Reunion
1,330
SO Somalia
1,266
IS Iceland
1,245
IE Ireland
1,217
BD Bangladesh
1,034
CM Cameroon
1,032
LV Latvia
1,030
AM Armenia
922
PE Peru
899
NG Nigeria
894
LI Liechtenstein
881
RS Serbia
794
CX Christmas Island
785
IL Israel
723
BG Bulgaria
721
KE Kenya
683
LU Luxembourg
679
FM Micronesia
646
LT Lithuania
620
NU Niue
615
BZ Belize
591
EG Egypt
462
BY Belarus
427
SA Saudi Arabia
422
GE Georgia
421
EC Ecuador
405
AL Albania
398
HR Croatia
390
NP Nepal
365
KZ Kazakhstan
329
MA Morocco
323
VE Venezuela
323
AD Andorra
321
QA Qatar
314
LK Sri Lanka
296
CR Costa Rica
291
GH Ghana
280
YT Mayotte
272
CY Cyprus
251
ST Sao Tome
239
MD Moldova
232
UY Uruguay
229
MT Malta
224
UG Uganda
204
AG Antigua
198
DO Dominican Republic
187
LB Lebanon
187
MN Mongolia
169
UZ Uzbekistan
160
MS Montserrat
160
TZ Tanzania
158
PA Panama
156
JO Jordan
149
GT Guatemala
135
PR Puerto Rico
130
PM Saint Pierre
120
DZ Algeria
111
BO Bolivia
110
SC Seychelles
110
DJ Djibouti
110
MK Macedonia
102
TN Tunisia
97
KH Cambodia
93
IQ Iraq
92
TJ Tajikistan
88
AZ Azerbaijan
87
KW Kuwait
86
OM Oman
86
BH Bahrain
84
GP Guadeloupe
81
CI Cote d'Ivoire
80
AF Afghanistan
79
GS South Georgia
76
ET Ethiopia
75
HN Honduras
71
SV El Salvador
70
BA Bosnia
67
NI Nicaragua
66
PY Paraguay
60
PS Palestine
57
NA Namibia
55
SX Sint Maarten
54
GL Greenland
53
NC New Caledonia
52
MU Mauritius
51
MM Myanmar
47
TC Turks
47
LC Saint Lucia
47
MV Maldives
44
MC Monaco
44
SN Senegal
43
MQ Martinique
42
FO Faroe Islands
40
GD Grenada
40
ZM Zambia
40
TG Togo
38
BS Bahamas
37
JM Jamaica
36
VI Virgin Islands
35
KG Kyrgyzstan
32
JE Jersey
31
TT Trinidad
27
HT Haiti
26
BM Bermuda
25
BW Botswana
24
GF French Guiana
23
TL East Timor
23
BB Barbados
21
SY Syria
21
MW Malawi
20
AW Aruba
20
KY Cayman Islands
16
FJ Fiji
14
ML Mali
14
BI Burundi
12
PG Papua New Guinea
10
VU Vanuatu
9
SM San Marino
8
PF French Polynesia
8
LY Libya
8
MH Marshall Islands
5

Financial

AMER AMER
735,204
ANZ ANZ
63,088
ANZUK ANZUK
243,599
APAC APAC
385,822
ASEAN3 ASEAN+3
188,472
ASEAN5 ASEAN+5
257,477
ASEAN6 ASEAN+6
293,531
FOURASIANTIGERS ASIAN TIGERS
58,554
AUKUS AUKUS
838,298
BASIC BASIC
116,748
BRIC BRIC
209,385
BRICS BRICS
220,524
CANZUK CANZUK
284,810
CIVETS CIVETS
94,781
CLMV CLMV
3,318
EMEA EMEA
930,152
FRIT FRIT
61,540
FRITES FRITES
105,120
IMEA IMEA
76,456
LATAM LATAM
103,167
LATCAR LATCAR
137,235
MART MART
138,261
MEA MEA
34,485
MENA MENA
13,734
MEP MEP
14,816
MIKTA MIKTA
105,777
MINT MINT
26,681
PIGS PIGS
75,468
SEA SEA
117,577
TIGERCUB Tiger Cub Economies
115,785
VISTA VISTA
36,782

Group

AFRICANUNION African Union
78,650
ARABLEAGUE Arab League
5,480
APEC Asia-Pacific Economic Cooperation
1,007,110
AER Assembly of European Regions
542,169
ACS Association of Caribbean State Members
71,210
ASEAN Association of Southeast Asian Nations
117,554
BIMSTEC Bay of Bengal Initiative for Multi-Sectoral Technical and Economic Cooperation
49,207
CARICOM Caribbean Community Members
3,065
CARIFTA Caribbean Free Trade Association
2,984
CAIS Central American Integration System
1,567
CEFTA Central European Free Trade Agreement Members
25,934
PARISCLUB Club de Paris
1,582,826
CIS Commonwealth of Independent States Members
106,053
COMMONWEALTH Commonwealth of Nations
368,337
CPLP Community of Portuguese Language Countries
32,128
COE Council of Europe
898,634
DAC Development Assistance Committee
1,487,670
EAC East African Community
1,272
EAP Eastern Partnership
564,427
INNERSIX EC Inner Six
442,897
OUTERSEVEN EC Outer Seven
229,998
ECO Economic Cooperation Organization
11,105
EAEU Eurasian Economic Union
105,486
EURONEST Euronest Parliamentary Assembly
563,913
EEA European Economic Area
565,212
EFTA European Free Trade Association
20,804
EU European Union
560,408
EUEEACH European Union, European Economic Area and Switzerland
581,212
FEAS Federation of Euro-Asian Stock Exchanges
22,320
G2 G2 Nations
633,042
G20 G20 Nations
1,549,186
G7 G7 Nations
1,219,391
3G Global Governance Group
152,215
G77 Group of 77
333,811
GUAM GUAM Organization for Democracy and Economic Development
2,670
GCC Gulf Cooperation Council
2,523
IBSA IBSA Dialogue Forum
80,694
ISA International Solar Alliance
1,187,940
IAO Interparliamentary Assembly on Orthodoxy
171,152
DACH Majority German-speaking states of Central Europe
354,229
NATO North Atlantic Treaty Organization
1,383,562
OECD Organisation for Economic Co-operation and Development
1,564,957
OECS Organisation of Eastern Caribbean States
33,360
OIC Organisation of Islamic Cooperation
39,141
OAS Organization of American States
743,960
OEI Organization of Ibero-American States
151,123
BSEC Organization of the Black Sea Economic Cooperation
118,806
OPEC Organization of the Petroleum Exporting Countries
8,667
TURKIC Organization of Turkic States
4,083
PACIFICALLIANCE Pacific Alliance
70,097
SCO Shanghai Cooperation Organisation
184,092
SAARC South Asian Association for Regional Cooperation
45,492
SADC Southern African Development Community
11,792
MERCOSUR Southern Common Market
30,945
NORAM Sovereign states and dependent territories of North America
680,117
USAN Union of South American Nations
539
V4 Visegrad Group
24,152

Region

AFRICA Africa
21,813
BALKANS Balkan States
38,761
BALTICS Baltic States
3,600
BENELUX Benelux Union
57,889
CENTRALAFRICA Central Africa
1,543
CENTRALAMERICA Central America
1,380
CENTRALASIA Central Asia
620
CENTRALEUROPE Central Europe
380,891
CHAPS CHAPS
38,913
EASTAFRICA East Africa
3,030
EASTASIA East Asia
75,313
EUROPE Europe
899,393
LEVANT Levant Narrow
4,863
MIDDLEEAST Middle East
13,134
NORDICS Nordics
19,939
CUSA North America
638,199
NORTHAFRICA Northern Africa
1,062
NORTHERNEUROPE Northern Europe
24,656
SCANDINAVIA Scandinavia
14,500
SOUTHAMERICA South America
95,781
SOUTHASIA South Asia
45,492
SOUTHERNAFRICA Southern Africa
11,225
SOUTHERNCONE Southern Cone
5,562
SOUTHERNEUROPE Southern Europe
103,754
SUBSAHARA Sub-Saharan Africa
19,506
WESTAFRICA Western Africa
1,469
WESTASIA Western Asia
14,564
WESTERNEUROPE Western Europe
631,677
Browse Technology Groups