Some Robots Disallow Usage Statistics

Some Robots Disallow

Blocks some but not all robots from indexing the website.

https://robotstxt.org

Robots.txt

Some Robots Disallow Customers

Get access to data on 66,639,436 websites Some Robots Disallow Customers. We know of 39,195,108 live websites using Some Robots Disallow and an additional 27,444,328 sites that used Some Robots Disallow historically and 1,258,211 websites in the United States.

Get a list of 66,639,436 websites using Some Robots Disallow which includes location information, hosting data, contact details, 39,195,108 currently live websites and an additional 393,589 domains that redirect to sites in this list. 27,444,328 sites that used this technology previouslyand 1,258,211 websites in the United States currently using Some Robots Disallow.

Site Totals

Total Live
39,195,108
393,589 additional website redirects?.
United States Live Sites
1,258,211
Live and Historical
66,639,436
Top 1m
2.62%
26,170
Top 100k
4.55%
4,551
Top 10k
7.03%
703

Countries

United Kingdom
1,664,025
United States
1,258,211
.co
390,398
.io
380,480
India
364,773
Canada
363,510
Australia
131,658
.me
112,488
Germany
84,611
Philippines
80,602
Cocos Islands
65,809
China
65,603
Mexico
59,674
.tv
54,821
Italy
54,799
Spain
54,323
Brazil
43,942
Switzerland
35,259
Netherlands
31,404
France
30,840
Russia
29,799
South Korea
23,221
Sweden
22,026
Anguilla
16,978
ZA South Africa
14,677
PL Poland
12,383
BE Belgium
11,706
ID Indonesia
10,949
NZ New Zealand
10,682
TW Taiwan
10,154
JP Japan
9,232
VG British Virgin Islands
8,519
TR Turkey
8,248
AT Austria
7,876
CZ Czech Republic
7,434
DK Denmark
7,278
TH Thailand
7,232
WS Samoa
6,775
SG Singapore
5,090
AE UAE
4,889
PW Palau
4,552
VN Vietnam
4,246
PE Peru
3,943
HK Hong Kong
3,498
AR Argentina
3,374
PK Pakistan
3,180
GG Guernsey
3,036
NO Norway
2,970
PT Portugal
2,635
CL Chile
2,616
MY Malaysia
2,446
GR Greece
2,127
IE Ireland
2,077
BZ Belize
1,983
RO Romania
1,954
SK Slovakia
1,853
FI Finland
1,727
UA Ukraine
1,662
HU Hungary
1,607
SH Saint Helena
1,453
IL Israel
1,428
EG Egypt
1,384
SA Saudi Arabia
1,329
FM Micronesia
1,265
SO Somalia
1,248
BD Bangladesh
1,245
KY Cayman Islands
1,218
BG Bulgaria
1,145
NG Nigeria
1,034
AG Antigua
1,018
CR Costa Rica
990
AM Armenia
820
IM Isle of Man
776
SI Slovenia
773
EC Ecuador
711
LK Sri Lanka
650
LC Saint Lucia
649
MS Montserrat
641
KE Kenya
617
LB Lebanon
587
VE Venezuela
583
QA Qatar
567
NP Nepal
566
PR Puerto Rico
560
NU Niue
554
GH Ghana
553
MT Malta
526
KW Kuwait
471
LV Latvia
460
GT Guatemala
456
PA Panama
454
MA Morocco
444
IR Iran
438
JO Jordan
430
HR Croatia
393
LT Lithuania
381
DO Dominican Republic
374
UY Uruguay
352
TO Tonga
351
UG Uganda
335
CY Cyprus
334
EE Estonia
331
LU Luxembourg
317
OM Oman
308
RS Serbia
294
BY Belarus
266
TZ Tanzania
255
GE Georgia
222
LI Liechtenstein
217
KH Cambodia
213
AF Afghanistan
208
TC Turks
191
BO Bolivia
185
SV El Salvador
183
IS Iceland
160
HN Honduras
154
RE Reunion
148
NA Namibia
140
TT Trinidad
140
AZ Azerbaijan
136
MU Mauritius
130
BS Bahamas
130
NI Nicaragua
130
KZ Kazakhstan
129
ZW Zimbabwe
122
DZ Algeria
107
MM Myanmar
106
MN Mongolia
100
TN Tunisia
98
SC Seychelles
97
MD Moldova
92
ZM Zambia
90
MV Maldives
90
NC New Caledonia
82
BM Bermuda
67
PM Saint Pierre
58
GP Guadeloupe
57
GD Grenada
56
UZ Uzbekistan
55
MC Monaco
53
MW Malawi
51
CI Cote d'Ivoire
50
PS Palestine
44
GY Guyana
44
ST Sao Tome
40
AW Aruba
40
TG Togo
40
FJ Fiji
37
GM Gambia
36
MG Madagascar
31
KG Kyrgyzstan
21
GL Greenland
19
SM San Marino
12
CK Cook Islands
7
TL East Timor
6
TJ Tajikistan
5

Financial

AMER AMER
2,072,523
ANZ ANZ
142,340
ANZUK ANZUK
1,806,365
APAC APAC
1,250,919
ASEAN3 ASEAN+3
208,960
ASEAN5 ASEAN+5
650,470
ASEAN6 ASEAN+6
716,073
FOURASIANTIGERS ASIAN TIGERS
97,412
AUKUS AUKUS
3,053,894
BASIC BASIC
488,995
BRIC BRIC
504,117
BRICS BRICS
518,794
CANZUK CANZUK
2,169,875
CIVETS CIVETS
429,902
CLMV CLMV
4,569
EMEA EMEA
2,236,749
FRIT FRIT
85,639
FRITES FRITES
139,962
IMEA IMEA
409,478
LATAM LATAM
509,366
LATCAR LATCAR
543,342
MART MART
74,504
MEA MEA
44,705
MENA MENA
21,735
MEP MEP
24,197
MIKTA MIKTA
233,750
MINT MINT
79,905
PIGS PIGS
113,884
SEA SEA
110,910
TIGERCUB Tiger Cub Economies
105,475
VISTA VISTA
41,494

Group

AFRICANUNION African Union
413,760
ARABLEAGUE Arab League
12,526
APEC Asia-Pacific Economic Cooperation
2,082,401
AER Assembly of European Regions
2,087,743
ACS Association of Caribbean State Members
459,777
ASEAN Association of Southeast Asian Nations
110,904
BIMSTEC Bay of Bengal Initiative for Multi-Sectoral Technical and Economic Cooperation
374,608
CARICOM Caribbean Community Members
7,005
CARIFTA Caribbean Free Trade Association
6,788
CAIS Central American Integration System
4,724
CEFTA Central European Free Trade Agreement Members
113,249
PARISCLUB Club de Paris
3,871,922
CIS Commonwealth of Independent States Members
31,323
COMMONWEALTH Commonwealth of Nations
2,635,671
CPLP Community of Portuguese Language Countries
46,736
COE Council of Europe
2,200,229
DAC Development Assistance Committee
3,836,724
EAC East African Community
1,366
EAP Eastern Partnership
346,518
INNERSIX EC Inner Six
213,677
OUTERSEVEN EC Outer Seven
1,742,069
ECO Economic Cooperation Organization
12,485
EAEU Eurasian Economic Union
31,035
EURONEST Euronest Parliamentary Assembly
346,116
EEA European Economic Area
346,667
EFTA European Free Trade Association
38,606
EU European Union
343,320
EUEEACH European Union, European Economic Area and Switzerland
381,926
FEAS Federation of Euro-Asian Stock Exchanges
19,026
G2 G2 Nations
1,323,814
G20 G20 Nations
4,222,475
G7 G7 Nations
3,465,228
3G Global Governance Group
269,503
G77 Group of 77
1,044,482
GUAM GUAM Organization for Democracy and Economic Development
2,112
GCC Gulf Cooperation Council
7,861
IBSA IBSA Dialogue Forum
423,392
ISA International Solar Alliance
3,803,242
IAO Interparliamentary Assembly on Orthodoxy
176,105
DACH Majority German-speaking states of Central Europe
127,746
NATO North Atlantic Treaty Organization
3,718,670
OECD Organisation for Economic Co-operation and Development
4,301,250
OECS Organisation of Eastern Caribbean States
29,965
OIC Organisation of Islamic Cooperation
42,749
OAS Organization of American States
2,136,733
OEI Organization of Ibero-American States
566,279
BSEC Organization of the Black Sea Economic Cooperation
46,409
OPEC Organization of the Petroleum Exporting Countries
9,135
TURKIC Organization of Turkic States
8,589
PACIFICALLIANCE Pacific Alliance
456,631
SCO Shanghai Cooperation Organisation
463,565
SAARC South Asian Association for Regional Cooperation
370,748
SADC Southern African Development Community
15,830
MERCOSUR Southern Common Market
47,797
NORAM Sovereign states and dependent territories of North America
1,718,966
USAN Union of South American Nations
847
V4 Visegrad Group
23,277

Region

AFRICA Africa
25,072
BALKANS Balkan States
127,797
BALTICS Baltic States
1,172
BENELUX Benelux Union
43,427
CENTRALAFRICA Central Africa
1,219
CENTRALAMERICA Central America
4,350
CENTRALASIA Central Asia
275
CENTRALEUROPE Central Europe
152,013
CHAPS CHAPS
31,153
EASTAFRICA East Africa
3,407
EASTASIA East Asia
111,812
EUROPE Europe
2,200,626
LEVANT Levant Narrow
11,100
MIDDLEEAST Middle East
21,017
NORDICS Nordics
34,233
CUSA North America
1,621,721
NORTHAFRICA Northern Africa
2,102
NORTHERNEUROPE Northern Europe
37,410
SCANDINAVIA Scandinavia
32,274
SOUTHAMERICA South America
446,906
SOUTHASIA South Asia
370,748
SOUTHERNAFRICA Southern Africa
14,873
SOUTHERNCONE Southern Cone
6,471
SOUTHERNEUROPE Southern Europe
228,892
SUBSAHARA Sub-Saharan Africa
21,728
WESTAFRICA Western Africa
1,984
WESTASIA Western Asia
22,195
WESTERNEUROPE Western Europe
1,868,477
Browse Technology Groups