Robots Disallow Usage Statistics

Robots Disallow

A robots disallow all directive with no other options.

https://robotstxt.org

Robots.txt

Robots Disallow Customers

We know of 7,971,308 live websites using Robots Disallow.

We know of at least 7,971,308 live websites using Robots Disallow.

Site Totals

Total Live
7,971,308
2,842,491 additional website redirects?.
USA Live Sites
882,287
Estimated
Top 1m
16.38%
163,830
Top 100k
30.54%
30,537
Top 10k
36.5%
3,650

Countries

United States
882,287*
Russia
157,911*
Germany
109,206*
United Kingdom
98,479*
France
64,510*
Netherlands
49,012*
Brazil
47,076*
Australia
44,525*
Canada
42,282*
Italy
37,784*
Japan
35,563*
India
33,217*
Spain
31,748*
Poland
27,683*
Switzerland
25,337*
China
23,788*
Sweden
20,793*
South Korea
17,092*
.io
16,944*
Indonesia
16,728*
Belgium
16,307*
Austria
15,703*
Denmark
15,600*
Turkey
14,428*
FI Finland
13,676*
CZ Czech Republic
13,551*
NO Norway
12,435*
UA Ukraine
12,423*
PT Portugal
10,704*
CO .co
10,100*
TW Taiwan
8,882*
RO Romania
8,631*
SG Singapore
8,529*
VN Vietnam
8,415*
AR Argentina
7,800*
MX Mexico
7,720*
HU Hungary
7,698*
ZA South Africa
7,698*
NZ New Zealand
7,641*
TH Thailand
6,240*
CL Chile
5,352*
MY Malaysia
5,283*
IE Ireland
5,158*
HK Hong Kong
5,135*
GR Greece
5,021*
ME .me
4,657*
SK Slovakia
4,657*
BG Bulgaria
4,532*
AE UAE
4,384*
IL Israel
3,905*
PH Philippines
3,575*
LT Lithuania
3,006*
CC Cocos Islands
2,926*
AI Anguilla
2,892*
BD Bangladesh
2,892*
HR Croatia
2,778*
KZ Kazakhstan
2,767*
EE Estonia
2,619*
SI Slovenia
2,539*
TV .tv
2,528*
SA Saudi Arabia
2,493*
LV Latvia
2,448*
PK Pakistan
2,425*
PE Peru
2,311*
LU Luxembourg
2,175*
BY Belarus
2,152*
RS Serbia
2,026*
UZ Uzbekistan
1,878*
IR Iran
1,867*
KE Kenya
1,582*
NG Nigeria
1,469*
MT Malta
1,389*
UY Uruguay
1,264*
CY Cyprus
1,229*
NP Nepal
1,184*
EC Ecuador
1,150*
PW Palau
1,127*
MA Morocco
979*
IS Iceland
967*
EG Egypt
956*
LK Sri Lanka
956*
AM Armenia
922*
GE Georgia
842*
TO Tonga
819*
GG Guernsey
785*
CR Costa Rica
740*
MD Moldova
728*
BO Bolivia
683*
RE Reunion
671*
QA Qatar
649*
SH Saint Helena
637*
MK Macedonia
626*
AZ Azerbaijan
614*
KW Kuwait
592*
DO Dominican Republic
569*
VE Venezuela
546*
LI Liechtenstein
489*
AL Albania
489*
KH Cambodia
489*
GH Ghana
489*
TN Tunisia
478*
TZ Tanzania
478*
PY Paraguay
478*
JO Jordan
455*
DZ Algeria
455*
GT Guatemala
421*
WS Samoa
409*
BZ Belize
387*
PA Panama
387*
BA Bosnia
364*
AF Afghanistan
364*
FM Micronesia
364*
NU Niue
364*
KG Kyrgyzstan
353*
LB Lebanon
353*
IM Isle of Man
341*
UG Uganda
341*
PR Puerto Rico
307*
MU Mauritius
296*
OM Oman
296*
IQ Iraq
296*
CX Christmas Island
261*
SO Somalia
261*
BH Bahrain
261*
BS Bahamas
261*
PS Palestine
239*
HN Honduras
239*
ST Sao Tome
227*
CI Cote d'Ivoire
227*
MZ Mozambique
227*
SV El Salvador
227*
ET Ethiopia
227*
NA Namibia
227*
ZW Zimbabwe
216*
MN Mongolia
216*
VC Saint Vincent
204*
SC Seychelles
204*
RW Rwanda
204*
MM Myanmar
204*
AG Antigua
182*
MC Monaco
170*
NC New Caledonia
170*
ZM Zambia
170*
AD Andorra
170*
MS Montserrat
159*
AO Angola
159*
BM Bermuda
159*
JM Jamaica
148*
TT Trinidad
136*
SN Senegal
136*
NI Nicaragua
136*
BW Botswana
136*
CD Congo
125*
GL Greenland
125*
TJ Tajikistan
113*
CM Cameroon
113*
GP Guadeloupe
102*
TM Turkmenistan
102*
GS South Georgia
102*
TC Turks
102*
MV Maldives
102*
MG Madagascar
102*
FO Faroe Islands
102*
BT Bhutan
102*
VI Virgin Islands
91*
GI Gibraltar
91*
MO Macao
91*
BJ Benin
91*
PM Saint Pierre
79*
BF Burkina Faso
79*
HT Haiti
79*
GM Gambia
68*
SX Sint Maarten
68*
FJ Fiji
68*
TG Togo
68*
GY Guyana
68*
GF French Guiana
68*
SM San Marino
68*
LS Lesotho
68*
AW Aruba
56*
PG Papua New Guinea
56*
VG British Virgin Islands
56*
TL East Timor
56*
MW Malawi
56*
SB Solomon Islands
56*

Financial

AMER AMER
996,607
ANZ ANZ
52,166
ANZUK ANZUK
150,646
APAC APAC
240,642
ASEAN3 ASEAN+3
125,946
ASEAN5 ASEAN+5
187,542
ASEAN6 ASEAN+6
211,330
FOURASIANTIGERS ASIAN TIGERS
54,546
AUKUS AUKUS
1,025,292
BASIC BASIC
111,780
BRIC BRIC
261,994
BRICS BRICS
269,692
CANZUK CANZUK
192,928
CIVETS CIVETS
50,822
CLMV CLMV
9,121
EMEA EMEA
850,914
FRIT FRIT
102,294
FRITES FRITES
134,043
IMEA IMEA
85,144
LATAM LATAM
80,316
LATCAR LATCAR
85,236
MART MART
209,838
MEA MEA
51,927
MENA MENA
34,458
MEP MEP
34,880
MIKTA MIKTA
100,495
MINT MINT
40,346
PIGS PIGS
85,258
SEA SEA
49,558
TIGERCUB Tiger Cub Economies
40,243
VISTA VISTA
55,070

Group

AFRICANUNION African Union
21,545
ARABLEAGUE Arab League
13,312
APEC Asia-Pacific Economic Cooperation
1,289,347
AER Assembly of European Regions
629,209
ACS Association of Caribbean State Members
15,282
ASEAN Association of Southeast Asian Nations
49,501
BIMSTEC Bay of Bengal Initiative for Multi-Sectoral Technical and Economic Cooperation
44,798
CARICOM Caribbean Community Members
1,844
CARIFTA Caribbean Free Trade Association
1,469
CAIS Central American Integration System
3,108
CEFTA Central European Free Trade Agreement Members
4,270
PARISCLUB Club de Paris
1,746,399
CIS Commonwealth of Independent States Members
167,443
COMMONWEALTH Commonwealth of Nations
265,968
CPLP Community of Portuguese Language Countries
58,475
COE Council of Europe
809,702
DAC Development Assistance Committee
1,620,145
EAC East African Community
2,801
EAP Eastern Partnership
497,853
INNERSIX EC Inner Six
278,995
OUTERSEVEN EC Outer Seven
199,054
ECO Economic Cooperation Organization
24,916
EAEU Eurasian Economic Union
164,106
EURONEST Euronest Parliamentary Assembly
495,086
EEA European Economic Area
494,061
EFTA European Free Trade Association
39,230
EU European Union
480,168
EUEEACH European Union, European Economic Area and Switzerland
519,399
FEAS Federation of Euro-Asian Stock Exchanges
51,130
G2 G2 Nations
906,075
G20 G20 Nations
1,650,595
G7 G7 Nations
1,270,114
3G Global Governance Group
95,257
G77 Group of 77
222,183
GUAM GUAM Organization for Democracy and Economic Development
14,610
GCC Gulf Cooperation Council
8,677
IBSA IBSA Dialogue Forum
87,991
ISA International Solar Alliance
1,411,024
IAO Interparliamentary Assembly on Orthodoxy
267,357
DACH Majority German-speaking states of Central Europe
150,247
NATO North Atlantic Treaty Organization
1,474,247
OECD Organisation for Economic Co-operation and Development
1,662,962
OECS Organisation of Eastern Caribbean States
3,780
OIC Organisation of Islamic Cooperation
66,856
OAS Organization of American States
1,005,956
OEI Organization of Ibero-American States
122,644
BSEC Organization of the Black Sea Economic Cooperation
206,547
OPEC Organization of the Petroleum Exporting Countries
12,309
TURKIC Organization of Turkic States
20,042
PACIFICALLIANCE Pacific Alliance
17,980
SCO Shanghai Cooperation Organisation
222,456
SAARC South Asian Association for Regional Cooperation
41,245
SADC Southern African Development Community
10,203
MERCOSUR Southern Common Market
56,619
NORAM Sovereign states and dependent territories of North America
940,842
USAN Union of South American Nations
1,332
V4 Visegrad Group
53,589

Region

AFRICA Africa
20,429
BALKANS Balkan States
41,473
BALTICS Baltic States
8,073
BENELUX Benelux Union
67,494
CENTRALAFRICA Central Africa
899
CENTRALAMERICA Central America
2,539
CENTRALASIA Central Asia
5,215
CENTRALEUROPE Central Europe
206,866
CHAPS CHAPS
69,293
EASTAFRICA East Africa
4,520
EASTASIA East Asia
90,770
EUROPE Europe
814,644
LEVANT Levant Narrow
20,634
MIDDLEEAST Middle East
32,454
NORDICS Nordics
63,747
CUSA North America
924,569
NORTHAFRICA Northern Africa
2,960
NORTHERNEUROPE Northern Europe
76,706
SCANDINAVIA Scandinavia
48,829
SOUTHAMERICA South America
69,885
SOUTHASIA South Asia
41,245
SOUTHERNAFRICA Southern Africa
8,164
SOUTHERNCONE Southern Cone
14,894
SOUTHERNEUROPE Southern Europe
95,837
SUBSAHARA Sub-Saharan Africa
17,206
WESTAFRICA Western Africa
2,835
WESTASIA Western Asia
34,834
WESTERNEUROPE Western Europe
386,722
Browse Technology Groups