Log In · Signup for Free

EN DE FR ES IT Pусский 한국어 日本語简体中文 عربي

Some Robots Disallow Usage Statistics

Top 10k

Top 100k

Top 1m

All Internet

Blocks some but not all robots from indexing the website.

https://robotstxt.org

Some Robots Disallow Customers

Get access to data on 62,270,899 websites Some Robots Disallow Customers. We know of 39,138,226 live websites using Some Robots Disallow and an additional 23,132,673 sites that used Some Robots Disallow historically and 557,025 websites in the United States.

Download Lead List

Some Robots Disallow Awards

2nd most popular on the Entire Internet in Robots.txt category.
4th most popular in the Top 10k sites in Robots.txt category.
4th most popular in the Top 100k sites in Robots.txt category.
7th most popular in the Top 1 Million sites in Robots.txt category.
The most popular in Philippines in Robots.txt category.
The most popular in Djibouti in Robots.txt category.
3rd most popular in .io in Robots.txt category.
3rd most popular in Christmas Island in Robots.txt category.
3rd most popular in Anguilla in Robots.txt category.

Download Lead List

Get a list of 62,270,899 websites using Some Robots Disallow which includes location information, hosting data, contact details, 39,138,226 currently live websites and an additional 1,044,078 domains that redirect to sites in this list. 23,132,673 sites that used this technology previouslyand 557,025 websites in the United States currently using Some Robots Disallow.

Site Totals

Total Live: 39,138,226
1,044,078 additional website redirects^?.
United States Live Sites: 557,025
Live and Historical: 62,270,899

Top 1m: 4.28%
42,752
Top 100k: 9.1%
9,102
Top 10k: 14.17%
1,417

Countries

Financial

Group

African Union: 78,522
Arab League: 4,483
Asia-Pacific Economic Cooperation: 994,128
Assembly of European Regions: 536,235
Association of Caribbean State Members: 71,230
Association of Southeast Asian Nations: 111,056
Bay of Bengal Initiative for Multi-Sectoral Technical and Economic Cooperation: 43,466
Caribbean Community Members: 2,821
Caribbean Free Trade Association: 2,726
Central American Integration System: 1,470
Central European Free Trade Agreement Members: 25,028
Club de Paris: 1,520,693
Commonwealth of Independent States Members: 142,220
Commonwealth of Nations: 332,736
Community of Portuguese Language Countries: 29,459
Council of Europe: 879,796
Development Assistance Committee: 1,388,265
East African Community: 1,021
Eastern Partnership: 529,030
EC Inner Six: 417,290
EC Outer Seven: 209,538
Economic Cooperation Organization: 6,452
Eurasian Economic Union: 141,642
Euronest Parliamentary Assembly: 528,481
European Economic Area: 529,799
European Free Trade Association: 17,956
European Union: 525,444
European Union, European Economic Area and Switzerland: 543,400
Federation of Euro-Asian Stock Exchanges: 14,980
G2 Nations: 597,549
G20 Nations: 1,502,584
G7 Nations: 1,138,479
Global Governance Group: 141,374
Group of 77: 316,759
GUAM Organization for Democracy and Economic Development: 2,537
Gulf Cooperation Council: 2,036
IBSA Dialogue Forum: 73,526
International Solar Alliance: 1,097,197
Interparliamentary Assembly on Orthodoxy: 204,223
Majority German-speaking states of Central Europe: 338,072
North Atlantic Treaty Organization: 1,285,447
Organisation for Economic Co-operation and Development: 1,465,118
Organisation of Eastern Caribbean States: 24,110
Organisation of Islamic Cooperation: 32,894
Organization of American States: 695,212
Organization of Ibero-American States: 144,871
Organization of the Black Sea Economic Cooperation: 153,173
Organization of the Petroleum Exporting Countries: 3,943
Organization of Turkic States: 3,827
Pacific Alliance: 70,244
Shanghai Cooperation Organisation: 219,465
South Asian Association for Regional Cooperation: 39,669
Southern African Development Community: 11,571
Southern Common Market: 30,299
Sovereign states and dependent territories of North America: 622,525
Union of South American Nations: 510
Visegrad Group: 23,723

Region

Browse Technology Groups