Log In · Signup for Free

EN DE FR ES IT Pусский 한국어 日本語简体中文 عربي

Some Robots Disallow Usage Statistics

Top 10k

Top 100k

Top 1m

All Internet

Blocks some but not all robots from indexing the website.

https://robotstxt.org

Some Robots Disallow Customers

Get access to data on 57,432,087 websites that are Some Robots Disallow Customers. We know of 39,138,226 live websites using Some Robots Disallow and an additional 18,293,861 sites that used Some Robots Disallow historically and 578,482 websites in the United States.

Download Lead List

Some Robots Disallow Awards

3rd most popular on the Entire Internet in Robots.txt category.
3rd most popular in the Top 10k sites in Robots.txt category.
4th most popular in the Top 100k sites in Robots.txt category.
7th most popular in the Top 1 Million sites in Robots.txt category.
The most popular in Philippines in Robots.txt category.
The most popular in Mayotte in Robots.txt category.
2nd most popular in Iran in Robots.txt category.
3rd most popular in South Korea in Robots.txt category.
4th most popular in .io in Robots.txt category.

Download Lead List

Get a list of 57,432,087 websites using Some Robots Disallow which includes location information, hosting data, contact details, 39,138,226 currently live websites and an additional 1,071,424 domains that redirect to sites in this list. 18,293,861 sites that used this technology previouslyand 578,482 websites in the United States currently using Some Robots Disallow.

Site Totals

Total Live: 39,138,226
1,071,424 additional website redirects^?.
USA Live Sites: 578,482
Live and Historical: 57,432,087

Top 1m: 3.98%
39,785
Top 100k: 9.24%
9,235
Top 10k: 15.02%
1,502

Countries

Financial

Group

African Union: 75,711
Arab League: 5,155
Asia-Pacific Economic Cooperation: 967,810
Assembly of European Regions: 531,016
Association of Caribbean State Members: 68,666
Association of Southeast Asian Nations: 116,136
Bay of Bengal Initiative for Multi-Sectoral Technical and Economic Cooperation: 47,123
Caribbean Community Members: 3,125
Caribbean Free Trade Association: 3,046
Central American Integration System: 1,509
Central European Free Trade Agreement Members: 25,046
Club de Paris: 1,544,368
Commonwealth of Independent States Members: 99,408
Commonwealth of Nations: 362,137
Community of Portuguese Language Countries: 30,320
Council of Europe: 883,360
Development Assistance Committee: 1,455,843
East African Community: 1,132
Eastern Partnership: 555,643
EC Inner Six: 438,682
EC Outer Seven: 230,441
Economic Cooperation Organization: 10,402
Eurasian Economic Union: 98,848
Euronest Parliamentary Assembly: 555,171
European Economic Area: 556,538
European Free Trade Association: 21,138
European Union: 551,873
European Union, European Economic Area and Switzerland: 573,011
Federation of Euro-Asian Stock Exchanges: 21,522
G2 Nations: 608,556
G20 Nations: 1,502,939
G7 Nations: 1,193,847
Global Governance Group: 151,368
Group of 77: 318,627
GUAM Organization for Democracy and Economic Development: 2,465
Gulf Cooperation Council: 2,350
IBSA Dialogue Forum: 77,639
International Solar Alliance: 1,159,868
Interparliamentary Assembly on Orthodoxy: 161,216
Majority German-speaking states of Central Europe: 351,857
North Atlantic Treaty Organization: 1,355,059
Organisation for Economic Co-operation and Development: 1,529,915
Organisation of Eastern Caribbean States: 34,813
Organisation of Islamic Cooperation: 36,019
Organization of American States: 718,519
Organization of Ibero-American States: 145,185
Organization of the Black Sea Economic Cooperation: 111,445
Organization of the Petroleum Exporting Countries: 8,141
Organization of Turkic States: 3,685
Pacific Alliance: 67,451
Shanghai Cooperation Organisation: 169,913
South Asian Association for Regional Cooperation: 43,673
Southern African Development Community: 11,864
Southern Common Market: 28,968
Sovereign states and dependent territories of North America: 660,554
Union of South American Nations: 547
Visegrad Group: 22,721

Region

Browse Technology Groups