Log In · Signup for Free

EN DE FR ES IT Pусский 한국어 日本語简体中文 عربي

Some Robots Disallow Usage Statistics

Top 10k

Top 100k

Top 1m

All Internet

Blocks some but not all robots from indexing the website.

https://robotstxt.org

Some Robots Disallow Customers

Get access to data on 59,067,545 websites that are Some Robots Disallow Customers. We know of 39,138,226 live websites using Some Robots Disallow and an additional 19,929,319 sites that used Some Robots Disallow historically and 525,939 websites in the United States.

Download Lead List

Some Robots Disallow Awards

3rd most popular on the Entire Internet in Robots.txt category.
3rd most popular in the Top 10k sites in Robots.txt category.
4th most popular in the Top 100k sites in Robots.txt category.
7th most popular in the Top 1 Million sites in Robots.txt category.
The most popular in Philippines in Robots.txt category.
The most popular in Mayotte in Robots.txt category.
2nd most popular in Iran in Robots.txt category.
3rd most popular in South Korea in Robots.txt category.
3rd most popular in Indonesia in Robots.txt category.

Download Lead List

Get a list of 59,067,545 websites using Some Robots Disallow which includes location information, hosting data, contact details, 39,138,226 currently live websites and an additional 1,021,044 domains that redirect to sites in this list. 19,929,319 sites that used this technology previouslyand 525,939 websites in the United States currently using Some Robots Disallow.

Site Totals

Total Live: 39,138,226
1,021,044 additional website redirects^?.
USA Live Sites: 525,939
Live and Historical: 59,067,545

Top 1m: 3.84%
38,359
Top 100k: 9.03%
9,034
Top 10k: 14.68%
1,468

Countries

Financial

Group

African Union: 76,420
Arab League: 5,149
Asia-Pacific Economic Cooperation: 918,804
Assembly of European Regions: 530,214
Association of Caribbean State Members: 69,191
Association of Southeast Asian Nations: 116,278
Bay of Bengal Initiative for Multi-Sectoral Technical and Economic Cooperation: 47,202
Caribbean Community Members: 3,009
Caribbean Free Trade Association: 2,930
Central American Integration System: 1,463
Central European Free Trade Agreement Members: 25,135
Club de Paris: 1,489,035
Commonwealth of Independent States Members: 100,565
Commonwealth of Nations: 360,300
Community of Portuguese Language Countries: 31,041
Council of Europe: 879,601
Development Assistance Committee: 1,399,026
East African Community: 1,173
Eastern Partnership: 553,646
EC Inner Six: 436,214
EC Outer Seven: 227,414
Economic Cooperation Organization: 10,525
Eurasian Economic Union: 100,000
Euronest Parliamentary Assembly: 553,155
European Economic Area: 554,457
European Free Trade Association: 20,600
European Union: 549,814
European Union, European Economic Area and Switzerland: 570,414
Federation of Euro-Asian Stock Exchanges: 21,555
G2 Nations: 557,513
G20 Nations: 1,450,062
G7 Nations: 1,137,329
Global Governance Group: 150,921
Group of 77: 322,054
GUAM Organization for Democracy and Economic Development: 2,520
Gulf Cooperation Council: 2,341
IBSA Dialogue Forum: 78,364
International Solar Alliance: 1,107,028
Interparliamentary Assembly on Orthodoxy: 163,059
Majority German-speaking states of Central Europe: 348,005
North Atlantic Treaty Organization: 1,298,579
Organisation for Economic Co-operation and Development: 1,473,796
Organisation of Eastern Caribbean States: 34,271
Organisation of Islamic Cooperation: 36,591
Organization of American States: 667,833
Organization of Ibero-American States: 146,543
Organization of the Black Sea Economic Cooperation: 112,711
Organization of the Petroleum Exporting Countries: 8,238
Organization of Turkic States: 3,760
Pacific Alliance: 68,094
Shanghai Cooperation Organisation: 172,636
South Asian Association for Regional Cooperation: 43,797
Southern African Development Community: 11,804
Southern Common Market: 29,738
Sovereign states and dependent territories of North America: 607,942
Union of South American Nations: 547
Visegrad Group: 23,162

Region

Browse Technology Groups