This article is about one of your first lines of defence – using an Access Control List (e.g. htaccess on Apache sites) to block unwanted visitors/bots from your site, by country IP location. Part 2 will look at blocking by User Agent.
Assumptions: you know what an IP address is, you are able to edit .htaccess, or have relevant role/permissions on IIS etc to deny IP addresses.
- Background
- Is blocking by country effective?
- Does blocking slow your website down?
- Are there any other disadvantages?
- Which countries should I block?
- Websites listing country IP ranges, and providing code to block them:
Background
Spam: enough said. Hacking: Even new and low visability sites attract hackers. If you regularly check your access logs, chances are you will see series of 404 not founds for similar sounding filenames in a variety of directories. Yep its another script kiddy trying to find vulnerabilities in your website using “off the shelf” black hat programs. They are not fussy, even if you have a static site without database or personal data it WILL be a target. In addition to the security risk, your bandwidth allowance is being used up.
Blocking “visitors” from specific countries – is it effective?
I remember one US ISP blocking all emails from the UK because of one spam email, and a technician who didn’t understand “top level” domains. So I don’t usually adopt the scatter gun approach; however ……
some countries are the origin of more bad than others; so if your aren’t interested in visitors from these places consider blocking them by IP location.
Country of origin can be “faked” and I expected blocking by IP location would only have a small impact on unwanted visits. I was totaly wrong, I blocked 4 countries on my first site and the volume of malicious visitors fell by 2/3rds (guesstimate).
So, it should reduce spam submission and hacking attempts; but you will still need to implement other decent security and spam prevention measures.
Does blocking slow your website down?
Probably not; and if you are subject to high volumes of spam/hacking/DOS from certain countries then it may actually improve speed. On various sites I’ve blocked 3 to 5 countries without any speed degradation. I suspect you can block many more without impact.
Bear in mind that Countries may have many non-consecutive IP ranges e.g. adding France to your ACL/htaccess will increase its size by about 85kb – a lot more file for your server to interpret. To minimize size use a list of allowed Country IP ranges if you want to deny most countries, or a list of denied countries if you want to allow most.
Are there any other disadvantages?
The IP details you obtain for blocking will probably be xx days old so may not be completely up to date; and IP ranges allocated to countries change (slowly) over time. I haven’t checked the rules, but I guess it is possible that a tiny fraction of the IP addresses you block may be re-allocated to countries you do want visitors from. An annual review of countries to block and with a refresh your blocking list is advisable.
Issues of completeness and freshness are even more important if you are going to use IP ranges to determine which visitors to allow rather than deny – you may end up disallowing some legitimate visitors.
Another point to remember is that blocking by country IPs will obviously block ALL traffic from these countries, so you won’t be crawled by their local search engines etc but this probably doesn’t matter if your site is of no legitimate interest to people from these locations.
Which countries should I block?
First check your logs (404 not founds etc), much of your unwanted trafffic may be from one or two countries. Ditto the IP address of spam comments.
You could also Google “the top countries for spam” etc. Freshness is important, a list from 3 years ago may not reflect the current sitution.
Countryipblocks.net publish top 10 (black)lists with reasonable regularity. They sometimes change page URL’s but you should be able to find these rankings in their list of articles and news.
The US and UK often appear in these lists, these are my key audience so I don’t block them (and I guess it will be the same for many of you).
Websites listing country IP ranges and providing code to block them:
When I first did this I spent ages trying to find a site with reasonably up to date listings of IP by country that were in a copyable form. I then had to spend time editing the copy to insert the relevant block command for each IP address range.
There are now user friendly sites (detailed below) that do much of the work for you. I’ve not commented on the freshness of data – you can check this on the individual sites.
My favourite is https://www.countryipblocks.net, which in addition to its lists identifying which countries originate most spam (see above), enables you to produce country allow or deny lists in a variety of formats to suit many different web-servers.
You can select a single or multiple countries make (hold down the CTRL key while selecting each country or the SHIFT key to select a range of countries) and choose the format you want.
There are a variety of formats for different systems including htaccess rules for Apache/Nix (just cut and paste the output into your own htaccess, upload, and your done). There is even an “htaccess allow rules format” useful if you have an Apache server and only want say to allow visitors from one or two countries.
ipinfodb.com has a tool to generate a basic CIDR list for one to 20 countries (selected from a list) in one go. It is also possible, via a laborious process, to generate iptable or htaccess rules for single countries.
ip2location.com will generate a list for a single country as CIDR, htaccess allow/deny, or iptables. If you register you can select up to 30 countries in one go.
ipdeny.com single country CIDR (simple list) format
blockacountry.com when I checked: “blockacountry.com is being configured. We’ll be back soon!”.
Author Andy W+
Hi there,
This is an amazing guide!
I have made a little tool that uses ipinfodb.com’s information to allow or deny different ranges to a IIS configuration.
LINK: http://www.ivault.dk/?p=749
Kind regards
Nikolaj Hansen
Hi there. What do you do if you know who’s in your system but don’t know their method? Can one entity be blocked? One person? Thanks. S.
Hi Susan
If “you know who is in your system”, and you commented here I guess you have access to your logs and “you know who” always has the same IP address. Then yes you can block individual IPs.
The follow up to this aticle covers other types of identifier that can also be blocked. e.g. User Agent (if it appears unique to the unwanted “user”).
Examples of blocking individual IPs & User Agents can be found here thesitewizard.com/apache/block-bots-with-htaccess.shtml However, individual checks like these are often ineffective; spammers often hide their real IP behind proxies that allow them to keep coming back with different IPs. User Agents are often spoofed to appear as a normal legitimate browser.
If by “you know who is in your system” you mean a registered user, then all the main CMS/packages like WordPress will provide you with some way of deleting the user – ask at the support forum for your package.
If you are talking about accessing the system without having registered/running scripts you are best off posting a question on a forum like Stackoverflow.com (there is always someone there who knows the answer). Use your logs to find out which scripts they are running – don’t give your site address when posting the qn either.
I installed the iptables country-blocking script from cyberciti recently, blocked just one country and have seen the number of requests on my webserver drop by 90%. That 90% was almost pure WordPress comment spam. From a ‘common good/Internet-is-eighth-wonder-of-the-world’ POV, it feels like a bad thing to do. It’s certainly not secure – all that a country-level block prevents is the lame scripts of the unwashed masses.
The obvious benefit is the loss of 5-20 emails a day announcing comments for moderation that the spam filter missed. We could wish for better spam filters, but I suspect the sheer volume of crap would mean an elevated number of false negatives creeping through.
Like anything else, it has pros and cons.