| > My site is currently being continuously downloaded by a
> disgruntled user, the hosting company reports that it is
> this tool or similiar that is causing the problem. Now I
> wish to stop this as the user is costing me 1gb transfer
> in under a day, can anyone help?
See also:
<http://www.httrack.com/HelpHtml/abuse.html#WEBMASTERS>
The best way:
- setup a robots.txt file which will exclude too big parts
of your site (images, movies..) like:
User-agent: *
Disallow: /images
Disallow: /movies
- if you want to block users who are passing through the
robots.txt, create invisible links like:
<a href="/images/block.cgi"></a>
<a href="/movies/block.cgi"></a>
in your main page.
Then, in /images/block.cgi, detect the user-IP address, and
ban it (this can be an INSERT into a mysql database, with a
timestamp, which will be removed after 5 minutes)
You can change the name of the script, of course, or use
other blocking methods.
I suggest, however, NOT to block ALL users from downloading
using offline browsers ; many users don't have permanent
connection, and are using offline browsers not to be
connected too much time. It greatly improves the life of
many users, even if a minotiry of (l)users are abusing
quite a bit.
If you have an apache server with Linux system and
ipchains, you can use an improved method such as this
script:
#!/bin/sh
#
test -n "$1" || exit 1
# First deny the abuser IP
/sbin/ipchains -I input 1 -p tcp -s $1 --dport 80 -j DENY
# Then, insert the lock in the database ('' = backquote)
time='''date +%s''
echo "INSERT into yourbase.yourtable(ip,time) values($1,
$time)" | mysql ...
And every 5 minutes (crontab), fetch all timed out values,
and do a
/sbin/ipchains -D input -p tcp -s $1 --dport 80 -j DENY
This is the general idea - I may create a working example
if I have time, and will put it on the FAQ.
| |