HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Bandwidth Thief
Author: Xavier Roche
Date: 07/12/2002 16:03
 
> My site is currently being continuously downloaded by a 
> disgruntled user, the hosting company reports that it is 
> this tool or similiar that is causing the problem. Now I 
> wish to stop this as the user is costing me 1gb transfer 
> in under a day, can anyone help?
See also:
<http://www.httrack.com/HelpHtml/abuse.html#WEBMASTERS>

The best way:

- setup a robots.txt file which will exclude too big parts 
of your site (images, movies..) like:
User-agent: *
Disallow: /images
Disallow: /movies

- if you want to block users who are passing through the 
robots.txt, create invisible links like:
<a href="/images/block.cgi"></a>
<a href="/movies/block.cgi"></a>
in your main page. 

Then, in /images/block.cgi, detect the user-IP address, and 
ban it (this can be an INSERT into a mysql database, with a 
timestamp, which will be removed after 5 minutes)

You can change the name of the script, of course, or use 
other blocking methods.

I suggest, however, NOT to block ALL users from downloading 
using offline browsers ; many users don't have permanent 
connection, and are using offline browsers not to be 
connected too much time. It greatly improves the life of 
many users, even if a minotiry of (l)users are abusing 
quite a bit.

If you have an apache server with Linux system and 
ipchains, you can use an improved method such as this 
script:

#!/bin/sh
#
test -n "$1" || exit 1
# First deny the abuser IP
/sbin/ipchains -I input 1 -p tcp -s $1 --dport 80 -j DENY
# Then, insert the lock in the database ('' = backquote)
time='''date +%s''
echo "INSERT into yourbase.yourtable(ip,time) values($1, 
$time)" | mysql ...

And every 5 minutes (crontab), fetch all timed out values, 
and do a 
/sbin/ipchains -D input -p tcp -s $1 --dport 80 -j DENY

This is the general idea - I may create a working example 
if I have time, and will put it on the FAQ.
 
Reply Create subthread


All articles

Subject Author Date
Bandwidth Thief

07/12/2002 14:06
Re: Bandwidth Thief

07/12/2002 15:22
Re: Bandwidth Thief

07/12/2002 16:03




3

Created with FORUM 2.0.11