| HTTrack is not the "best" tool to leech images or other
king of material. It is mainly used to backup sites and
make archives of live content. Default settings are
generally fine (a maximum of 25KB/s, spider signature and
default robots.txt following), but a minotity of bad users
can easily clobber a website. As they can clobber an ftp
site with a 10-threading ftp leecher.
The <http://www.httrack.com/html/abuse.html#WEBMASTERS>
page contains some hints and advise, I would suggest the
following:
- no javascript/form hacks
- robots.txt to prevent spidering large sections (images,
for example)
- hidden links that point to these sections ("fake" images)
that automatically bans the incoming IP for a given period
of time (1 hour, for example)
The abuse faq contains an example of script (and the script
can be hidden using apache rewriting rule, or php4 module
rule that allow to transform scripts into "folder names")
| |