> Are you familiar with the HTTrack options?> I will make a note to write up
some of the important 
> settings that should be set by default.
> What do you think, Xavier?
Okay, some default options should definitely be set. What I 
suggest:

- default number of connection from 8 to 4, this should be 
less aggressive on small cgi's. 

- default keep-alive activated, this should limit the 
overload (less connection, less daemon spawning on the 
server) especially for "test phases"

- default bandwidth limiter to something like 25KB: this 
option will have no effect on modem users, but will prevent 
cable/broadband users from clobbering fast networks like @ 
500KB/s

(and of course still robots.txt by default)

Besides, the GUI may prompt an alert to users who wants to 
disable robots.txt, or set higer connection limits ; 
basically these are the two major options to control the 
overall speed

What do you think?