| Xavier,
I agree that tools like yours are useful and that
people can abuse it. HOWEVER, if you create a
potential dangerous tool - and as your message shows
you are fully aware of it - you also need to accept
responsiblity for what people are doing with it.
I would be no problem at all to build a mechanism into
it that will read a file e.g. named httrack.txt that
doubles the robots.txt functionality but can not be
overwritten. Sure, people that can compile and modify
the source won't be stopped. But in my experience,
these are not the people abusing such tools. So you
could at least prevent most desasters from occuring.
Rainer | |