| I am trying to mirror a site that does not allow
crawling at faster than 8 hits / minute (average). I
have tried various settings, even setting the
simultaneous connections to 1, but still get errors on
my mirror indicating that I am crawling too fast.
This is there requirement:
Current crawling limitations per host (IP address):
hits/hour:300, hits/minute (averaged):8, simultaneous
open connections:10
Is there a way to force httrack to follow these
paramters?
Any help would be great appreciated.
Thanks,
-alan | |