> What robots.txt lines are recognized and honored by
> httrack, including extensions?  I once had to throw
> Crawl-Delay in to stave off Yahoo Slurp! on a
> previous server--does httrack support similar lines?
It supports the standard exclusion rules by default, but can be overriden. I
don't know about other options. Perhaps Xavier will reply.

Also by default httrack only runs 4 connections 10 con/s 100kbs max (security
limits) If you are getting contention, perhaps you need to optimize. Some
webmasters use httrack to convert their 'real' dynamic pages to static public
ones.

<http://www.httrack.com/html/abuse.html>