Set spider option not to follow robots.txt rules. In Winhttrack, it is in the Set Options/ Spider tab. sN follow robots.txt and meta robots tags (0=never,1=sometimes,* 2=always) (--robots[=N])
asdf
Kumar
Created with FORUM 2.0.11