| > 06:33:45 Warning: Link xxx not scanned (follow robots meta
> tag).
> I try to change Browser identity to IE Explorer 5.50, but
> result is still the same.
You can tell HTTrack to not obey those rules and act more like the browser.
Go to the options:
Spider > Spider: > change to "no robots.txt rules"
Please note that some sites use robots.txt to stop programs like HTTrack from
hammering their sites, so set your options to also have few "Number of
connections" and also limit the "Max transfer rate"
| |