| > WinHTTrack Website Copier 3.30-RC-12 (+swf)
> Still a small problem with robot.txt rules:
> When I use Winhttrack locally
> <http://127.0.0.1/site/index.php>) it follows meta tags,
but
> seems not to read robots.txt.
Strange.. did you select any specific option when running
httrack? Any robots.txt reference in hts-log.txt ?
| |