| Subject: Re: Can't find the option to ignore the robots.txt! |
Author: Xavier Roche |
Date: 11/04/2001 11:37 |
| | > Quite possible I'm stupid or blind but I can't find
> the option where I can tell Httrack to ignore the
> robots.txt file. Thuis the site I'm trying to
download
> can't be accessed. Where can I turn this off?
Options/Spider/Spider: "no robots.txt rules"
or, using commandline release: option -s0
| |
|
|
|
|
6
Created with FORUM 2.0.11