| Subject: Re: unable to backup certain folders |
Author: Xavier Roche |
Date: 08/15/2005 14:42 |
| | > I am getting this error when backing up a site but am not sure which options
to change:
> due to www.abc.com remote robots.txt rules, links
> begining with these path will be forbidden:
The site has a robots.txt policy that prevent you from downloading some parts
of it. You can override these policies (see Options / Spider), but only with
great care (reduce the number of connections to 2, for example, and lower the
maximum transfer rate from 25000 to 10000 bytes/seconds)
| |
|
|
|
|
8
Created with FORUM 2.0.11