| > I am getting this error when backing up a site but am not sure which options
to change:
> due to www.abc.com remote robots.txt rules, links
> begining with these path will be forbidden:
The site has a robots.txt policy that prevent you from downloading some parts
of it. You can override these policies (see Options / Spider), but only with
great care (reduce the number of connections to 2, for example, and lower the
maximum transfer rate from 25000 to 10000 bytes/seconds)
| |