HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: unable to backup certain folders
Author: Xavier Roche
Date: 08/15/2005 14:42
 
> I am getting this error when backing up a site but am not sure which options
to change:
> due to www.abc.com remote robots.txt rules, links
> begining with these path will be forbidden:

The site has a robots.txt policy that prevent you from downloading some parts
of it. You can override these policies (see Options / Spider), but only with
great care (reduce the number of connections to 2, for example, and lower the
maximum transfer rate from 25000 to 10000 bytes/seconds)
 
Reply Create subthread


All articles

Subject Author Date
unable to backup certain folders

08/14/2005 18:53
Re: unable to backup certain folders

08/15/2005 14:42
Re: unable to backup certain folders

04/03/2013 07:47




8

Created with FORUM 2.0.11