HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Remote Robot
Author: William Roeder
Date: 10/09/2007 17:04
 
> robots.txt rules, links begining with these path
> will be forbidden: /js/, /wvw/, /Imaging/ (see in
> the options to disable this)"

options -> Spider = no robots.txt rules
 
Reply Create subthread


All articles

Subject Author Date
Remote Robot

10/09/2007 16:18
Re: Remote Robot

10/09/2007 17:04
Re: Remote Robot

10/10/2007 07:43
Re: Remote Robot

10/10/2007 09:54
Re: Remote Robot

11/07/2007 07:55
Re: Remote Robot

11/07/2007 10:29




c

Created with FORUM 2.0.11