HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: PHP based websites
Author: Xavier Roche
Date: 11/23/2003 18:33
 
> I am frustrated with innormous settings and a night spent 
> in figuring out how to get past the damn robot.txt... 

Set Options / Spider / Spider: no robots.txt

Please use bandwith limits, too, to avoid to clobber the 
website, THANKS.

 
Reply Create subthread


All articles

Subject Author Date
PHP based websites

11/23/2003 12:58
Re: PHP based websites

11/23/2003 18:33
Re: PHP based websites

11/24/2003 09:41
Re: PHP based websites

11/26/2003 02:56




0

Created with FORUM 2.0.11