HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Protection other than robots.txt ?
Author: D_A
Date: 03/04/2006 18:09
 
Changing the browser user agent seems to solve the problem.
I could download all the subdirectories (not to overload servers, I also use
limits by default -1 connection/second, slow max transfer rate ... it may be
necessary, I didn't test) 
 
Reply Create subthread


All articles

Subject Author Date
Protection other than robots.txt ?

03/04/2006 07:55
Re: Protection other than robots.txt ?

03/04/2006 07:58
Re: Protection other than robots.txt ?

03/04/2006 18:09
Re: Protection other than robots.txt ?

03/05/2006 06:52
Re: Protection other than robots.txt ?

03/05/2006 10:46




4

Created with FORUM 2.0.11