HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Error: "" (500) after 2 retries at link...
Author: Tom
Date: 05/20/2009 14:12
 
> robots.txt says:
> User-agent: *
> Disallow: /cgi-bin/
> 
> The site doesn't want you to. You'll have to
> override robots.txt setting
> 
As I wrote in my initial posting: I already tried "ignore rules in
robots.txt".
Any workaround?
 
Reply Create subthread


All articles

Subject Author Date
Error: "" (500) after 2 retries at link...

05/14/2009 01:28
Re: Error: "" (500) after 2 retries at link...

05/14/2009 01:33
Re: Error: "" (500) after 2 retries at link...

05/14/2009 16:37
Re: Error: "" (500) after 2 retries at link...

05/14/2009 20:34
Re: Error: "" (500) after 2 retries at link...

05/15/2009 04:29
Re: Error: "" (500) after 2 retries at link...

05/20/2009 14:12
Re: Error: "" (500) after 2 retries at link...

05/20/2009 16:48
Re: Error: "" (500) after 2 retries at link...

05/20/2009 19:12




3

Created with FORUM 2.0.11