| Subject: Re: Error: "" (500) after 2 retries at link... |
Author: Tom |
Date: 05/20/2009 14:12 |
| | > robots.txt says:
> User-agent: *
> Disallow: /cgi-bin/
>
> The site doesn't want you to. You'll have to
> override robots.txt setting
>
As I wrote in my initial posting: I already tried "ignore rules in
robots.txt".
Any workaround? | |
|
|
|
|
3
Created with FORUM 2.0.11
