| Subject: Re: I do not understand how filters work |
Author: William Roeder |
Date: 08/25/2010 18:31 |
| | > I don't know why it is forced external, shouldn't
> be.
Found it. The page contains <meta name="robots" content="index, nofollow">
So you have to set options -> spider -> spider = no robot.txt rules
| |
|
|
|
|
8
Created with FORUM 2.0.11