| > Note for example missing flash titles, and racoon
> logo, which is referenced int he html file as:
>
> ../../www.rush.co.uk/sites/default/files/images/Raco
> on_Logo.jpg
<http://rush.co.uk/robots.txt> includes:
Disallow: /scripts/
Disallow: /sites/
you'll have to override robots.txt (options -> spider) to get the first
section of disallows. You probably should then filter out the bottom section:
-*/admin/* -*/comment/reply/* -*/contact/*
-*/logout/* -*/node/add/* -*/search/* -*/user/register/*
-*/user/password/* -*/user/login/* -*?q=*
| |