| Subject: Re: www.Direct.gov.uk |
Author: William |
Date: 09/26/2005 21:50 |
| | If the articles are in the /assetRoot/ directory on this site, here is the
reason: (www.direct.gov.uk robots.txt file)
User-agent: UKOnline
Disallow:
User-agent: *
Disallow: /assetRoot/
You need to Mirror, Modify Options and click the Spider tab. in the
robots.txt combo box, select "no robots.txt rules" | |
|
|
|
|
9
Created with FORUM 2.0.11
