| Subject: Re: Thousands of files and I don't understand why |
Author: William Roeder |
Date: 07/02/2012 14:25 |
| | > Directory Listing:
> <http://www.freeimagehosting.net/3afnf>
Shows lots of search pages.
The robots.txt said not to get them:
User-agent: *
Disallow: /search
Allow: /
But you overrode robots: -s0
So you got them. | |
|
|
|
|
8
Created with FORUM 2.0.11
