Subject: Re: Thousands of files and I don't understand why
Author: William Roeder
Date: 07/02/2012 14:25
> Directory Listing:
> <http://www.freeimagehosting.net/3afnf>
Shows lots of search pages.
The robots.txt said not to get them:
User-agent: *
Disallow: /search
Allow: /
But you overrode robots: -s0
So you got them.
