HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: weird stuff mirroring a site
Author: William Roeder
Date: 05/31/2012 03:34
 
> I can not get these 2 files:
> www.evilbible.com/images/redwhitebluegradient.jpg
> www.evilbible.com/images/evil_bible_banner.jpg
> 
When I clicked on the above links in FF, the site returned forbidden. After
opening the page, the links worked.
The problem is not robots.txt, but something funky the site is doing, perhaps
with the referring tag.
Keep robots = none, and just update the site. Worked for me.
 
Reply Create subthread


All articles

Subject Author Date
weird stuff mirroring a site

05/30/2012 21:01
Re: weird stuff mirroring a site

05/31/2012 03:34
Re: weird stuff mirroring a site

06/01/2012 01:41
Re: weird stuff mirroring a site

06/04/2012 22:50




6

Created with FORUM 2.0.11