HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: A tip for people having issues copying sites
Author: shawn
Date: 06/26/2008 19:23
 
Your suggestions work well and I've been using them for a while. Another option
is to have httrack pretend to be a web spider like Googlebot. Some sites let
the bots in while unregistered users don't get in. I should point out that
some sites don't behave the same while pretending to be a spider so I wouldn't
suggest you use it all of the time.

I have found one site that is proving rather interesting to copy. It's an
adult site that displays fine if you use a browser and click on one of the
thumbnails, but if you try to go directly to one of the images you will get a
Forbidden message. It's at www.creampie-4you.co.uk/avidols/akira/akira05.html
I haven't given up on it since I consider this a challenge but it is proving
rather challenging. Presumably there's some code running on the server that
detects how you are entering the site.
 
Reply Create subthread


All articles

Subject Author Date





0

Created with FORUM 2.0.11