HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re:
Author: Xavier Roche
Date: 07/17/2001 22:52
 
> Occasionally when it grabs a 
> webpage, *supposedly* their http server will send 
back 
> a re-director to a different URL (usually some sort 
of 
> useless page, or advertisement) in place of the page 
> you think you are getting. This is evident in the 
log 
> file 'Warning: page has moved from xxxx to yyyyy'. 
> I thought at first it might be the Browser ID

Might be. Might be a problem of n# of simultaneous 
connection - try to avoid to "overload" these servers, 
using 1 simultaneous connection, and limiting the 
bandwidth. Some servers are setting stricter limits 
(per requests or bandwidth) for each incoming IP
If you have a permanent connection, using 1 connection 
and limiting the bandwidth to 1-2KB is fine, you can 
leave the engine working one night if necessary, and 
this won't cause bandwidth problems

>Another theory...the HTTP referral? I don't know of 
>any site ripping software that supports HTTP 
>Referral...maybe that's the trick. 

HTTP referer is supported by HTTrack :)

 
Reply Create subthread


All articles

Subject Author Date
"File Moved" -- anti-mirroring trick?

07/17/2001 21:38
Re:

07/17/2001 22:04
Re:

07/17/2001 22:52




4

Created with FORUM 2.0.11