| > Occasionally when it grabs a
> webpage, *supposedly* their http server will send
back
> a re-director to a different URL (usually some sort
of
> useless page, or advertisement) in place of the page
> you think you are getting. This is evident in the
log
> file 'Warning: page has moved from xxxx to yyyyy'.
> I thought at first it might be the Browser ID
Might be. Might be a problem of n# of simultaneous
connection - try to avoid to "overload" these servers,
using 1 simultaneous connection, and limiting the
bandwidth. Some servers are setting stricter limits
(per requests or bandwidth) for each incoming IP
If you have a permanent connection, using 1 connection
and limiting the bandwidth to 1-2KB is fine, you can
leave the engine working one night if necessary, and
this won't cause bandwidth problems
>Another theory...the HTTP referral? I don't know of
>any site ripping software that supports HTTP
>Referral...maybe that's the trick.
HTTP referer is supported by HTTrack :)
| |