HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Repeated scanning same page.
Author: Xavier Roche
Date: 04/10/2002 20:57
 
> I tried with your first suggest, does not work, 
could 
> be a bug.
> 
> Please try <http://www.cclife.org/htdocs/cclife.nsf>
> I add the filter -www.cclife.org/htdocs/cclife.nsf*
> and it still scans it a lot of times.

Argh, this site is really terrible, it has md5-like 
random URLs AND md5-like random query strings... and 
of course random embedded links, therefore impossible 
to detect
The only solution I see is to limit the depth, say, to 
2 or 3, and if this is not sufficient, augment it and 
restert the mirror using 'Continue an interrupted 
mirror' (NOT the update feature).
 
Reply Create subthread


All articles

Subject Author Date
Repeated scanning same page.

04/10/2002 18:25
Re: Repeated scanning same page.

04/10/2002 19:01
Re: Repeated scanning same page.

04/10/2002 20:02
Re: Repeated scanning same page.

04/10/2002 20:57
Re: Repeated scanning same page.

04/10/2002 21:10
Re: Repeated scanning same page.

04/12/2002 01:29




f

Created with FORUM 2.0.11