| > I tried with your first suggest, does not work,
could
> be a bug.
>
> Please try <http://www.cclife.org/htdocs/cclife.nsf>
> I add the filter -www.cclife.org/htdocs/cclife.nsf*
> and it still scans it a lot of times.
Argh, this site is really terrible, it has md5-like
random URLs AND md5-like random query strings... and
of course random embedded links, therefore impossible
to detect
The only solution I see is to limit the depth, say, to
2 or 3, and if this is not sufficient, augment it and
restert the mirror using 'Continue an interrupted
mirror' (NOT the update feature).
| |