| >The only reason (if the problem does not generate any
alert
>in hts-log.txt) I see is that the server did not give any
>Content-length, and broke the connection. Ensure that you
>are crawling the site using reasonnable settings (2 or 3
>simultaneous connection, maximum), and also ensure that
you
>are using a recent (example: 3.22 or 3.23) release.
I am using version: 3.22-3 (from about), no errors in the
log, I have it set to only have 1-2 (usualy 2) connections
open (I don't wan't to bog the site down, heh). It sounds
like what you sugjest with the server not sending a content-
length that the page would end in different spots. Every
single solitary time I do an attempt at downloading the
page is cut off on the same byte as previous attempts. I
haven't seen a common character or I would assume that its
an equivalent eof character.
It is only the letter b, index2, and 1 or 2 other pages (I
don't remember the other pages as I figured fix bindex2 and
others would be fine). I would assume some server side
problem if the cutoff was random rather than exactly same
spot each scan. A thought occurs to me though, might it be
that the server is giving an inacurate Content-lenth (too
small) rather than not send one at all? That would
certainly explain the cutoff point being the same all the
time. | |