| > Actualy the file extention is shtml but
> anyway. It works fine for 99.9% of the files but there
are
> maybe 3 files on the whole website that come in
incomplete
> (litteraly if you look at the html raw it looks like it
> just stoped downloading the page).
The only reason (if the problem does not generate any alert
in hts-log.txt) I see is that the server did not give any
Content-length, and broke the connection. Ensure that you
are crawling the site using reasonnable settings (2 or 3
simultaneous connection, maximum), and also ensure that you
are using a recent (example: 3.22 or 3.23) release.
| |