HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: parsing html files taking too long
Author: Xavier Roche
Date: 12/27/2004 21:49
 
> i get around 35mb, then the
> active connections drops from 10 to 1, and stays that 
way. I
> have read in other posts and they were told to change
> something in the 'mime' configuration.

Hum, this is unfortunately the typical case where httrack 
is a bit confused: most html files do not have any type 
(instead of 'www.example.com/foo.html', you just have 
'www.example.com/foo') and because httrack has to "know" 
the filetype *before* naming it (because of a questionnable 
design choice :p), it spends a lots of time testing each 
link.
There isn't any real solution to speedup the process - 
these cases are rares, but quite annoying for httrack.

I'll rewrite the heap download routines one day, with a 
post-download naming system. This should speedup the whole 
thing (but this will require some hard and rough coding)

 
Reply Create subthread


All articles

Subject Author Date
parsing html files taking too long

12/27/2004 09:46
Re: parsing html files taking too long

12/27/2004 21:49
Re: parsing html files taking too long

12/28/2004 12:48




5

Created with FORUM 2.0.11