HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Parse speed / download efficiency
Author: Xavier Roche
Date: 06/17/2003 18:52
 
> The 'parsing' and validation of links seems to take too
> long, and hence bandwidth is wasted because of a blocking

- User recent releases (such as 3.30-RCxx)

- Use 'Set Options' / 'Mime Types'
to set default mime types to dynamic files ; this should 
speed up greatly the processing

> Why have parse phases and download phases together?
This is the case ; but type testing are still slow anyway

> Why validate links while parsing on new downloads
> anyhow?
This is only done on "unknown" files such as asp, php.. to 
name the local file. Again, MIME types should fix this 
issue.

> I seem to recall that some servers support serverside
> compression of html on the fly, which again, speeds up
> performance significantly.

Supported by httrack now (gzip and deflate compression), 
among with other cool things (Keep-alive, Chunking, Etag, 
SSL..)


 
Reply Create subthread


All articles

Subject Author Date
Parse speed / download efficiency

06/17/2003 16:58
Re: Parse speed / download efficiency

06/17/2003 18:52




a

Created with FORUM 2.0.11