HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Bad CGI-based pages
Author: Xavier Roche
Date: 12/04/2003 19:55
 
> I used HTTrack with a website which uses CGI. It happens 
> that the internal CGI mechanism which generates html 
files 
> from the database (placed on the server) - sometimes 
fails. 
> It means, you can sometimes see a site with the text 'the 
> dataserver is temporarily busy' instead of the contents 
you 
> expected.

Humm, the suggestion is interesting, but the best solution 
would be to generate on the server-side 5XX errors (500 
error code, for example), with the html message inside.
I'm not yet sure what to do with 500 error codes, by the 
way: I used to skip them and did not store then in cache, 
but this can be problematic.

Another way to discard the "bogus" pages is to mangle their 
entries in the hts-cache/new.ndx text file ; something that 
can be done with scripting.
 
Reply Create subthread


All articles

Subject Author Date
Bad CGI-based pages

12/04/2003 10:37
Re: Bad CGI-based pages

12/04/2003 19:55




3

Created with FORUM 2.0.11