HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Fixing pending issues on 3.42
Author: William Roeder
Date: 08/18/2008 18:30
 
> - the "updating a mirror redownload too much data"
> bug is hard to reproduce, if you have reproductible
> steps, please provide it. none of the tested sites
> (including dynamic ones) could raise the problem
> using latest 3.42 builds ("no files updates" message
> at the end of the mirror, even after several
> re-updates)

Now that I have DSL, I could spend some more time on diagnosting.  The
previous 3.4x repeatedly downloaded the exe's from www.httrack.com. The latest
3.42-3 did not.

> - the "only one connection is sometimes being used"
> issue is not really a bug, but a design consequence
> of the link scanner, which is less parallelized, to
> solve complex redirect/mime types issues (such as an
> html link redirecting to an image after several
> loops, and which shall be solved for local
> filesystem issues)

Updating httrack.com only used one connection. For html and gif, ico.  Only if
I deleted the .exe's and updated did it use multiple connections.

> - the "many temporary files, especially .delayed
> files, are left on disk" should be a 3.41 bug fixed
> in 3.42 -- please also feel free to provide
> reproductible steps if the bug is still sometimes
> floating around

Didn't see any.

> William will probably spot more pending issues :)
I compared the logs and files from the two versions. 3.32 found more links.
(Took longer because of a timeout issue.)

> HTTrack Website Copier/3.42-3 mirror complete in 3 minutes 3 seconds : 160
links scanned, 156 files written (13411687 bytes overall), 32 files updated
[212334 bytes received at 1160 bytes/sec], 337822 bytes transfered using HTTP
compression in 23 files, ratio 35%, 1.2 requests per connection

< HTTrack Website Copier/3.32 mirror complete in 4 minutes 13 seconds : 167
links scanned, 167 files written (20855939 bytes overall), no files updated
[235368 bytes received at 930 bytes/sec], 378673 bytes transfered using HTTP
compression in 27 files, ratio 35%, 1.1 requests per connection

3.42-3 renamed all favicon.ico to .html:
> 10:31:36	Info: 	engine: transfer-status: link updated:
www.httrack.com/page/1/en/favicon.ico -> %BASE%/web/favicon-4.html
< 10:48:57	Info: 	engine: transfer-status: link updated:
www.httrack.com/page/1/en/favicon.ico -> %BASE%/web/ico/favicon-5.ico
Mirrored from www.httrack.com/page/21/en/favicon.ico by HTTrack Website
Copier/3.x [XR&CO'2004], Mon, 18 Aug 2008 14:32:09 GMT
But the content was "Snapshots of WebHTTrack Website Copier"

3.42-3 renamed all fade.gif to .html:
> 10:32:15	Info: 	engine: transfer-status: link updated:
www.httrack.com/page/fade.gif -> %BASE%/web/fade-4.html
< 10:49:13	Info: 	engine: transfer-status: link updated:
www.httrack.com/page/fade.gif -> %BASE%/web/gif/fade-6.gif

3.4x correctly renamed
10:30:26	Debug: 	Testing link type (from cache)
www.pkware.com/products/enterprise/white_papers/appnote.txt
10:30:26	Debug: 	Changing link extension
www.pkware.com/products/enterprise/white_papers/appnote.txt to .html

TITLE Page has moved
META HTTP-EQUIV="Refresh" CONTENT="0;
URL=../external.html?link=http://www.pkware.com/software/enterprise/white_papers/appnote.txt

Where as 3.32 kept the extention as .txt:
Mirrored from www.pkware.com/products/enterprise/white_papers/appnote.txt by
HTTrack Website Copier/3.x [X...
Page has moved
META HTTP-EQUIV="Refresh" CONTENT="0;
URL=../../external.html?link=http://www.pkware.com/software/enterprise/white_papers/appnote.txt
 
Reply Create subthread


All articles

Subject Author Date
Fixing pending issues on 3.42

08/18/2008 11:56
Re: Fixing pending issues on 3.42

08/18/2008 18:30
Re: Fixing pending issues on 3.42

08/18/2008 20:45
Re: Fixing pending issues on 3.42

08/18/2008 20:58
Re: Fixing pending issues on 3.42

08/19/2008 00:05
Re: Fixing pending issues on 3.42

08/19/2008 10:33
Re: Fixing pending issues on 3.42

08/19/2008 11:08
Re: Fixing pending issues on 3.42

08/19/2008 17:26
Re: Fixing pending issues on 3.42

08/19/2008 17:31
Re: Fixing pending issues on 3.42

08/19/2008 17:57
Re: Fixing pending issues on 3.42

08/19/2008 18:01
Re: Fixing pending issues on 3.42

08/20/2008 04:00
Re: Fixing pending issues on 3.42

08/20/2008 09:32
Re: Fixing pending issues on 3.42

08/20/2008 09:51
Re: Fixing pending issues on 3.42

08/20/2008 15:01
Re: Fixing pending issues on 3.42

08/24/2008 01:45
Re: Fixing pending issues on 3.42

08/25/2008 15:10
Re: Fixing pending issues on 3.42

08/25/2008 15:32




c

Created with FORUM 2.0.11