HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: recursive scanning is a problem
Author: joseph
Date: 12/13/2002 06:55
 
Thank you for checking. Luckily when a project or site was 
already downloaded, then the files were not downloaded 
again as you indicated should be the case. But I noticed 
when the project or site was downloaded for the first time, 
files may get downloaded more than once during the scan. 
Perhaps Httrack is not keeping an accurate record of files 
downloaded while scanning was in progress.

You may check the following link in the sample below. I 
picked it to demonstrate that files are downloaded more 
than once when links go back to the same pages in a new 
project. They are not downloaded again if the project 
already exists.

By scanning the following link with some options set, I get 
the same files downloaded three times. With my slow modem 
connection, I had time to monitor the status in Httrack and 
the files on the hard drive. It seems Httrack did not check 
for an existing file because it kept overwriting the same 
files in the same folders, for 
example, /software/miditype.exe, midipgms.exe, 
midiview.exe, etc. I watched the file timestamps and file 
sizes as they were being added to folders.

Sample:
a new project was created that did not exist
url, <http://www.borg.com/~jglatt/progs/software.htm>
action = download web site
limits, mirroring = 2
limits, external = 0
primary scan = store all files
travel mode = up & down
global travel = same domain
rewrite links = relative/absolute
spider = no robot
links, detect all links = yes
links, get non-html = yes
links, get html first = yes
etc.


Thanks. Any suggestions you may have is appreciated.
Joe
 
Reply Create subthread


All articles

Subject Author Date
recursive scanning is a problem

11/29/2002 01:25
Re: recursive scanning is a problem

11/29/2002 07:37
Re: recursive scanning is a problem

12/13/2002 06:55
Re: recursive scanning is a problem

12/16/2002 22:13
Re: recursive scanning is a problem

12/23/2002 18:14
Re: recursive scanning is a problem

02/16/2003 15:04
Re: recursive scanning is a problem

03/01/2003 22:24




9

Created with FORUM 2.0.11