HTTrack Website Copier
Free software offline browser - FORUM
Subject: Q: Never download existing Binary Files again.
Author: Mani
Date: 12/07/2004 12:43
 
Hi all,

first of all thank you for this great software. Very 
usefull!!!

But I have a problem with HTTrack (I use 3.32-2 (+swf)) and 
I don't know if it is just a question of configuration or 
if HTTrack has not implement such a feature.

I want to mirror parts (only the Helicoptes for Flight 
Simulator 2004) of the links in the URL:
<http://www.simviation.com/sitemap.html>

So I use this URL as the WebAddress.
I configured following rules:

1) -*
2) +*fs2004helicopters*
3) +*.zip +*.tar +*.tgz +*.gz +*.rar +*.z +*.exe
4) +www.simviation.com/files/1helicopters/*
5) +63.217.28.218/1helicopters/*
6) +www.simviation.com/pageimages/*

Explanation:
1) As there are so many links on the Startpage (a Sitemap) 
above I decided to exclude everything first.
2) Include all links with fs2004helicopters in the filename
3) Include all archives and binaries I need
4-6) Include only links from this path

Other settings:

Limits:
"Max connections/seconds = 1"

Flow Control:
As the FTP-Site 63.217.28.218 only accept maximum 2 
connections I set
"Number of connections=1" (with 2 I still got errors)
"KeepAlive=0" (So that the connection should be closed 
after a download)
"Retry=2"

Links:
Attempt to detect all links => checked

Spider (I think this is default):
Accept Cookies => checked
Parse java files => checked
Update hack => checked

Log files, Index, Cache:
Create Log files => checked
Make an index => checked

So these settings are mirroring exact what I want. 
But sometimes I got errors. 

The log have some entries:
Error 1:
14:32:04	Error: 	"Unexpected write error" (-1) at 
link <ftp://63.217.28.218/1helicopters/xxxxxx.zip> (from 
www.simviation.com/fs2004helicopters2.htm)  

What does this "Unexpected write error" mean? I have no 
problems with my harddisk and I have enough space free.

Error 2:
14:32:14	Error: 	"Bad user name: Sorry, the maximum 
number of clients (2) from your host are already 
connected." (-1) at link 
<ftp://63.217.28.218/1helicopters/xxxx.zip> (from 
www.simviation.com/fs2004helicopters2.htm)

But that is why I limit the maximum number of connections 
to 1! There is no other application running which is 
connected to this URL. Could that be that the connection is 
not closed correctly sometimes or that HTTrack allready 
opens the next connection if the first is not closed?
Error 3:
15:01:08	Error: 	"" (0) at link 
<ftp://63.217.28.218/1helicopters/x1x1x1x1.zip> (from 
www.simviation.com/fs2004helicopters4.htm) 

What does this error message tells me?
All these errors are normaly no major errors if the update 
feature of HTTrack would work correctly. So if I start the 
mirroring again (using the action "*Update existing 
download") then all links on the ftp-site (which are only 
binaries) will be mirrored again. It doesn't matter if this 
file was allready downloaded. Binaries which are linked to 
the http-Site are not downloaded again.
In this context I found out that HTTrack couldn't retrieve 
the filesize if it download a file from the ftp-site 
<ftp://63.217.28.218>. 
Could that be the reason why HTTrack isn't able to validate 
the local file with the one of the server and that is why 
it started the download again?That would be astonishing because if I download
the link 
using other applications (ex. Internet Explorer or Download 
Accelerator Plus) these applications have no problem to 
find out the size of the files.

Last question (which would be the solution for me): Is 
there a setting in HTTrack to prevent it from downloading 
binaries again if the filename allready exist on the local 
machine (doesn't matter if the file has the wrong size.)?
Thank you for support and thank you if you read all this to 
this end.

Regards,
Manfred

PS: Yes I use the search Option in the forum but I only 
found some entries which belong to similar problems (not 
with ftp):
<http://forum.httrack.com/readmsg/8193/index.html>
<http://forum.httrack.com/readmsg/4047/index.html>
 
Reply


All articles

Subject Author Date
Q: Never download existing Binary Files again.

12/07/2004 12:43
Re: Q: Never download existing Binary Files again.

07/22/2007 22:23




5

Created with FORUM 2.0.11