HTTrack Website Copier
Free software offline browser - FORUM
Subject: Local conversion with httrack - speed
Author: Localhost
Date: 06/05/2014 01:18
 
Here is the scheme: .warc archive on hdd -> warc-proxy/localhost interface ->
httrack

I've been trying to make a local 122gb mirror for a week now, but annoying
speed limitations in both 3.33 and 3.48 drive me crazy. In the latter you
CANNOT disable this idiotic lock with the flag as advertised, nothing helps: I
tried putting it in a console, url and scan rules fields, shortcut target -
nothing changes. I'm so mad, I can't convert 122gb with measly 200-300 kbit.
Httrack cant't even convert the archive completely, it thinks the job done,
but actually acquires only 4-5gb. Although that's a different issue.

In 3.33 there are, apparently, no "limits" so playing with --max-rate, number
of connection, connections per second can give me a couple of megabits at
best. 

I can get the archive converted to html with cygwin's wget v15-1 in a day, the
speed is insane, it fully utilizes my hdd, but the dumb thing apparently can't
handle the mirror size and won't start link conversion afterwards.

This is so ironic: i don't need any advanced functions the program has, but do
need the basic thing it doesn't - speed. I need megabytes of speed, not
"unlocked" megabits. Put any number of zeroes after -A/--max--rate and nothing
changes.

The program is really not optimized for local environment it seems, is that
intentional?
 
Reply


All articles

Subject Author Date
Local conversion with httrack - speed

06/05/2014 01:18
Re: Local conversion with httrack - speed

06/05/2014 20:28
Re: Local conversion with httrack - speed

06/06/2014 00:49
Re: Local conversion with httrack - speed

06/07/2014 14:37




8

Created with FORUM 2.0.11