HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: How to increase download speed?
Author: Sergei Kulagin
Date: 12/06/2020 19:51
 
> WinHTTrack comes also with the commandline version
> of it.
> Perhaps you are more successful with your download
> job when using the command line approach.
> 
> Have a look/get inspiration at examples to increase
> the speed like here:
> 
> <https://techienotes.blog/2016/07/28/the-fastest-way>-
> to-download-a-website-offline/
> 
> <https://archiveteam.org/index.php?title=HTTrack_opti>
> ons
> 
> 

Thanks, Trakker, I followed second link >
<https://archiveteam.org/index.php?title=HTTrack_options> and wrote this
command: 

> httrack <http://www.tekkenzaibatsu.com/> -O
"H:\Different\HTTrackCommandLine\TekkenZaibatsu" --connection-per-second=50
--sockets=80 --keep-alive --display --verbose --advanced-progressinfo
--disable-security-limits -n -i -s0 -m -A100000000 -#L500000000 +*.png +*.gif
+*.jpg +*.jpeg +*.css +*.js -ad.doubleclick.net/* -mime:application/foobar
-*editpost.php?action=editpost&postid=*
-*newreply.php?action=newreply&postid=* -*showthread.php?postid=*#post*
-*/online.php/* -*search.php?s=&action=showresults&searchid=* -*search.php?s=*
-*private.php?action=newmessage&userid=*
-*member2.php?action=addlist&userlist=buddy&userid=*
-*newreply.php?action=newreply&threadid=*
-*newthread.php?action=newthread&forumid=*

It works and started downloading the website into the specified folder with -O
argument. Added all those - wildcards to exclude many pages on the forums. Now
it downloads with 50~60 KiB/s speed: <https://i.imgur.com/l3TuS3e.png>

It downloaded 25 MB in almost 6 minutes and fully downloaded 18 pages from
what I see in the console(formatting is glitched or something). If my math is
correct and website has 30000 pages, and it'll download one page in 20
seconds, then it will take 600 000 seconds to download the website, which is
almost 7 days of continuous work(the website will stop working by then
probably). Website's speed and my own is just fine. Site loads instantly and I
get 100 Mbit/s on speedtests and while downloading in browsers and in torrent
clients.

I wonder, why is it still so slow? Browsers open new pages in less than one
second. I think it would be just fine if httrack opened one page in one
second, it would still be too slow, but it would be 20 times faster than it is
right now.
 
Reply Create subthread


All articles

Subject Author Date
How to increase download speed?

12/05/2020 14:08
Re: How to increase download speed?

12/05/2020 15:11
Re: How to increase download speed?

12/05/2020 21:31
Re: How to increase download speed?

12/06/2020 19:51
Re: How to increase download speed?

12/07/2020 02:07
Re: How to increase download speed?

12/07/2020 04:00
Re: How to increase download speed?

12/09/2020 20:13
Re: How to increase download speed?

12/09/2020 21:29
Re: How to increase download speed?

12/14/2020 02:31
Re: How to increase download speed?

12/14/2020 02:57
Re: How to increase download speed?

12/14/2020 02:58
Re: How to increase download speed?

12/14/2020 03:00
Re: How to increase download speed?

12/16/2020 10:59
Re: Digital Marketing Experts in India

01/11/2021 17:40
Re: How to increase download speed?

01/11/2021 17:41
Re: How to increase download speed?

01/11/2021 17:41




5

Created with FORUM 2.0.11