| I am using httrack to make an offline copy of
<https://literotica.com>
I have disabled the security limits and set max connections/sec to 50, max
transfer rate 200 and number of connections 30.
when its scanning <https://literotica.com/s> where most of the stories are, it
scans one link at a time, which is taking forever and there are a couple of
million links on site.
but the thing is when it scans <https://literotica.com/tags/>
it uses the full 30 connections, so why the discrepancy?
could MIME settings or something else be used to speed up this process.
any help would be appreciated
thanks.
| |