HTTrack Website Copier
Free software offline browser - FORUM
Subject: Can I limit requests per second?
Author: James
Date: 04/03/2021 13:11
 
I run a command like the following
httrack <https://www.website.com/data/> -O /home/tmp/  -*master*  -c1 -%c1 -v 

However, after a while, I get kicked off with a 403 error because the
webmaster explicitly states no more than 10 requests per second, and I see
"11.6 requests per connection" in the output.

I know I have one socket  ( −cN, −−sockets[=N]) that only has one
connection per second ([−%cN,−−connection−per−second[=N] ] ), 

Is there any way to further limit requests? I have tried
--connection-per-second=0.1 but this still does not seem to target requests
per second. 


 
Reply


All articles

Subject Author Date
Can I limit requests per second?

04/03/2021 13:11




e

Created with FORUM 2.0.11