| Subject: how to have maximum performance and not hammer URL |
Author: Jim Zaba |
Date: 01/05/2005 01:27 |
| | I need to download e.g. 500 URLs
I do not want to hammer them. Can I slow down spider with
some sleep (10 min.)
10 min. is time for access the URL
Is this SW smart that I can set 10 theards.
They grab first page from each URL and then wait x min.
(10 min - time of execution)
Thanks,
Jim | |
|
|
|
|
d
Created with FORUM 2.0.11