I need to download e.g. 500 URLs
I do not want to hammer them. Can I slow down spider with
some sleep (10 min.)
10 min. is time for access the URL
Is this SW smart that I can set 10 theards.
They grab first page from each URL and then wait x min.
(10 min - time of execution)
Thanks,
Jim |