| Subject: DON't download big websites! |
Author: Michael_G |
Date: 10/18/2016 14:59 |
| | Is it possible to specify website size limit ?
I have 30 000 urls I need to scan through, I need only HTML Files. BUT Some of
the websites are very huge, so I want to skip them.
Limits -> Site size limit is very bad, it just tells the engine download 30
megabytes in total and end everything. | |
|
|
|
|
e
Created with FORUM 2.0.11