| Is it possible to specify website size limit ?
I have 30 000 urls I need to scan through, I need only HTML Files. BUT Some of
the websites are very huge, so I want to skip them.
Limits -> Site size limit is very bad, it just tells the engine download 30
megabytes in total and end everything. | |