| Subject: Re: [N00B] getting all *.jpg's in directory |
Author: Xavier Roche |
Date: 03/13/2004 09:20 |
| | > is there an
> option to get all *.jpg or *.gif and override this?
The only solution is to override robots.txt ('Set
Options' / 'Spider' / spider: no robots.txt)
But ensure that you don't clobber the bandwidth by setting
up limits (Set Options / Limits and Set Options / Flow
control)
You can also add additional scan rules (Set Options / Scan
rules):
+*.jpg +*.gif
But these rules should be activated by default
| |
|
|
|
|
1
Created with FORUM 2.0.11