> Is there a way to allow it to scan the entire site
> but only save files with a certain format?>
> I tried doing this:
>
> -*
> +users/*/
>
> I'm trying to save all the files in the users
> directory BUT the index page (the start location)
> does NOT have a list of all the users. It should
> crawl throughout all the pages and save only when it
> hits a page with that url format. Is it possible to
> do this?
That filter prevents the crawl so you get nothing.
If you add +*.htm +*.asp etc then you'll get all the pages plus everything on
the users
Also make that: +*users/* |