| Subject: Re: scan 1 folder, while only grabbing another |
Author: William Roeder |
Date: 04/01/2008 23:44 |
| | > In summary, I simply want to
> * crawl: site.com/a/*
> * download (and not parse): site.com/b/*
Only allow one level of b:
-* +*/a/* +*/b/* -*/b/*/* | |
|
|
|
|
|
