| Subject: Re: Bug? Escaping spider. |
Author: William Roeder |
Date: 08/03/2010 22:53 |
| | > This is supposed to grab the 5 files about stirling
> engines, plus all the HTML it points to (external
> depth=1), plus all non-HTML any of these pages
> points to (. But in reality it mirrors all of
> *.howstuffworrks (where * ~= auto).
*.howstuffworks where * is not auto are BY DEFINITION external to
auto.howstuffworks and you said to GET ALL externals (to depth one) Doing
exactly what you asked it to do.
| |
|
|
|
|
0
Created with FORUM 2.0.11