> Scan rules depend on what you find in robots.txt, and the
> site you want to mirror.
> Which site is it and which folder do you need ?
Not sure what you mean by 'what you found in robots.txt' I 
was trying to download www.learningpage.com but I couldn't 
get it to work. The next time I watched the screen 
intently as I started the download and 'error  
=robots.txt' flashed up on the screen then disappeared. 
the other times I had tried to download the site I had 
been distracted from the screen and had missed this.

So I disabled robots.txt and have managed to download most 
of the site. I have encountered the %pdf message on some 
of the files though, that I see someone else has also had, 
but I have been unable to find if there was ever a 
solution to that problem.