> I am frustrated with innormous settings and a night spent 
> in figuring out how to get past the damn robot.txt... 

Set Options / Spider / Spider: no robots.txt

Please use bandwith limits, too, to avoid to clobber the 
website, THANKS.