> I am frustrated with innormous settings and a night spent > in figuring out how to get past the damn robot.txt... Set Options / Spider / Spider: no robots.txt Please use bandwith limits, too, to avoid to clobber the website, THANKS.
zer0bit
Xavier Roche
flood
Created with FORUM 2.0.11