In the options menu of your task there is a tab "Spider". Inside there is a ComboBox with the Label "Spider" which is blank by default. There you can select wheteher you want to follow robots.txt or not.
Luca
anonymous
Created with FORUM 2.0.11