| Subject: Re: robots.txt bypass |
Author: anonymous |
Date: 02/25/2022 07:52 |
| | In the options menu of your task there is a tab "Spider". Inside there is a
ComboBox with the Label "Spider" which is blank by default. There you can
select wheteher you want to follow robots.txt or not. | |
|
|
|
|
6
Created with FORUM 2.0.11
