Re: Using Options to disable remote robots.txt rul

Subject: Re: Using Options to disable remote robots.txt rul

Author: William Roeder

Date: 09/17/2007 04:28

> robots.txt rules, links begining with these path

options -> Spider -> Spider = no robots txt rules

Also try 3.32

All articles

Subject	Author	Date
Using Options to disable remote robots.txt ruls	Stan Bucci	09/17/2007 00:11
Re: Using Options to disable remote robots.txt rul	William Roeder	09/17/2007 04:28
Re: Using Options to disable remote robots.txt rul	Anil kumar mohanty	04/20/2016 20:34
Re: Using Options to disable remote robots.txt rul	Stan Bucci	09/24/2007 01:51
Re: Using Options to disable remote robots.txt rul	Anil kumar mohanty	04/20/2016 18:57
Re: Using Options to disable remote robots.txt ruls	SV	11/26/2017 14:03
Re: Using Options to disable remote robots.txt ruls	SJ	05/08/2020 20:43
Re: Using Options to disable remote robots.txt ruls	Andrew	04/05/2025 17:00