HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Using Options to disable remote robots.txt ruls
Author: SJ
Date: 05/08/2020 20:43
 
Seems SV solution is working.

Click on set option -> Go to Spider ->Do Not Follow Robots.txt 

Done.


Thanks you
 
Reply Create subthread


All articles

Subject Author Date
Using Options to disable remote robots.txt ruls

09/17/2007 00:11
Re: Using Options to disable remote robots.txt rul

09/17/2007 04:28
Re: Using Options to disable remote robots.txt rul

09/24/2007 01:51
Re: Using Options to disable remote robots.txt rul

04/20/2016 18:57
Re: Using Options to disable remote robots.txt ruls

11/26/2017 14:03
Re: Using Options to disable remote robots.txt ruls

05/08/2020 20:43




2

Created with FORUM 2.0.11