HTTrack Website Copier
Free software offline browser - FORUM
Subject: Disabling robots.txt file
Author: Rebekah
Date: 04/18/2004 09:10
 
Quote from FAQ's <Info: Note: due to www.foobar.com remote 
robots.txt rules, links begining with these path will be 
forbidden: /cgi-bin/,/images/ (see in the options to 
disable this) If you want to disable them, just change the 
corresponding option in the option list!>

Does 'see in the options' refer to the 'set options' 
button? If so where is the 'corresponding option in the 
option list' that you change to override it?
If not, where is the 'options' referred to?
 
Reply


All articles

Subject Author Date
Disabling robots.txt file

04/18/2004 09:10
Re: Disabling robots.txt file

04/18/2004 10:29
Re: Disabling robots.txt file

04/19/2004 00:41
Re: Disabling robots.txt file

04/19/2004 09:16
Re: Disabling robots.txt file

04/26/2004 00:08
Re: Disabling robots.txt file

07/10/2009 13:22
Re: Disabling robots.txt file

04/26/2013 06:06




6

Created with FORUM 2.0.11