HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Disabling robots.txt file
Author: D_A
Date: 04/19/2004 09:16
 
San rules depend on what you find in robots.txt, and the
site you want to mirror.
Which site is it and which folder do you need ?
 
Reply Create subthread


All articles

Subject Author Date
Disabling robots.txt file

04/18/2004 09:10
Re: Disabling robots.txt file

04/18/2004 10:29
Re: Disabling robots.txt file

04/19/2004 00:41
Re: Disabling robots.txt file

04/19/2004 09:16
Re: Disabling robots.txt file

04/26/2004 00:08
Re: Disabling robots.txt file

07/10/2009 13:22
Re: Disabling robots.txt file

04/26/2013 06:06




6

Created with FORUM 2.0.11