Re: Disabling robots.txt file - HTTrack Website Copier Forum

Free software offline browser - FORUM

Subject: Re: Disabling robots.txt file

Author: D_A

Date: 04/19/2004 09:16

San rules depend on what you find in robots.txt, and the
site you want to mirror.
Which site is it and which folder do you need ?

Create subthread

All articles

Subject	Author	Date
Disabling robots.txt file		04/18/2004 09:10
Re: Disabling robots.txt file		04/18/2004 10:29
Re: Disabling robots.txt file		04/19/2004 00:41
Re: Disabling robots.txt file		04/19/2004 09:16
Re: Disabling robots.txt file		04/26/2004 00:08
Re: Disabling robots.txt file		07/10/2009 13:22
Re: Disabling robots.txt file		04/26/2013 06:06

6

Created with FORUM 2.0.11