HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Problem with robots.txt - pls help
Author: D_A
Date: 11/28/2004 17:52
 
If you want to modify how httrack handles robots.txt rules
Set options / Spider then select the best option in Spider:
 
Reply Create subthread


All articles

Subject Author Date
Problem with robots.txt - pls help

11/28/2004 13:44
Re: Problem with robots.txt - pls help

11/28/2004 17:52




1

Created with FORUM 2.0.11