Re: Problem with robots.txt - pls help - HTTrack Website Copier Forum

Subject: Re: Problem with robots.txt - pls help

Author: D_A

Date: 11/28/2004 17:52

If you want to modify how httrack handles robots.txt rules
Set options / Spider then select the best option in Spider:

All articles

Subject	Author	Date
Problem with robots.txt - pls help	Ian	11/28/2004 13:44
Re: Problem with robots.txt - pls help	D_A	11/28/2004 17:52