Subject: Re: httrck error when downloading this site
Author: William Roeder
Date: 02/07/2008 16:25
> 11:26:11 Info: Note: due to www.allenware.com
> remote robots.txt rules, links begining with these
> path will be forbidden: /cf/ (see in the options to
> disable this)
options -> Spider -> Spider = "no robots.txt rules"
