| Subject: Re: httrck error when downloading this site |
Author: William Roeder |
Date: 02/07/2008 16:25 |
| | > 11:26:11 Info: Note: due to www.allenware.com
> remote robots.txt rules, links begining with these
> path will be forbidden: /cf/ (see in the options to
> disable this)
options -> Spider -> Spider = "no robots.txt rules" | |
|
|
|
|
7
Created with FORUM 2.0.11
