| Subject: Re: *** MIRROR ERROR! *** Why? |
Author: Bo Hovgaard Thomasen |
Date: 06/29/2007 13:11 |
| | No I did not test it... and it did not work for me either, to just disable
robots.txt rules.
However, I succesfully mirrored the web page in question by telling HTTrack
to:
1: Ignore robots.txt rules
2: Set the user agent to "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)"
From the command line it is done with these parameters:
-s0 -F "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)" | |
|
|
|
|
f
Created with FORUM 2.0.11