| > > Your log shows your mirroring blackviper.com not
> www.blackviper.com.
> Doesn't matter. I also tried with www, but with no
> luck.
It matters if the site contains absolute paths as httrack won't mirror
external sites by default.
> > You override this with options -> spider.
> Could you please tell me which exactly option
> overrides robots.txt protection?I assumed you were using winHttrack.
<http://httrack.com/html/fcguide.html> contains the entire httrack manual.
-sN follow robots.txt and meta robots tags (0=never,1=sometimes,* 2=always)
(--robots[=N])
-s0
> still gettings "403" error which says access to the
> sites is forbidden.
>
> If I can access the web site using my web browser,
> may be I should try to change httrack's browser ID
> to that one Konqueror uses?I tried the site, it rejects browser ID
containing httrack. Worked fine with:
-F "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"
| |