HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Failed to get password protected site?
Author: William Roeder
Date: 05/29/2011 16:51
 
> lt.aspx not scanned (follow robots meta tag)
> what does that mean?The site's robots.txt says not to follow the link

> How do i fix it?set option to override robots.

You also forgot to filter out the logout url.
 
Reply Create subthread


All articles

Subject Author Date
Failed to get password protected site?

05/28/2011 22:16
Re: Failed to get password protected site?

05/29/2011 16:51




5

Created with FORUM 2.0.11