HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: No index.htm or .html to save, HTTrack rejects
Author: Vladimir
Date: 11/05/2006 05:00
 
> The author doesn't want the site to be copied.
> The access is filtered by the Browser ID.
> Ask for autorisation then change option.

Thanks. I thought that Browser ID can be changed in HTTrack, tried doing that
in settings - no result.  

Here's what's in the robots.txt:

User-agent: *
Host: www.goltis.info

User-Agent: DISCo Pump, Wget, WebZIP, Teleport Pro, WebSnake, Offline
Explorer, Web-By-Mail, Teleport Pro/1.29, Scooter-W3-1.0
Disallow: /

And the robots.txt from www.goltis.info:

User-Agent: *Disallow: /com/
Host: www.goltis.info

And here we don't even have any user-agent descriptors for downloaders. I
don't see the reason for HTTrack to refuse this site.  If user-agent can be IE
or Netscape, then why doesn't HTTrack download the site?  

P.S.  Most authors don't want their sites to be downloaded.  But hey, these
offline browsers are very vital to modem users, who don't have permanent
connections to the Net.  

 
Reply Create subthread


All articles

Subject Author Date
No index.htm or .html to save, HTTrack rejects

11/04/2006 12:10
Re: No index.htm or .html to save, HTTrack rejects

11/04/2006 14:26
Re: No index.htm or .html to save, HTTrack rejects

11/05/2006 05:00
Re: No index.htm or .html to save, HTTrack rejects

11/05/2006 11:29




8

Created with FORUM 2.0.11