HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Unable to download (any) site
Author: Dan
Date: 04/26/2004 18:45
 
You have "s2" in your options.  This tells httracker to 
follow the rules outlined in the webmaster's "robots.txt" 
file (in the web server's root directory).  Unless the 
webmaster allows wide open access to his server (i.e. puts 
an "*" in the robots file next to "user_agent") or 
specifically allows the agent that you are "impersonating" 
to access his web site, your s2 option will crash on 
hitting the robots.txt entry for "user_agent."  

I think that's probably your problem and changing s2 to s0 
(ignore robots.txt file) might do the trick.

Cheers,
Dan.
 
Reply Create subthread


All articles

Subject Author Date
Unable to download (any) site

04/25/2004 22:12
Re: Unable to download (any) site

04/26/2004 18:45
Re: Unable to download (any) site

04/26/2004 22:18
Re: Unable to download (any) site (solved)

04/26/2004 22:44




0

Created with FORUM 2.0.11