HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: *** MIRROR ERROR! *** Why?
Author: Bo Hovgaard Thomasen
Date: 06/29/2007 07:40
 
Your effort to make a mirror is blocked by the robots.txt of the website. 

The page <http://www.computerhope.com/robots.txt> has the following entry:  

User-agent: HTTrack
Disallow: /

You will have to tell HTTrack to ignore robots.txt rules to mirror the web
pages.   

Read more about robot.txt rules here:
<http://www.httrack.com/html/faq.html#Q1b1> and here
<http://en.wikipedia.org/wiki/Robots.txt>
 
Reply Create subthread


All articles

Subject Author Date
*** MIRROR ERROR! *** Why?

06/28/2007 18:38
Re: *** MIRROR ERROR! *** Why?

06/29/2007 07:40
Re: *** MIRROR ERROR! *** Why?

06/29/2007 08:59
Re: *** MIRROR ERROR! *** Why?

06/29/2007 13:11
Re: *** MIRROR ERROR! *** Why?

07/02/2007 10:52
Re: *** MIRROR ERROR! *** Why?

10/15/2007 10:29




2

Created with FORUM 2.0.11