HTTrack Website Copier
Free software offline browser - FORUM
Subject: Can't find the option to ignore the robots.txt!
Author: chetwood
Date: 11/04/2001 11:23
 
Quite possible I'm stupid or blind but I can't find 
the option where I can tell Httrack to ignore the 
robots.txt file. Thuis the site I'm trying to download 
can't be accessed. Where can I turn this off? It's a 
mirror of my old homepage that I wanna download and I 
know there's only html and gifs in.

I'm using Winhttrck 3.09c. Thanks for any info.
 
Reply


All articles

Subject Author Date
Can't find the option to ignore the robots.txt!

11/04/2001 11:23
Re: Can't find the option to ignore the robots.txt!

11/04/2001 11:37




6

Created with FORUM 2.0.11