HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: robots.txt
Author: William Roeder
Date: 05/06/2012 21:28
 
> Hi. I tried downloading this website but it seems
> robots.txt is preventing me but even when I disable
> robots.txt in the options I can't download the
> website.

> HTTrack Website Copier/3.45-4 mirror complete in 1
> seconds : 1 links scanned, 1 files written (859

Some sites do not like a HTT browser ID. I only run using msie6
 
Reply Create subthread


All articles

Subject Author Date
robots.txt

05/06/2012 15:25
Re: robots.txt

05/06/2012 21:28
Re: robots.txt

05/06/2012 23:06
Re: robots.txt

05/07/2012 15:24
Re: robots.txt

05/07/2012 22:19




1

Created with FORUM 2.0.11