HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: --near option
Author: Xavier Roche
Date: 01/27/2003 22:37
> I don't know why but this option has never worked (in
> but this only gets the index.html and not the images such
> as logo.gif  Could somebody shed light on this matter.  

.. because robots.txt prevent you from doing that. (and if 
you override default robots.txt setings, PLEASE limit the 
crawl scope AND bandwidth settings, because mirroring a 
search engine is rather dangerous due to scope problems)
Reply Create subthread

All articles

Subject Author Date
--near option

01/24/2003 20:47
Re: --near option

01/27/2003 22:37


Created with FORUM 2.0.11