HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: How to exclude HTTrack from a site
Author: Xavier Roche
Date: 07/27/2001 17:07
 
> > Does HTTrack check robot.txt to see if it's 
allowed 
> to get a website or not?> 
> Yes, this is the default behaviour

See also:

<http://www.httrack.com/HelpHtml/abuse.html>
 
Reply Create subthread


All articles

Subject Author Date
Re: How to exclude HTTrack from a site

07/27/2001 09:25
Re: How to exclude HTTrack from a site

07/27/2001 17:07




c

Created with FORUM 2.0.11