HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Dead link
Author: Xavier Roche
Date: 10/11/2001 14:37
 
> Hallo, I need a hint. I'm starting to use this 
> program. What I didn't understand is if I can use it 
> for the spidering a list of URL (directory as 
Yahoo). 

Many dynamic pages (including some yahoo groups) are 
very hard to handle. For some sites, you'll need to 
use the 'capture url' feature, or define additional 
filters. Note that some websites also setup robots.txt 
rules to prevent from spidering them.

 
Reply Create subthread


All articles

Subject Author Date
Dead link

10/11/2001 12:44
Re: Dead link

10/11/2001 14:37




1

Created with FORUM 2.0.11