Subject: Re: Trouble to download wordpress MU website
Author: William Roeder
Date: 06/02/2008 21:48
> On the websites I deleted it <meta name='robots'
> content='noindex,nofollow' />
Tell httrack to ignore it: options -> Spider -> Spider = "no robots.txt except
wizard" or "no robots.txt rules"
