> That's right, but the page at > <http://mostgraveconcern.com/freebsd/> contains the following > HTML code: > <META NAME='ROBOTS' CONTENT='NOFOLLOW'> > ... and httrack checks both robots.txt AND the robots meta- > tag :) Aha! :)
Steve
Leto
Xavier Roche
Created with FORUM 2.0.11