have a look there :
<http://forum.httrack.com/readmsg/10359/index.html>
and there :
<http://forum.httrack.com/readmsg/9815/index.html>
and the other threads about W3C and robots.txt...