Could you please confirm that the user agent name for WinHTTrack in robots.txt
is
"httrack".
Also, could you please add this information to the documentation so that those
wishing to know hoe to enable parts of a website specifically for WinHTTtrack
can easily find out how to do so.
thanks
|