Always post the command line used (log file line two)
> I am trying to copy www.healthteacher.com. What is
> all this other stuff?
> <http://img141.imageshack.us/img141/3510/unledfk.png>
Your image shows a bunch of robots.txt from other sites.
Not all sites are fully contained on the initial site. Some store images and
javascript files on other sites.
HTT has to read a robots.txt from each it downloads from.
|