> robots txt?????
Rules defined by the remote server to prevent spiders from 
downloading parts of the website that could be useless.

See the hts-log.txt log file, there must be something 
relevant inside.