| > 17:26:22 Info: Note: due to www.ted.com remote
> robots.txt rules, links begining with these path
> will be forbidden: /index.php/profiles/browse (see
> in the options to disable this)
Self explanatory.
<http://www.ted.com/robots.txt> says:
User-agent: *
Disallow: /index.php/profiles/browse
> 17:26:24 Warning: File not parsed, looks like
> binary: www.ted.com/
> 17:26:24 Error: "Open error when decompressing"
> (-1) at link www.ted.com/ (from primary/primary)
I've seen this on some sites that don't like the default browser id. Change to
something else. I use:
Mozilla/4.0 (compatible; MSIE 5.0; Win32) | |