I am having the same issue with the same behaviour as the OP.

This is not a robots.txt issue - I'm trying to download from blogspot.ca