Yes it does but not with all websites, for example I believe it does respect a
robots.txt "no robots" directive and I think sites created with certain
technologies are problematic.