| Hi,
I am trying to scrape ALL pages from within
<http://www.iov.co.uk/index.pl?n=999;section=54>
This is a password protected area of the site - for which I legitimately have
the credentials.
I have amended the Browser ID to "Mozilla/4.0 (compatible; MSIE 6.0; Windows
NT 5.0)" (just a random choice) - & also selected "no robots.txt rules". I
have copied my cookies to the project file.
I am able to retrieve (over 100+) public pages from this site which is great.
I and can also pull pages from the protected area, however these are not
correct; I simply get the with the logo centered horizontally at the top of
the page.
Does anybody have any ideas of what I am doing wrong?
Many thanks in advance,
Andy | |