| Hi!
I'm trying to download;
<http://forums.station.sony.com/mxo/forums/list.m>
in its entirety. A job that I will throttle and run over a few weeks..
I've had problems setting up HTTrack (first time user..).
I've set the rules;
+http://forums.station.sony.com/mxo/posts/*
+http://forums.station.sony.com/mxo/forums/*
+*.css +*.js -ad.doubleclick.net/* -mime:application/foobar
+*.gif +*.jpg +*.png +*.tif +*.bmp
+*.zip +*.tar +*.tgz +*.gz +*.rar +*.z +*.exe
But the problem is that HTTrack doesn't pick up the posts.
For example if I use that address above, it only grabs that one page. If I put
in the address of each forum, it grabs the front page of that, but doesn't
crawl the individual topics and posts, and I guess doesn't follow the pages
1,2,3 etc etc.
I'm just wondering what I'm doing wrong, or if this has been covered before,
could someone point me to a post that deals with full depth crawling?
Many Thanks!
Andrew | |