HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Another Forum Thread Help
Author: William Roeder
Date: 03/01/2012 17:17
 
> I tried putting in 
> www.bay12forums.com/smf/index.php?topic=19239.
> as the web address and under the scan rules I have
19239 and 19239. both show the same page. You got lucky.
> 
> -*
> +www.bay12forums.com/smf/index.php?topic=19239.*
> +mime:image/*
Clicking on page two the url is
<http://www.bay12forums.com/smf/index.php?PHPSESSID=ff0cbee5542db8cd6ef5293421bfea89&topic=19239.15>
so your filter does not work. Try:
-* +*=19239.* +mime:image/*
> 
> and put not to follow robots.txt. Should I leave max

robots contains a lot of restrictions. You should have followed it. You don't
want to mirror these:
# Default SMF Actions
Disallow: /smf/index.php?action=activate
Disallow: /smf/index.php?action=admin
Disallow: /smf/index.php?action=calendar
Disallow: /smf/index.php?action=emailuser
Disallow: /smf/index.php?action=findmember
Disallow: /smf/index.php?action=help
Disallow: /smf/index.php?action=helpadmin
Disallow: /smf/index.php?action=login
Disallow: /smf/index.php?action=logout
Disallow: /smf/index.php?action=mlist
Disallow: /smf/index.php?action=modifykarma
Disallow: /smf/index.php?action=pm
Disallow: /smf/index.php?action=post
Disallow: /smf/index.php?action=printpage
Disallow: /smf/index.php?action=profile
Disallow: /smf/index.php?action=recent
Disallow: /smf/index.php?action=register
Disallow: /smf/index.php?action=reminder
Disallow: /smf/index.php?action=search
Disallow: /smf/index.php?action=theme
Disallow: /smf/index.php?action=unread
Disallow: /smf/index.php?action=unreadreplies
Disallow: /smf/index.php?action=verificationcode
Disallow: /smf/index.php?action=who
Disallow: /smf/index.php?theme
do you?
 
Reply Create subthread


All articles

Subject Author Date
Another Forum Thread Help

03/01/2012 16:51
Re: Another Forum Thread Help

03/01/2012 17:17
Re: Another Forum Thread Help

03/01/2012 17:54




4

Created with FORUM 2.0.11