| Subject: Re: same URL but different contents |
Author: William Roeder |
Date: 03/09/2011 13:48 |
| | > The problem is that HTTrack believes that all pages
> with URL www.mydomain.com/theme/index.php?prev (or
> ?next) are the same...and then stops the 'spiding
> process' on the 4th link it finds...
As well as every search engine. Webmasters that do that don't understand why
their site doesn't get much traffic.
> Is there a way of telling HTTrack to 'read' the Html
> file before saying 'I already have downloaded this
> page...
NO. How would it know when to stop?
Look for a site map or search results page. | |
|
|
|
|
3
Created with FORUM 2.0.11