| Subject: Handling soft error pages |
Author: Nilesh Chaudhari |
Date: 03/26/2010 12:02 |
| | Is there a way to handle soft 404 error pages in httrack?
I am crawling a website which does not return 404 for missing pages. Instead
it returns a 200 OK with a page which says -- file not found.
I searched for this problem on the internet and found an interesting example
of how a spider could handle soft 404s. Check this link --
<http://bit.ly/9Cm1rH>
-Nilesh | |
|
|
|
|
6
Created with FORUM 2.0.11