Is there a way to handle soft 404 error pages in httrack?
I am crawling a website which does not return 404 for missing pages. Instead
it returns a 200 OK with a page which says -- file not found.
I searched for this problem on the internet and found an interesting example
of how a spider could handle soft 404s. Check this link --
<http://bit.ly/9Cm1rH>
-Nilesh |