| Subject: Re: Request: 403.9 error recognition |
Author: Xavier Roche |
Date: 09/20/2002 07:23 |
| | > Does the spider consider all the 403.* error as fatal
> ones, like the 404 ones?
As fatal errors ; 403 (unauthorized) is used to indicate
that the page access was refused (auhtentication)
> I am having problem downloading some pages from a website
> with high traffic, the ones you get 'HTTP 403.9 - Access
> Forbidden: Too many users are connected
> Internet Information Services' from.
Argh.. this is a problem indeed, because this will
generate "random holes" in the website you are crawling..
try not to update the website during working hours ; the
server will maybe be slightly less overloaded.
| |
|
|
|
|
5
Created with FORUM 2.0.11