HTTrack Website Copier
Free software offline browser - FORUM
Subject: Message board retry question
Author: Michael
Date: 02/12/2010 00:40
 
I am trying to mirror a message board and I am occasionally getting the
following message in testing:

"The server is temporarily unable to service your request due to maintenance
downtime or capacity problems. Please try again later."

Is this something that can be cleared up with expanding the timeout or retry
count?  I am obeying robots.txt >

User-agent: MJ12bot
Disallow: /
User-agent: Slurp
Crawl-delay: 1
User-agent: msnbot
Crawl-delay: 1
User-agent: BecomeBot
Disallow: /
User-agent:	*
Crawl-delay: 1
Disallow:
 
Reply


All articles

Subject Author Date
Message board retry question

02/12/2010 00:40
Re: Message board retry question

02/12/2010 01:06




f

Created with FORUM 2.0.11