> > I am running windows server 2012 and am currently
> > trying to back up a very large website for my
> > employer and have run into an issue with using
> > multiple instances (each instance is responsible
> for
> > one section of the site). For several days the
> CPU
> > usage of each instance is within reason, but then
> a
> > single instance's CPU usage spikes to over 70% of
> > the CPU of my computer. The CPU spike doesn't
> seem
> > to be correlated to file parsing and doesn't seem
> to
> > be linked to large file fetching.
> 
> How many links are being processed ? httrack is
> known not to be good beyond 100,000 links (millions
> links will probably cause performance issues in
> various places), and this might be the cause.
> 
> 

The total number of seeds is 6161 split into 616/instance. I don't know how
many links there are per seed, but it varies from seed to seed. Is there a way
to figure out the average or some sort of information about the state of the
crawler?Some additional info: I have only one connection/instance to keep
traffic down (we cannot sacrifice speed or processing time on the host
server). The action status is 'ready'.