| I noticed in website copier that when you specify a
maximum number of active connections, the number of
real connections is often less than this number. At
some moment, it even falls down to one for a long
time, which dramatically slows down the crawling
process (for example, <http://www.tin.it> with a maximum
of 32 connections in parrallel + level of links = 5).
I wonder why we don't always have the maximum number
of connections open at the same time ? | |