Won't it be possible to think about an option which allows 
not to let the engine retest 
all links to see if they have been updated - or not ?
I'm trying to download an huge site (NASA for hubble 
pictures) an with a little modem I cannot obtain all the 
site in one time. But when I ask for update, it take 3 or 4 
hours to retest and when it really begin to download new 
files, I have to stop downloading.

So in a future version ?