| The Time is unkown, because it is unknowable. When downloading a single file
the server will tell us the size and from that we can give a time estamite.
When downloading a website We don't know anything about number of files or
number of links or the overall size of all the files.
We CRAWL a site, and Literally click on every link, and then click on every
like at those links. Limited only by the 'Filter Rules' that we put in place
to stop from downloading the entire Internet.
So there is no way to know, from the outside how big a website is, so there
fore we cannot esitamet a time to completion. | |