| > I am trying to save some of the Library of Congress
> to disk.
> In particular 16th Congress 2nd session.
> They debated "citizen". I am interested saving the
> pics rather than the data bases although they may
> help me also.
It seems to work AFAIKS, when disabling default robots.txt rules (please setup
lower transfer limits, however, such as a low bandwidth limit of 5KB/s)
You may want to use scan rules, however, to limit the scope of the mirror
> Eventuraly I would like to collect the whole site
> at:
> <http://lcweb2.loc.gov/ll/>
Beware, this site might be huge. My advise is to setup "low" bandwidth limit
(5KB/s) and take the time to mirror everything, not to clobber the server's
bandwidth.
| |