HTTrack Website Copier
Free software offline browser - FORUM
Back to the forum

Subject: Re: copying sites from http://web.archive.org
Author: Xavier Roche
Date: 04/10/2010 19:01
 
> Any workarround?
For the records, yes, a tricky one, using the libbaselinks module to rewrite
links. However, it is strongly advised to dramatically reduce the number of
connections (one connection each 10 seconds for example), not to stress the
archive site, which is often overloaded.

Example:
httrack --sockets=1 --connection-per-second=0.1 \
    --wrapper
'/usr/lib/httrack/libtest/libbaselinks.so.1.0.0,http://web.archive.org/web/THE_DATE/
\
<http://web.archive.org/web/THE_DATE/http://example.com/index.html> \
    -* +web.archive.org/web/THE_DATE/http://example.com/*.html

(beware not to allow everything under web/ or you'll download the entire
archive)
 
Reply Create subthread


All articles

Subject Author Date
copying sites from http://web.archive.org

04/26/2003 08:37
Re: copying sites from http://web.archive.org

04/27/2003 08:44
Re: copying sites from http://web.archive.org

04/29/2003 09:58
Re: copying sites from http://web.archive.org

09/24/2004 00:19
Re: copying sites from http://web.archive.org

10/01/2004 10:54
Re: copying sites from http://web.archive.org

04/10/2010 19:01
Re: copying sites from http://web.archive.org

08/27/2010 19:43
Re: copying sites from http://web.archive.org

04/07/2015 16:52
Re: copying sites from http://web.archive.org

10/06/2017 13:32
Re: copying sites from http://web.archive.org

03/08/2018 10:12
Re: copying sites from http://web.archive.org

03/11/2018 05:28
Re: copying sites from http://web.archive.org

11/30/2018 13:14
Re: copying sites from http://web.archive.org

01/11/2019 16:33
Re: copying sites from http://web.archive.org

02/13/2020 12:04
Re: copying sites from http://web.archive.org

04/16/2020 17:23




8

Created with FORUM 2.0.11