HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: download 100,000+ files
Author: Xavier Roche
Date: 08/11/2013 21:14
 
> I am using a RESTful server to lookup specific
> values from a database. Each query is a unique URL.
> The response is usually an XML or JSON file of 1 - 5
> KB. On many occasions I need to look up more than
> 100,000 values (using more than 100,000 URLs).
> However, HTTrack stops after processing ~ 99,000
> URLs.

Add -#L1000000 to the list of URLs

See
<http://forum.httrack.com/readmsg/676/671/index.html?q=100000>
 
Reply Create subthread


All articles

Subject Author Date
download 100,000+ files

08/10/2013 21:42
Re: download 100,000+ files

08/11/2013 21:14




8

Created with FORUM 2.0.11