HTTrack Website Copier
Free software offline browser - FORUM
Subject: download 100,000+ files
Author: tom_swift
Date: 08/10/2013 21:42
 
I am using a RESTful server to lookup specific values from a database. Each
query is a unique URL. The response is usually an XML or JSON file of 1 - 5
KB. On many occasions I need to look up more than 100,000 values (using more
than 100,000 URLs). However, HTTrack stops after processing ~ 99,000 URLs.

Is there a way to increase or remove the limit?
I see reference on the site by folks that say they have increased the limit,
but I don't see their explanation of how they did it.

Thank you in advance for your advice.
 
Reply


All articles

Subject Author Date
download 100,000+ files

08/10/2013 21:42
Re: download 100,000+ files

08/11/2013 21:14




8

Created with FORUM 2.0.11