HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Get only urls
Author: Matt
Date: 10/01/2017 02:27
 
To Crawl a site, you start with one page, download it, then parse it for links,
then download all those pages, then parse all them, repeat.....


So Can you get only a list of URLS without downloading the whole site... NO

IF you use the "Make an index" then you would have a list of all the local
files, which were all the links that were followed. So you should be able in
most cases to extract the urls from that. and modify them back to web urls
from the local version.
 
Reply Create subthread


All articles

Subject Author Date
Get only urls

09/08/2017 14:49
Re: Get only urls

10/01/2017 02:27
Re: Get only urls

03/07/2020 12:55




4

Created with FORUM 2.0.11