HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: Only downloading first 5 and last page of forums
Author: IrishMarty
Date: 01/18/2025 18:04
 
I found a solution.

I grabbed the link for every individual page from the forum, and set the max
internal mirror depth to 1. The number of links was over 29,000. I used a
scraping tool called ParseHub and a Firefox extension called "Click Copy Link"
to speed up this process.

In order to achieve the same appearance of the site offline, I had to add a
couple of external links to the scan rules. To find the necessary external
sites I checked the network GET requests on the Inspect browser tools.

The main one was googletagmanager, which seemed to be responsible for the
site's appearance.
 
Reply Create subthread


All articles

Subject Author Date
Only downloading first 5 and last page of forums

01/15/2025 15:54
Re: Only downloading first 5 and last page of forums

01/18/2025 18:04




3

Created with FORUM 2.0.11