| Subject: Re: Only downloading first 5 and last page of forums |
Author: IrishMarty |
Date: 01/18/2025 18:04 |
| | I found a solution.
I grabbed the link for every individual page from the forum, and set the max
internal mirror depth to 1. The number of links was over 29,000. I used a
scraping tool called ParseHub and a Firefox extension called "Click Copy Link"
to speed up this process.
In order to achieve the same appearance of the site offline, I had to add a
couple of external links to the scan rules. To find the necessary external
sites I checked the network GET requests on the Inspect browser tools.
The main one was googletagmanager, which seemed to be responsible for the
site's appearance. | |
|
|
|
|
3
Created with FORUM 2.0.11