HTTrack Website Copier
Free software offline browser - FORUM
Subject: I can't get Maximum Mirrorring Depth to work
Author: SP
Date: 08/10/2024 15:41
 
Hi, I am trying to download a fandom.com wiki, which are typically a cluttered
mess full of junk, but is unfortunately the wiki platform of choice for a lot
of games. Because of all that clutter I tried to keep it as basic as possible;
I browsed "Special:All Pages" and then included every page from that, manually
because pagination is not numbered, which turned out to be just seven:

<https://sunlesssea.fandom.com/wiki/Special:AllPages>
<https://sunlesssea.fandom.com/wiki/Special:AllPages?from=Another+Day%3A+Where+The+Wood+Was>
<https://sunlesssea.fandom.com/wiki/Special:AllPages?from=Fortune+%26+Fortune>
<https://sunlesssea.fandom.com/wiki/Special:AllPages?from=Never+Knew+Ye>
<https://sunlesssea.fandom.com/wiki/Special:AllPages?from=Secure+Compartment>
<https://sunlesssea.fandom.com/wiki/Special:AllPages?from=The+Man+who+Stole+the+Sun>
<https://sunlesssea.fandom.com/wiki/Special:AllPages?from=You+have+defeated+a+Beloved>

these are my base web addresses. Then I set:

- Maximum mirroring depth to 2 (catch every link these seven pages link to)
- Maximum external depth to 2 (in my desperation, trying to see if maybe
HTTrack is considering the links I want to be external pages because the
linked urls are not sister nodes, not child nodes, of those seven parents)

and the these scan rules:
+static.wikia.nocookie.net/sunlesssea_gamepedia/*
(all non-junk image files are hosted here)
-*.fandom.com/*
(don't download all the spam from other wikis that have nothing to do with
this one)
+sunlesssea.fandom.com/wiki/*
(overwrite an exception for the wiki I actually want)
+sunlesssea.fandom.com/*.php*
(overwrite an exception for these parameterized php atrocities that they use
to download every css and js file)

It doesn't do what I was hoping for. Setting depth 2 basically acts like I
just set zero depth and spiders almost nothing at all. It downloads the seven
pages, some junk external links but not others, and at this point I'm at a
loss about what else to try. Anyone knows what am I doing wrong?
 
Reply


All articles

Subject Author Date
I can't get Maximum Mirrorring Depth to work

08/10/2024 15:41




6

Created with FORUM 2.0.11