| | Hey guys,
I'm trying to mirror a website, which I've been mostly successful in except
for a very specific problem I keep having where some sections of a page are
not mirrored successfully. I'll go more in depth about exactly what's going
wrong, but before that there's need for a warning: The site in question is
mild NSFW (no nudity or anything). The page we'll use in example is chosen
because it is basically entirely safe.
Now, to get to the problem. The page I'm trying to rip in this example is:
<https://www.theduchy.com/building-blocks-junctions/>
Though all of the menus and most of the pictures get downloaded without
problems; you'll see that on this page, there are 3 "expansion" menus, which
you click in order to load a new section. On this particular page, the first
two of these drop downs ("The Big Picture", and "Video") work as intended. But
the third does not ("Pictures & Text"). Instead, it drops down to a 3-dotted
loading image. On most pages, none of the drop down menus work effectively.
I've been struggling with finding a way to include these sections of the pages
but to no avail. So far, it seems like there's some sort of javascript
interaction which httrack fails to follow.
The options, which are essentially just defaults, of the logfile are as
follows (should be reproduceable):
HTTrack3.49-2+htsswf+htsjava launched on Tue, 28 Jan 2020 11:51:59 at
<https://www.theduchy.com/building-blocks-junctions/> +*.css +*.js
-ad.doubleclick.net/* -mime:application/foobar +*.gif +*.jpg +*.jpeg +*.png
+*.tif +*.bmp +*.zip +*.tar +*.tgz +*.gz +*.rar +*.z +*.exe +*.mov +*.mpg
+*.mpeg +*.avi +*.asf +*.mp3 +*.mp2 +*.rm +*.wav +*.vob +*.qt +*.vid +*.ac3
+*.wma +*.wmv
(winhttrack -qwr3C2%Ps2u1%s%uN0%I0p3DaK0H0%kf2A25000%f#f -F "Mozilla/4.5
(compatible; HTTrack 3.0x; Windows 98)" -%F "<!-- Mirrored from %s%s by
HTTrack Website Copier/3.x [XR&CO'2014], %s -->" -%l "en, *"
<https://www.theduchy.com/building-blocks-junctions/> -O1
"C:\Users\ArthursMyrdin\Desktop\tempy\duchytest" +*.css +*.js
-ad.doubleclick.net/* -mime:application/foobar +*.gif +*.jpg +*.jpeg +*.png
+*.tif +*.bmp +*.zip +*.tar +*.tgz +*.gz +*.rar +*.z +*.exe +*.mov +*.mpg
+*.mpeg +*.avi +*.asf +*.mp3 +*.mp2 +*.rm +*.wav +*.vob +*.qt +*.vid +*.ac3
+*.wma +*.wmv )
My question, then, is how to get this third header to also be mirrored
correctly. I would love your advice on how to get this to work! The remainder
of the logfile is copied down below. Doesn't seem like there's too much
interesting in there.
--------------------------------------------------------
Information, Warnings and Errors reported for this mirror:
note: the hts-log.txt file, and hts-cache folder, may contain sensitive
information,
such as username/password authentication for websites mirrored in this
project
do not share these files/folders if you want these information to remain
private
11:57:15 Warning: Permanent Redirect for pinterest.com/robots.txt
11:57:15 Warning: Redirected link is identical because of 'URL Hack' option:
pinterest.com/robots.txt and <https://pinterest.com/robots.txt>
11:57:15 Warning: Warning moved treated for pinterest.com/robots.txt (real
one is <https://pinterest.com/robots.txt>)
11:57:56 Warning: Permanent Redirect for
pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/&media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg
11:57:56 Warning: Redirected link is identical because of 'URL Hack' option:
pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/&media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg
and
<https://pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/&media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg>
11:57:56 Warning: File has moved from
pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/&media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg
to
<https://pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/&media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg>
11:57:57 Warning: Moved Permanently for
<https://www.theduchy.com/building-blocks-junctions/?feed=rss2&withoutcomments=1>
11:57:57 Warning: File has moved from
<https://www.theduchy.com/building-blocks-junctions/?feed=rss2&withoutcomments=1>
to <https://www.theduchy.com/building-blocks-junctions/feed/?withoutcomments=1>
11:59:48 Error: "Not Found" (404) at link
<https://www.theduchy.com/wp-content/themes/agama-pro/assets/css/images/icons/iconalt.svg>
(from
<https://www.theduchy.com/wp-content/themes/agama-pro/assets/css/style.min.css?ver=1.4.8>)
12:06:12 Warning: Permanent Redirect for <https://pinterest.com/robots.txt>
12:06:12 Warning: Redirected link is identical because of 'URL Hack' option:
<https://pinterest.com/robots.txt> and <https://www.pinterest.com/robots.txt>
12:06:12 Warning: Warning moved treated for <https://pinterest.com/robots.txt>
(real one is <https://www.pinterest.com/robots.txt>)
12:06:16 Warning: Permanent Redirect for
<https://pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/%26media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg>
12:06:16 Warning: Redirected link is identical because of 'URL Hack' option:
<https://pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/%26media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg>
and
<https://www.pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/%26media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg>
12:06:16 Warning: File has moved from
<https://pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/%26media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg>
to
<https://www.pinterest.com/pin/create/button/?url=https://www.theduchy.com/building-blocks-junctions/%26media=https://www.theduchy.com/wp-content/uploads/2019/10/building-blocks-junctions-800x800.jpg>
HTTrack Website Copier/3.49-2 mirror complete in 19 minutes 14 seconds : 142
links scanned, 135 files written (2865398 bytes overall), 5 files updated
[120253 bytes received at 104 bytes/sec], 250753 bytes transferred using HTTP
compression in 6 files, ratio 23%, 1.1 requests per connection
(1 errors, 14 warnings, 0 messages)
| |
