| we are using httrack 3.33 (commandline at debian) to create a mirror, and zip
that mirror every night.
but not all pages are mirrored, and httrack is really slow.
we call httrack using the next line:
sudo httrack <http://example.website/> -O "/full/path/to/mirror_location" -I0
--user-agent "new_user_agent" -%U test_webuser
after httrack is being finished, we zip the mirror.
i read the hts-log.txt file, and there were just 2 lines in it:
22:00:04 Info: Note: due to example.website remote robots.txt rules, links
begining with these path will be forbidden: /error/, /stats/ (see in the
options to disable this)
22:00:08 Warning: Unexpected 412/416 error (Requested Range Not Satisfiable)
for example.website/img/intra_navsub_arrow_active.gif
No files purged
i tried searching for a fix, but could not find an answer to this particular
problem.
can anyone please help us? thanks! | |