| Hi,
I was just reading the "plugging function" docs at
<http://www.httrack.com/html/plug.html> and didn't understand how to implement
my function, specifically for mirroring binary files (images).
I want to do this: for a given URL, HEAD request the first N bytes, md5sum
them, and using this md5sum decide whether or not to proceed with a full GET
of all bytes. It looks like "check_link" is close to what I want, but it
doesn't appear to have the referring page context I need (as many web servers
block requests with an empty or invalid "Referer" header to prevent
hot-linking).
It looks like I might be able to use one of the html processing plugs
(check_html, preprocess, postprocess), but this doesn't really seem
appropriate ; I'm not trying to modify any HTML, just make a decision of
whether or not to follow/retrieve a particular link.
-Allen | |