HTTrack Website Copier
Free software offline browser - FORUM
Subject: Re: plugging function for HEAD requests
Author: Xavier Roche
Date: 05/05/2012 10:30
 
> I want to do this: for a given URL, HEAD request the
> first N bytes, md5sum them, and using this md5sum
> decide whether or not to proceed with a full GET of
> all bytes.

Humm, this isn't a HEAD request, but a regular GET request. (HEAD requests
have no data associated)

I don't think this is something possible, because it would require some
hacking deep in the code ; but in theory the only change would be to download
the first bytes in memory, checksum the file, and decide to disconnect the
socket ending (sparing bandwidth ; only few additional TCP packets would be
sent/received) upon identical checksum. 

Note that the current "update hack" include a more basic heuristic: the GET
request is issued, but immediately closed, if the file size is identical to
the previous one. This is probably not less or more safe than checksuming the
first bytes.
 
Reply Create subthread


All articles

Subject Author Date
plugging function for HEAD requests

05/04/2012 22:48
Re: plugging function for HEAD requests

05/05/2012 10:30
Re: plugging function for HEAD requests

05/05/2012 12:03




6

Created with FORUM 2.0.11